Enhancements
What's New
This section consists of the enhancements by services introduced in this release.
Ambari
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1816 | Implemented the Ranger Kafka plugin for both Kafka and Kafka 3. | |
ODP-1817 | Set the Ambari and Infra Solr versions. | |
ODP-1813 | Removed Oozie and Spark 2 from the Ambari stack. | |
ODP-1627 | Added the keyadmin LDAP user. | |
ODP-1690 | Added the Knox Homepage UI to Knox via quicklinks.json. | |
ODP-1530 ODP-1612 | The Ambari changes for JDK11 Runtime Support for Hadoop, Hive, Infra-Solr, Yarn, and Druid. | |
ODP-1782 | Added the ODP-3.3 stack in the Ambari stack. | |
ODP-1724 ODP-1849 | Added the ODP stack JDK11 branch versions. | |
ODP-1336 | Added the cluster admin permission to allow user management access and fixed Python3 warnings. | |
ODP-1545 ODP-887 | Added JDK11 support for 3.x stacks. | |
ODP-1995 | Updated the Ranger version to 2.5.0. | |
ODP-1913 | Upgraded the Java library versions for CVE reconciliation. | |
ODP-2186 | Updated the jdk version in the ambari-server setup | |
ODP-1873 | AMBARI-2611 | AMBARI-26119 | AMBARI-26122 | AMBARI-2510 | AMBARI-25956 | AMBARI-26130 | AMBARI-26093 | Fixed a few security issues and components getting restarted in maintenance mode with the restart of all. |
ODP-1951 | AMBARI-26073 | AMBARI-26116 | AMBARI-26109 | AMBARI-26141 | AMBARI-26136 | Added DFSRouter via the 'Actions Button' on the HDFS summary page (#118). |
ODP-2369 | Removed the spark2-client dependency for Zeppelin from parameters. | |
ODP-2311 | Added the 3.4 stack for future ODP upgrade. |
Druid
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1904 | The ODP stack and CVE libraries versions reconciliation. |
Hadoop
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1104 | Updated the snappy-java version to 1.1.10.4 andsnappy-java version to 1.1.10.4-v2 | |
ODP-1781 | HADOOP-19116 | Updated the Zookeeper client version to 3.8.4 to fix CVE-2024-23944. |
ODP-1103 | HADOOP-15327 | Upgraded MR ShuffleHandler to use Netty4. |
ODP-1103 | Upgraded the netty4 version to 4.1.94. |
HBase
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1498 | Added the support for JDK11. | |
ODP-1900 | HADOOP-19116 | Upgraded the Guava version for CVE reconciliation. |
Hive
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1672 | Updated the LLAP Daemon log4j2 routing properties. | |
ODP-1950 | Upgraded the Hive schema version from 3.1.4 to 4.0.0. | |
ODP-1904 | ODP stack & CVE libraries versions reconciliation. |
Impala
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1642 | Upgrade the Hive schema from 3.1.4 to 4.0.0. | |
ODP-1950 | HIVE-28226 | Split Replication Commands from HiveParser.g. |
ODP-2162 | HIVE-28399 | Improved the fetch size in HiveConnection. |
ODP-2377 | Built Impala using the ODP jars. | |
ODP-1926 | Improved the Impala Ranger Authorizer with auto-populated ranger service cache and SSL management. |
Livy
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1906 | Livy-ODP and CVE versions reconciliation. | |
ODP-2095 | Fixed the Repl folder missing and netty4 version issues. Also, updated hadoop.major-minor.version as 3. | |
ODP-2117 | Added additional Java imports for pyspark. | |
ODP-1630 | Livy - porting ODP-main now commits into ODP-0.8.0-jdk11 | |
ODP-1351 | Fixed the Spark3 livy0.7 TypeError: required field "type_ignores" missing from Module and other python3 fixes. |
NiFi
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-2101 | Fixed the nifi-asana-services issue due to the error: the package com.google.gson.annotations does not exist. | |
ODP-1880 | Enabled the endpoint override option for S3FileResourceService to allow internal object store custom URLs. |
Ozone
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-2142 | Added the Ozone JAVA_GC_ ARGS for JAVA 11 compatibility. |
Phoenix
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1920 | Upgraded Commons-codec, Curator, and Thrift versions for CVE reconciliation. |
Ranger
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1816 | Added the Kafka Ranger environment for Kafka 3.0 compatibility. | |
ODP-2127 | Upgraded Bump Hbase to 2.5.8 and Zookeeper to 3.8.4 in Ranger 2.5.0 to support the ODP stack. | |
ODP-2116 | Added the missing jars for the Kafka and Ozone plugins. |
Schema Registry
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1908 ODP-2051 | Upgraded snakeyml, jackson, and scala versions for CVE reconciliation. |
Spark 3
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1486 | Built Spark 3.5.1 with JDK11. | |
ODP-756 | Included Open Table formats such as Hudi, Delta, and Iceberg to spark jar. | |
ODP-1262 | Added the support for Python 3.11. | |
ODP-1486 ODP-1303 | SPARK-45732 | Upgraded commons-text to 1.11.0. |
ODP-1699 | Fixed the spark-core test failures. | |
ODP-2020 ODP-1095 | Fixed the critical CVE patches. | |
ODP-1916 | SPARK-39653 | Done partial reverting to add populate fn for handling WritableColumnVector. |
ODP-2118 | Upgraded Hudi, DeltaLake, and Iceberg versions for open table clients. | |
ODP-1971 | Spark3 - reverted the Kafka version to 3.4.1. |
Tez
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1237 | Excluded multiple slf4j binding warns for Tez. |
Zeppelin
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1829 | Updated the pom.xml files. | |
ODP-1644 | Added two variables that were previously missed. | |
ODP-1644 | Hive 4.0.0 now supports JDBC. | |
ODP-1644 | Added libthrift and updated the Hadoop and Phoenix versions. | |
Updated to the Spark Version. | ||
Removed the unsupported interpreters. |
Kafka-2
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1921 | Updated jackson, netty, and the snappy versions in dependencies.gradle. |
Cruise Control-2
ODP JIRA | Apache JIRA | Description |
|---|---|---|
ODP-1649 | Updated the Zookeeper and netty versions in gradle.properties. |