Acceldata
ODP

Enhancements

What's New

This section consists of the enhancements by services introduced in this release.

Ambari v2.7.8.2-3

Bug ID

Apache JIRA

Description

ODP-1336

Added the cluster admin permission to allow user management access and fix python3 warnings.

ODP-1629

Changed the Ambari as per knox log4j2 format in ODP-3.2.2.0-2.

ODP-1690_python3

Added the Knox Homepage UI to Knox via quicklinks.json (#102)

ODP-1627

Added the key admin ldap user.

ODP-1873

AMBARI-26118 AMBARI-26119 AMBARI-26122 AMBARI-25107 AMBARI-25956 AMBARI-26130 AMBARI-26093

Fixed a few security issues and component getting restarted in the maintenance mode with restart all.

ODP-1816

Implemented the Ranger Kafka plugin for both Kafka and Kafka 3 (#120).

ODP-2359

Fixed the webhdfs url handling for SSL enabled hdfs clusters.

ODP-1926

Hive now populates the Ranger service cache for Impala fix.

ODP-2375

Added the Druid SSL Properties.

ODP-1353

Added the ODP-3.3 the Ambari stack.

ODP-1913

Upgraded the Java library versions for CVE reconciliation.

ODP-2577

Added the Kafka Mirrormaker 2 observability for JMX port. (#154)

ODP-2519

Added the Kafka connect observability for JMX.

ODP-2491

Migrated from init.d scripts to an alternative system for Ambari server and agent (#157).

ODP-2428

Added the support for Ubuntu22 and RHEL 9 (#156)

ODP-2617

Updated stack 3.3 and added stack 3.4.

ODP-2527

Removed the init.d script for ranger-kms.

ODP-2428

Added the support for Ubuntu 22 and RHEL 9 for new stacks.

ODP-2425

Enabled the Solr auditing for Kudu (Python 3).

ODP-2822

Added the default Ozone admin user for Ranger KMS service (#174).

ODP-2805

Fixed the Python 3 candidates and version check for ambari-python-wrap.

ODP-2719

Updated the script shebangs to use ambari-python-wrap (#168).

ODP-2752

Removed the stack ODPWIN.

ODP-2747

Removed the stacks older than 3.0 (#164)

ODP-2977

Handled the task Name Node restart timeout during the rolling upgrade.

ODP-3206

Fixed the kerberos deprecated encryption types (#186).

ODP-787

Fixed the Oozie directory permissions to support custom Oozie user (#183).

ODP-3198

Updated the Python version to 3.11.

ODP-3432

Made Ambari Python 3 compatible by replacing iteritems() with items() (#200).

ODP-3530

Added the Systemd support for Ambari and fixed the executable issue (#208).

ODP-3540

Applied multiple checks before the Ranger setup for impala to fix the Hive Client Installation failures.

Hadoop

Bug ID

Apache JIRA

Description

ODP-2583

HADOOP-11616

Upgraded Curator to version 5.2.0. #33

ODP-2635

HADOOP-16167

Fixed the Hadoop shell script for Ubuntu 18. (#44)

ODP-2634

YARN-10352

Skips scheduling on not heartbeated nodes in Multi Node Placement.(#43)

ODP-2636

MAPREDUCE-7431

Refactored ShuffleHandler and fixes issues observed after Netty4 upgrade. (apache#5311). (#47)

ODP-2638

MAPREDUCE-7433

Removed the unused mapred or LoggingHttpResponseEncoder.java (apache#5388) (#48).

ODP-2640

HADOOP-16647

Supports OpenSSL 1.1.1 LTS - patch-0.

ODP-2831

MAPREDUCE-7373

This release now uses CMAKECXX standard instead of -std=c++11 directly.

ODP-2768

Refactored shebangs for python scripts to ambari-python-wrap.

ODP-3570

Provides the NVIDIA GPU Support with MIG Compatibility for YARN 3.2.3.

Airflow

Bug ID

Apache JIRA

Description

ODP-3406

Automates quick links in case of SSL enablement.

ODP-3317

Limits the Airflow logs creation.

ODP-2757

Supports Airflow installation through tarball.

Druid

Bug ID

Apache JIRA

Description

ODP-2564 ODP-2412

DRUID-17451:

Added support for using Dropwizard and parameterized emitters together in the Druid component.

ODP-2718 ODP-2759

Refactored shebangs for Python scripts to ambari-python-wrap (#20).

Flink

Bug ID

Apache JIRA

Description

ODP-2676

Fixed the flink-shaded-zookeeper-3 version.

ODP-2774

Refactored shebangs for Python scripts to ambari-python-wrap.

HBase

Bug ID

Apache JIRA

Description

ODP-2583

Upgraded Curator to 5.2.0.

ODP-2641

Downgraded Jackson for Hadoop compatibility.

ODP-2761

Refactored shebangs for Python scripts to ambari-python-wrap.

Hive

Bug ID

Apache JIRA

Description

ODP-3080

Apache HIVE#4887

Added the cookie signature verification to HiveServer2.

HIVE-27952

Uses SslContextFactory.Server() instead of SslContextFactory (apache#4947) (#62).

ODP-2718|

ODP-2733

Refactored shebangs for Python scripts to ambari-python-wrap(#61).

Hue

Bug ID

Apache JIRA

Description

ODP-2265

Updated the Hue version to 4.11.0.

ODP-2428

Added the support for RHEL 9 and Ubuntu 22.

Impala

Bug ID

Apache JIRA

Description

ODP-3004

Defined IMPALA_SYSTEM_PYTHON {2,3} once before checking its value.

ODP-2739

Refactored to use ambari-python-wrap.

Kafka

Bug ID

Apache JIRA

Description

ODP-1921

Updated the jackson,netty and snappy versions in dependencies.gradle.

ODP-2577

Introduced Kafka Mirrormaker 2 observability for jmx port.

ODP-1924

Introduced the Kafka connect observability for jmx port.

ODP-2654 ODP-2480

Refactored the Kafka-connect jar names based on the JDK version.

ODP-2772

Refactored shebangs for Python scripts to ambari-python-wrap.

Kafka 3

Bug ID

Apache JIRA

Description

ODP-2428

Added support for RHEL 9 and Ubuntu 22.

ODP-3602 ODP-2577

Added the JMX ports for Kafka Connect 3 and Mirrior Maker 3.

Cruise Control

Bug ID

Apache JIRA

Description

ODP-3451

Added log rotation for Cruise-Control 2 and 3.

Knox

Bug ID

Apache JIRA

Description

ODP-1550

KNOX-1462

Added commons-text as an undeclared dependency and fixed Knox Log4j vulnerability (CVE-2019-17571) detected after ODP upgrade to 3.2.2.0-2.

ODP-2583

Upgraded Curator to 5.2.0.

ODP-2718 ODP-2773

Refactored shebangs for Python scripts to ambari-python-wrap.

Livy

Bug ID

Apache JIRA

Description

ODP-2050

Upgraded bootstrap to 4.6.2

ODP-2050

Upgraded jQuery to 3.7.1.

ODP-2050

LIVY-903

Addressed Netty upgrades for version 0.8.0.

ODP-2050

Upgraded jQuery from 1.10.15 to 1.13.11.

ODP-2795

Refactored Livy to use ambari-python-wrap.

NiFi

Bug ID

Apache JIRA

Description

ODP-1880

Enabled the endpoint override option for S3FileResourceService to allow internal object store custom URLs

ODP-2306

NIFI-12219 NIFI-12858 NIFI-13532 NIFI-12524

Added the Xodus Flow History Storage with migration.

ODP-2058

NIFI-12969 NIFI-12232

Ensured that if a Process Group lacks a Versioned Component ID, we use ComponentIdLookup to generate one based on its Instance ID, aligning with the approach used during flow serialization.

This guarantees consistent IDs when synchronizing flows across the cluster.

Additionally, performed code cleanup, improving failure handling during startup.

ODP-2771

Refactored shebangs for Python scripts to ambari-python-wrap.

Oozie

Bug ID

Apache JIRA

Description

ODP-2711

Updated the curator version to 5.2.0 in oozie.

ODP-2707

Removed the ZKLockService class to make Oozie compatible with curator 5.2.0.

ODP-3117

Migrated jquery-1.8.3 to jquery-3.7.1.

ODP-2070

Fixed the Log4j 1.* issue in Oozie.

ODP-2760

Refactored Oozie to use ambari-python-wrap for all scripts.

Ozone

Bug ID

Apache JIRA

Description

ODP-2583

Updated the curator version to 5.2.0 in Ozone.

ODP-2147

Fixed the log4j properties file names for all components to match the ODP stack.

ODP-2767

Refactored shebangs for Python scripts to ambari-python-wrap.

Ranger

Bug ID

Apache JIRA

Description

ODP-2583

Updated the curator version to 5.2.0 in Oozie.

ODP-1816

Added the Kafka Ranger environment for KAFKA3 compatibility.

ODP-2523

Updated the ODP stack project dependencies in ranger as target version.

ODP-1394

Updated Ozone Service Check User Policy to delegate service check user as admin only within the policy.

ODP-1456

Enabled the Kudu plugin in Ranger.

ODP-2274

Fixed the jackson-core version in kms

ODP-3190

Added a message to Kudu ranger plugin connection test.

ODP-3062

Updated try-catch block to catch IOException in EagerKeyGeneratorKeyProviderCryptoExtension.java.

ODP-3395

RANGER-4915

Fixed the default SSL ciphers for the User Sync service to enhance security.

ODP-2781

Refactored shebangs for Python scripts to ambari-python-wrap.

ODP-3326

RANGER-4775

Fixes an issue where Ranger KMS fails when using the Oracle 23 database.

ODP-3039

Added the inclusion of bcprov-jdk15on and the versioned bcpkix-jdk15on jar to the Ranger webapp Ozone libraries.

Spark 2

Bug ID

Apache JIRA

Description

ODP-2667

Added the scala-reflect dependency to core/pom.xml.

ODP-2061

Replaced log4j with reload4j in Spark2.

ODP-2656

Upgraded Jetty, Jackson, and netty-all to match Hadoop.

ODP-2583

Upgraded Apache Curator to version 5.2.0

Schema Registry

Bug ID

Apache JIRA

Description

ODP-2505

Integrated Schema Registry observability JMX ports into the ODP stack.

ODP-2765

Refactored Schema Registry to use ambari-python-wrap.

ODP-2105

Fixed Scala 2.13 and jackson 2.14.2 compatibility.

Sqoop

Bug ID

Apache JIRA

Description

ODP-1987

This release uses asciidoctor tool to compile docs instead of asciidoc and xmlto (#6).

ODP-2764

Refactored Sqoop to use ambari-python-wrap for all scripts.

ODP-2848

Fixed the Sqoop compilation for RHEL 9.

Tez

Bug ID

Apache JIRA

Description

ODP-2642

TEZ-4580

Addressed slow preemption of new containers when re-use is enabled (unmerged).

ODP-2643

TEZ-4488

This release ensured TaskSchedulerManager is initialized when the first DAG arrives.

ODP-2645

TEZ-4460

Improved Shuffle Handler reliability by correcting the usage of EMPTY_LAST_CONTENT and channel write to prevent read timeouts.

ODP-2646

TEZ-4455

Improved debuggability by adding LoggingHandler to the ShuffleHandler pipeline.

ODP-2647

TEZ-4450

Improved shuffle data fetching reliability when using CompositeRoutedDataMovementEvent for data transfer.

ODP-2648

TEZ-4383

Upgraded to Mockito 4.3.1.

ODP-2649

TEZ-4394

Optimized Netty4 ShuffleHandler by configuring it to use a single boss thread.

ODP-2650

TEZ-4334

Resolved potential deadlock in ShuffleScheduler between ShuffleScheduler.close() and the ShufflePenaltyReferee thread.

ODP-2651

TEZ-4336

Improved ShuffleScheduler to report the original exception when shuffle becomes unhealthy.

ODP-2718 ODP-2763

Refactored shebangs for Python scripts to ambari-python-wrap.

Zeppelin

Bug ID

Apache JIRA

Description

ODP-3111

Uses reload4j instead of log4j.

ODP-2782

Refactored Zeppelin to use ambari-python-wrap.

Zookeeper

Bug ID

Apache JIRA

Description

ODP-2632

Downgraded the Jetty and Jackson version to match Hadoop.

ODP-2769

Refactored shebangs for python scripts to ambari-python-wrap.

Mpack Enhancements

Hue (Mpack)

Bug ID

Apache JIRA

Description

ODP-2728

Enabled the SSL configuration for Hue through the Ambari UI.

ODP-2535

Added the HDFS automatic configurations to Hue.

ODP-3057

Added the missing package definition for Hue's MPack.

ODP-2815

Removed the compat-openssl dependency.

ODP-2826

Enabled the automatic configuration for Kerberos in Hue.

ODP-2476

Enabled the automatic configuration of Impala when it is enabled in Hue.