ODP

Search documentation… ⌘K

Acceldata Open Source Data Platform Support Matrix

Version

↑↓ navigate ↵ open esc close

Release Notes

Release Notes ODP 3.2.3.6-3

Supported Component Versions New Features Enhancements Bug Fixes Fixed CVEs Known Limitations

Release Notes ODP 3.2.3.6-3

Supported Component Versions New Features Enhancements Bug Fixes Fixed CVEs Known Limitations

What is ODP

Introduction to Open Source Data Platform (ODP) Introduction to Open Source Data Platform (ODP)

Installation

Getting Started

Meet Minimum System Requirements Collect Information Setting up the Environment

Getting Started

Meet Minimum System Requirements Collect Information Setting up the Environment

Accessing Acceldata Repositories Accessing Acceldata Repositories Deploying ODP In Production Data Centers With Firewalls Deploying ODP In Production Data Centers With Firewalls Using an Existing or Installing a Default Database Using an Existing or Installing a Default Database

Installing Ambari

Set Up the Ambari Server Installing, Configuring, and Deploying a Cluster Working with Management Packs Installing Ambari Agents Manually Obfuscating LDAP Bind Password in Ambari

Installing Ambari

Set Up the Ambari Server Installing, Configuring, and Deploying a Cluster Working with Management Packs Installing Ambari Agents Manually Obfuscating LDAP Bind Password in Ambari

Using Oracle Database with ODP

Using Ambari with Oracle Using Hive with Oracle Using Oozie with Oracle Using Ranger and RangerKMS with Oracle

Using Oracle Database with ODP

Using Ambari with Oracle Using Hive with Oracle Using Oozie with Oracle Using Ranger and RangerKMS with Oracle

Advanced Installation

Installing Oozie on ODP Installing Oozie on ODP

Installing Knox on ODP

Configure Okta SSO for Apache Knox

Installing Knox on ODP

Configure Okta SSO for Apache Knox

NiFi Setup, Security, and Registry Configuration NiFi Setup, Security, and Registry Configuration

Druid

Enabling Kerberos in Druid Enabling LDAP on Druid Enabling SSL on Druid Obfuscating LDAP Bind Password for Druid

Druid

Enabling Kerberos in Druid Enabling LDAP on Druid Enabling SSL on Druid Obfuscating LDAP Bind Password for Druid

Kafka

Kafka Connect Kafka MirrorMaker 2 Cruise Control

Kafka

Kafka Connect Kafka MirrorMaker 2 Cruise Control

Kafka 3

Kafka MirrorMaker 3 Kafka Connect 3 Kafka Cruise Control 3 Zookeeper to Kraft Migration Deployment of Standalone Kafka

Kafka 3

Kafka MirrorMaker 3 Kafka Connect 3 Kafka Cruise Control 3 Zookeeper to Kraft Migration Deployment of Standalone Kafka

Ozone

Ozone Configurations Ozone Installation Ozone Command Line Interface Working with Ozone File System Using Ozone S3 Gateway

Ozone

Ozone Configurations Ozone Installation Ozone Command Line Interface Working with Ozone File System Using Ozone S3 Gateway

Airflow

Single Node Installation MultiNode Installation Advanced Configurations Installing Custom Packages in Apache Airflow Airflow Log Clean-up and DAG Management Apache Airflow Logging Guide Running Apache Airflow Commands via CLI Set up Apache Airflow-Redis for Celery Redis Sentinel as Celery Broker for Airflow Scheduler and Webserver High Availability for Airflow Basic Authentication for Flower UI in Airflow

Airflow

Single Node Installation MultiNode Installation Advanced Configurations Installing Custom Packages in Apache Airflow Airflow Log Clean-up and DAG Management Apache Airflow Logging Guide Running Apache Airflow Commands via CLI Set up Apache Airflow-Redis for Celery Redis Sentinel as Celery Broker for Airflow Scheduler and Webserver High Availability for Airflow Basic Authentication for Flower UI in Airflow

Schema Registry Schema Registry

Modern Open Table Format integration with ODP

Deploy the Delta-Hive Connector

Modern Open Table Format integration with ODP

Deploy the Delta-Hive Connector

Kudu Kudu

Pinot

Introduction to Apache Pinot Apache Pinot Sizing Guide Install Pinot via Mpack Install Knox Batch Import using Local File Streaming Ingestion Example with Kafka 2/Kafka 3 Configure Schema and Tables in a Kerberized Environment Apache Pinot Batch Import using HDFS Operational Workflow of Pinot Pinot Known Limitations Accessing Swagger UI

Pinot

Introduction to Apache Pinot Apache Pinot Sizing Guide Install Pinot via Mpack Install Knox Batch Import using Local File Streaming Ingestion Example with Kafka 2/Kafka 3 Configure Schema and Tables in a Kerberized Environment Apache Pinot Batch Import using HDFS Operational Workflow of Pinot Pinot Known Limitations Accessing Swagger UI

Trino

Introduction to Trino Trino Capacity Planning Install Trino via Mpack Set up Trino SSL Set up Trino Knox Trino Supported Connectors Configure JMX Troubleshooting Trino Configure Trino client Trino Known Limitations

Trino

Introduction to Trino Trino Capacity Planning Install Trino via Mpack Set up Trino SSL Set up Trino Knox Trino Supported Connectors Configure JMX Troubleshooting Trino Configure Trino client Trino Known Limitations

JupyterHub

Jupyter Prerequisites Install JupyterHub via Mpack Handle HDFS and YARN Permissions Configure YarnSpawner and HDFSCM Debug Issues with YarnSpawner and HDFSCM Enable SSL for JupyterHub Configure Databases for JupyterHub Jupyter Authentication Documentation for Spark Notebook Examples Error in Submitting YARN Job Custom Packages Installation in JupyterHub Notebook Custom User for JupyterHub

JupyterHub

Jupyter Prerequisites Install JupyterHub via Mpack Handle HDFS and YARN Permissions Configure YarnSpawner and HDFSCM Debug Issues with YarnSpawner and HDFSCM Enable SSL for JupyterHub Configure Databases for JupyterHub Jupyter Authentication Documentation for Spark Notebook Examples Error in Submitting YARN Job Custom Packages Installation in JupyterHub Notebook Custom User for JupyterHub

ClickHouse

Understand ClickHouse Explore the Architecture Follow Best Practices Install ClickHouse Manage Configuration Files Apply Security and Governance Connect Integrations Use Interfaces and Tools

ClickHouse

Understand ClickHouse Explore the Architecture Follow Best Practices Install ClickHouse Manage Configuration Files Apply Security and Governance Connect Integrations Use Interfaces and Tools

Gluten with Velox Gluten with Velox

MLflow

Get Started with MLflow Manage the ML Lifecycle using MLflow Use MLflow in Real-World Projects Set Up MySQL as MLflow's Backend Store Set Up PostgreSQL as MLflow’s Backend Store Install MLflow using Ambari Mpack Run MLflow in Local or Remote Mode Store MLflow Artifacts in HDFS Store MLflow Artifacts in S3 Install NGINX and Configure for MLflow Configure Basic Authentication for MLflow

MLflow

Get Started with MLflow Manage the ML Lifecycle using MLflow Use MLflow in Real-World Projects Set Up MySQL as MLflow's Backend Store Set Up PostgreSQL as MLflow’s Backend Store Install MLflow using Ambari Mpack Run MLflow in Local or Remote Mode Store MLflow Artifacts in HDFS Store MLflow Artifacts in S3 Install NGINX and Configure for MLflow Configure Basic Authentication for MLflow

Spark RAPIDS Spark RAPIDS

Spark 3

Key Benefits Spark 3 Use Cases Install Multiple Spark 3 Versions Using Multiple Spark Versions Manage Spark 3 Configurations Troubleshooting Spark 3 Spark 3 FAQs Spark 3 Installation Conclusion Spark 3 Appendix: Quick Reference

Spark 3

Key Benefits Spark 3 Use Cases Install Multiple Spark 3 Versions Using Multiple Spark Versions Manage Spark 3 Configurations Troubleshooting Spark 3 Spark 3 FAQs Spark 3 Installation Conclusion Spark 3 Appendix: Quick Reference

Livy 3

Key Benefits Livy 3 Use Case Install Multiple Livy 3 Using Multiple Livy Versions Submitting Batch Jobs to Specific Versions Configure Client Application Manage Configurations Manage Sessions Configure Security Livy 3 Limitations Troubleshooting Livy 3 Session and Connection Pooling Monitoring and Metrics Best Practices Livy 3 FAQs Livy 3 Migration Guide Livy Appendix: Quick Reference Livy 3 Takeaways

Livy 3

Key Benefits Livy 3 Use Case Install Multiple Livy 3 Using Multiple Livy Versions Submitting Batch Jobs to Specific Versions Configure Client Application Manage Configurations Manage Sessions Configure Security Livy 3 Limitations Troubleshooting Livy 3 Session and Connection Pooling Monitoring and Metrics Best Practices Livy 3 FAQs Livy 3 Migration Guide Livy Appendix: Quick Reference Livy 3 Takeaways

Spark2 Thrift Server Spark2 Thrift Server

Zeppelin

Obfuscating LDAP Bind Password for Zeppelin

Zeppelin

Obfuscating LDAP Bind Password for Zeppelin

Ranger

Why Ranger-S3: Addressing Current Limitations Ranger Prerequisites Ranger Implementation Ranger Limitations

Ranger

Why Ranger-S3: Addressing Current Limitations Ranger Prerequisites Ranger Implementation Ranger Limitations

Hive Hive

Configuration and Management

Configuring Ports Configuring Ports Controlling ODP Services Manually Controlling ODP Services Manually

Upgrade

Master Upgrade to ODP 3.2.3.6-3

Upgrade Prerequisites

Increase the NameNode Timeout Value Create a Separate Database and Store Configs (Recommended) Create Database and Store Configs Common Steps to Perform Before the Upgrade Backup Mpack Configuration

Upgrade

Upgrade to 3.2.3.6-3 on JDK 11

ODP Stack Upgrade

Ambari Upgrade to JDK 11

Upgrade Ambari Server Upgrade Ambari Agents Upgrade Infra Solr

Configure JDK 11 Environment Changes

Configure the Java 11 Flags Automatically Using a Script

Configure the Java 11 Flags Manually

Hadoop YARN HBASE Tez Hive Oozie Druid

Resume and Finalize the ODP Stack Upgrade

Upgrade to 3.2.3.6-3 on JDK 8

Ambari Upgrade to JDK 8

Upgrade Ambari Server Upgrade Ambari Agents Upgrade Infra Solr

ODP Stack Upgrade

Master Upgrade to ODP 3.2.3.6-3

Upgrade Prerequisites

Increase the NameNode Timeout Value Create a Separate Database and Store Configs (Recommended) Create Database and Store Configs Common Steps to Perform Before the Upgrade Backup Mpack Configuration

Upgrade

Upgrade to 3.2.3.6-3 on JDK 11

ODP Stack Upgrade

Ambari Upgrade to JDK 11

Upgrade Ambari Server Upgrade Ambari Agents Upgrade Infra Solr

Configure JDK 11 Environment Changes

Configure the Java 11 Flags Automatically Using a Script

Configure the Java 11 Flags Manually

Hadoop YARN HBASE Tez Hive Oozie Druid

Resume and Finalize the ODP Stack Upgrade

Upgrade to 3.2.3.6-3 on JDK 8

Ambari Upgrade to JDK 8

Upgrade Ambari Server Upgrade Ambari Agents Upgrade Infra Solr

ODP Stack Upgrade

Upgrade Standalone Kafka Upgrade Standalone Kafka Upgrade Known Limitations Upgrade Known Limitations

USER GUIDES

Ambari Admin Guide

Using the Administrator Role in Ambari Web Setting up Ambari to use an Internet proxy server Managing Cluster Roles Managing Versions Managing Local Users Ambari High Availability Setup

Ambari Admin Guide

Using the Administrator Role in Ambari Web Setting up Ambari to use an Internet proxy server Managing Cluster Roles Managing Versions Managing Local Users Ambari High Availability Setup

Administration

Administering HDFS Managing and Monitoring a Cluster Managing High Availability

Administration

Administering HDFS Managing and Monitoring a Cluster Managing High Availability

Storage

Configuring ADLS Gen2 with ODP Configuring ODP with GCS Configuring ODP with AWS S3 using S3A Configure Apache Hive with S3A Configure ODP with Ceph S3 Ceph Using S3A

Storage

Configuring ADLS Gen2 with ODP Configuring ODP with GCS Configuring ODP with AWS S3 using S3A Configure Apache Hive with S3A Configure ODP with Ceph S3 Ceph Using S3A

Service Tuning

HDFS: Separate RPC Queues and Enable Lifeline Protocol

Service Tuning

HDFS: Separate RPC Queues and Enable Lifeline Protocol

Enabling Oozie High Availability Enabling Oozie High Availability

Troubleshooting ODP

Ambari Troubleshooting HDFS NameNode Heap Estimation Troubleshooting premature swapping Troubleshooting the Kerberos Configuration Issues in Ambari Troubleshooting JVM: JStack, Heap Dumps and JFR

Troubleshooting ODP

Ambari Troubleshooting HDFS NameNode Heap Estimation Troubleshooting premature swapping Troubleshooting the Kerberos Configuration Issues in Ambari Troubleshooting JVM: JStack, Heap Dumps and JFR

Troubleshooting ODP SSL, Keystore, and Truststore

What is SSL? Key Concepts: Keystore and Truststore Types of Certificates How to Check the Certificate Type? JKS Creation and Management Extracting Certificates and Keys Checking Subject Alternative Name (SAN) and Extensions Creating a Keystore in PKCS12 Format TLS Verification Checking SSL Ciphers Additional SSL Troubleshooting Commands Troubleshooting Ambari SSL and LDAPS Setup Services SSL and Truststore Configuration Configure the Ranger Plugin Truststore SSL SSL Troubleshooting for Hadoop Clusters SSL/TLS Security Best Practices for Hadoop Clusters

Troubleshooting ODP SSL, Keystore, and Truststore

What is SSL? Key Concepts: Keystore and Truststore Types of Certificates How to Check the Certificate Type? JKS Creation and Management Extracting Certificates and Keys Checking Subject Alternative Name (SAN) and Extensions Creating a Keystore in PKCS12 Format TLS Verification Checking SSL Ciphers Additional SSL Troubleshooting Commands Troubleshooting Ambari SSL and LDAPS Setup Services SSL and Truststore Configuration Configure the Ranger Plugin Truststore SSL SSL Troubleshooting for Hadoop Clusters SSL/TLS Security Best Practices for Hadoop Clusters

Erasure Coding for Data Durability

Understanding Erasure Coding Policies Comparing Replication and Erasure Coding Best Practices for Rack and Node Setup for EC Prerequisites for Enabling Erasure Coding Limitations of Erasure Coding Using Erasure Coding for Existing Data Using Erasure Coding for New Data Advanced Erasure Coding Configuration Erasure Coding CLI Commands Erasure Coding Examples

Erasure Coding for Data Durability

Understanding Erasure Coding Policies Comparing Replication and Erasure Coding Best Practices for Rack and Node Setup for EC Prerequisites for Enabling Erasure Coding Limitations of Erasure Coding Using Erasure Coding for Existing Data Using Erasure Coding for New Data Advanced Erasure Coding Configuration Erasure Coding CLI Commands Erasure Coding Examples

Guidelines for Enabling Ranger Auditing on Supported Plugins

Enable Ranger HDFS Audits Enable Ranger Solr Audits

Guidelines for Enabling Ranger Auditing on Supported Plugins

Enable Ranger HDFS Audits Enable Ranger Solr Audits

Security

Authentication

Replacing Knox Self-Signed Certificate with CA Certificate Enabling SSL for Services Using Bash Script Enabling Kerberos in an ODP Cluster Bash Scripts for Automated Tasks Securing NiFi with Existing CA Certificates Configuring SSL for the Ambari Server Configuring Ranger Admin and Usersync with SSSD or Centrify Enabling Cross-Cluster and Cross-Realm Kerberos Authentication for Hadoop Data Migration

Authentication

Replacing Knox Self-Signed Certificate with CA Certificate Enabling SSL for Services Using Bash Script Enabling Kerberos in an ODP Cluster Bash Scripts for Automated Tasks Securing NiFi with Existing CA Certificates Configuring SSL for the Ambari Server Configuring Ranger Admin and Usersync with SSSD or Centrify Enabling Cross-Cluster and Cross-Realm Kerberos Authentication for Hadoop Data Migration

Authorization

Configuring Ambari Authentication for LDAP/AD Securing HiveServer2 with LDAP Authentication Configuring Ranger Admin High Availability (HA)

Authorization

Configuring Ambari Authentication for LDAP/AD Securing HiveServer2 with LDAP Authentication Configuring Ranger Admin High Availability (HA)

Encryption

Disabling High Availability (HA) in ODP Ranger KMS Copying the Encrypted Data Between Two ODP Clusters Using Ranger KMS

Encryption

Disabling High Availability (HA) in ODP Ranger KMS Copying the Encrypted Data Between Two ODP Clusters Using Ranger KMS

ODP Password Obfuscation Guide ODP Password Obfuscation Guide

Uninstall ODP

Complete Uninstallation of ODP Complete Uninstallation of ODP

ODP › Upgrade › Ambari Upgrade to JDK 8

Ambari Upgrade to JDK 8

Follow the steps below to upgrade Ambari.

Upgrade Ambari Server
Upgrade Ambari Agents
Upgrade Infra Solr

Was this page helpful?

0 helpful · 0 not helpful

On this page