Acceldata
 ODP
Search documentation… ⌘K

Installing Custom Packages in Apache Airflow

  • Access the Node: Log into the node where the Apache Airflow Scheduler and Webserver are installed.
  • Navigate to the Airflow Home Directory: Move to the directory designated as the Airflow home.
  • Activate the Virtual Environment: Activate the virtual environment used by Airflow to ensure package installations are isolated and consistent.
source <virtual_environment_path>_bin_activate
  • Install the Custom Packages: Use the appropriate package manager (e.g., pip) to install any required custom packages. For example:
pip install <package_name>
  • Verify Installation: Ensure the installed packages are available by checking their version or importing them in a Python script.
  • Deactivate the Virtual Environment: After installing the packages, deactivate the virtual environment by running:
deactivate
  • Restart Airflow Services: Restart the Airflow services using the Ambari UI to apply the changes.