Bulk Workflow Migration

To convert all workflows in a single operation, utilize the setup_o2a_bulk.sh script. Ensure that all workflows are placed in a single directory, as demonstrated below.

<PARENT-DIR>/
 |- workflow1
 |- workflow.xml. - - Oozie workflow xml 
 |- configuration.properties
 |- job.properties
 |- workflow2 
 |- workflow.xml - Oozie workflow xml 
 |- configuration.properties
 |- job.properties
 |- ... - additional folders required to be copied to HDFS

The setup_o2a_bulk.sh script automates the setup of workflows and their associated configuration files into the required input directory structure, as illustrated below.

[root@airflowdemonode01 o2a]# bash setup_o2a_bulk.sh -f ~/bulk_job
[2025-04-25 14:22:39] [INFO] Starting conversion process for /root/bulk_job
[2025-04-25 14:22:39] [INFO] Created directory structure in /root/bulk_job_CONVERTED
[2025-04-25 14:22:39] [WARN] configuration.properties not found
[2025-04-25 14:22:39] [INFO] Processed subworkflow action: hive2_action
[2025-04-25 14:22:39] [WARN] configuration.properties not found
[2025-04-25 14:22:39] [INFO] Processed shell action: sehaj_errors_sample
[2025-04-25 14:22:39] [INFO] Setup completed for /root/bulk_job. You can now run ./run_o2a_bulk.sh to perform the conversion.
[2025-04-25 14:22:39] [INFO] Log files are stored in: /root/bulk_job_CONVERTED/logs

<APPLICATION>/
 |- job.properties - job properties that are used to run the job
 |- hdfs - folder with application - should be copied to HDFS
 | |- workflow.xml - Oozie workflow xml (1.0 schema)
 | |- ... - additional folders required to be copied to HDFS
 |- configuration.template.properties - template of configuration values used during conversion
 |- configuration.properties - generated properties for configuration values

Once the directory structure is established, initiate the conversion process by executing the run_o2a_bulk.sh script.

[root@airflowdemonode01 o2a]# ./run_o2a_bulk.sh
[2025-04-25 14:22:45] [INFO] Starting bulk conversion process
[2025-04-25 14:22:45] [INFO] Converting shell action: sehaj_errors_sample
[2025-04-25 14:22:45] [INFO] Input directory: /root/bulk_job_CONVERTED/shell_actions/sehaj_errors_sample
[2025-04-25 14:22:45] [INFO] Output directory: /root/bulk_job_CONVERTED/output/shell_sehaj_errors_sample
[2025-04-25T14:23:40.819+0530] {oozie_converter.py:173} INFO - Adding error handlers
[2025-04-25T14:23:40.819+0530] {oozie_converter.py:155} INFO - Converting relations between tasks groups.
[2025-04-25T14:23:40.819+0530] {oozie_converter.py:150} INFO - Converting dependencies.
[2025-04-25T14:23:40.819+0530] {renderers.py:104} INFO - Saving to file: /root/bulk_job_CONVERTED/output/shell_sehaj_errors_sample/sehaj_errors_sample.py
Fixing /root/bulk_job_CONVERTED/output/shell_sehaj_errors_sample/sehaj_errors_sample.py
[2025-04-25 14:23:41] [INFO] Conversion successful for sehaj_errors_sample
[2025-04-25 14:23:44] [INFO] Bulk conversion completed

After the conversion completes successfully, your Airflow DAG will be available in the specified output directory, such as bulk_job_CONVERTED.