Acceldata
 Pulse
Search documentation… ⌘K

Upgrade to Version 3.4.2

Prerequisite

Upgrade the Docker version to 20.10.x or above.

Steps to upgrade the Docker version:

  1. Uninstall the current Docker:
yum remove -y docker-ce docker-ce-cli containerd.io
  1. Install the latest version of Docker:
yum install -y docker-ce docker-ce-cli containerd.io
  • To install a specific version of Docker:
yum install -y docker-ce-24.0.5 docker-ce-cli-24.0.5 containerd.io
  • Replace 24.0.5 with the appropriate version string
  • To list specific versions of docker
yum list docker-ce --showduplicates | sort -r
  1. Start Docker:
systemctl start docker

Migration Steps from Version 3.3.x to 3.4.2

Pulse Upgrade Steps (Core Services Server / Addon Services Server)

  1. Update Configuration:
    • Update the ImageTag in the $AcceloHome/config/accelo.yml file to 3.4.2.
  2. Push Updated Configuration to Database:
    • Run the following command:
accelo admin database push-config
  1. Download Necessary Images:
    • Ensure you have downloaded or loaded all the necessary Docker images for version 3.4.2.
  2. Update Accelo Binary:
    • Download the new Accelo binary.
    • Replace the older Accelo binary with the new one.
  3. Run Migration Command:

Important

  • Make sure you run the command from $AcceloHome.
  • Do not change the value of -v from 3.4.0 unless explicitly advised by Engineering. The following command needs to run on all Pulse nodes including standalone nodes i.e. nodes where PULSE_SA_NODE is true. Examples of such nodes include but aren’t limited to Logsearch, FS Analytics V2, Dashplot .
  • -i : This indicates the version you are migrating from. For example, to migrate from 3.3.8 to 3.4.0, run:
accelo migrate -v 3.4.0 -i 3.3.8 -b
  • For Multi-Cluster:
accelo migrate -v 3.4.0 -i 3.3.8 -a -b
  1. Handle Migration Issues:
    • If any issues occur during migration, rerun the same command.
  2. Restart Containers:
    • Restart all containers to use the updated images:
accelo restart all -d
  1. Uninstall Hydra Agent:
    • Run the following command:
accelo uninstall remote
  1. Install Hydra Agent:
    • Run the following command:
accelo deploy hydra
  1. If Logsearch is deployed in standalone mode:
    • Check if yarn_app_es_urls is present in override.yml
    • If the above is present, change the port number from 19013 to 19051.
yarn_app_es_urls: <http://<LOG_SEARCH_HOSTNAME>>:19051
  1. Reconfigure Cluster:
    • To get the updated configuration for agents, run:
accelo reconfig cluster -a
  1. Push Updated Configuration to Database:
    • Run the following command:
accelo admin database push-config -a
  1. Configure MongoDB Cleanup & Compaction:
    • To configure when Gauntlet Snap will run MongoDB cleanup and compaction, run:
accelo config retention
  1. Create Indices for Database Collections:
    • Finally, run the following command:
accelo admin database index-db

Migration Steps from Version 3.0.x to 3.4.2

Cluster Specific Changes

The following components are introduced in version 3.4.0 and are not present in version 3.0.X:

  • ad-events
  • ad-gauntlet

To accommodate the ad-events component, you need to make certain changes related to hooks on the cluster side. For configuring Hive/Tez hooks, refer to the document, [Configure HDP for Pulse Deployment](Configure HDP for Pulse Deployment).

Pulse Upgrade Steps (Core Services Server / Addon Services Server)

  1. Update Configuration:
  • Update the ImageTag in the $AcceloHome/config/accelo.yml file to 3.4.2.
  1. Push Updated Configuration to Database:
  • Run the following command:
accelo admin database push-config
  1. Download Necessary Images:
    • Ensure you have downloaded or loaded all the necessary Docker images for version 3.4.2, including:
191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:migrate-4.4.20
191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:migrate-5.0.16
  1. Update Accelo Binary:
    • Download the new Accelo binary.
    • Replace the older Accelo binary with the new one.
  2. Run Migration Command:

Important

  • Make sure you run the command from $AcceloHome.
  • Do not change the value of -v from 3.4.0 unless explicitly advised by Engineering. The following command needs to run on all Pulse nodes including standalone nodes i.e. nodes where PULSE_SA_NODE is true. Examples of such nodes include but aren’t limited to Logsearch, FS Analytics V2, Dashplot .
  • -i : This indicates the version you are migrating from. For example, to migrate from 3.0.16 to 3.4.0, run:
accelo migrate -v 3.4.0 -i 3.0.16 -b
  • For Multi-Cluster:
accelo migrate -v 3.4.0 -i 3.0.16 -a -b
  1. Handle Migration Issues:
    • If any issues occur during migration, rerun the same command.
  2. Generate Notifications Configuration:
    • Skip this step on nodes that don't have Core services running on them.
    • After migration completes successfully, generate notifications.yml by running:
accelo config actions notifications
  1. Deploy Core Containers:
    • Skip this step on nodes that don't have Core services running on them.
    • Deploy all core containers by running:
accelo deploy core
  • If you see the following error, just answer y and proceed:
ERROR: Looks like core setup is missing the services '[ad-events ad-gauntlet]' from the stack 'default'.
  1. Restart Containers:
    • Restart all containers to use the updated images:
accelo restart all -d
  1. Uninstall Hydra Agent:
    • Run the following command:
accelo uninstall remote
  1. Install Hydra Agent:
    • Run the following command:
accelo deploy hydra
  1. If Logsearch is deployed in standalone mode:
    • Check if yarn_app_es_urls is present in override.yml
    • If the above is present, change the port number from 19013 to 19051.
yarn_app_es_urls: <http://<LOG_SEARCH_HOSTNAME>>:19051
  1. Reconfigure Cluster:
    • To get the updated configuration for agents, run:
accelo reconfig cluster -a
  1. Push Updated Configuration to Database:
    • Run the following command:
accelo admin database push-config -a
  1. Configure MongoDB Cleanup & Compaction:
    • To configure when Gauntlet Snap will run MongoDB cleanup and compaction, run:
accelo config retention
  1. Create Indices for Database Collections:
    • Finally, run the following command:
accelo admin database index-db

Migration Steps from Version 3.2.x to 3.4.2

Pulse Upgrade Steps (Core Services Server / Addon Services Server)

  1. Update Configuration:
    • Update the ImageTag in the $AcceloHome/config/accelo.yml file to 3.4.2.
  2. Push Updated Configuration to Database:
    • Run the following command:
accelo admin database push-config
  1. Download Necessary Images:
    • Ensure you have downloaded or loaded all the necessary Docker images for version 3.4.2, including:
191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:migrate-4.4.20
191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:migrate-5.0.16
  1. Update Accelo Binary:
    • Download the new Accelo binary.
    • Replace the older Accelo binary with the new one.
  2. Run Migration Command:

Important

  • Make sure you run the command from $AcceloHome.
  • Do not change the value of -v from 3.4.0 unless explicitly advised by Engineering. The following command needs to run on all Pulse nodes including standalone nodes i.e. nodes where PULSE_SA_NODE is true. Examples of such nodes include but aren’t limited to Logsearch, FS Analytics V2, Dashplot .
  • -i : This indicates the version you are migrating from. For example, to migrate from 3.2.14 to 3.4.0, run:
accelo migrate -v 3.4.0 -i 3.2.14 -b
  • For Multi-Cluster:
accelo migrate -v 3.4.0 -i 3.2.14 -a -b
  1. Handle Migration Issues:
    • If any issues occur during migration, rerun the same command.
  2. Restart Containers:
    • Restart all containers to use the updated images:
accelo restart all -d
  1. Uninstall Hydra Agent:
    • Run the following command:
accelo uninstall remote
  1. Install Hydra Agent:
    • Run the following command:
accelo deploy hydra
  1. If Logsearch is deployed in standalone mode:
    • Check if yarn_app_es_urls is present in override.yml
    • If the above is present, change the port number from 19013 to 19051.
yarn_app_es_urls: <http://<LOG_SEARCH_HOSTNAME>>:19051
  1. Reconfigure Cluster:
    • To get the updated configuration for agents, run:
accelo reconfig cluster -a
  1. Push Updated Configuration to Database:
    • Run the following command:
accelo admin database push-config -a
  1. Configure MongoDB Cleanup & Compaction:
    • To configure when Gauntlet Snap will run MongoDB cleanup and compaction, run:
accelo config retention
  1. Create Indices for Database Collections:
    • Finally, run the following command:
accelo admin database index-db

Migration Steps from Version 3.3.2x to 3.4.2

Pulse Upgrade Steps (Core Services Server / Addon Services Server)

  1. Update Configuration:
    • Update the ImageTag in the $AcceloHome/config/accelo.yml file to 3.4.2.
  2. Push Updated Configuration to Database:
    • Run the following command:
accelo admin database push-config
  1. Download Necessary Images:
    • Ensure you have downloaded or loaded all the necessary Docker images for version 3.4.2, including:
191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:migrate-4.4.20
191579300362.dkr.ecr.us-east-1.amazonaws.com/acceldata/ad-database:migrate-5.0.16
  1. Update Accelo Binary:
    • Download the new Accelo binary.
    • Replace the older Accelo binary with the new one.
  2. Run Migration Command:

Important

  • Make sure you run the command from $AcceloHome.
  • Do not change the value of -v from 3.4.0 unless explicitly advised by Engineering. The following command needs to run on all Pulse nodes including standalone nodes i.e. nodes where PULSE_SA_NODE is true. Examples of such nodes include but aren’t limited to Logsearch, FS Analytics V2, Dashplot .
  • -i : This indicates the version you are migrating from. For example, to migrate from 3.3.20 to 3.4.0, run:
accelo migrate -v 3.4.0 -i 3.3.20 -b
  • For Multi-Cluster:
accelo migrate -v 3.4.0 -i 3.3.20 -a -b
  1. Handle Migration Issues:
    • If any issues occur during migration, rerun the same command.
  2. Restart Containers:
    • Restart all containers to use the updated images:
accelo restart all -d
  1. Uninstall Hydra Agent:
    • Run the following command:
accelo uninstall remote
  1. Install Hydra Agent:
    • Run the following command:
accelo deploy hydra
  1. If Logsearch is deployed in standalone mode:
    • Check if yarn_app_es_urls is present in override.yml
    • If the above is present, change the port number from 19013 to 19051.
yarn_app_es_urls: <http://<LOG_SEARCH_HOSTNAME>>:19051
  1. Reconfigure Cluster:
    • To get the updated configuration for agents, run:
accelo reconfig cluster -a
  1. Push Updated Configuration to Database:
    • Run the following command:
accelo admin database push-config -a
  1. Configure MongoDB Cleanup & Compaction:
    • To configure when Gauntlet Snap will run MongoDB cleanup and compaction, run:
accelo config retention
  1. Create Indices for Database Collections:
    • Finally, run the following command:
accelo admin database index-db

Migration Steps from Version 3.4.x to 3.4.2

  1. Update Configuration:
    • Update the ImageTag in the $AcceloHome/config/accelo.yml file to 3.4.2.
  2. Push Updated Configuration to Database:
    • Run the following command:
accelo admin database push-config -a
  1. Take a backup of the accelo file and download the latest version for the accelo.linux CLI.
  2. Move and make the latestaccelo CLI file as an executable. Also, check the updated Pulse information for the following:
mv accelo.linux accelo
chmod +x ./accelo
  1. For the Online installation option, run the following command and press 'Y' at the prompt to get the most recent binaries.
accelo login docker
accelo pull all
  1. For the Offline Installation option, download all the latest image packs and run the following command to upload the latest images.
docker load -i
  1. Run the below command to update and add new keys in vars.yml.
accelo reconfig cluster
  1. Run the below command to push acceldata.conf and other updated files to MongoDB.
accelo admin database push-config
  1. Restart all containers including database(s) with the following command.
accelo restart all -d
  1. Uninstall the hydra agent using the following command.
accelo uninstall remote
  1. Install the hydra agent using the following command.
accelo deploy hydra
  1. Reconfigure the cluster to get the updated configurations for agents.
accelo reconfig cluster -a