Acceldata
 ODP
Search documentation… ⌘K

Manage Spark 3 Configurations

Version-Specific Configurations

Each Spark version has independent configuration files:

/etc/spark3/conf/ # Spark 3.5.5
/etc/spark3_3_3_3/conf/ # Spark 3.3.3
/etc/spark3_3_5_1/conf/ # Spark 3.5.1

Modifying Configurations via Ambari

  1. Navigate to Services → Select Spark version
  2. Click the "Configs" tab
  3. Modify settings as needed
  4. Save changes
  5. Restart affected components

Example: Increase History Server memory for Spark 3.3.3

  1. Go to Spark3 3.3.3 service
  2. Configs → Advanced spark3-3.3.3-env
  3. Set spark_daemon_memory = 2048 (MB)
  4. Save and restart the History Server