Configure ODP Spark 2 and 3

This page helps you enable Spark metrics via the Graphite sink, allowing Pulse to capture Spark application and JVM metrics.

Configure Spark Metrics Sink

In the Ambari UI:

1. Navigate to Spark > Configs > Advanced spark2-metrics-properties.
2. Update the following parameters:

# Graphite sink class
*.sink.graphite.class=org.apache.spark.metrics.sink.GraphiteSink
# Location of your graphite instance
*.sink.graphite.host=localhost
*.sink.graphite.port=12003
*.sink.graphite.protocol=tcp
*.sink.graphite.prefix=spark.metrics
*.sink.graphite.period=20
master.source.jvm.class=org.apache.spark.metrics.source.JvmSource
worker.source.jvm.class=org.apache.spark.metrics.source.JvmSource
driver.source.jvm.class=org.apache.spark.metrics.source.JvmSource
executor.source.jvm.class=org.apache.spark.metrics.source.JvmSource

Result

• Spark 2.x/3.x sends metrics (including JVM metrics) to Graphite for Pulse ingestion.