Update Real-time Monitoring Parameters of HDFS

This page describes how to enable and configure real-time monitoring of HDFS in Pulse.

Step 1: Edit Configuration File

On the Pulse node, navigate to the following path and edit the configuration file:

$AcceloHome/config/acceldata_<clustername>.conf

Step 2: Configure Real-Time Monitoring

By default, the following parameter is enabled:

fsanalytics.inotify.enabled=true

This setting enables real-time monitoring of HDFS using the inotify mechanism.

You may update other related parameters in the configuration file as needed, based on your environment and monitoring requirements.

{
 name = "fsanalytics-connector"
 trigger = "scheduled"
 fsanalytics.off.heap.storage = "/etc/fsanalytics"
 fsanalytics.metastore {
 # HIVE METASTORE is ENABLED during the setup
 enabled = true
 encrypted = true
 DAYS = 7
 }
 hive.schema.prefix = "hive."
 # Inotify configuration
 fsanalytics.inotify.enabled = true # Enable/Disable Inotify feature
 fsanalytics.inotify.poll.interval.ms = 1000 # Time between consecutive polls to HDFS Inotify
 fsanalytics.inotify.poll.timeout.ms = 1000 # Sets poll timeout
 fsanalytics.inotify.es.flush.interval.ms = 1
 fsanalytics.inotify.es.batch.size = 100
 fsanalytics.inotify.es.buffer.capacity = 10000
 slick-mysql {
 profile = "slick.jdbc.MySQLProfile$"
 db {
 dataSourceClass = "slick.jdbc.DriverDataSource"
 properties = {
 driver = "org.mariadb.jdbc.Driver"
 # Check the server timezone
 # Check & validate the metastore DB Name
 url = "jdbc:mysql://odp311.acceldata.dvl:3306/hive?serverTimezone=UTC&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&useUnicode=true"
 user = "hive"
 password = "ColrOFzIvF+y0RK0807jpg=="
 }
 }
 }
}