Update Real-time Monitoring Parameters of HDFS

This page describes how to enable and configure real-time monitoring of HDFS in Pulse.

Step 1: Edit Configuration File

On the Pulse node, navigate to the following path and edit the configuration file:

$AcceloHome/config/acceldata_<clustername>.conf

Step 2: Configure Real-Time Monitoring

By default, the following parameter is enabled:

fsanalytics.inotify.enabled=true
This setting enables real-time monitoring of HDFS using the inotify mechanism.
You may update other related parameters in the configuration file as needed, based on your environment and monitoring requirements.

{
name = "fsanalytics-connector"
trigger = "scheduled"
fsanalytics.off.heap.storage = "/etc/fsanalytics"
fsanalytics.metastore {
# HIVE METASTORE is ENABLED during the setup
enabled = true
encrypted = true
DAYS = 7
}
hive.schema.prefix = "hive."
# Inotify configuration
fsanalytics.inotify.enabled = true # Enable/Disable Inotify feature
fsanalytics.inotify.poll.interval.ms = 1000 # Time between consecutive polls to HDFS Inotify
fsanalytics.inotify.poll.timeout.ms = 1000 # Sets poll timeout
fsanalytics.inotify.es.flush.interval.ms = 1
fsanalytics.inotify.es.batch.size = 100
fsanalytics.inotify.es.buffer.capacity = 10000
slick-mysql {
profile = "slick.jdbc.MySQLProfile$"
db {
dataSourceClass = "slick.jdbc.DriverDataSource"
properties = {
driver = "org.mariadb.jdbc.Driver"
# Check the server timezone
# Check & validate the metastore DB Name
url = "jdbc:mysql://odp311.acceldata.dvl:3306/hive?serverTimezone=UTC&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&useUnicode=true"
user = "hive"
password = "ColrOFzIvF+y0RK0807jpg=="
}
}
}
}