Pulse Add-on Services

Acceldata SQL Analysis Service

This add-on provides details about Hive tables and performs analysis based on them. The data becomes available only after the successful loading of the HDFS fsimage.

Alerts

This service is responsible for creating and managing alerts, including predefined alerts and user-defined alerts.

Actions

This service is responsible for running actions on the Hadoop cluster nodes. The port used by this service is 19999.

Core Connector

This service comprises two components: ad-connectors and ad-sparkstats.

• ad-connectors: Parses all Hive, Tez, and MapReduce queries, as well as YARN applications, once they are loaded into NATS.
• ad-sparkstats: Processes Spark history logs stored in HDFS and provides detailed analysis of Spark jobs.

Druid Analyzer

This service is responsible for fetching Druid-related queries and JMX metrics. The port used by this service is 19090.

Dashplot

This service includes two components: ad-pg and ad-dashplot.

• ad-pg: Serves as the database for storing all created dashboards.
• ad-dashplot: Facilitates the creation of custom charts and dashboards using data from various input sources within the Pulse datastore.

FSAnalytics V2

The ad-fsanalyticsv2-connector service fetches the fsimage, parses it, and then pushes the data to Elasticsearch.

Facelift UI

This service consists of three components: ad-pulse-ui, ad-pg-ui, and ad-dashplot-ui. The Facelift UI service is responsible for deploying the new user interface with an updated user experience (UX).

FS Elastic

This service is used when you want to deploy FSAnalytics on a separate node and segregate the FSimage stats and logs into different Elasticsearch instances.

HA Graphql

This is an add-on that enables high availability (HA) for the Pulse UI.

Hydra

This add-on is used for communication with the Hydra agent to synchronize the configuration of all agents with the vars.yml file.

Impala-Connector

This add-on service fetches Impala query data from ad-events, processes the data, and then sends it to the appropriate databases.

Kafka Connector(Both)

These add-ons connect to the Kafka admin client to fetch Kafka related data.

LDAP

This service is responsible for setting up LDAP for the Pulse UI. It queries the customer's LDAP server to be used by the RBAC (Role-Based Access Control) feature of the UI.

Logsearch

This service consists of three components: ad-logsearch-curator, ad-logstash, and ad-elastic.

• ad-logsearch-curator: Handles the auto purging and retention of Elasticsearch indices.
• ad-logstash: Used for parsing and formatting the logs sent by the PulseLogs agent.
• ad-elastic: The Elasticsearch container responsible for storing and indexing the log data.

Notifications

This service sends notifications to the configured external sources.

Oozie Connector

This service is used for connecting to the Oozie db and collecting Oozie related jobs and data.

Ozone Connector

This service fetches data about Ozone Service which is a Object storage for Hadoop data.

Proxy

This add-on is used for setting up the proxy for Pulse UI (SSL/TLS for pulse UI).

Config Server DB

This service is used for creating a Mongo Sharded cluster.

Query Router DB

This service is used for querying data from a Mongo Sharded cluster.

Recommendation Service

This service is used for calculating recommendations for various Hadoop services for query recommendations.

Shard Server DB

This service is used for actually storing the data in a Mongo Sharded cluster.

Standalone Connector

This service is used for deploying our ad-connector service in a standalone node.

Yarn Optimizer

This service is used for optimizing the YARN Cluster for resources.