Acceldata
Pulse

Changelog

Pulse 3.6.x
Pulse v3.6.610th June, 2025

HDFS

  • Pulse supports processing larger DS and NS quota values up to exabytes (19 digits), enhancing its ability to handle high-volume data workloads.
Pulse v3.6.51st June, 2025

Alerts

  • Added alerting support for the namenode_topuser_opcounts_count metric to track high RPC activity per user and call type.

Dashplots

  • Custom Dashplots now support conditional row/column highlighting with configurable color codes.
  • Added visualization for HDFS DS and NS quota usage in Path Analysis charts.

Impala

  • Added Statistics Corrupt and Statistics Missing metrics to the Impala Queries pages.

Encryption Support

  • Added support for encrypting LDAP bind passwords with the encryptedPassword=true parameter and updating them in configuration files.
  • Added support for encrypting SSL passphrases for GraphQL and updating them in configuration files.
Pulse v3.6.325th March, 2025

Dashplots

  • Custom Dashboards: The Storage I/O Metrics and HDFS I/O Metrics dashboards are now available by default on the Dashplots page.
Pulse v3.6.220th March, 2025

Impala

  • Optimized performance for the Impala Connector.
Pulse v3.6.118th March, 2025

Enhancements

  • YARN now honors the configured Node memory limit in Ambari to reduce OOM risks.
  • Logs page now supports page number editing to jump directly to the specific page and first/last page navigation icons.
  • Pulse is certified for JDK-11, bringing better performance and compatibility.
  • Support for Spark 3.5.2 in standalone environments is now available.
Pulse v3.6.0February 21, 2025

New Pulse Alerts

  • The new alerts have been introduced to enhance system monitoring and ensure optimal performance:
  • The Pulse Alerts now support regex matching in filters.

Reports and Dashboard

  • Introduced a capability to generate and download a Pulse Stock Report to verify the number of applications running, their status, the hosts on which they are running, and other relevant details.
  • Introduced the NiFi API Summary Dashboard, which distinguishes between external and internal API requests.

Actions and Executions

  • Introduced the Yarn Application Auto-Killer action, which automatically terminates Yarn applications that exceed predefined conditions.
  • Enhanced Action Execution Tracking for killed queries by highlighting them in red, enabling detailed insights and easy navigation to related components.
  • The “Move Pending App Queue” action now supports multiple destination queues, intelligently selecting the one with the most available resources while ensuring job submission permissions.
  • Enables cleanup of HDFS and local files by configuring the last access date (atime) in the respective actions, streamlining file management.
  • Introduced Cron Generator on Actions to simplify scheduling and managing tasks with precise time intervals.
  • Several new parameters have been added to the HDFS Balancer Playbook, enhancing its customizability and scalability.

Notification

  • Enables to configure multiple Xmatters groups and keys using the accelo config alerts notifications command.
  • Enhanced Quantum Notification details by including environment or cluster information in the ticket when an alert is raised.

Security

  • Enhanced security by masking the spark.authenticate.secret in the Pulse UI > Spark > Jobs Compare Metrics section.

Visualization and User Experience

  • Enables to create dashplot or visualization for Impala Kill Queries and YARN Kill Applications, providing deeper insights into query termination patterns.
  • Enhanced the "View in Context" option in Logs UI for a more comprehensive and seamless log viewing experience.

Other Enhancements

  • Optimized the ad-alerts component with various mechanisms to enhance the overall stability and performance of the system.
  • Supports downloading over 10K rows of data in CSV or XLSX format for modules like Tez Queries, Spark Jobs, Impala Queries, and Yarn Application Explorer without RM Live.
  • Displays the total driver runtime for Spark jobs in the Spark Jobs Timeline section, enhancing visibility into driver execution runtime.
  • Introduced the Queue-Level Job Details to display the top applications utilizing the most queue capacity within the selected interval.
Pulse 3.5.x
Pulse v3.5.330th January, 2025

Enhancements

  • The utiliseFQDNYarnOptimise flag has been introduced in the configuration file of Yarn App Metrics.
Pulse v3.5.214 January, 2025

Enhancements

  • Pulse now allows you to override specific information in the default-endpoints.yml file.

New Changes

  • Upgraded the Ubuntu Base Image of the containers (Logstash, Elastic, and MongoDB) from version 23.04 to 24.04
Pulse v3.5.122nd November, 2024

Security Enhancements

  • This release strengthens security on the Pulse Agent by enforcing strict file ownership and permissions within the directory.
  • This release enhances security by removing the default plain text passwords from the template or configuration files to avoid access to unauthorized users.

Other Enhancements

  • This release supports Yarn Optimizer on the CDP distribution.
  • This release splits the Agents summary into individual Agents Summary.
  • Improved the Yarn Optimization workflow by allowing you to configure the OverCommit Connection Timeout Value to the pipeline.

New Changes

  • Upgraded the Postgres version from 12.X to 16.4.
Pulse v3.5.031st October, 2024

New and Enhanced Observability

  • Airflow
  • Pulse now enhances the real-time observability capabilities for the NiFi and Impala services.

New Capabilities

  • Introduced the Yarn Optimizer capability designed to enhance the performance of Yarn Clusters by optimizing the allocation of memory for yarn jobs, leading to a significant performance improvement.
  • Introduced the Auditing capability to show the Agent and Pulse statistics such as hosts where the Agent status is down, page login activity, the most viewed pages of Pulse, etc.
  • Pulse collects the Impala coordinator metrics to create dashboards and alerts.
  • Introduced the new action runbooks such as Perform HDFS Non Admin Operations, HDFS Top 10 Directories, and Kerberos to automate tasks.

Enhancements

  • Pulse supports notifications via the Big Panda notification channel.
  • Enhanced the HDFS Filtering Records Capability to filter files and folders.
  • The Actions Service workflow has been improved.

New Changes

  • The new steps are available to deploy Playbooks using the command: accelo deploy playbooks.
Pulse 3.4.x
Pulse v3.4.529th October 2024

Enhancements

  • Enhanced Pulse Agent Security – Restricts access to /opt/pulse files for improved security.
  • Secured Template Files – Removes default plain-text passwords to prevent unauthorized access.
  • Yarn Optimizer Support for CDP – Enables port configuration for Yarn Optimizer on the CDP distribution.
  • Improved Yarn OverCommit Timeout Handling – Ensures stability by reverting memory if optimization fails.
Pulse v3.4.327th August 2024

Enhancements

  • Standalone Pulse Add-ons Deployment – Deploy Ad-alerts, Ad-notifications, and Ad-director on standalone nodes for faster processing.
  • Total Directory Size in HDFS Explorer – View the total size of the current directory dynamically on the HDFS > File Explorer page.
  • Multi-KDC Multi-Node Deployment Support – Enables deployment of ad-connectors in GraphQL across multiple KDCs and nodes.
  • Executor Placement Strategy for Yarn Optimizer – Distributes workloads based on free memory, reducing allocation skew in Yarn scheduling.
  • Queue Capacity Strategy for Yarn Optimizer – Triggers overallocation only when Yarn queues are heavily loaded or memory is low.
  • Optimized Yarn Optimization Calculation – Excludes removed or unhealthy nodes from Yarn Optimization to enhance accuracy.
Pulse v3.4.15th July 2024

Enhancements

  • Upgraded HDFS File Explorer – Introduces infinite scrolling and batch data retrieval for seamless browsing.
  • Enhanced Visibility Controls – Adds a toggle to reveal hidden or empty folders in the file system.
Pulse v3.4.018th June 2024

New Features

  • (Beta) Introduced Yarn Optimizer – Optimizes memory usage on the Yarn cluster for better performance.
  • Pulse on Kubernetes – Enables deployment on Kubernetes (v1.26+), simplifying orchestration.

Enhancements

  • Kafka Mirror Maker
  • Improved Actions Service Architecture – Enables OS-agnostic execution of actions without SSH dependency.
  • Database Backup & Restore – Supports manual backups for MongoDB and VictoriaDB to ensure data recovery.
  • New Notification Channels – Supports Quantum and xMatters for enhanced alerting.
  • Elasticsearch Security Upgrade – Strengthens security to prevent information disclosure vulnerabilities.
  • Enhanced Yarn Search Capability – Provides dynamic search suggestions and advanced filtering options.
  • Improved Hive Visibility – Adds charts for Hive Metastore and Hive Server2 insights in the Tez Dashboard.
  • Yarn NodeManager Metrics Alerts – Allows alert creation based on specific Yarn NodeManager metrics.
  • Real-time Impala Runtime Metrics – Enables dynamic monitoring with enhanced performance insights.
  • Automated Elastic Data Migration – Introduces accelo migrate command for seamless version upgrades.
  • (Beta) New UI Facelift – Introduces a fresh, intuitive, and user-friendly interface.
Pulse 3.3.x
Pulse v3.3.2326th June, 2024

Enhancements

  • Pulse now allows you to override specific information in the default-endpoints.yml file.
Pulse v3.3.223rd June, 2024

Enhancements

  • Enabled persistent configuration storage and seamless YARN RM node switching with MongoDB-backed algorithm behavior.
Pulse v3.3.927th May, 2024

Enhancements

  • Enhanced ES Client Connection: GraphQL ES client now supports a comma-separated list of hosts without ports.
  • Multi-Node Elasticsearch Support: Enabled multi-node Elasticsearch connectivity via updated configurations.
  • New Oozie Schema Field: Added support for non-default Oozie database schemas in configuration files.
  • Zookeeper Automation in NiFi: Automated Zookeeper Service configuration for NiFi Standalone via Pulse CLI.
  • Pulse Java Services Performance: Improved MongoDB performance with a shared connection pool per service.
Pulse v3.3.2115th May, 2024

Enhancements

  • Added HDFS Utilization chart to display used, total storage capacity, and usage percentage.
Pulse v3.3.831st March, 2024

Enhancements

  • xMatters Integration: Added xMatters notification channel for streamlined incident alerts and actions. New Stock Alert: Introduced YARN_LONG_RUNNING_JOB_WITH_FILTERS alert for monitoring long-duration Yarn jobs. MapReduce Query Trends Enhancement: Improved Query Trends chart performance by removing the DelayTime field. Table Usage Chart Update: Added a date picker to filter data by specific date ranges on the Databases page.
Pulse v3.3.2025th March, 2024

Enhancements

  • MongoDB Shard Cluster: Implemented shard clustering for improved data distribution, availability, and scalability.
  • Query Count Chart Enhancement: Added a date picker to filter data by specific date ranges.
  • Yarn Optimizer & Metrics Agent: Enhanced security and configuration for better deployment flexibility.
  • Retention Logging: Config changes now logged in logs/accelo.log for better tracking.
  • Tez Query Filtering: Enabled exclusion of specific connection verification queries for streamlined data collection.
  • Storage Efficiency: Optimized yarn_yarnapp_lifecycle collection to reduce storage usage.
Pulse v3.3.66th March, 2024

Enhancements

  • Impala Queries Runbook (CDP): Introduced a runbook to terminate Impala queries based on predefined conditions.
  • Kill Impala Queries by Query ID: Enabled termination of Impala queries using query IDs.
  • IMPALA_NOT_ALLOWED_QUERIES Alert: Added an alert to detect and trigger actions on restricted Impala query patterns.
Pulse v3.3.57th February, 2024

Enhancements

  • Enhanced Pulse Data Visualization: Improved Pulse with better stability, UI/UX, and data pipelines across key components like NiFi, Tez, Yarn, HDFS, and more.
  • Advanced Spark Jobs Metrics: Upgraded Spark Jobs Metrics pipeline for improved efficiency and deeper job insights.
  • Enhancements for ODP: Pulse v3.3.5 is now officially verified for Acceldata’s Open Data Platform (ODP).
Pulse v3.3.45th December, 2023

Enhancements

  • Optimized Pulse UI Performance: Improved UI responsiveness for smoother navigation of long-duration data on the Spark Job Details page.
Pulse v3.3.323rd November, 2023

Enhancements

  • Improved Spark Standalone Installation: Simplified integration of Spark Standalone clusters into Pulse with a consistent setup process.
  • View Spark Standalone Application Logs: Added support to view Spark Standalone stage logs directly in the LogSearch page.
  • Introduced Node Monitoring: Enabled infrastructure monitoring for individual nodes in Pulse.
  • Introduced .Log Download Format: Logs can now be downloaded in .log format with extended context up to 200 lines.
Pulse v3.3.21st December, 2023

Enhancements

  • Optimized Pulse UI Performance: Improved UI performance for smoother viewing of long-duration data in Yarn Chargeback Reports.
  • Enhanced Container Security: Added support to enable readOnlyRootFSEnabled in accelo.yml for tighter container security.
Pulse v3.3.011th October, 2023

Tez

  • View YARN Schedule Delay Time: Added a Delay Time widget on Tez Query Details to show YARN scheduling delays.
  • Highlight Tables with Small Files: Identifies tables with small files across query pages with recommendations provided.
  • View Long-Running Applications: Displays start and end times for long-running YARN apps in Application Explorer.
  • Group Tez Queries by Queue: Added ability to group Tez and MapReduce queries by Queue in the Group By menu.
  • New Improved DAG: Updated DAG visualization on Tez Query Details for better query execution insights.
  • New Mapper Scan Duration Column: Shows min, max, and average vertex runtime for each query UID.
  • View User Details: Added a Users column on Tez Table Details to list users accessing the tables.

Kafka

  • View Consumer Rebalancing Data: Pulse now displays reasons for Kafka consumer rebalancing events for better visibility and troubleshooting.

Dashplots and Dashboards

  • View Long-Running Containers: View and filter long-running containers that exceed threshold limits in Pulse.
  • View Delta Files: Displays the count of delta files in tables on the Database Table Details page.
  • Download Data from Dashboards: Enables downloading of chart data across YARN, Tez, Spark, and more.
  • Configure Step Interval: Allows configuration of step intervals (minutes to weeks) for global time selection.
  • Compare Chart Data: Compare chart data across multiple time periods, like current vs. previous month.

Alerts and Actions

  • Pause Alert Notifications: Allows pausing of alert notifications for a defined time period to suppress noise.
  • New Predefined Alerts: Introduced KAFKA_CONSUMER_GROUP_NO_CONSUMPTION and FS_SNAPSHOT_ANALYSIS_REPORT as new predefined alerts.

YARN

  • New Queue Usage By User Chart: Added a bar graph on the YARN Summary page to show queue consumption per user.
  • New Predefined Alerts: Introduced KAFKA_CONSUMER_GROUP_NO_CONSUMPTION and FS_SNAPSHOT_ANALYSIS_REPORT as new predefined alerts.

Databases

  • New “Usage Count” Column: Shows how many times each column was used across all queries in the Column Details table.
  • New “Hive Locks” Section: Displays real-time lock information on tables in the new Hive Locks section.
  • New Columns and Widgets: Added detailed metrics like table size, file count, partitions, and columns to the Database Table Details view.
Pulse 3.2.x
Pulse v3.2.1416 February, 2024

Enhancements

  • Quantum Ticketing Integration: Integrated Pulse Alerts and Actions with Quantum Ticketing to auto-raise and clear incidents on the Quantum platform.
Pulse v3.2.09 May, 2023

Spark

  • New Concurrent Jobs Section: Displays parallel applications running in the same queue on the Spark Job Details page.
  • New Concurrent Jobs Section: Displays parallel applications running in the same queue on the Spark Job Details page.
  • View Reason for Blank Job Details: Provides reasons and guidance when Spark Job Details page appears blank.
  • Compare Any Two Spark Executions: Allows comparison between any two Spark job executions, not limited to the last 10.
  • New DAG Chart: Visualizes the task execution flow in a DAG hierarchy on the Spark Job Details page.
  • New Nodes Tab: Shows a list of unhealthy nodes associated with the application in the new Nodes tab.

Hive

  • View Hive Audit Logs: Introduced a new page in Pulse to display Hive audit logs for enhanced auditability.

Alerts and Actions

  • New Stock Alerts: Added HIVE_QUERIES_HDFS_BYTES_READ and HIVE_QUERIES_HDFS_BYTES_WRITTEN as new stock alerts.
  • New Predefined Alerts: Introduced five predefined alerts including YARN_QUEUE_CAPACITY_USAGE and SPARK_EXECUTOR_NODE_BLACKLISTED.
  • New Runbooks: Added runbooks to kill YARN apps exceeding thresholds and to move pending apps to different queues.
  • Clear JIRA Status Automatically: Pulse now auto-sets JIRA ticket status to "Done" once the related incident is resolved.
  • Pass Arguments from Alerts to Actions: Supports passing dynamic alert values to override default action configs for auto-actions.
  • Export and Import Actions: Allows exporting actions as JSON and importing them across clusters with generic value mappings.

YARN

  • New Queue Usage By User Chart: Added a bar chart on the YARN Capacity page to show per-user queue consumption.

HDFS

  • New Last Modified File Count Chart (HDFS): Added a chart on the HDFS Analytics page to show file modification counts based on selected time periods.

Databases

  • Filter Tables by Created Time (Databases): Enabled filtering of tables by creation time on the Database page for easier data exploration.

Dashplots and Dashboards

  • Improved Search Bar: Enhanced search functionality with partial string match and NOT IN operator across multiple pages.
  • Retain Visualization after Dashboard Deletion: Visualizations are retained if the Deep Delete checkbox is disabled when deleting a dashboard.
  • New Filter Operators on Dashplot: Introduced equals, not equals, and NOT IN filter options on the Dashplot page.
  • Reporting Feature: Added report scheduling for dashboards and visualizations with hourly to monthly frequencies via dashplots-studio.
  • Pass Arguments from Alerts to Actions: Supports passing dynamic alert values to override default action configs for auto-actions.
  • Download Visualization Data: Enabled downloading of data directly from visualizations on dashboards.

CLI

  • Ability to Use Gauntlet: Introduced Gauntlet component for deleting Elasticsearch indices and purging/compacting MongoDB collections.
  • New CLI Commands: Added accelo admin override, accelo config actions notifications, and accelo admin database migrate.
  • New Option in Kerberos Config Command: Added -u flag to update Kerberos user in Pulse container files via CLI.
  • Deploy Monitoring Stack: Introduced optional Pulse Monitoring Stack for enhanced system monitoring and observability.
Pulse 3.1.x
Pulse v3.1.26 June, 2023

Enhancements

  • Support for Postgres Database: Pulse now supports using Postgres as the Hive metastore database.
Pulse v3.1.01 April, 2023

Platform

  • View Hive Audit Logs: Introduced a new page in Pulse to display Hive audit logs for enhanced auditability.

CLI

  • New CLI Command for Missing Data: Added accelo report application to generate reports on missing data in Pulse.
  • Additional FSAnalytics Variables: Introduced new configurable variables for the FSAnalytics container.
  • New -a Option in Migrate Command: Enables multi-cluster migration automation using the -a flag in accelo migrate.
  • Hyphen Support in Cluster Names: Cluster names can now include hyphens without breaking the reconfig cluster command.
Pulse 3.0.x
Pulse v3.0.025 November, 2022

Oozie

  • Oozie Observability: Pulse now integrates with Oozie, providing dashboards and metrics for Bundle, Coordinator, and Workflow jobs.
  • Enhanced Oozie Features: Added capabilities to map Oozie IDs, debug workflows, view job definitions, and manage SLA requirements.

Dashboard

  • Dashboard Reporting Capabilities: Users can now schedule and email dashboards as attachments via Dashplots.
  • New PULSE-AGENT-STATS Dashboard: Introduced a dashboard to monitor Pulse agent CPU, memory, state, and configuration updates.
  • New HIVE-SERVICE-SUMMARY Dashboard: Added a new dashboard to visualize critical Hive service metrics.

Others

  • Application Log Link in Alerts: Alerts now include a direct link to application logs in the Pulse Log section.
  • Redirection to Consumer Group: Group Details button on Topic Details now redirects to the selected consumer group’s details.
  • Type-in Search for Kafka Lag Chart: Kafka Dashboard's lag chart now supports time-based validation and type-in search.
  • Improved Search Behavior: Enhanced search functionality for more precise and one-click results.
  • Terminate Jobs by Memory/Vcore: New action added to kill YARN apps exceeding memory or Vcore thresholds.
  • Color-coded HDFS Capacity Trend: HDFS usage trends now displayed with color-coding based on consumed capacity.
Pulse 2.2.x
Pulse v2.2.522 December, 2022

Enhancements

  • Pulse is now improved with Tez and LLAP latency enhancements.
Pulse v2.2.127 July, 2022

Enhancements

  • Hive Connection Count Alerts: Pulse now supports alerts for Hive Connection Count metrics.
  • YARN Long-Running Job Alerts: New alert added for YARN jobs running beyond a defined threshold time.
  • Zombie LLAP Daemon Alert: Alert introduced to detect and notify about Zombie Hive LLAP Daemon processes.
Pulse v2.2.020 July, 2022

YARN

  • YARN Summary Tab: A new Yarn Summary tab provides key performance indicators for clusters and nodes.
  • Live Queue Distribution: Real-time hierarchy and metric values now shown via the Live Queue Distribution graph.

Application Explorer

  • YARN Application Detail Page: Clicking on App ID in YARN Application Explorer now opens detailed view of running applications.

Tez

  • Query Execution Summary Chart: A new chart shows time taken at each execution stage on the Tez Query Details page.
  • DAG Comparison: You can now compare two DAGs side-by-side to optimize query flows.
  • New Tez Filters: Memory and VCore filters added to the Sankey chart in the Tez Dashboard.

Logs

  • Download More Log Data: You can now download up to 2000 rows of log data as a spreadsheet from the Pulse Log page.

Kafka

  • Total Replica Fetcher Lag Metric: A new metric on the Kafka Topic page shows the lag in messages per follower replica.

HBase

  • New HBase Replication Metrics: Pulse now displays detailed replication metrics on the HBase Region Server listing page.
  • Cluster-Level Compaction Trends: A new graph on the HBase Dashboard shows compaction trends across the cluster with adjustable view options.

Others

  • Data Migration from InfluxDB to VictoriaMetrics: Pulse now supports migrating data from InfluxDB to VictoriaMetrics for improved performance and scalability.
  • New Table Temp Chart in Hive File Analytics: A bar chart named Table Temp has been added to show file temperature based on modification dates.
  • Password-Based SSH Authentication: Users can now create SSH keys with password-based authentication in the SSH tab under User Management.
Pulse 2.1.x
Pulse v2.1.028 April, 2022

Actions

  • Kerberos Support for Actions: Pulse now allows enabling Kerberos authentication for Actions to support secure environments.
  • Display of Kerberos-Enabled List: A new checkbox in Create Actions displays all Kerberos profiles configured, making selection easier.
  • New Playbook to Kill Long-Running Tez Queries: A new playbook is available to automatically terminate long-running Tez queries.

Alerts

  • New YARN Alert Group by App Name: Alerts can now be grouped by yarn_application_name for better organization.
  • CLI Supports Multiple Emails: You can now add multiple email addresses for notifications using the CLI.
  • HBase Master Failover Alert: New predefined alert HBASE_MASTER_FAILOVER added to track master switch events.
  • Hive Interactive Endpoint Check: New alert HIVE_INTERACTIVE_ENDPOINT_CHECK monitors HiveServer2 interactive availability.
  • Hive JVM Heap Usage Alert: Pulse raises alerts if Hive interactive server heap usage exceeds thresholds.
  • Global Default Email for Alerts: Use a global default email config in the alert notification wizard.
  • Bulk Enable/Disable Alerts: Alerts can now be enabled or disabled in bulk from the listing page.
  • Bulk Enable/Disable Notifications: Notifications can now be toggled in bulk using checkboxes.
  • Jira Integration for Alerts: Pulse now supports Jira integration directly from the Create Alerts Notification wizard.

Other Enhancements

  • YARN Capacity Metric Options: Users can now view avg, min, and max for VCore and Memory usage per user on the YARN Capacity dashboard.
  • Help Links in UI: Pulse UI now includes direct links to help guides and API documentation.
  • Impala Query Grouping: Impala Queries page now supports grouping by Resource Pool and User.
  • Keytab Upload Support: Pulse now allows uploading Keytab files for streamlined Kerberos authentication.
  • Node Page Search Bar: A new search bar has been added to the Nodes page for easier filtering.
  • Saved Tez Query Searches: Tez Queries page now supports saving search parameters for quicker access.
Pulse 2.0.x
Pulse v2.0.014 March, 2022

Hydra Agent

  • Introduction of Hydra for SSH Access: Pulse 2.0 introduces Hydra, a major enhancement designed to simplify and secure SSH access to cluster nodes. The Hydra Agent supports both CDP and non-CDP platforms, offering easy deployment and maintenance for users. This marks a significant milestone in Pulse’s evolution.

Alerts

  • New Notification Channels: Pulse now supports Telegram, Line, and ServiceNow as additional notification channels for alerts.
  • Tooltips for Predefined Alerts: A description tooltip now appears under the Edit Alert section for predefined alerts, helping users understand the alert variables.
  • YARN_LONG_RUNNING_JOB – Triggers on unusually long-running YARN jobs.

Kafka

  • Kafka Capacity Planning now suggests required broker count based on utilization.
  • New Kafka Offline Partitions page displays topics with offline partitions.
  • Kafka Consumers page now includes a new column called Time Lag.
  • Two new charts added to Kafka Topic Details: Topic Partition and Broker Disk Distribution.

Impala

  • Impala Queries page now supports filtering by Resource Pool.
  • Impala Daemon page now includes Memory Limit and Memory Over Limit charts.
  • Impala Dashboard shows average admission wait time for queries.
  • Impala data is now grouped and displayed by Resource Pool.
  • New Impala Resource Pool properties are now viewable in the Impala Dashboard.

CLI

  • CLI now supports setting custom retention time for execution logs.
  • CLI can now install/uninstall Hydra agent and deploy Hydra server as an addon.
  • CLI now allows purging MongoDB data after a specified time period.
  • VictoriaMetrics data now has a default retention policy of 31 days.
  • adpulse is now the only user-added agent component for Hadoop node installations.

Actions

  • New HDFS Balance Runbook helps balance data load across HDFS nodes.
  • New DNS Lookup Runbook checks forward and reverse DNS resolution for hosts.
  • New Hive Canary Check Runbook monitors daily Hive query runtimes and triggers alerts on discrepancies.

Tez

  • New tab added in Tez Query Details to compare configuration differences between runs.
  • HDFS Read and Write data now visible in Query Trends chart on Tez Query Details page.
  • Table details in Tez Table Details now redirect to the Database Table Details page.

Other Enhancements

  • Pulse now retains Error logs for up to 30–45 days and Info logs for 7 days.
  • Nodes page heat map can now be sorted by Host utilization.
  • Dropdown filters in Application Explorer now support arrow and enter key navigation.
Pulse 1.8.x
Pulse v1.8.010 December, 2021

Enhancements

  • Tez Query Details page now shows Hive Configurations via a new Hive Config option.
  • JVM GC charts now include metrics for hive_server_2 and hive_meta_store.
  • HBase dashboard includes a new RPC Count chart.
  • Column widths are now manually adjustable in Kafka Brokers and Nodes pages.
  • Inactive Kafka brokers can now be deleted from the Kafka Brokers page.
  • Application Explorer includes a new search option.
  • Predefined Alerts are now grouped under Stock Alerts on the Alerts page.
  • New API added to create roles and add bulk users.
  • LLAP Daemons page now shows host IP and details.
  • A new Refresh Configuration button is available on the Landing Dashboard.
  • CLI now prompts for Kerberos settings when configuring agents for HDP clusters.
  • Victoriametrics is now supported as a metrics DB to reduce InfluxDB load.
  • New CLI command compact-db added to free disk space after purge-db.
  • Kafka Brokers page now includes Broker Partition, Disk Distribution, and Leader Imbalance Ratio charts.