Compute | Databricks
Databricks Compute in ADOC gives you visibility into cluster health, job performance, and cost across your Databricks environment, so you can monitor and optimize resources without leaving ADOC.
What's in Databricks Compute
Page | What It Covers |
|---|---|
Databricks Compute | Cluster states, resource utilization, job performance, and DBU consumption across the Overview, Clusters, Job Studio, All Purpose Cluster, Job Runs, and DLT Pipelines tabs. |
Databricks Cost | A cost breakdown by cluster type, individual cluster, and cloud vendor service. |
Databricks Query Studio | Historical and real-time query monitoring, with query fingerprinting to group similar queries together. |
Downloading visualization data
Any visualization in Databricks Compute can be downloaded as a CSV file. The file name includes "DB" (for Databricks), the visualization name, and the date and time of download. If you applied global calendar filters before downloading, the exported data reflects those filters.
Firewall requirements
Databricks clusters need outbound HTTPS (port 443) access to download the agent that collects data for ADOC. Without it, clusters can't retrieve the required files during startup or restart, which disrupts data collection.
Allow outbound access to:
Purpose | Destination |
|---|---|
Agent binaries, downloaded during cluster startup or restart |
|
Agent files required for data collection (for example, the Spark listener JAR) |
|
To configure this:
Identify the network environment your Databricks clusters run in (for example, the Databricks subnet).
Add outbound HTTPS rules for the destinations above.
Test connectivity to confirm the clusters can reach them.
Restart your Databricks clusters so they pick up the agent files on the next startup.
These settings are in addition to the Control Plane IPs already required for your ADOC deployment. Review firewall rules periodically to confirm continued connectivity as your security policies evolve.
What's next
Databricks Compute – Explore cluster, job, and pipeline metrics.
Databricks Cost – Break down Databricks and cloud vendor spend.
Databricks Query Studio – Investigate query performance and cost.
Send a comment