Azure Databricks Costs: DBUs, VMs and Auto-Termination
Every running Databricks cluster sends two bills, one from Databricks and one for the VMs beneath it.
Two meters on every cluster
Azure’s Databricks pricing page prices each workload as a DBU count and DBU price plus a separate VM price, and shows the total as their sum. The DBU rate depends on the workload type (all-purpose, jobs, SQL, serving) and features such as Photon, which the same page lists with a higher DBU emission rate than non-Photon compute. The VM side is ordinary Azure compute running in your subscription.
Both meters stop when the compute terminates. Microsoft’s compute management guide explains the two ways that happens: you terminate it manually, or you set automatic termination, an inactivity period, measured as a number of minutes, after which the compute terminates if no command has run. It also warns that terminated compute is permanently deleted 30 days after termination unless an admin pins it.
What ZopNight covers
On Azure, Databricks rides the existing Azure subscription connection: workspaces are discovered from Resource Graph and their clusters, pools and SQL warehouses through the workspace API, once the “Zopnight Databricks Access” workspace-admin role is granted. ZopNight sees each compute object’s size, worker configuration, autoscaling and auto-termination settings and cost tags.
Start and stop. Workspaces, clusters, instance pools and SQL warehouses are on ZopNight’s start/stop list, using the same schedules, groups and overrides as VMs. SQL warehouses and all-purpose clusters can stop nights and weekends, and pools can scale down outside working hours. A cluster Stop is a terminate, so it is destructive in Databricks terms. Jobs and model-serving endpoints are discovered but not schedulable.
Recommendations. The Databricks family runs on Azure under its own rule range and covers:
- Cluster Missing/Weak Auto-Termination and SQL Warehouse Auto-Stop Disabled/Excessive
- Serving Endpoint Always-On (Scale-to-Zero Disabled)
- Instance Pool Min-Idle Underutilized
- Databricks Cluster Oversized, Cluster Autoscaling Disabled and Photon Not Enabled
- Job on All-Purpose Compute, which flags jobs running at all-purpose rates
- Cluster Workers On-Demand Instead of Spot
- Cluster Without Cluster Policy and missing cost-allocation tags on clusters, warehouses and jobs
- Orphaned Job (No Runs in 30 Days) and Orphaned SQL Warehouse (No Queries in 30 Days)
Remediation. A Databricks cluster is on the remediation pause roster as a reversible step, so a finding can be paused through the Remediate workflow instead of edited by hand.
Databricks on AWS and GCP connects as a standalone Databricks account instead; see the Databricks integration. Azure fact sheet: Azure Databricks; schedules: scheduling Azure Databricks. Deeper reading: Databricks cost optimization.
Questions we get a lot.
Does stopping a Databricks cluster from ZopNight delete it?
A cluster Stop is a terminate. The configuration is kept, but Databricks permanently deletes terminated compute after 30 days unless an admin pins it.
Can ZopNight schedule Databricks jobs?
No. Jobs and model-serving endpoints are discovered and covered by recommendations, but only workspaces, clusters, pools and SQL warehouses are schedulable.