Interactive clusters with auto-termination off or set too high
What does ZopNight detect here?
ZopNight flags an interactive Databricks cluster whose auto-termination is disabled or set above 120 minutes. An all-purpose cluster left running bills DBUs plus the underlying cloud VMs for every idle minute, and the saving is prorated against that 120-minute ceiling rather than assumed total.
Signal and threshold
| Field | Value |
|---|---|
| Rule IDs | RC-2301 · RC-2401 · RC-2201 |
| Category | schedule |
| Severity | medium |
| Metric | none — pure configuration read |
| Source | cluster_autotermination.go |
Where it applies
Two failure modes, one rule
Auto-termination set to zero means disabled, and the cluster runs until somebody stops it. That is the expensive case: a cluster started for an afternoon’s exploration and forgotten bills through the night and the weekend.
Auto-termination set high, say 480 minutes, is the quieter case. It terminates eventually, but only after eight idle hours, and it resets on every stray interaction. In practice a cluster with a long timeout and an open notebook rarely terminates at all.
The 120-minute ceiling and how the saving is derived
The rule treats 120 minutes as the upper bound for a healthy setting. Above it, or at zero, the finding fires.
The saving is not the cluster’s whole cost. It is prorated: the reported figure scales by
1 - (120 / configured minutes), so a cluster set to 480 minutes yields three-quarters of the
idle cost rather than all of it. That reflects what tightening to the ceiling actually recovers,
not what deleting the cluster would.
What you are paying for while idle
Databricks charges DBUs for the cluster, and your cloud provider charges separately for the underlying VMs. An idle all-purpose cluster accrues both. That double meter is why interactive clusters are consistently the largest avoidable line in a Databricks bill.
Listing autotermination_minutes per cluster
databricks clusters list --output json | jq -r '.clusters[] | select(.cluster_source=="UI" or .cluster_source=="API") | [.cluster_name, .autotermination_minutes, .state] | @tsv'Anything reporting 0, or a value above 120, is what this rule sees.
Where to set it instead
Ten to thirty minutes suits interactive work: long enough to survive a coffee break, short enough that a forgotten notebook costs little. Cluster policies can enforce a maximum so the setting is not left to whoever creates the cluster.
Job clusters are exempt from this finding entirely: they terminate when the job ends, which is the behaviour auto-termination is approximating.