# Databricks Cluster Oversized

> An autoscale minimum of 4 or more workers is a floor billed 24/7. The cluster can never shrink below it, even overnight. ZopNight recommends lowering the floor to 2 only after the derived cluster CPU shows at least 10% of a 30-day window under 20% load, and values only the idle share.

Source: https://zop.dev/integrations/databricks/recommendations/databricks-cluster-oversized
Updated: 2026-08-19

---

## The minimum is the commitment; the maximum is free

Autoscale bounds are routinely read backwards. A high maximum costs nothing until
demand summons those workers; a high minimum is a standing purchase: the cluster
holds that many workers through every idle hour, billing VMs and DBUs for capacity
that exists because a configuration line says so. This rule targets interactive
clusters whose floor is 4 workers or more, a shape that usually starts as "keep it
snappy for the demo" and survives long after.

## Config gets it flagged; measurement gets it a number

Autoscaling must be on (the fixed-size case belongs to a sibling rule, and the two
can never both fire on one cluster), with the minimum at 4 or above. Then the same
derived per-cluster CPU series used across ZopNight's Databricks rules must show a
sustained idle window: at least 10% of the 30-day series under 20% CPU. A genuinely
busy min-4 cluster shows no such window and is left alone; telling it to drop to 2
workers would trade real capacity for imagined savings, and an earlier version of
this rule that fired on configuration alone was corrected for exactly that reason.

## Why the delta is halved

Pricing values the cluster at its autoscale midpoint: the average of min and max
workers, plus the driver. Lowering the floor from its current value to the target of
2 moves that midpoint by half the floor reduction, so the reclaimed node count is
(min − 2) / 2, priced at the per-node rate and then multiplied by the measured idle
fraction. Only the idle share of the freed floor is claimed as savings; the busy
share was doing work.

## Survey your floors

```bash
databricks api get /api/2.1/clusters/list | jq '
  .clusters[]
  | select((.autoscale.min_workers // 0) >= 4)
  | {cluster_name, autoscale, state}'
```

## Before lowering the number

Interactive latency is the one real cost: a floor exists so notebooks respond without
waiting for scale-up. Check who uses the cluster at which hours, drop the minimum to
2, and watch the first week. The autoscaler will still reach the old worker counts
whenever demand is real, just no longer at 3 a.m.