Model serving endpoints that never scale to zero
What does ZopNight detect here?
A Databricks model serving endpoint with scale-to-zero disabled holds provisioned compute for all 730 hours in a month, whether or not it serves a request. The rule prices the saving against measured active hours and abstains when either the rate or the activity data is missing.
Signal and threshold
| Field | Value |
|---|---|
| Rule IDs | RC-2304 · RC-2404 · RC-2204 |
| Category | idle |
| Severity | low |
| Metric | none — pure configuration read |
| Source | serving_always_on.go |
Where it applies
Provisioned readiness is the whole cost
Model serving bills for provisioned concurrency, not for inference volume. An always-on endpoint pays for the capacity to answer instantly, continuously, regardless of traffic.
For an endpoint behind a production application with steady traffic that is correct: cold starts are real and user-facing. For one backing an internal tool, a batch process, or an experiment that ended months ago, it is 730 hours of readiness nobody consumes.
How the figure is derived
servingRate × (730 − measuredActiveHours). The rule measures how many hours the endpoint was
genuinely active and prices only the remainder.
If either input is unavailable (the rate or the activity measurement), it abstains. The source records that this replaced earlier behaviour which emitted a fabricated $0 under an advisory category, described there as advisory-dumping a cost finding.
Severity is low deliberately. The mechanism is real, but per-endpoint figures are usually
modest and enabling scale-to-zero carries a genuine latency consequence.
The trade being accepted
Scale-to-zero means the first request after an idle period waits for compute to start. That is seconds, not milliseconds. That is unacceptable on a user-facing inference path, and entirely fine for an internal dashboard, a nightly scoring job, or a demo endpoint.
Most always-on endpoints nobody questions are in that second group.
Checking scale_to_zero_enabled on each endpoint
databricks serving-endpoints list --output json \ | jq -r '.endpoints[] | [.name, (.config.served_entities[0].scale_to_zero_enabled // false)] | @tsv'Anything reporting false is holding compute continuously.
Check what keeps it warm first
A synthetic health check polling every minute keeps an endpoint permanently active, so enabling scale-to-zero saves nothing while that check runs. Fix the probe interval first, or the setting will look ineffective and get reverted.