Scheduling Azure Databricks Workspace
Can ZopNight schedule Azure Databricks Workspace?
Workspace-level schedules fan out to 3 child resource types with different treatments: interactive clusters are terminated, instance pools drop to 0 idle instances, and SQL warehouses are stopped. Job clusters are deliberately untouched, so overnight scheduled jobs still run, and both the DBU and VM meters go quiet.
How the stop works
| Field | Value |
|---|---|
| Behaviour | workspace-level scheduling fans out to child resources: interactive clusters are terminated, pools are scaled to 0 idle instances, and SQL warehouses are stopped. |
Example schedules
- 0 8 * * 1-5 — Business Hours Start: Start at 8:00 AM on weekdays
- 0 18 * * 1-5 — Business Hours Stop: Stop at 6:00 PM on weekdays
- 0 22 * * * — Night Shutdown: Stop at 10:00 PM every day
- 0 6 * * 1-5 — Morning Startup: Start at 6:00 AM on weekdays
- 0 20 * * 5 — Weekend Shutdown: Stop at 8:00 PM on Friday
- 0 7 * * 1 — Weekend Startup: Start at 7:00 AM on Monday
One schedule, three verbs
A Databricks workspace is not a single stoppable thing. It is a container for clusters, pools and warehouses that each idle differently. The workspace schedule applies the right verb to each: Terminate for interactive clusters, an idle-floor of 0 for instance pools, Stop for SQL warehouses. One calendar entry covers the sprawl a workspace accumulates, instead of a per-resource schedule for every cluster a notebook user ever created.
The harshest treatment goes to interactive clusters
Termination is destructive: execution contexts, cached data and runtime-installed libraries on those clusters are gone, and a restart builds fresh compute. For clusters left attached to notebooks at 7pm this is almost always the right call: their state was abandoned, not in use. The cluster definitions survive, so morning users restart from configuration in a few minutes.
Pools and warehouses get gentler handling
Instance pools are not stopped at all. Their minimum idle count drops to 0, releasing the standby VMs held warm. Nothing that uses the pool breaks; cold starts just replace warm ones. SQL warehouses take a clean stop and restart on demand or on schedule. Neither loses data, and neither needs morning attention.
Scheduled jobs sail through
Job clusters are exempt on purpose. Databricks jobs provision their own ephemeral compute, run, and tear it down, so a 2am ETL job creates its cluster at 2am regardless of the workspace schedule and cleans up after itself. The schedule silences idle waste without touching the production pipeline calendar, which is what makes workspace-level scheduling safe to adopt broadly.
Two meters stop together
Every silenced resource was billing twice, in DBUs to Databricks and VM hours to Azure. The overnight saving compounds across both lines, which is why idle interactive clusters are routinely the largest single waste item on a data platform bill.
The fan-out also keeps pace with sprawl: a cluster created next month falls under the same workspace schedule automatically, with no per-resource wiring for anyone to remember.