Interactive clusters running SQL and Delta work without Photon's engine
What does ZopNight detect here?
Photon bills roughly 2 to 2.9 times the DBU rate while often finishing SQL and Delta workloads in far fewer DBU-hours, a trade that only pays where vectorized execution applies. ZopNight flags non-Photon interactive clusters but suppresses ML runtimes and unknown workloads, promising no savings without a benchmark.
Signal and threshold
| Field | Value |
|---|---|
| Rule IDs | RC-2311 · RC-2411 · RC-2211 |
| Category | performance |
| Severity | low |
| Metric | none — pure configuration read |
| Source | photon_disabled.go |
Where it applies
A higher rate that can still lower the bill
Photon is Databricks’ vectorized execution engine, and enabling it raises the DBU rate to roughly 2 to 2.9 times the standard multiplier, depending on compute type. The bet is on the other axis: for SQL and Delta-heavy workloads, vectorized execution can finish the same work in a fraction of the DBU-hours, and rate times hours is what you actually pay. When the runtime shrinks by more than the rate grows, Photon is a cost reduction wearing a price increase. When it does not, Photon is just a price increase, which is why this rule refuses to promise numbers.
What the runtime strings reveal
Photon presence is read from two signals: a runtime engine value of PHOTON, or a
Spark version string carrying the -photon- image infix. Either one means the
cluster is already accelerated and the rule moves on. Two suppression gates then do
the real safety work. A Spark version containing the -ml- infix marks a Databricks
ML runtime, and Photon does not accelerate ML, UDF, RDD, or streaming workloads. It
would raise their DBU rate for no speedup, so ML runtimes are never flagged. And an
empty Spark version means the workload type is unknowable, so the rule stays silent
rather than risk recommending a cost increase. Stopped clusters and system-managed
job, pipeline, SQL, and serving clusters are out of scope throughout.
Enumerate engines per cluster
databricks api get /api/2.1/clusters/list | jq ' .clusters[] | {cluster_name, runtime_engine, spark_version}'Why the finding carries no dollar amount
Whether Photon wins depends on how vectorizable your specific queries are, and nothing in cluster metadata can know that. The finding is filed as a performance advisory with a savings figure of 0: a deliberate refusal to fabricate a percentage that marketing pages elsewhere are happy to invent.
Benchmark, then decide
Enable Photon on a test clone of the cluster, run a representative workload (the real nightly pipeline, not a synthetic query) and compare total DBU consumption against the non-Photon baseline. Roll it out where the DBU total drops and skip it where it does not; a week of measurement settles what no rule of thumb can.