Dataproc clusters labeled as running without autoscaling
What does ZopNight detect here?
Dataproc clusters labeled autoscaling=false are flagged by ZopNight rule RC-1211 as medium severity. Detection is attestation-based: the rule reads the cluster's own labels, not its autoscalingConfig, so an unlabeled fixed-size cluster is never flagged. Fixing it means creating an autoscaling policy with min/max worker counts and attaching it.
Signal and threshold
| Field | Value |
|---|---|
| Rule IDs | RC-1211 |
| Category | compliance |
| Severity | medium |
| Metric | none — pure configuration read |
| Source | dataproc_autoscaling.go |
Where it applies
A label, not a cluster inspection
Honesty first: RC-1211 does not examine a Dataproc cluster’s autoscalingConfig. Its gate
is the cluster’s own labels: it fires when a label named autoscaling carries the exact
value false, and abstains in every other case, including the common one where no such
label exists at all. Detection is attestation-based: a team that labels its fixed-size
clusters is telling ZopNight about them, and the rule surfaces that admission as a standing
finding rather than letting it sit unread in a label nobody queries.
What the rule would like to prove but cannot
The provable version of this check would read config.autoscalingConfig.policyUri from the
cluster and flag its absence. Today’s discovery path stamps Dataproc clusters from Cloud
Asset Inventory (roles/cloudasset.viewer) with enrichment reads under
roles/dataproc.viewer (dataproc.clusters.list), and the labels ride along with the
asset. Until the policy URI is part of that enrichment, an unlabeled cluster with no
autoscaling policy produces no finding. That is a false negative the rule accepts in exchange
for never false-flagging a cluster it has not actually inspected.
Fixed-size clusters and the bill they imply
The rule sits in the compliance category and carries a $0 figure, but the underlying concern is capacity governance. A Dataproc cluster with a fixed worker count is sized for its peak job; every hour below that peak, the idle workers bill at full price. Autoscaling policies bound the shape instead, setting a floor for baseline work and a ceiling for bursts. Dataproc adds and removes workers as jobs arrive and drain, which is the behavior the label is supposed to certify.
Inspect the autoscaling policy directly
gcloud dataproc clusters describe CLUSTER_NAME --region=REGION \ --format="value(config.autoscalingConfig.policyUri)"Empty output means no policy is attached, whatever the labels claim.
Attach a policy, then flip the label
Create an autoscaling policy with min and max worker counts matched to observed job
patterns, attach it to the cluster, and update the label to autoscaling=true so the
finding clears on the next discovery pass. The label update matters as much as the policy:
in an attestation-gated rule, the label is the sensor, and leaving it stale keeps the
recommendation open against a cluster that has already been fixed.