Skip to main content
Idle resources · Medium severity

GCP Dataproc Cluster Idle

A Dataproc cluster bills for every VM in it for as long as it runs, and a cluster between jobs is paying full rate for finished work.

Sign inExplore the playground

Free to start. No card. The playground just needs your work email.

Google Cloudprovider
Mediumseverity
Savingswhere it shows up
Guidedfix, after a review

Google Cloud

Found, explained, fixed.

Findings with a dollar figure attached, idle, oversized, orphaned, unscheduled and undiscounted spend.

01

Detect

ZopNight checks this automatically across Google Cloud, with read-only access to the account.

02

Explain

Every finding says exactly what to change: Pause the idle Dataproc cluster. Where the saving can be proven it is priced; where it cannot, the finding says so.

03

Fix

After a guided review, where you confirm the change, ZopNight pauses the service, then checks the change took.

Applies to
Dataproc Clusters on Google Cloud
The fix, by hand
  1. Check Cloud Scheduler / Composer / Cloud Workflows for jobs that target the resource.
  2. Confirm no submitted jobs in the last 30 days: gcloud dataproc jobs list --cluster=‹name› --region=‹region›.
  3. For ephemeral workflows, prefer Dataproc Workflow Templates or Serverless Spark. They tear down automatically.
  4. If you must keep a long-lived cluster, enable idle TTL: gcloud dataproc clusters update ‹name› --region=‹region› --max-idle=30m.
  5. Otherwise delete: gcloud dataproc clusters delete ‹name› --region=‹region›.

ZopNight does this for you after a guided review.

Category
Idle resources. Resources running but doing no useful work: zero invocations, no connections, no traffic.
Where it appears
The Savings tab of Recommendations, with every affected resource listed.
Rule ID
RC-1210
Full reference

GCP Dataproc Cluster Idle

Related checks

GCP Dataproc Cluster Autoscaling DisabledGoogle CloudAzure Firewall IdleAzureAzure ML Online Deployment IdleAzureBedrock Marketplace Endpoint IdleAWSBedrock Provisioned Throughput IdleAWSGCP Cloud Spanner Instance IdleGoogle Cloud

See the idle resources in your account.

Connect a read-only role and the first pass runs on your own estate. This check, and the rest of the catalogue, with it.

Start freeExplore the playground

Prefer to talk it through first? Book 20 minutes with the team.

  • $30M+annualised cloud spend under management
  • 550K+resources tracked since launch
  • 20-60%off the bill in the first month
  • SOC 2Type II report, plus ISO 27001

Figures published on zop.dev.

Multi-cloud automation· Production-ready in 30 min· SOC 2 · ISO 27001· 20–60% off the bill, first month· 4 platforms · 1 console· Multi-cloud automation· Production-ready in 30 min· SOC 2 · ISO 27001· 20–60% off the bill, first month· 4 platforms · 1 console·