# AWS Glue Job DPU Rightsizing

> AWS Glue jobs whose peak needed executors fall below allocated capacity get a DPU rightsizing recommendation. ZopNight requires a remediable MaxCapacity class, a coverage-checked 30-day peak-executor gauge, and an aggregator-priced saving of at least $5: the delta (configured DPU minus peak-needed DPU) times the Glue DPU rate and measured DPU-hours.

Source: https://zop.dev/integrations/aws/recommendations/aws-glue-job-dpu-rightsizing
Updated: 2026-08-19

---

## What makes a Glue job over-provisioned and remediable

All three producers landed and the rule now correctly implements rules.MetricsAware, so RC-098 emits a real recommendation when an over-provisioned, remediable Glue job carries a real aggregator-priced saving; it abstains (returns nil) on any missing input. The retired implementation fired on a configured-DPU>10 tag and priced savings off a hardcoded 30% fraction, a WRONG_BINDING_CONSTRAINT; that is gone. Firing requires all of:
1. Configured DPU (remediable class). The discoverer's glue:GetJobs enrichment (GlueProvider, glue.go) stamps configured_dpu (MaxCapacity, or NumberOfWorkers × worker-type-DPU) and dpu_remediable (the MaxCapacity class the provisioner can resize through RecommendTarget → glueMaxCapacity, resize/aws/glue_job.go; worker-type jobs fail closed). The rule gates on getMetadataFloat("configured_dpu")>0 and metadataBool("dpu_remediable")==true (see discoverer LIVE.md).
2. A trusted DPU-utilization peak. numberMaxNeededExecutors (peak executors the load needed) vs numberAllExecutors (allocated) is the authoritative over-provisioning signal, and both counters come straight from [the executor gauges Glue publishes to CloudWatch](https://docs.aws.amazon.com/glue/latest/dg/monitoring-awsglue-with-cloudwatch-metrics.html). The rule requires HasSufficientMaxCoverage() (MaxCoverageDays >= MinMaxCoverageDays) on the peak gauge before believing the job is over-provisioned; an untrusted peak abstains. The gauge is fetched via RequiredMetrics() (gluePeakNeededExecutorsMetric, 30-day lookback), which is required for both halves of rules.MetricsAware: it is the reason SetMetrics actually gets called by service.go's injection loop, and it is what admits the gauge into requiredMetricNames()'s union / the aggregator's metric_names allow-set.
3. A real aggregator-priced saving. glue-job moved off calcFree to calcGlueDPU (aggregator), which prices the job (configuredDPU × measured DPU-hours × the AWSGlue glue-dpu rate, since Glue [bills a job by DPU-hour](https://aws.amazon.com/glue/pricing/)) and emits the dedicated GlueDPURightsizeSavingsUSD quantity delta (configuredDPU − peakNeededDPU) × rate × DPU-hours, where peakNeededDPU = configuredDPU × clamp(maxNeeded/all, 0..1). It is injected here via the GlueDPURightsizeSavingsAware seam (SetGlueDPURightsizeSavings, mirroring the S3 retier seam) → glueDPUSavingsMap[uid]. RC-098 emits exactly that saving (clamped to the priced cost) and abstains when it is absent or below the $5 floor, never a fraction.

## The aggregator quantity delta, not a fraction

SavingsUSD = ComputedCostEntry.glueDpuRightsizeSavingsUsd (the aggregator quantity delta), clamped to the priced CurrentCostUSD. This is not a rate delta and not a fraction of the bill. See aggregator COST.md / METRICS.md for the producer.

## Reducing MaxCapacity incrementally

1. Open the Glue job and review the current MaxCapacity (DPU count)
2. Check the Glue job run history for executor utilization
3. Enable Glue job metrics and review glue.driver.ExecutorPool.numberAllExecutors vs. numberActiveExecutors
4. Reduce MaxCapacity incrementally (e.g., by 25%) and compare run times
5. Enable Glue auto-scaling if available for the job type to let AWS optimise DPUs dynamically

## The peak-needed executors gauge

glue.driver.ExecutorAllocationManager.executors.numberMaxNeededExecutors (peak-need gauge that gates firing; coverage-checked), numberAllExecutors + cost/GlueElapsedTimeMs (consumed in the aggregator cost path)