AWS Glue Job

An AWS Glue job runs serverless ETL on Spark or Python shells, billed per DPU-hour consumed. Over-provisioned DPU settings and frequently retried jobs inflate ETL spend without changing output.

DPU-hours: workers multiplied by runtime

A Glue job bills per DPU-hour, metered per second with a 1-minute minimum while the job runs. A data processing unit bundles vCPU and memory. Cost is therefore a simple product: number of workers times how long they run. There is no idle charge between runs; a Glue job that never executes costs nothing, which concentrates all waste inside the runs themselves.

Oversized by template

Worker counts get set once, usually by copying another job’s configuration, and never revisited. A job allocated 20 workers that Spark cannot parallelize beyond 4 pays five times the necessary rate every run, and because each run completes successfully, nothing ever looks wrong. The reverse error also bills: too few workers stretch runtime, and DPU-hours are indifferent to which factor grew.

Retries re-bill the whole run

A failing job on a schedule with automatic retries bills every attempt in full. An ETL job that runs 50 minutes and then fails on a bad output permission, retried 3 times nightly, bills roughly 3 hours of DPU-time per night producing nothing. Failure-retry patterns are cost signals here, not just reliability ones.

Per-job attribution

Glue jobs are discovered via a dedicated provider on the 6-hour cycle, with per-job cost from Cost Explorer or CUR 2.0. Recommendations cover DPU rightsizing against observed parallelism and flagging retry loops. There is nothing to stop or schedule at the resource level; the job definition is dormant configuration between runs.

Run history in the console

AWS Glue console, then ETL jobs, then a job’s Runs tab. Each run row shows DPU-hours consumed and status side by side. A column of failed runs each carrying full DPU-hours is the retry loop made visible.