Release EMR Serverless Pre-Initialized Capacity at Night

Idle EMR Serverless applications only cost money when they hold pre-initialized workers. Here is how to release them on a timer.

An EMR Serverless application with pre-initialized capacity keeps a warm pool of workers so jobs start in seconds. AWS is explicit that you pay for those provisioned workers even when the application is idle. The overnight lever is therefore not the jobs; it is the warm pool. stop-application releases the initial capacity, and start-application initializes it again.

Find applications holding warm workers

aws emr-serverless list-applications --states STARTED \
  --query "applications[].{id:id,name:name,type:type}" --output table

aws emr-serverless get-application --application-id 00f1abcdexample \
  --query "application.{state:state,initialCapacity:initialCapacity,autoStop:autoStopConfiguration,autoStart:autoStartConfiguration}"

An empty initialCapacity means the application has no warm pool and nothing to release on a schedule.

Check the auto-stop and auto-start settings first

Two settings decide whether a schedule is even needed:

Auto-stop is enabled by default and stops the application after 15 idle minutes. The idle timeout can be set from 1 to 10080 minutes. If someone raised it to a week, the warm pool never goes away on its own.
Auto-start is enabled by default and starts the application when a job is submitted. A nightly job or a stray test submission will start a stopped application and re-initialize its pre-initialized workers.

AWS recommends keeping auto-stop on when you use pre-initialized workers. To put it back to the default:

aws emr-serverless stop-application --application-id 00f1abcdexample
aws emr-serverless update-application --application-id 00f1abcdexample \
  --auto-stop-configuration enabled=true,idleTimeoutMinutes=15

The stop comes first because an application has to be in the stopped or created state before it can be updated.

Stop it on a schedule

All scheduled and running jobs must be completed or cancelled before the application can stop, so pick a time after the last pipeline finishes.

aws emr-serverless stop-application --application-id 00f1abcdexample
aws emr-serverless get-application --application-id 00f1abcdexample \
  --query "application.state" --output text

The state moves through STOPPING to STOPPED. In the morning:

aws emr-serverless start-application --application-id 00f1abcdexample

STARTING then STARTED means the warm pool is back.

Or shrink the pool instead of removing it

If overnight jobs do run but do not need a warm start, keep the application but set a smaller pool while it is stopped, then restore it before business hours:

aws emr-serverless update-application --application-id 00f1abcdexample \
  --initial-capacity '{"DRIVER":{"workerCount":1,"workerConfiguration":{"cpu":"2vCPU","memory":"4GB"}},"EXECUTOR":{"workerCount":2,"workerConfiguration":{"cpu":"2vCPU","memory":"4GB"}}}'

Jobs that need more than the pool still get extra workers, up to the application’s maximum capacity; they just wait for them to be provisioned.

Permissions

emr-serverless:ListApplications, emr-serverless:GetApplication, emr-serverless:StopApplication, emr-serverless:StartApplication, and emr-serverless:UpdateApplication if you change capacity or auto-stop.

Verify

Next morning, get-application should report STARTED with the full initialCapacity map. If the application is STARTED at 02:00, a job submission woke it through auto-start; aws emr-serverless list-job-runs --application-id 00f1abcdexample shows which one.

ZopNight

EMR Serverless applications are on ZopNight’s start and stop list, so the stop and start above can run from a schedule with cron expressions in an IANA timezone. A force-off override with an expiry keeps an application stopped through a quiet week even if the regular schedule would start it. Details on the resource are on the EMR Serverless scheduling page.