# SageMaker Batch Transform Job Over-Provisioned

> Flags batch transform jobs that under-used their ml.* instance and prices a one-size-smaller instance for the next run.

Source: https://zop.dev/integrations/aws/recommendations/sagemaker-batch-transform-job-over-provisioned

---

## A transform job pays for its instance for the whole run

Batch transform is charged for the instance type you choose, based on the duration of use, according
to [SageMaker AI pricing](https://aws.amazon.com/sagemaker/ai/pricing/). A job that runs on an
instance twice the size it needs pays roughly twice as much for the same predictions. Because
transform jobs usually run on a schedule, an oversized instance type in the job definition repeats
the waste every run.

## How to read transform job CPU correctly

The `/aws/sagemaker/TransformJobs` namespace reports `CPUUtilization` as the sum across cores: the
[SageMaker metrics reference](https://docs.aws.amazon.com/sagemaker/latest/dg/monitoring-cloudwatch.html)
gives 0% to 400% as the range on a four-CPU instance. `MemoryUtilization` is already 0 to 100%.
Divide CPU by the vCPU count before comparing it with any threshold.

```bash
aws sagemaker list-transform-jobs --status-equals Completed \
  --query 'TransformJobSummaries[].[TransformJobName,CreationTime]' --output table

aws sagemaker describe-transform-job --transform-job-name my-transform-job \
  --query '[TransformResources.InstanceType,TransformResources.InstanceCount,TransformStartTime,TransformEndTime]'
```

The job metrics use a `Host` dimension of the form `transform-job-name/instance-id`.

## What has to hold for a finding

1. ZopNight resolves the instance's vCPU count from its `ml.*` type and divides CPU by it. If the
   count is unknown, it does not guess.
2. Normalised average CPU is below 10% and memory averages below 50%, within the 30-day window.
3. The normalised CPU peak is below 80%, so a job with a heavy phase is not shrunk.
4. The job has a positive run cost.
5. A concrete one-size-smaller `ml.*` type exists and ZopNight has rates for both types.

## Jobs that do not get a recommendation

GPU instance types are never downsized by this rule, and nor are the smallest sizes of a family or
types outside the `ml.*` naming. When either rate is missing, or the smaller type is not cheaper,
there is no finding. The job has already finished, so the advice applies to its next run; ZopNight
does not change the job definition.

## Per-run saving from the rate ratio

```text
saving = run cost x (1 - smaller type hourly rate / current type hourly rate)
```

This assumes the job takes about as long on the smaller instance. A CPU-light job usually does.

## Right-sizing the next run

1. Set a smaller `InstanceType` in the `TransformResources` of the next `CreateTransformJob` call,
   or in your SDK Transformer configuration.
2. If the job used several instances, try lowering `InstanceCount` as well.
3. Tune `MaxConcurrentTransforms`, `BatchStrategy` and `MaxPayloadInMB` so the smaller instance stays
   busy. [Batch transform](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html) caps
   `MaxConcurrentTransforms x MaxPayloadInMB` at 100 MB.
4. Compare the run time and cost of the next run with this one.