# Over-Provisioned SageMaker HyperPod Cluster

> ZopNight flags a HyperPod cluster as over-provisioned when node CPU averages between 5% and 10% while memory stays under 50% across 30 days, on a fleet of 2 or more nodes. Removing one of N nodes saves roughly cost/N; single-node clusters and bursty fleets are skipped.

Source: https://zop.dev/integrations/aws/recommendations/over-provisioned-sagemaker-hyperpod-cluster
Updated: 2026-08-19

---

## Why the CPU band starts at 5%

A HyperPod cluster whose nodes average under 5% CPU is not over-provisioned. It is idle, and a
separate rule owns that case with a teardown recommendation. This rule begins where that one
stops: node CPU averaging at least 5% but under 10%, with node memory under 50%, sustained over
a 30-day lookback. The floor makes the two rules disjoint by construction, so the same monthly
reading can never produce both a "delete it" card and a "shrink it" card for one cluster.

## Two nodes minimum, by definition

Shedding a node only makes sense if a node remains. A single-node cluster cannot get smaller
and keep running, so the rule never fires below 2 nodes. A node count of zero or unset means
discovery could not size the cluster; that also lands below the floor and skips, because the
rule refuses to claim a saving on a fleet whose real size it does not know.

## The cost/N figure is deliberately approximate

HyperPod [bills per node instance-hour](https://aws.amazon.com/sagemaker/pricing/), so removing one of N nodes returns roughly one Nth of
the monthly cost. Roughly, because clusters can mix instance groups with different instance
types: cost/N prices the removal of one average node, and the exact figure depends on which
group you actually scale down. The recommendation states this caveat rather than presenting
the estimate as exact.

## Peak, coverage, and pricing abstains

Both of [the cluster's node utilisation series](https://docs.aws.amazon.com/sagemaker/latest/dg/monitoring-cloudwatch.html) (`node_cpu_utilization` and
`node_memory_utilization`) must exist for the cluster with enough coverage days; a missing or thin series produces no finding. The trusted CPU peak must
also sit below the safe ceiling, because a fleet that bursts near saturation would be starved
if shrunk, however low its average. And when no positive monthly cost is available from pricing
data, the rule emits nothing rather than a zero-dollar card.

## Sizing the instance groups yourself

```bash
aws sagemaker describe-cluster --cluster-name my-hyperpod \
  --query 'InstanceGroups[].[InstanceGroupName,InstanceType,CurrentCount,TargetCount]' \
  --output table
```

Compare each group's node count against its per-group utilisation in CloudWatch before
deciding where the surplus lives.

## Scaling down without hurting training

HyperPod has no in-place resize, so the finding is advisory: you apply it by lowering an
instance group's count through UpdateCluster. Pick the group whose nodes are actually idle,
not the most expensive one (they are often different groups), and watch training throughput
and queue wait on the smaller fleet before shrinking again.