SageMaker endpoint fleets of 2+ instances doing one instance's work
What does ZopNight detect here?
SageMaker bills every instance behind an endpoint per hour, so a 4-instance fleet at 3% CPU is mostly paid-for silence. ZopNight recommends shedding exactly one instance, saving cost/N precisely, after normalizing CPU by vCPU count, since the endpoint namespace sums utilization across cores.
Signal and threshold
| Field | Value |
|---|---|
| Rule IDs | RC-1624 |
| Category | rightsizing |
| Severity | medium |
| Metric | none — pure configuration read |
| Source | sagemaker_endpoint_instance_count.go |
Where it applies
The normalization most dashboards miss
CPU utilization in the SageMaker endpoint namespace is summed across vCPUs: a 4-vCPU instance reads 0–400%, not 0–100%. Read naively, a fleet running at a tenth of capacity looks four times busier than it is, and an earlier reading of this data under-fired on every multi-core family for exactly that reason. The rule resolves the instance type’s published vCPU count and divides before judging. It abstains entirely when the count cannot be resolved, because an un-normalizable number cannot be compared to a 10% threshold safely. Memory needs no such treatment; it is genuinely 0–100 in this namespace.
Who qualifies
In-service endpoints running at least 2 instances on a single production variant. One-instance endpoints cannot shed capacity without going dark; that is the idle rule’s territory. Multi-variant endpoints (A/B, blue-green, shadow) abstain because one utilization series cannot describe several independent fleets. The trigger is normalized CPU under 10%; memory under 50% acts as a veto when its series exists, and since memory here is an agent-dependent metric, its absence falls back to the CPU-plus-peak gates rather than silently exempting the fleet. A normalized peak near saturation suppresses everything: bursty fleets keep their headroom.
Arithmetic with no estimation in it
Endpoint cost is the per-instance hourly rate × instance count, so dropping one of N saves exactly cost/N: a fifth of a five-instance fleet’s bill, half of a two-instance fleet’s. No utilization-to-dollars model, no fabricated fraction. That is also why the rule abstains when the endpoint’s measured cost is missing.
Observe a fleet yourself
aws cloudwatch get-metric-statistics \ --namespace /aws/sagemaker/Endpoints --metric-name CPUUtilization \ --dimensions Name=EndpointName,Value=my-endpoint Name=VariantName,Value=AllTraffic \ --start-time "$(date -u -v-30d +%Y-%m-%dT%H:%M:%SZ)" \ --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ --period 86400 --statistics Average MaximumDivide the values by the instance’s vCPU count before comparing to any threshold.
Applied by you, not by automation
Capacity on an inference endpoint is an SLA decision involving traffic peaks, latency budgets, and autoscaling floors, so this recommendation is advisory by design: no automated resize is offered. Lower the variant’s desired instance count (or the autoscaling minimum), watch latency against SLA on the smaller fleet, and repeat next month if utilization still says so.