Dev/test SageMaker endpoints worth tearing down outside working hours
What does ZopNight detect here?
Real-time endpoints bill their ML instances 24/7, and a dev/test endpoint used 8 hours a weekday is idle roughly two-thirds of the week. ZopNight quantifies the saving from each endpoint's measured weekly idle heatmap, never a flat assumption, and abstains for endpoints with no usage history.
Signal and threshold
| Field | Value |
|---|---|
| Rule IDs | RC-1611 |
| Category | schedule |
| Severity | medium |
| Metric | none — pure configuration read |
| Source | scheduling_sagemaker_endpoint.go |
Where it applies
No native off switch, and the rule is honest about it
Most schedulable resources stop and start. SageMaker real-time endpoints do not; the only “off” is deleting the endpoint and recreating it from its saved configuration. That is why this recommendation ships as advisory rather than a one-click schedule: the remediation describes the delete-and-recreate cycle (EventBridge plus a small Lambda, or your CI), keeping the EndpointConfigName so recreation is byte-identical each morning. The endpoint config and model objects persist through the cycle; only the hourly-billed instances come and go.
Measured idleness or nothing
The saving is the endpoint’s measured cost multiplied by the idle fraction its weekly CPU heatmap actually shows: an endpoint idle 65% of the week saves 65%. Without heatmap data for the specific endpoint there is no number worth printing, and the rule abstains rather than assume a nights-and-weekends pattern. One measurement caveat runs in the safe direction: endpoint CPU is summed across vCPUs, so multi-core endpoints can read busier than they are and the idle fraction under-measures. The rule under-fires or understates, never inflates.
The eligibility fence
Only dev/test-named, non-production-tagged, in-service endpoints on a single production variant qualify. Prod names or tags veto outright; endpoints with any schedule-related tag are assumed handled; Savings Plan or reservation coverage vetoes too, because tearing down committed capacity saves nothing, since the commitment bills anyway. Multi-variant endpoints abstain since the heatmap reflects only the first variant’s activity, and deleting a whole endpoint on one variant’s silence could drop another variant’s live traffic.
See the weekly shape yourself
aws cloudwatch get-metric-statistics \ --namespace /aws/sagemaker/Endpoints --metric-name CPUUtilization \ --dimensions Name=EndpointName,Value=my-dev-endpoint Name=VariantName,Value=AllTraffic \ --start-time "$(date -u -v-14d +%Y-%m-%dT%H:%M:%SZ)" \ --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ --period 3600 --statistics AverageHourly datapoints over two weeks make a business-hours pattern obvious at a glance.
When serverless beats a schedule
An endpoint whose traffic is genuinely sporadic (minutes of inference scattered through the day) may be better migrated to a serverless endpoint than scheduled: per-inference billing removes the teardown machinery entirely. The schedule wins when usage is predictable and sustained inside the window; serverless wins when it is not.