Vertex AI endpoints serving zero predictions on always-on nodes
What does ZopNight detect here?
Vertex AI endpoints keep dedicated serving nodes running regardless of prediction traffic. ZopNight flags endpoints whose VertexPredictionCount shows 0 activity over the window, preferring a scheduled off-hours undeploy when a measured idle heatmap exists, and otherwise recommending teardown of the deployment, since there is no native stop for an endpoint.
Signal and threshold
| Field | Value |
|---|---|
| Rule IDs | RC-1212 |
| Category | idle |
| Severity | medium |
| Metric | none — pure configuration read |
| Source | vertex_idle.go |
Where it applies
Serving nodes never sleep on their own
Deploying a model to a Vertex AI endpoint allocates dedicated serving machines that bill
by the node-hour from deployment until undeploy. Prediction traffic does not modulate the
charge: a model that answered its last request in March costs the same as one under load.
An endpoint harvests 12 different monitoring series, so the rule reads
VertexPredictionCount strictly by name and fires only when both average and maximum are
0 across the harvested window; any measured prediction clears it.
Two levers, schedule preferred
When the aggregator has built a weekly usage heatmap for the endpoint, the recommendation becomes a recurring off-hours undeploy/redeploy window, priced as the endpoint’s cost times its measured idle fraction. That is the safe lever. Serving is unchanged during working hours, and nothing is destroyed. Only when no schedule data exists does the rule fall back to the full teardown recommendation, with the entire monthly cost as the recoverable amount.
No stop API means undeploy and redeploy
Vertex endpoints cannot be paused. “Off” is implemented as undeploying the model, via the
aiplatform.endpoints.undeploy permission in roles/aiplatform.user, with
aiplatform.endpoints.deploy restoring it from the saved DeployedModel configuration. It
is typically automated with Cloud Scheduler and a small Cloud Function. Sporadic traffic
is better served by moving to an autoscaling deployment with a zero minimum where the
model allows it.
Inspect the endpoint’s deployments
gcloud ai endpoints describe ENDPOINT_ID --region us-central1 \ --format="value(deployedModels.id,deployedModels.dedicatedResources.minReplicaCount)"A non-empty deployment list with a flat prediction graph is the exact state this rule prices.
The pricing catch on Vertex serving
Vertex serving SKUs are only attributed to an endpoint when billing-export data ties spend to its UID; ZopNight’s calculated rack-rate path deliberately prices these endpoints at $0 rather than guess a rate, so the rule fires for billing-connected accounts. It also abstains when the prediction metric is absent or when any traffic is observed. An unmeasured endpoint is never torn down on assumption.