Feature Online Stores provisioned with nodes nobody queries
What does ZopNight detect here?
Feature Online Stores on the Bigtable-backed tier bill fixed nodes hourly even at 0 queries per second. ZopNight anchors idleness on the continuous optimized-nodes gauge and treats a missing request counter as idle-consistent: the request series is a delta counter that writes no datapoints while the store is idle.
Signal and threshold
| Field | Value |
|---|---|
| Rule IDs | RC-1218 |
| Category | idle |
| Severity | medium |
| Metric | none — pure configuration read |
| Source | vertex_idle.go |
Where it applies
Fixed nodes under a serverless name
A Feature Online Store sounds like managed infrastructure that scales itself away, and the Optimized tier does autoscale, but the Bigtable-backed tier runs a fixed node count that bills hourly regardless of serving traffic. A store stood up for an ML experiment that ended keeps its nodes running for nothing. Even a never-queried Optimized store is wasted setup: its FeatureViews sync data that no application reads back.
A gauge anchors, a counter vetoes
The idle decision anchors on VertexFeatureOnlineStoreNodes, the continuous
storage/optimized-nodes gauge that exists for a provisioned store even at 0 QPS. The
request counter, VertexFeatureOnlineStoreRequestCount, only vetoes: if that series is
present and shows any traffic at all (any positive average or maximum), the store is
serving and no recommendation appears.
Why absence of data is evidence here
The request series is a delta counter, and Cloud Monitoring writes no datapoints for a
delta counter that has nothing to count. A “present and zero” request series is therefore
impossible for an idle store. An earlier revision that demanded one could never fire on
exactly the resources it existed to catch. Treating the counter’s absence as
idle-consistent, while requiring the nodes gauge to be present, is what makes the rule
both live and safe. Both series arrive through monitoring.timeSeries.list in
roles/monitoring.viewer.
List your online stores
gcloud beta ai feature-online-stores list --region us-central1 \ --format="table(name,createTime,bigtable.autoScaling.minNodeCount)"Confirm with the owning team that no service reads online features from a flagged store, then delete the store and its FeatureViews together.
Billing attribution limits the firing
Pricing a Vertex serving type would mean inventing a per-node rate, so ZopNight’s calculated cost path carries no rack rate for these types and the rule only fires when GCP billing export attributes real spend to the store’s UID. It also abstains when the nodes gauge itself is absent. When it does fire, the full attributed monthly cost is the recoverable figure, since an unqueried store has no partial value to preserve.