Cloud Cost Anomaly Detection: A Practical Guide
Budgets catch spend that crosses a line you drew. Anomaly detection catches spend that stops looking like itself.
Cloud cost anomaly detection is the automated comparison of recent spend against what that spend would be expected to be, given its own history, so that unexpected changes are flagged even when they stay inside any budget. The FinOps Foundation’s Anomaly Management capability describes the goal as detecting, identifying, alerting on, and managing unexpected or unforecasted cost and usage irregularities in a timely manner. A budget answers “are we over the number”; anomaly detection answers “is this different from normal”, which is why a spike early in the month can be an anomaly long before it is a budget problem.
What the providers offer
AWS Cost Anomaly Detection uses machine learning models on net unblended cost, the cost after applicable discounts. It runs about 3 times a day after billing data is processed, but because it relies on Cost Explorer data, which can lag by up to 24 hours, detection can take up to 24 hours after the usage. A new monitor takes 24 hours to start detecting, and a new service needs 10 days of history before anomalies can be detected for it. Monitors can cover services independently or specific accounts, cost allocation tags, or cost categories, and root causes are ranked by dollar impact across service, account, Region, and usage type. Alerts go to email or an SNS topic.
Azure Cost Management evaluates subscriptions daily, comparing the day’s usage to a forecast built from the last 60 days, and runs 36 hours after the end of the day (UTC) so the data is complete. Anomalies appear as insights in Cost Analysis, and anomaly alerts can email a summary of the top resource group changes.
The lag matters. Provider anomaly detection finds yesterday’s surprise today or tomorrow; it is not real-time monitoring.
Tuning for signal
- Pick the right grain. An organization-wide monitor hides a team’s spike under everyone else’s steady spend. Per-account or per-team monitors surface it, at the cost of more alerts.
- Set a floor. Percentage changes on tiny costs are noise. Require a minimum absolute cost before an anomaly counts.
- Respect seasonality. Weekly patterns such as quiet weekends and busy Mondays are normal. Methods that learn from several weeks of history handle them better than a comparison with yesterday.
- Route by owner. An alert to a shared channel is read by nobody in particular. Send each scope’s anomalies to the team that owns it.
Investigating an anomaly
Work from the anomaly to a cause in a fixed order: which scope, which service, which resources, and what changed. Causes worth checking first are a new resource, a resize or price change on an existing one, a commitment that expired and left usage billed at on-demand rates, a schedule that failed to stop something, or usage that simply grew. Record the cause and whether it was expected, because the same pattern will recur.
Anomaly detection in ZopNight
ZopNight evaluates cost anomalies daily across organization, cloud account, resource type, resource group, resource, and team, plus Azure resource group and tenant hierarchy. It uses two methods, percent deviation from a 7-day rolling average and z-score, and takes the higher severity: warning from 30 percent deviation, critical from 100 percent, and emergency from 500 percent. Guards reduce noise: at least 4 data points, at least $1 a day of cost, and a standard deviation above 10 percent of the mean. When cost moves between teams with a net change under 20 percent, both team anomalies are suppressed as a redistribution. Each anomaly carries a root cause analysis that looks for instance resizes, new resources, reservation or savings plan expiry, schedule failures, and unscheduled usage increases.
Evaluation does not depend on notifications: anomalies appear on the cost trend chart and in the anomaly report whether or not a channel is configured, and subscribing later is purely additive. Subscriptions can be scoped by dimension and severity, are de-duplicated by date and entity, and re-notify when severity escalates.
Related: what cloud budgets actually do, and the step-by-step how to set up cost anomaly alerts.
Questions we get a lot.
If yours isn't here, email us and we'll answer directly.
How fast can cost anomaly detection alert?
It depends on billing data latency. AWS Cost Anomaly Detection can take up to 24 hours after usage because Cost Explorer data lags, and Azure runs detection 36 hours after the end of each UTC day. ZopNight evaluates anomalies once a day.
What is the difference between an anomaly and a budget alert?
A budget alert fires when spend crosses a threshold you set. An anomaly fires when spend departs from its own recent pattern, which can happen well inside a budget.
Do I need to configure a notification channel before ZopNight finds anomalies?
No. Anomalies are evaluated and stored for every organization and appear on the cost trend chart and in the anomaly report. Notifications are an optional layer on top.