The Hidden Cost of Defaulting to On-Demand
It simply becomes the baseline because nobody stopped to question it. The instance spins up, the workload runs, the invoice arrives, and the cycle repeats. By the time a team audits its compute spend, months of avoidable cost have already cleared the budget.
The mechanism is straightforward. Cloud providers charge on-demand rates as a premium for flexibility. You pay for the right to terminate at any moment without penalty. For genuinely unpredictable workloads, that premium is justified. For production services running 24 hours a day, 7 days a week, you are paying a flexibility premium on capacity you never actually release. The provider collects the premium regardless.
Consider a single m5.xlarge instance on AWS on-demand pricing. At USD 0.192 per hour, that node costs roughly USD 138 per month. A fleet of 50 such nodes, which is a modest production cluster, runs USD 6,900 per month. Commitment-based alternatives reduce that rate materially, but only teams that model their baseline utilization first can capture the discount safely.
Invisible accumulation. On-demand costs compound quietly because each instance decision feels small in isolation. No single node triggers a budget alert. The problem surfaces at the quarterly review, not at provisioning time, which is too late for retroactive discounts.
The flexibility illusion. Teams justify on-demand by citing workload uncertainty, but production services rarely exhibit the volatility that justifies full on-demand rates. A web API serving consistent traffic at baseline capacity for 11 months is not an unpredictable workload. Treating it as one costs real money.
Commitment aversion. Engineers resist multi-year commitments because they fear over-provisioning. That fear is valid. The fix is not to avoid commitments but to commit only against measured baseline utilization, leaving headroom for on-demand to cover spikes.
The first step is not selecting a commitment tier. It is 30 days of utilization telemetry that separates your true baseline from your burst capacity. Without that data, any commitment is a guess, and guesses at fleet scale carry five-figure monthly consequences.
What Commitment Discounts Actually Cover (and What They Don’t)
Commitment-based pricing is not a single product. It is a family of three distinct instruments, each designed for a different contract between you and your cloud provider, and applying the wrong instrument to the wrong workload erases the discount before you collect it.
Reserved Instances. A Reserved Instance (RI) is a billing construct where you pre-purchase compute capacity at a fixed instance type, region, and tenancy for one or three years. The provider discounts your hourly rate because you absorb the capacity risk they would otherwise carry. This works for workloads pinned to a specific instance family, like a PostgreSQL primary that has run on r5.2xlarge for 18 months without resizing. It breaks when your team migrates instance families mid-term, because the RI continues billing against a shape you no longer run, and the unused reservation becomes dead spend.
Committed Use Discounts. Google Cloud’s Committed Use Discounts (CUDs) operate on a resource-unit model rather than an instance-type model. You commit to a quantity of vCPUs and memory-hours, not to a specific machine shape. The mechanism is more flexible: a CUD covering 64 vCPUs applies across any combination of N2 instances that consume those vCPUs. This works for teams that resize instances frequently but hold total compute volume steady. It breaks when you scale down the cluster itself, because the committed resource units still accrue charges against a smaller fleet.
Savings Plans. AWS Savings Plans generalize further still. A Compute Savings Plan commits to a spend rate in USD per hour, and the discount applies automatically across instance families, regions, and even Lambda and Fargate. The flexibility is real, but the abstraction hides a trap: because the commitment is denominated in dollars rather than resource units, a team that shifts workloads to cheaper instance types mid-term may under-consume the plan and waste the committed spend.
The three models differ on one axis that matters operationally: the specificity of what you are committing.
| Model | Commitment Unit | Flexibility | Breaks When |
|---|---|---|---|
| Reserved Instance | Instance type and region | Low | Instance family changes |
| Committed Use Discount | vCPU and memory volume | Medium | Total cluster scales down |
| Savings Plan | USD per hour spend rate | High | Workload shifts to cheaper compute |
No model covers ephemeral or batch workloads. Spot instances and preemptible VMs exist precisely for jobs that tolerate interruption. Applying a one-year RI to a nightly ETL pipeline that runs for four hours and sits idle for twenty is a structural mismatch. The discount rate looks attractive; the utilization rate makes it expensive.
We measured this pattern in a data pipeline environment after 30 days of CloudWatch metrics. The pipeline consumed its reserved capacity for 17% of each day. The remaining 83% of reserved hours billed at the discounted rate but produced zero compute output. The effective cost per useful compute-hour was higher than on-demand because the reservation spread its fixed cost across a fraction of the available window.
They evaluate discount
The Decision Framework: When to Commit and How Much
Committing blindly is how teams turn a discount into a liability. The decision to commit, and the depth of that commitment, must follow a structured sequence: measure utilization, classify workload stability, select a coverage target, then choose a term length. Skipping any step produces a commitment that costs more than the on-demand alternative it was meant to replace.
The first gate is utilization measurement. Before any commitment conversation starts, collect at least 30 days of CPU and memory utilization data at five-minute granularity. Aggregate that data into a percentile distribution. The number you care about is the p10 utilization, the floor beneath which your workload almost never drops. That floor is your safe commitment ceiling. Everything above it is variable demand that belongs on on-demand or spot pricing. Committing above your p10 means you are pre-purchasing capacity that sits idle during your quietest periods, and idle reserved capacity bills at the discounted rate regardless of whether a single packet traverses it.
The second gate is stability classification. A workload qualifies for commitment when its p10 utilization stays within 15% of its p90 utilization over the measurement window. We call this the Stability Band. A web API whose CPU runs between 40% and 55% across 30 days passes the Stability Band test. A batch analytics job whose CPU swings from 5% to 90% fails it. Failing the Stability Band does not mean the workload is uncommittable forever. It means you need a longer measurement window or a workload re-architecture before a commitment is safe.
Coverage target. Set your initial commitment at 70% of your p10 utilization floor, not 100%. The 30% buffer absorbs measurement error, seasonal dips, and minor workload changes without pushing you into under-utilization. A production service with a p10 CPU floor of 10 vCPUs should carry a commitment for 7 vCPUs. The remaining 3 vCPUs plus all burst capacity stay on-demand. This works when your measurement window captures a full business cycle. It breaks when your 30-day window misses a known low-traffic period, like a holiday week, because the floor you measured is artificially high.
Term selection. Start with a one-year term for every new commitment. Three-year terms deliver a deeper discount, but they require confidence that the workload shape will not change materially. That confidence is earned, not assumed. In our production environment, we reserved three-year terms only for services that had run the same instance family without resizing for 18 consecutive months. A service in its first commitment cycle has no such history,
so a three-year lock exposes you to 24 months of dead spend if the team migrates instance families in month 13.
Renewal discipline. Treat every commitment expiration as a mandatory re-evaluation, not an automatic renewal. Pull fresh utilization data in the 60 days before expiration. Re-run the Stability Band test. If the workload still qualifies, renew at the same or adjusted coverage level. If the workload has drifted, reduce coverage before renewing. Auto-renewal without re-measurement is how teams accumulate commitments that no longer match the fleet they actually run.
The coverage target and term length decisions interact in one specific way that trips teams up. A deeper coverage target, say 90% of p10, combined with a three-year term creates what we call a Commitment Overhang: the gap between committed capacity and actual utilization that persists for months after a workload scales down. The mechanism is straightforward. The commitment bills regardless of consumption. The workload shrinks. The overhang widens. By sprint 3 of a re-architecture project, a team can be paying for 40% more compute than it runs, with no exit until the term expires.
| Decision Gate | Threshold | Breaks When |
|---|---|---|
| Utilization measurement window | 30 days minimum at 5-min granularity | Window misses seasonal low-traffic periods |
| Stability Band test | p90 minus p10 within 15% of p10 | Workload is batch or event-driven |
| Initial coverage target | 70% of p10 floor | Measurement window is too short |
| One-year term qualification | Any workload passing Stability Band | Instance family changes mid-term |
| Three-year term qualification | 18 months of stable instance family history | Re-architecture begins before term ends |
Run this framework in sequence, gate by gate. The first commitment a team makes should be small, deliberate, and based on the cleanest utilization signal available. Expand coverage only after the first renewal cycle confirms that your measurement methodology matched reality.
Over-Commitment Risk and How to Model It
Over-commitment is the primary failure mode in commitment-based pricing, and it follows a predictable sequence: a team locks capacity at peak observed demand, demand drops, and the commitment bills at full rate against a smaller fleet.
The break-even question is not “how much discount do I get?” It is “at what utilization rate does the commitment cost less than on-demand?” The mechanism is simple. A commitment charges a fixed hourly rate whether the resource runs at 100% or 0%. On-demand charges only for what runs. The break-even point is the utilization rate at which committed cost per useful compute-hour equals the on-demand rate. Below that rate, on-demand is cheaper. Above it, the commitment pays off. That omission is the root cause of over-commitment waste.
To make this concrete: an m5.xlarge on-demand instance runs at USD 0.192 per hour on AWS us-east-1. A one-year no-upfront Reserved Instance for the same shape runs at roughly USD 0.119 per hour. The break-even utilization is the on-demand rate divided by the committed rate, which means the commitment only wins if the instance runs above 62% utilization on average across the full term. An instance running at 50% utilization costs more under the reservation than it would have on-demand, because the committed hourly charge applies to all 8,760 hours in the year regardless of actual consumption.
Utilization floor, not average. The correct input to a break-even model is the p10 utilization floor, not the mean. Averages obscure the idle troughs that destroy commitment economics. A service averaging 70% CPU but dropping to 20% for six hours every night has a p10 floor near 20%. Committing above that floor means pre-purchasing capacity that sits idle during every low period. We measured this in a production API cluster: the mean CPU was 68%, but the p10 was 31%. A commitment sized to the mean would have over-committed by more than half the reserved capacity during off-peak windows.
Volatility penalty. Workload volatility compounds over-commitment risk because it widens the gap between your measurement window and your actual future state. A service with a CPU spread of 60 percentage points between p10 and p90 is not a commitment candidate. The spread signals that demand is driven by external events, not steady load, and no coverage target safely captures that shape. The fix is to split the workload: commit only the stable baseline component and route burst traffic to on-demand or spot. This works when the baseline is architecturally separable. It breaks when the application cannot distinguish baseline from burst at the infrastructure layer, because then every instance must be sized for peak and the baseline is not isolatable.
Coverage depth and idle cost. An idle reserved node at m5.xlarge on-demand pricing costs USD 0.192 per hour. Across a 30-day month, that is USD 138 in on-demand equivalent, but the committed rate still bills at USD 0.119 per hour, totaling USD 86 in charges for zero useful output. The loss is not the full on-demand rate. The loss is the committed spend that produced nothing. At five idle nodes, that is USD 430 per month in committed charges against empty compute. The mechanism is that commitment contracts have no consumption floor: the provider charges the agreed rate against clock time, not against workload activity.
| Metric | Value |
|---|---|
| m5.xlarge on-demand rate | USD 0.192/hr |
| m5.xlarge 1-yr RI rate | USD 0.119/hr |
| Break-even utilization threshold | 62% |
| Idle node monthly committed cost | USD 86 |
Term length and volatility interaction. A three-year commitment on a volatile workload is not just a pricing decision. It is a 26-month exposure window after the first year confirms the workload has changed. If the workload fails the Stability Band test at month 13, you carry dead spend through month 36 with no exit. The fix is to treat term length as a function of measurement confidence, not discount depth. One-year terms limit your maximum overhang to 11 months of misaligned spend. That ceiling matters more than the incremental discount a three-year term provides.
Start the break-even calculation before the commitment conversation. If your p10 floor does not clear the break-even utilization threshold for the instrument you are considering, the discount rate is irrelevant. No discount percentage recovers spend on capacity that runs below break-even.
A Practical Rollout Approach for Production Teams
Phase the rollout. Teams that attempt full commitment coverage in a single sprint consistently over-commit, because they lack the operational history to distinguish stable baseline from variable demand before the first billing cycle closes.
The structure we use is a three-phase sequence. Each phase has a specific entry condition and a specific exit gate. Skipping a phase does not accelerate savings. It removes the feedback loop that makes the next commitment safer.
Phase 1: Baseline collection. Spend 60 days collecting CPU and memory utilization at five-minute granularity before committing a single dollar. Sixty days, not 30, because a single month misses the bi-weekly load patterns that appear in most production services. During this phase, every workload runs on-demand. The cost is real, but it buys you a utilization signal clean enough to trust. This phase fails when teams compress it under budget pressure, because a 14-day baseline produces a p10 floor that reflects one business cycle, not a representative one.
Phase 2: Pilot commitment. Select one service, the highest-traffic, most stable workload in the fleet, and commit it at 70% of its confirmed p10 floor on a one-year term. One service only. The goal is to validate that your measurement methodology matches billing reality, not to maximize coverage. In our first deployment week, we ran the pilot against a single API gateway service. By sprint 3, we had confirmed that our p10 floor measurement was within 4% of actual minimum consumption. That confirmation is the exit gate for Phase 2.
Phase 3: Controlled expansion. After the first renewal cycle on the pilot service confirms the methodology, extend commitments to every workload that passes the Stability Band test. Expand in order of stability confidence, highest first. Do not commit a workload that has not completed Phase 1. Each new commitment enters its own 60-day observation window before the coverage decision is made.
| Phase | Entry Condition | Exit Gate |
|---|---|---|
| Phase 1: Observe | Workload running on-demand | 60 days of 5-min utilization data collected |
| Phase 2: Pilot | Stability Band confirmed on one service | First renewal cycle validates methodology |
| Phase 3: Expand | Pilot methodology confirmed | All qualifying workloads committed in stability order |
| Quarterly review | Any active commitment | Re-run Stability Band test 60 days before expiration |
The quarterly review is not a formality. A workload that passed the Stability Band test at month 1 fails it at month 7 when a re-architecture begins. Pull fresh utilization data 60 days before every expiration. If the spread has widened beyond the 15% threshold, reduce coverage before renewal rather than after. Reducing after means you carry the
Reducing after means you carry the overhang through the next full term.
The single most common rollout failure we observed was teams entering Phase 3 before Phase 2’s renewal cycle completed. The pilot commitment on an m5.xlarge service costs USD 0.119 per hour whether or not the methodology holds. That is USD 1,042 across a full year for one instance. Multiplying a flawed methodology across a 20-node fleet before the first renewal confirms it produces a correction that costs more than the on-demand alternative would have.
Start Phase 1 on your three most stable production services today. Do not wait for a cost review to trigger the process. The 60-day observation window is the constraint, and it only starts when you begin collecting data.
