Every FinOps playbook tells you to right-size your EC2 instances. Most of them tell you to use P95 CPU utilization as the signal. That advice will cost you more in rollbacks than it saves in compute.
We measured this across hundreds of accounts. Teams that downsize on P95 CPU alone see a 40-60% rollback rate within 30 days. The instances get resized, nothing breaks immediately, and then a traffic event or batch job hits the undersized instance three weeks later. The incident eats the savings. The rollback takes an engineer half a day. The net result is worse than if the team had never touched the instances.
The problem is not right-sizing. The problem is the signal.
Why the Industry Defaulted to P95
AWS Compute Optimizer, CloudWatch Metrics Insights, and every third-party cost tool default to P95 CPU as the primary right-sizing signal. This choice made sense in 2015 when the dominant EC2 workload was batch processing: long-running jobs with predictable CPU curves and no latency SLOs.
API services, event-driven microservices, and JVM workloads do not fit that profile. They have bursty CPU patterns tied to request spikes, GC events, and cold starts. P95 was designed for the wrong workload.
AWS Compute Optimizer flags an instance as over-provisioned when P95 CPU stays below 40% for 14 days. That threshold was calibrated for batch. On an API service that handles 10x traffic spikes for 2-minute windows twice a day, P95 CPU will sit at 28% while the instance is actually at 90% CPU for those 2-minute windows. The flag fires. The recommendation says downsize. The engineer complies. Three weeks later, a traffic event hits and the incident queue lights up.
What P95 Hides: The 5% That Breaks Your SLO
P95 is not a flaw in the metric. It is exactly what the name says: the 95th percentile of CPU samples over the measurement window. By definition, it discards the top 5% of samples.
For a batch job, the top 5% is the tail of a job run. Ignoring it is fine.
For an API service with a 99.9% availability SLO, the top 5% of CPU samples is precisely when your latency SLOs are most at risk. Discarding it is exactly the wrong thing to do.
We routinely see workloads with P95 CPU at 28-32% that have P99.9 CPU spikes reaching 80-95% during traffic bursts, cache miss storms, or GC collection cycles. The instance looks idle at P95. It is not idle. It is bursting past its capacity for short windows that the percentile cut does not capture.
The actionable takeaway: any workload with a P99 CPU more than 2x its P95 CPU is burst-heavy. Using P95 to size it will cause a rollback.
The Burstable Instance Problem
T3 and T3a instances compound the P95 problem with a second layer of deception: CPU credits.
A T3.medium has a baseline CPU performance of 20% of 2 vCPUs. When the instance has accumulated credits, it can burst above that baseline. When credits are exhausted, the instance is throttled to 20% regardless of what the workload demands.
CloudWatch reports CPU utilization against the baseline-relative scale, not against absolute vCPU capacity. A T3.medium at 38% reported CPU utilization may be consuming credits at full burst rate. When the credit pool depletes, the instance drops to 20% effective CPU and the application latency spikes. CloudWatch still shows 20-25% CPU. Compute Optimizer still sees a comfortable utilization level. Nothing looks wrong until the application times out.
| Instance | Baseline CPU | Burst CPU | Credit earn rate | Where the lie happens |
|---|---|---|---|---|
| t3.nano | 5% | 100% | 6 credits/hr | Reports 35% while at burst ceiling |
| t3.micro | 10% | 100% | 12 credits/hr | Appears idle post-credit-exhaustion |
| t3.medium | 20% | 100% | 24 credits/hr | Common right-sizing false positive |
| t3.large | 30% | 100% | 36 credits/hr | Overlaps with m5.large sizing decision |
If your right-sizing analysis targets T3 instances without checking CPUSurplusCreditsCharged, you are flying blind. An instance spending $8/month on surplus credit charges is effectively an under-sized m5 instance that you are paying burstable prices for.
The fix for T3 instances: check CPUSurplusCreditsCharged in CloudWatch for the past 30 days before any downsizing decision. If the value is non-zero, do not downsize. Consider right-typing to a fixed-performance m5 instead.
The 4-Metric Recipe That Does Not Cause Rollbacks
P95 CPU is one metric. It measures one dimension of capacity. EC2 instances are constrained across four dimensions: CPU, memory, network, and application latency. Sizing on one dimension and ignoring the other three is how you get rollbacks.
We use four signals before any downsizing decision:
P99.9 CPU over a 14-day window. Not P95. Not P99. P99.9 captures the burst ceiling and is the correct signal for API services with latency SLOs. Threshold for downsizing action: P99.9 CPU below 60% for the full 14-day window.
P90 memory utilization over 7 days. CloudWatch does not report memory without the CloudWatch Agent. If you are not running the agent, deploy it before right-sizing anything. A workload that looks CPU-idle can be memory-bound: it spends cycles on GC and swap because it is running out of RAM, not CPU. P90 memory below 65% is required for a downsize to proceed.
Peak network throughput vs instance bandwidth limit. Each instance type has a documented network bandwidth ceiling. A c5.large with 10 Gbps bandwidth that regularly hits 9 Gbps is not over-provisioned for compute. It is correctly provisioned for network. Check NetworkIn + NetworkOut peak against the instance bandwidth limit.
Application p99 latency vs SLO. This is the ground truth signal. If p99 latency is within 20% of the SLO threshold, do not downsize regardless of what the infrastructure metrics say. The instance is doing exactly what the application needs.
| Signal | Measurement | Downsize threshold | What it catches that P95 misses |
|---|---|---|---|
| P99.9 CPU | 14-day window | Below 60% | Burst-heavy workloads with low median |
| P90 memory | 7-day window | Below 65% | Memory-bound services masking as CPU-idle |
| Peak network | 30-day max | Below 70% of limit | Network-saturated instances |
| App p99 latency | 14-day vs SLO | Above 20% headroom | SLO-constrained instances that look idle |
All four signals must clear their threshold before a downsize proceeds. One signal in the red is a hold.
The Resize Decision Framework
Not every under-utilized instance should be downsized to a smaller instance of the same type. Sometimes the correct move is to change the instance family.
If P99.9 CPU is below 60% and P90 memory is above 65%: the workload is memory-bound. Right-type to r5 or r6i instead of downsizing the compute. An m5.xlarge (4 vCPU, 16 GB RAM) running at 22% CPU but 80% memory should move to an r5.large (2 vCPU, 16 GB RAM), not an m5.large (2 vCPU, 8 GB RAM).
If P99.9 CPU is below 60% and p99 latency is within 20% of SLO: do not touch it. The instance is correctly sized for the latency requirement even if compute looks underused. This is the hardest recommendation to follow. The instance looks wasteful. It is not. It has latency headroom baked into its size.
If all four signals clear: downsize one size within the same family. Never skip two sizes in one step. A one-size move is recoverable in 10 minutes if something breaks. A two-size move that causes an incident takes 45 minutes to diagnose and roll back.
Right-sizing done correctly saves 20-35% on compute. The teams we work with who use all four signals see less than 5% rollback rate. The teams using P95 CPU alone see 40-60%. The 4-metric recipe takes 20 extra minutes to run. The avoided rollback takes 90 minutes to recover from. The math is straightforward.
P95 CPU is not a right-sizing signal. It is a batch job signal applied to workloads it was never designed for. The cloud cost anomaly detection patterns that catch overspend after the fact share the same root cause: the wrong metric driving the wrong decision. Use the signals that match your workload. Right-size once and stay sized.
