Kubernetes Cost and Reliability Rules: A Practical Guide

A Kubernetes cost rule and a Kubernetes reliability rule frequently point at the same manifest field. Resource requests set both how much node capacity a workload reserves and how the kubelet treats it under pressure. Replica counts set both spend and fault tolerance. Autoscaler and disruption settings decide both whether capacity shrinks at night and whether a node can be drained safely. Treating these as two separate backlogs leads to fixes that undo each other.

Four places cost and reliability meet

Requests. The scheduler fits pods onto nodes by summing requests, and node consolidation also reasons about requests rather than real usage. Over-requesting therefore holds nodes you pay for. Under-requesting, or omitting requests, has a reliability cost: under node resource pressure the kubelet evicts pods whose usage exceeds their requests first, ranked by priority and by how far they exceed the request. A pod whose containers set no CPU or memory requests or limits is BestEffort, so any usage at all exceeds its request.

Autoscaling signals. The Horizontal Pod Autoscaler computes utilization as a percentage of each container’s resource request. If some containers in a pod lack the relevant request, CPU utilization for that pod is not defined and the autoscaler does not act on that metric. A missing request is therefore not only a packing problem; it can silently disable scaling.

Replicas. One replica is the cheapest configuration and the least available one. Any voluntary disruption, such as a node drain for an upgrade or a scale-down, takes the service to zero.

Disruption budgets. A PodDisruptionBudget limits how many pods of a replicated application are down at once during voluntary disruptions, and tools that drain nodes are expected to respect it through the Eviction API. A budget that allows no disruption at all protects the workload by blocking every drain, which also stops the node autoscaler from removing the node it runs on.

Rules that encode these trade-offs

ZopNight’s Kubernetes workload rules run per provider on EKS, GKE, and AKS and cover reliability, security, idle, orphan, and rightsizing areas. A few show how cost and reliability are handled together:

• Missing CPU or memory requests and missing limits name every offending container, not just the first.
• Single replica deployment prefers an environment tag over name heuristics to decide whether the workload is production. Its fix, scaling from one to two replicas through a manifest edit, is guided-only: ZopNight never auto-scales a workload the customer chose to run as a singleton.
• HPA at max capacity flags autoscalers pinned at their ceiling, a sign the workload is capacity-constrained.
• CPU over-provisioned fires when an HPA reports it wants fewer replicas than its minimum allows while targeting CPU. For StatefulSets backed by persistent volumes the recommendation carries a caution to reduce gradually.
• Memory over-provisioned currently abstains. An HPA replica signal carries no per-pod memory evidence, and a memory request set too low leads to OOM kills, so the rule emits nothing until a memory-usage signal is available.
• PDB rules flag budgets that block all voluntary disruptions, set maxUnavailable to zero, or match no pods.

An order of work

1. Add requests where they are missing, sized from observed usage. This unblocks both autoscaling and packing.
2. Fix disruption budgets that block every eviction, so drains and consolidation can proceed.
3. Give production services a second replica and a PDB that allows one pod down at a time.
4. Only then reduce over-sized requests, starting with stateless workloads.

Related reading: pod requests and limits, the Horizontal Pod Autoscaler, Kubernetes workload rules, and the PodDisruptionBudget resource page.