# ResourceQuota Exhausted

> Flags ResourceQuotas where any resource's used amount has reached its hard limit, so new pods or objects in the namespace are refused.

Source: https://zop.dev/integrations/kubernetes/recommendations/resourcequota-exhausted

---

## What happens once a namespace hits its quota

A ResourceQuota caps what a namespace may consume: CPU and memory requests and limits, storage,
or counts of objects such as pods, Services and LoadBalancers. The
[resource quotas documentation](https://kubernetes.io/docs/concepts/policy/resource-quotas/) says
that when creating or updating a resource would violate a quota, the control plane rejects the
request with HTTP 403 Forbidden and a message naming the constraint.

The failure is easy to misread. A Deployment that asks for more than the quota allows is still
created; the same page explains that the Deployment may simply fail to get all of its pods, and
you have to inspect its status to see why. An exhausted quota therefore shows up as replicas that
never appear, a rollout that hangs or an HPA that cannot add pods, not as a clear error in the CI
log. Existing workloads keep running, because quota changes and contention do not affect objects
already created.

## Reading used against hard

```bash
kubectl describe resourcequota -A
```

Each quota prints a `Used` and `Hard` column per resource. For a quick filter on count-style
entries, which are plain integers:

```bash
kubectl get resourcequota -A -o json | jq -r '
  .items[] | .metadata as $m | .status as $s
  | ($s.hard // {}) | to_entries[]
  | select(($s.used[.key] // "") | test("^[0-9]+$"))
  | select(.value | test("^[0-9]+$"))
  | select((.value | tonumber) > 0 and ($s.used[.key] | tonumber) >= (.value | tonumber))
  | "\($m.namespace)/\($m.name) \(.key) \($s.used[.key])/\(.value)"'
```

CPU and memory entries use suffixes such as `m` and `Gi`, so read those from the `describe`
output.

## How ZopNight scores the quota

For every resource that appears in both `hard` and `used`, ZopNight converts the two values from
Kubernetes quantity notation and computes used divided by hard. The quota fires when the worst
ratio is 1.0 or more. One exhausted resource is enough, even if everything else in the quota is
nearly empty. There is no lookback window; the ratio is read fresh each evaluation.

## Entries it ignores

Resources listed in `hard` with no matching `used` value, values it cannot parse, and entries
whose hard limit is zero are skipped. That last point matters: a quota such as
`services.loadbalancers: "0"`, used to forbid an object type entirely, is not reported as
exhausted. Quotas between 85% and 100% are reported by
[ResourceQuota near limit](https://zop.dev/integrations/kubernetes/recommendations/resourcequota-near-limit)
instead.

## Blocked deploys rather than dollars

This high-severity governance finding has no savings estimate. The cost is delivery: releases and
autoscaling in the namespace stop until someone frees capacity or raises the limit.

## Freeing or raising the quota

1. Run `kubectl describe resourcequota -n <namespace>` to find the resource at its limit.
2. List what consumes it, for example `kubectl get pods -n <namespace>` for a pods count, or the
   largest requests for `requests.cpu`.
3. Remove what is finished: failed Jobs, idle Deployments, unused claims.
4. If the demand is real, raise the value under `spec.hard` with `kubectl edit resourcequota`,
   keeping cluster capacity in mind, since quotas are independent of how much the nodes can supply.