Kubernetes Requests Are an Economic Model

Aug 28

By

kubernetescloud infrastructurereliabilitycapacity planningtechnical leadership

Copied resource YAML creates two kinds of waste: idle nodes paid for by inflated requests, and outages caused by limits that were never tested against real workload behaviour.

Kubernetes requests and limits are not tuning details. Requests express the capacity the scheduler must reserve. Limits constrain consumption. Together with replicas and autoscaling, they encode the service’s economic and failure model.

Requests buy placement

The scheduler uses requests to decide whether a Pod fits on a node. A request is therefore a claim on cluster capacity even when the process is idle.

resources:
  requests:
    cpu: 250m
    memory: 512Mi
  limits:
    memory: 768Mi

If 100 replicas each request 1 CPU but normally use 100m, the organisation reserves roughly ten times observed steady-state CPU before headroom. Lowering requests blindly can improve packing until synchronized traffic arrives and every Pod competes for CPU at once.

Set requests from representative distributions, not averages. CPU can often begin near a sustained high percentile plus startup and burst evidence. Memory should include working-set peaks, runtime behaviour, and a safety margin because memory exhaustion is not gracefully throttled.

CPU and memory fail differently

CPU is compressible. When demand exceeds available CPU or a configured limit, work slows and latency rises. A restrictive CPU limit can produce throttling even when a node has spare capacity, turning a protection mechanism into a latency incident.

Memory is incompressible. Exceeding the memory limit can terminate the container. Without a limit, a leak can pressure the node and trigger eviction elsewhere.

That leads to a useful default for many services: specify CPU requests based on scheduling needs, be cautious with CPU limits, and set measured memory requests and limits with an explicit OOM recovery model. This is not universal—multi-tenant or untrusted compute may require hard CPU ceilings—but the decision should be intentional.

QoS is incident ordering

Kubernetes assigns Pods to Guaranteed, Burstable, or BestEffort classes based on requests and limits. Under node pressure, eviction preference considers those classes. QoS is therefore a statement about which workloads the platform sacrifices first.

  • BestEffort: no CPU or memory requests/limits; suitable only for genuinely disposable work.
  • Burstable: some resources specified or requests differ from limits; common for elastic services.
  • Guaranteed: CPU and memory requests equal limits for every container; strongest reservation and least flexible packing.

Do not make everything Guaranteed to feel safe. It can reserve large amounts of idle capacity and reduce the room the scheduler has to respond. Match class to business criticality and workload shape.

Autoscaling needs compatible signals

Horizontal Pod Autoscaler CPU utilisation is measured relative to requested CPU. Change the request and the same workload produces a different utilisation percentage. A team can “fix” autoscaling by changing requests while silently changing cluster reservation.

For queue consumers, queue age or drain time is often a better scaling signal than CPU. For APIs, combine concurrency, latency, and saturation evidence. Scaling on a lagging signal after capacity is exhausted creates replica storms that compete for the same database connections.

Model the whole path:

new replicas
  -> startup CPU and image pulls
  -> readiness delay
  -> connection pool growth
  -> downstream request growth
  -> actual useful capacity

Autoscaling cannot manufacture downstream capacity.

Establish a resource policy

A platform policy should require:

  1. requests for every production container, including sidecars;
  2. measured memory limits and an OOM alert;
  3. startup, readiness, and liveness probes with different purposes;
  4. Pod disruption budgets for quorum and availability needs;
  5. a maximum replica count derived from dependency capacity;
  6. periodic right-sizing using production percentiles;
  7. documented exceptions for CPU limits and Guaranteed QoS.

Use admission policy to reject missing requests, not to impose one universal value. Defaults are useful for development and dangerous as permanent production assumptions.

Review cost and reliability together

For each service, report:

  • requested versus used CPU and memory;
  • throttling and OOM events;
  • pending time and scheduling failures;
  • replicas versus useful throughput;
  • node headroom during peak and failure;
  • downstream saturation during scale-out;
  • cost per successful business operation.

High reservation with low use suggests packing waste. High usage near requests with latency growth suggests capacity risk. OOMs suggest either an incorrect limit or an application memory problem; raising the limit without a heap profile is not diagnosis.

The executive question

Ask why a service requests the capacity it does. “The template said so” means the cluster has no defensible capacity model.

A production resource policy should let engineering explain how much demand a replica serves, how the fleet behaves during a node loss, what dependency becomes limiting first, and how much safety margin the company is buying. At that point, YAML becomes an operating contract rather than decoration.

References

>