A PodDisruptionBudget Is a Maintenance Contract

Sep 5

A PodDisruptionBudget (PDB) does not promise that your service stays available. It limits how many selected pods an eviction-aware actor may voluntarily disrupt at once. Hardware failure, resource pressure, direct deletion, and workload rollouts do not become harmless because a PDB exists.

That narrower contract is valuable—provided the team designs the rest of the system around it.

A PDB protects maintenance coordination, not the application from every cause of unavailability.

Start from serving capacity

Suppose a service has six replicas. Four are required to sustain peak critical traffic with acceptable tail latency. One replica may already be unavailable during ordinary operation. The maintenance budget is therefore one additional eviction, not “25% because that seems safe.”

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: checkout-api
spec:
  maxUnavailable: 1
  unhealthyPodEvictionPolicy: AlwaysAllow
  selector:
    matchLabels:
      app: checkout-api

minAvailable and maxUnavailable are mutually exclusive. Use a number derived from workload capacity, quorum, or recovery cost. Percentages round and may behave unexpectedly at low replica counts, so calculate concrete cases during review.

Understand which actions participate

Kubernetes documents PDBs as constraints on voluntary disruptions performed through the Eviction API. kubectl drain uses that API and retries rejected evictions. Directly deleting a pod or deployment can bypass the budget. Application rollouts are controlled by the workload’s own update strategy rather than by the PDB.

This creates an ownership boundary:

MechanismGoverning control
Node drain or autoscaler evictionPDB plus Eviction API
Deployment rolloutmaxUnavailable / maxSurge
Node lossReplication and topology
OOM or disk pressureRequests, limits, node capacity
Direct pod deletionAccess controls and operating procedure

A platform team cannot infer application safety from a PDB alone. An application team cannot assume every platform action respects one.

Avoid the impossible budget

minAvailable: 100% can block every voluntary eviction. That may be appropriate for a deliberately manual workload, but on an ordinary service it can prevent node maintenance indefinitely. A broken pod can also block a drain when policy requires it to be healthy before eviction.

The AlwaysAllow unhealthy-pod eviction policy helps drains make progress when a pod is already unhealthy. It does not repair insufficient replicas or bad placement. Document the trade-off: maintenance progress versus preserving a possibly recoverable unhealthy instance.

Capacity without topology is fragile

Six replicas placed on two nodes do not provide six independent failure units. Combine the budget with topology spread constraints or anti-affinity, adequate node headroom, and readiness that reflects the ability to serve.

Readiness is part of the disruption calculation. If a replacement pod becomes ready before caches are warm or dependencies are established, the controller may permit the next eviction too soon. Test readiness against actual critical-path behavior. The probe design guide covers this failure policy in detail.

Coordinate with rollouts and autoscaling

The deployment strategy and PDB should express compatible assumptions. If a rollout permits two unavailable replicas while peak capacity tolerates only one, the PDB will not save the rollout. If the HPA scales down near a drain window, a previously safe absolute replica count can disappear.

For critical workloads, define a maintenance floor separate from the demand-driven scaling floor. Check disruption allowance before planned changes, but do not use the current allowance as the only gate: status can trail reality.

Run the drain game day

In a staging environment that resembles production placement:

  1. generate representative traffic;
  2. drain one node with a realistic timeout;
  3. observe rejected and allowed evictions;
  4. confirm replacements become genuinely ready;
  5. measure p99 latency and error budget consumption;
  6. introduce one unhealthy replica and repeat;
  7. simulate a simultaneous involuntary node loss.

The final step matters because real incidents overlap with planned work. A PDB that supports a clean drain but leaves no tolerance for one node failure encodes an optimistic world.

Track blocked drains, disruptionsAllowed, unavailable replicas, scheduling latency, and service SLOs together. The operational outcome is the product of all five.

A good PDB makes a precise promise to operators: under stated capacity and health assumptions, this many coordinated evictions may proceed. Everything beyond that promise belongs to topology, workload policy, and application resilience.

References

>