Cache Invalidation Is a Consistency Protocol

Aug 26

By

cachingdistributed systemsdatabase engineeringreliability

A cache is a replica with a weaker update protocol. Once that is acknowledged, cache design stops being “pick a TTL” and becomes a consistency decision.

For every cached value, define the maximum acceptable staleness, the authority, the invalidation path, and what happens when that path fails.

Name the guarantee

Different data deserves different contracts:

DataAcceptable behavior
article pagestale for minutes
product pricestale briefly, revalidated at checkout
authorizationshort bounded staleness or fail closed
account balanceprimary read for financial decision

A TTL is not a universal consistency model. It is an upper bound only if clocks, refresh behavior, and fallback paths obey it.

State the contract as an invariant:

A revoked permission is no longer accepted within 30 seconds, and every destructive operation rechecks the authoritative policy version.

That statement can be tested. “We use Redis” cannot.

Follow the cache-aside race

The common read path is:

GET cache → miss → SELECT database → SET cache

The common write path updates the database and deletes the cache entry. Redis documentation describes invalidation after the primary write as the simple cache-aside approach. But concurrency still matters:

reader: cache miss
reader: reads database value v1
writer: commits v2
writer: deletes cache
reader: writes stale v1 into cache

The stale value survives until expiry.

Mitigations include:

  • short TTL where bounded staleness is acceptable;
  • versioned values and compare-before-set;
  • delete after write, then a delayed second delete for known race windows;
  • change-stream invalidation;
  • write-through under one owner;
  • bypassing cache for correctness-critical reads.

No technique removes every failure mode. Choose against the actual invariant.

Put versions in the value

Store source version with cached data:

{
  "entity_version": 481,
  "cached_at": "2026-08-26T08:20:00Z",
  "value": {"plan":"enterprise"}
}

An invalidation or refresh carrying version 480 cannot overwrite 481. Versioning also makes stale observations diagnosable.

For immutable or expensive derived objects, put the version in the key:

report:customer-42:revision-481

Publish the active revision separately. Old values expire without in-place mutation, reducing races during recomputation.

Treat invalidations as at-least-once messages

Invalidations can be delayed, duplicated, reordered, or lost during disconnects. Handlers should be idempotent; deleting an already absent key is success.

Redis client-side caching tracks keys read by clients and sends invalidation messages when those keys change. It can efficiently reduce database reads, but applications must evict entries on notification and handle reconnects. After a gap, assume local state may be stale and flush or revalidate it.

For durable correctness requirements, ephemeral pub/sub is not sufficient evidence. Use a durable change log or version check so a disconnected consumer can catch up.

Measure:

  • source commit to invalidation latency;
  • invalidation consumer lag;
  • reconnect flushes;
  • cached versus authoritative version gaps;
  • stale reads detected at the write boundary.

Prevent stampedes

When a popular key expires, thousands of requests may miss together and overload the source.

Use request coalescing:

first miss → becomes loader
other misses → wait on same in-flight result
loader writes cache → wake waiters

Add TTL jitter so many keys do not expire simultaneously. Consider stale-while-revalidate for non-critical reads: serve a recently expired value while one worker refreshes it.

Bound the loader. A distributed lock with no expiry can make a cache miss permanent; a lock with no fencing can let an expired owner overwrite a newer result. Version checks remain necessary.

Negative caching needs shorter rules

Caching “not found” protects the database from repeated misses and enumeration. It can also hide a newly created resource.

Use a distinct short TTL, include tenant and authorization scope in the key, and invalidate on creation. Never reuse an unauthenticated negative result for an authenticated subject if visibility differs.

Cache keys are part of the security model:

bad:  profile:user-7
good: profile:tenant-42:user-7:policy-19

The correct key encodes every dimension that can change the response.

Design degradation

A cache outage should not automatically redirect its full peak load to the database. That converts one failure into two.

Apply admission control, per-key coalescing, bounded concurrency, and selective degradation. Serve stale public content where safe. Reject expensive optional views. Preserve capacity for checkout or writes.

Test cache loss at realistic peak traffic. A successful steady-state benchmark says little about miss-storm behavior.

CTO review

  1. What staleness is allowed for each data class?
  2. Which source is authoritative at the decision boundary?
  3. Can a slow reader repopulate an older value after a write?
  4. Are invalidations replayable after disconnect?
  5. Does every key include tenancy, authorization, and representation dimensions?
  6. How are stampedes coalesced and bounded?
  7. What happens to the database when the cache disappears?
  8. Which metric proves the cache meets its consistency contract?

A cache earns its latency improvement by accepting a consistency cost. Production design makes that cost explicit, bounded, and observable.

References

>