Adding Kafka Partitions Changes the Ordering Contract

“Add partitions” sounds like a capacity change. For a keyed event stream, it can be a semantics change.

Kafka orders records within a partition, not across a topic. Producers commonly map a key to a partition. If the partition count changes, that mapping may change. Events for the same customer can land in the old and new partitions during the transition, and consumers can observe them in an order the business never anticipated.

Write the real invariant #

“Messages are ordered” is not precise enough. Useful invariants look like:

For one account, balance mutations are applied in source sequence.
For one device, configuration version n+1 never applies before n.
For one order, cancelled never transitions back to paid.

Now identify the mechanism enforcing each invariant. If the answer is “Kafka ordering,” record the key, partitioner, producer behavior, and consumer-side checks. A topic-wide statement is almost certainly wrong.

Why partition growth is not transparent #

Suppose a default key hash behaves conceptually as:

partition = hash(key) % partition_count

Changing the count from 12 to 24 does not merely create empty lanes. Many keys receive a different result. Old records remain in their original partitions while new records go elsewhere.

More partitions also change operational limits:

• maximum useful consumer concurrency rises;
• per-partition traffic can become more skewed;
• rebalances coordinate a larger assignment set;
• broker metadata, files, and replication work increase;
• downstream systems may receive more parallel requests.

Capacity moved; the failure and ordering surfaces moved too.

Safer migration patterns #

Create a new versioned topic #

Publish to account-events-v2 with the desired partition count. Dual-write or replay through a controlled cutover, validate consumer positions, then retire the old topic. This costs migration work but gives the contract a visible boundary.

Route through stable virtual shards #

Map entity keys to a fixed number of logical shards, then map shards to physical partitions. The extra indirection lets operators move shard ownership deliberately. It is useful when key affinity is valuable enough to justify a routing layer.

Make consumers reject stale transitions #

Carry an aggregate sequence, source version, or monotonic update number. A consumer can then detect gaps and reject regressions even if transport ordering changes.

UPDATE account_projection
SET balance = :balance, source_version = :version
WHERE account_id = :id
  AND source_version = :version - 1;

A zero-row update is not “just retry.” It signals a duplicate, a gap, or concurrent processing that needs reconciliation.

Decision table #

Common mistakes #

• Increasing partitions in production without replaying an entity-order test.
• Treating a producer acknowledgement as proof that a projection applied the event.
• Choosing high-cardinality keys without measuring hot-key skew.
• Scaling consumers beyond the downstream database’s concurrency budget.
• Assuming an idempotency key repairs an out-of-order state transition.

Trade-offs #

A versioned-topic migration consumes temporary storage and doubles operational paths. Stable virtual shards add routing complexity. Consumer-side versions require domain support and gap recovery. Each is more work than changing one integer, because each preserves an invariant that the integer can break.

Partitions are simultaneously a throughput unit, an ordering scope, and a parallelism boundary. Change them only after deciding which of those contracts the product is allowed to change.

Further reading #

• Apache Kafka concepts and guarantees
• Kafka producer partitioner configuration
• Why a rebalance is a coordination protocol