A bounded retry stops when its attempt limit or caller deadline is exhausted. In TypeScript, exponential backoff spaces attempts apart, while jitter varies the delay so callers are less likely to retry together.
The example below demonstrates the retry mechanism. Before using it, define which failures are retryable and ensure the underlying operation observes cancellation. A timeout does not establish whether a remote write succeeded.
Bounded Retry with Backoff + Jitter
A retry spends the same dependency capacity that may already be failing.
import { setTimeout as sleep } from 'node:timers/promises'
async function retry<T>(operation: (signal: AbortSignal) => Promise<T>, retryable: (error: unknown) => boolean, caller: AbortSignal, deadlineMs = 2_000) {
const deadline = Date.now() + deadlineMs
const signal = AbortSignal.any([caller, AbortSignal.timeout(deadlineMs)])
for (let attempt = 0; ; attempt++) {
try { return await operation(signal) }
catch (error) {
if (signal.aborted) throw signal.reason
const delay = Math.min(50 * 2 ** attempt, 500) * (0.5 + Math.random())
if (!retryable(error) || Date.now() + delay >= deadline || attempt >= 4) throw error
await sleep(delay, undefined, { signal })
}
}
}Invariant: Never retry after the caller’s deadline.
Use when: A transient dependency can recover, but retries must respect a deadline.
Why this boundary matters
Unclassified retries amplify incidents, duplicate ambiguous writes, and consume time after the result can no longer help the caller.
Failure policy
| Boundary | Action |
|---|---|
| Connection timeout | Retry only when the operation is safe and budget remains |
| 429 throttling | Honor Retry-After, capped by the remaining deadline |
| Selected 5xx | Retry from an explicit allowlist with backoff and jitter |
| 4xx validation or authorization | Do not retry |
| Caller cancellation | Stop immediately |
| Ambiguous write | Retry only with a stable idempotency key |
| Deadline exhausted | Fail; do not start another attempt |
Trade-offs
Retries recover transient faults but add load when a dependency may already be unhealthy. Multiple retrying layers can turn one request into a storm. Put retry ownership at one boundary, measure exhausted budgets, and reconcile ambiguous writes.
Decision rule: Retry only when the failure is plausibly transient, the operation is safe to repeat, and enough deadline remains for a useful result.
Further reference
When should a retry stop? #
Stop on cancellation, an exhausted deadline, a permanent validation error, or an attempt limit. Retry a transient failure only when another attempt has time to finish. For ambiguous writes, use an idempotency contract before allowing automatic retries.
What should the retry budget include? #
Count the operation time and the sleep between attempts against the same caller budget. A loop bounded to three attempts can still be too slow if each attempt has its own long timeout. Read the deadline-budget walkthrough, then explore load amplification in the architecture workbench.
Further reading: AWS Builders’ Library on timeouts, retries, backoff, and jitter.
Browse all engineering snippets · Read about deadline budgets