OpenTelemetry baggage is attractive because it carries application-defined key-value context across service boundaries. Put a tenant identifier at ingress and every downstream span can become easier to search.
It is also propagated through request headers, can reach services you do not control, and has no built-in integrity guarantee. Those properties make baggage useful telemetry context—and a dangerous place for trust.
Baggage may describe a request. It must never authorize one.
Separate claims from decisions #
An inbound header such as baggage=tenant.id=acme,plan=enterprise is a claim made by the caller. If a downstream service uses plan=enterprise to unlock a feature, the client has acquired an authorization interface.
Authentication and policy must produce trusted server-side context. You may then emit a safe derivative into telemetry, but consumers must not reconstruct authority from it.
untrusted request headers
|
v
sanitize propagation context -----> tracing only
|
v
authenticate + load policy --------> authorization decisionTreat trace IDs similarly: they correlate work; they do not prove caller identity, ownership, or causality by themselves.
Use an allowlist at ingress #
Do not automatically accept arbitrary baggage from the public internet. Decide which keys may cross each boundary, validate length and syntax, and drop everything else. For an external request, the safe list may be empty.
At trusted internal ingress, create low-sensitivity identifiers specifically for observability. Prefer an opaque account cohort or irreversible keyed hash to an email address, access token, health identifier, or raw customer ID.
OpenTelemetry explicitly warns that baggage can be propagated to unintended third parties. Automatic instrumentation makes boundary review important: a service that calls a payment provider may forward context unless propagation is restricted.
Scrub at egress too #
Ingress filtering protects against forged data. Egress filtering protects against accidental disclosure.
Maintain an outbound policy by destination class:
| Destination | Trace context | Baggage |
|---|---|---|
| Owned internal service | Allowed, filtered | Small allowlist |
| Vendor with tracing agreement | Explicit decision | Usually none |
| Arbitrary customer URL or webhook | Regenerate or omit | None |
Clear baggage before calling untrusted endpoints. Do not rely exclusively on the collector to repair a secret after it has already crossed the network.
Control telemetry cardinality #
Even non-sensitive baggage can be operationally hazardous. Copying request IDs, URLs, or unrestricted customer values onto every span creates high-cardinality attributes and expensive indexes.
Define a schema for propagated observability context:
- owner for every key;
- allowed source and destinations;
- sensitivity class;
- maximum encoded size;
- whether it becomes a span, log, or metric attribute;
- retention period;
- expected cardinality.
Baggage does not automatically become a span attribute. That explicit conversion is a useful control point. Enrich only the signals that answer a known operational question.
Defense in depth at the collector #
OpenTelemetry’s security guidance recommends data minimization and documents attribute, filter, redaction, and transform processors for managing sensitive telemetry. Use an allowlist or deletion policy near collection as a second boundary.
But collector redaction is not sufficient if application logs already copied the value elsewhere, exporters bypass the collector, or baggage went to an external API. Prevent, restrict, and then redact.
Test the propagation graph #
Build automated boundary tests:
- send forged baggage from a public client;
- verify forbidden keys are absent downstream;
- send approved internal context;
- verify it appears only on intended telemetry;
- call a captured third-party endpoint;
- assert no internal baggage header arrived;
- inject oversized and malformed values;
- confirm authorization outcomes never change.
Also inventory auto-instrumentation during upgrades. Propagator defaults and HTTP client coverage can change even when application code does not.
The architectural rule is small: observability context is data moving through the system, so it needs ownership, classification, boundaries, and tests. When baggage is treated as a convenient global variable, it quietly becomes both a leakage path and a shadow control plane.