NATS and container boundary: carry traceparent across messages and into task containers #256
Notifications
Due Date
No due date set.
Depends on
Reference: sam/archie#256
Reference in new issue
Block a user
Traces stop at every NATS hop and at the container boundary. natsrpc.Request/Respond send bare payloads with no header; eventbus.Message has no Header(). Context is dropped on request paths: forgerpc (7 forge-client calls on context.Background), worktreerpc handlerContext, taskactions Apply, webui api_config, staterpc task_grants; events.Bus.Publish takes no ctx. task.json and taskrun.Request carry no trace field, containers inherit no daemon env, and Kit containers can reach only the egress relay.
Design
No NATS instrumentation exists in nats.go v1.54.0 or otel contrib; hand-roll it.
Carrier (nats.Header is case-sensitive; propagation.HeaderCarrier canonicalises to "Traceparent" and misses lowercase senders; nats.Header has no Keys()):
type natsCarrier nats.Header
func (c natsCarrier) Get(k string) string { return nats.Header(c).Get(k) }
func (c natsCarrier) Set(k, v string) { nats.Header(c).Set(k, v) }
func (c natsCarrier) Keys() []string { return slices.Collect(maps.Keys(c)) }
Set panics on a nil map: build messages with nats.NewMsg.
API switches: natsrpc Call uses nc.RequestMsgWithContext (RequestWithContext sends no headers); Respond uses msg.RespondMsg; logrpc and agenttransport bypasses likewise. JetStream: js.PublishMsg with headers, eventbus.Message gains Header() backed by jetstream.Msg.Headers(). Request/reply: CLIENT/SERVER span pair. JetStream consume: SpanKindConsumer started with a link to the producer span context, not as its child (producer span has usually ended). messaging.* attrs (semconv/v1.43.0) are Development stability.
Request-path ctx drops to fix: forgerpc (7 forge-client calls), worktreerpc handlerContext, taskactions Apply, webui api_config, staterpc task_grants; events.Bus.Publish takes ctx.
Container trace context: otel-go has no TRACEPARENT env carrier. Add
Trace map[string]string json:"trace,omitempty"to the task brief (/.git/task.json) or taskrun.Request:c := propagation.MapCarrier{}; otel.GetTextMapPropagator().Inject(ctx, c); brief.Trace = c // daemon
ctx = otel.GetTextMapPropagator().Extract(ctx, propagation.MapCarrier(brief.Trace)) // archie-agent
Sampling follows the parent flag (ParentBased), so the container keeps the daemon decision.
Container export over NATS (compile-checked against v1.47.0; the ReadOnlySpan->proto converter is internal, but not needed):
Containers get no exporter config, no egress rule, no collector reachability.
Acceptance criteria
One task shows as a single trace from event intake through dispatch, the container's agent loop and the result, including spans produced inside a Kit container, with no new egress rule.
Notes
Part A landed (
74b11b33): NATS header propagation, JetStream consumer links, ctx-drop fixes, taskrun.Request.Trace injected/extracted. Remaining: agent otlptrace NATS exporter, archied relay to otlptracegrpc, agent provider+shutdown (after fzb3x.3). Subscribe handlers get no ctx, so only the consumer span exists there.Close reason
Part A
74b11b33, part B812f1bdd: container spans exported over ARCHIE_SPANS and relayed by archied to OTLPImported from beads
archie-core-fzb3x.4· GitHub #1852 · type feature · priority P2 · created 2026-10-25 · closed 2026-10-25