LLM calls: trace model and tool calls in gateway and archie-agent #254
Notifications
Due Date
No due date set.
Depends on
Reference: sam/archie#254
Reference in new issue
Block a user
archie-core uses neither ai-sdk's middleware nor its telemetry package, and cannot: ai-sdk runtime builds providers internally with no transport or middleware option. LLM calls run in archie-gateway (chat turns, curators, MCP sampling) and archie-agent (modelloop LoopRunner). The real fix is in ai-sdk (ai-sdk-161: core emits OTel spans with gen_ai attributes, runtime accepts a transport).
Design
After ai-sdk-161 ships: bump ai-sdk; LLM calls already receive the request/task ctx (gateway chat_setup Chat/ChatStream, modelloop agentloop.Run), so spans parent automatically once the provider from fzb3x.3 is installed. Pass a transport via the new runtime.Config option: otelhttp.NewTransport(base) outermost, where base in archie-agent is the egress.ModelTransport that agentworker currently installs as http.DefaultTransport (replace that global swap with the option). Message content capture stays off.
Interim if ai-sdk work slips: otelhttp.NewTransport around http.DefaultTransport in gateway and agent startup gives bare HTTP client spans with no model/token data.
Acceptance criteria
A chat turn and a workflow stage each produce model-call spans with model, token usage and finish reason, and tool-call child spans, parented to the gateway request or task span.
Close reason
b1ef0be3: ai-sdk v0.5.0, otelhttp model transport in gateway and agent; gen_ai spans parent to caller span (verified with a throwaway SDK test)Imported from beads
archie-core-fzb3x.2· GitHub #1853 · type feature · priority P2 · created 2026-10-25 · closed 2026-10-25