epic: task observability — capture and surface agent/gate output #955

Closed
opened 2026-10-27 13:38:26 +00:00 by sam · 0 comments
Owner

Design doc: docs/architecture/observability.md (this branch/PR).

Problem

A task's real work happens in an ephemeral archie-agent container over
NATS. Four things currently destroy the evidence of what it did:

  1. cmd/archie-agent/main.go logs to stderr only, and
    internal/container/pool.go creates containers with AutoRemove: true —
    the log dies with the container.
  2. StageBaselineGate (internal/domain/workflow/implement.go) captures
    full gate output via CombinedOutput(), clips it to 200 chars into a
    Warn log, and discards the rest — the park reason carries only a status
    string.
  3. agentexec.Result has no step-level transcript (tool calls, durations,
    errors).
  4. agentexec.SubjectForSystem is documented as the return channel for
    exactly this ("log dumps, health... the daemon reads these for
    observability") but has zero publishers and zero consumers anywhere in
    the codebase.

Consequence: a task parked today with
stage baseline: baseline red -- go build ./... fails and builder could not auto-fix (status: parked) and there was no way, from the dashboard or from
chat, to see the actual go build error — the container that had it was
already gone.

This matters beyond convenience: archied is meant to run unattended for
weeks (CLAUDE.md deployment model) and potentially for users other than
its original operator. A system only debuggable by someone watching
docker ps live doesn't meet that bar.

Decision

NATS is the only transport this targets. [agent] mode = "nats" is the
supported deployment and the agent must already reach NATS to report any
stage result — a container that can't reach NATS is a distinct, narrower
failure (connectivity) than this epic addresses (a connected container's own
output being unrecoverable). No new functionality goes into the in-process
runner, which is being migrated away.

Subtasks

See linked sub-issues.


Imported from beads archie-core-1786637501090-427-c465db57 · GitHub #450 · type epic · priority P2 · created 2026-08-09 · closed 2026-08-09

Design doc: `docs/architecture/observability.md` (this branch/PR). ## Problem A task's real work happens in an ephemeral `archie-agent` container over NATS. Four things currently destroy the evidence of what it did: 1. `cmd/archie-agent/main.go` logs to stderr only, and `internal/container/pool.go` creates containers with `AutoRemove: true` — the log dies with the container. 2. `StageBaselineGate` (`internal/domain/workflow/implement.go`) captures full gate output via `CombinedOutput()`, clips it to 200 chars into a `Warn` log, and discards the rest — the park reason carries only a status string. 3. `agentexec.Result` has no step-level transcript (tool calls, durations, errors). 4. `agentexec.SubjectForSystem` is documented as the return channel for exactly this ("log dumps, health... the daemon reads these for observability") but has zero publishers and zero consumers anywhere in the codebase. Consequence: a task parked today with `stage baseline: baseline red -- go build ./... fails and builder could not auto-fix (status: parked)` and there was no way, from the dashboard or from chat, to see the actual `go build` error — the container that had it was already gone. This matters beyond convenience: archied is meant to run unattended for weeks (`CLAUDE.md` deployment model) and potentially for users other than its original operator. A system only debuggable by someone watching `docker ps` live doesn't meet that bar. ## Decision NATS is the only transport this targets. `[agent] mode = "nats"` is the supported deployment and the agent must already reach NATS to report any stage result — a container that can't reach NATS is a distinct, narrower failure (connectivity) than this epic addresses (a connected container's own output being unrecoverable). No new functionality goes into the in-process runner, which is being migrated away. ## Subtasks See linked sub-issues. --- _Imported from beads `archie-core-1786637501090-427-c465db57` · GitHub #450 · type epic · priority P2 · created 2026-08-09 · closed 2026-08-09_
sam added the mediumepic labels 2026-10-27 13:38:26 +00:00
sam self-assigned this 2026-10-27 13:38:26 +00:00
sam closed this issue 2026-10-27 13:38:26 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#955