A remotely-shipped task log entry was re-emitted through the sink with
slog.Logger.LogAttrs, which stamps time.Now(): the entry's own event time
was discarded and replaced with the daemon's receipt time. Because a
container's records arrive in bursts, a multi-minute run collapsed onto a
few milliseconds of receipt timestamps and its timing became
unrecoverable -- which stage ran when, how long an iteration took, whether
a stage stalled.
Write now builds the record from entry.Time (falling back to receipt time
only for an entry that carries none) and dispatches it through the same
handler, preserving the sink's level gate. Receipt time stays available
separately in the record's received_at field, so event time and arrival
time cannot be confused. The agent-side producer already carried the
record's own time; a test now pins that half too.
The events table had no attempt column, so two attempts of the same task were
indistinguishable: a retried task's stage events interleaved into one timeline and
nothing could say which run produced a given result.
- events gains an `attempt` column through the existing presence-gated
migrateTasks loop rather than a version bump, so user_version stays 3 and an
existing archie.db is migrated in place. A legacy-database test builds the
pre-change schema and proves both that existing rows survive unattributed
(attempt = 0) and that a new value round-trips.
- Event.attempt = 11 and ReadTaskLogRequest.stage = 9 are field additions to
existing messages: no new RPC, no new table, and no change to any canonical error
message that crosses the state-store wire.
- Every task event the workflow engine emits now carries the attempt. Producers
that have no attempt yet (task_queued, binding_dispatch_failure) stay
unattributed rather than guessing one.
- Changed files are captured as a durable changes_captured event at commit and push
time, reusing the existing three-dot diff primitive: paths from FilePatches(),
counts from Stats(), base_sha recorded as the merge base actually diffed rather
than the base tip, a 200-file cap with totals kept complete. Consumption goes
through an unexported optional interface so workflow.Trees -- the yaegi capability
surface for interpreted stage plugins -- is not widened. A capture is also taken
once the pull request exists, so the capture can link it.
- The per-attempt configuration is captured as a durable config_captured event
carrying the non-secret config.TaskConfig, instead of a new table plus two new
state-store RPCs: events are already this system's per-attempt provenance stream,
so a table would have given one feature two provenance mechanisms. Redaction is
proven against canary secrets. Measured payload: 539 bytes typical, 2333 bytes at
20 models.
- Captures carry a schema marker and are refused unless their key set matches the
declared payload shape, so drift surfaces as "recorded but could not be read"
rather than as a capture of zeroes.
- Three read endpoints serve the attempt rail, the changed-file captures and a raw
debug view, mapping to UI-owned response types rather than dumping store types.
- The task-log read gains a stage filter, bound to the stage logger for that
stage's own lines; agent and tool output records no stage.
A task-scoped grant may already insert events on its own task, so this provenance
is exactly as trustworthy as the rest of that task's event stream. That limit is
documented rather than papered over.
Write built its own context.Background() for the handler call, which meant
any new caller with a context to offer could not hand it over. Taking the
caller's context is what the next commit's park-reason writer needs, and it
is what contextcheck requires of a daemon-side writer.
No behaviour change: the context is used for the handler's own attribute
extraction and never consulted for cancellation -- a park is recorded at
exactly the moment a run's context has been cancelled, so a cancelled
context must not suppress the entry.
The dashboard's attempt-log panel answered every read with "task logging is
optional and was not enabled for this run". That message was false: logging is
wired unconditionally in the daemon, and the log files exist. The dashboard
process simply had no reader -- composition left TaskLogs absent because no
contract carried it -- so the handler saw a nil registry and reported the
disabled shape, which the page rendered as a claim about configuration.
internal/logging now answers whether an attempt has a log at all: Tail sets
Result.Found, so "this attempt has no log" is a value rather than an empty
list indistinguishable from a read that never happened. Its registry gains
TaskLog (a decoded page) and TaskLogContent (the file verbatim, streamed,
because a log is unbounded input), and an absent reader reports
ErrTaskLogsUnavailable rather than an empty page -- a process that cannot read
logs must not implicate the attempt.
Those two answers travel the State Store contract as storecontract.TaskLogStore
(ReadTaskLog, StreamTaskLogContent) and are rehydrated on the client by
(Unavailable, canonical message), so archie-ui composes the reader from its own
store client instead of opening another process's files. The daemon's registry
is wired as the server-side reader in the State Store process only.
On top of that read, GET /api/tasks/{id}/logs/download serves the attempt's log
as a Content-Disposition attachment named task-<id>-attempt-<n>.log, and the
panel offers it. The JSON route keeps its shape and gains found, so the page
states the attempt has no log instead of blaming a setting.
Task log files stay owned by their directory's owner: archiving still removes a
task's log directory, and that removal is the daemon's, not the dashboard's.
rotate() closed the current file before attempting the rename+reopen, so
when the reopen failed (a full disk, a permission change) every write
afterward went to a closed *os.File and was silently dropped -- the sink
stayed permanently broken rather than recovering once the transient
condition cleared. Keep the old descriptor open across the rename/reopen
and only close it once the new one is confirmed, so a write during (or
after) a failed rotation always lands somewhere.
FeedHandler.WithAttrs/WithGroup only tracked bound attrs for the live
feed copy; Handle forwarded the bare record to the wrapped handler, so
component (and any other With-bound attr) never reached stderr/file
JSON sinks or survived restart for historical log filtering. Now
WithAttrs/WithGroup also derive the wrapped handler via its own
With*, matching agentexec.SystemLogHandler's existing pattern.
CodeQL's Go analyzer (go/uncontrolled-allocation-size, alert #6) can't
trace min(limit, MaxTailLines) as a bound on the request-derived limit
before make([]Entry, 0, limit). The clamp was already correct and has
been in place since 5e24072 -- confirmed by rescanning at current HEAD,
the alert still fires -- so this is an analyzer gap, not a live bug.
Same clamp, written as an explicit if so static analysis (and readers)
can verify the bound directly.
task_logs could only read the newest N entries with no way to walk the
rest, so a task's log beyond the page cap was unreachable and the agent
had no honest way to say what was missing. Add logging.Page -- a
byte-offset cursor over the tail-most 8 MiB scan window -- plus
Query.Since/Until time bounds, and wire the gateway's task_logs tool to
expose level/since/until/after_id with cursor and more_available in the
result.
The reader now reads the scan window into a single in-memory buffer and
walks it by exact slice offsets. Offsets into a contiguous buffer are
trivially correct, which an incremental reader over the file is not:
bufio.Reader buffers ahead, so its underlying file position runs one
buffer past its logical position and cannot be used as a cursor. The
same rewrite removes f.Read in favour of f.ReadAt (full-or-error), since
a silent short read would drop the newest entries and still report
success.
The scan window is a documented hard ceiling: for a log over 8 MiB the
oldest bytes are unreachable by any cursor -- that is what Truncated
reports, and send_file remains the path for a complete archive. The
tool description teaches the model the difference between more_available
(more this call did not return) and truncated (older entries the scan
cap hides).
Closes archie-core-vq92 (#625).
SystemLogHandler and FeedHandler both json.Marshal Entry.Fields
directly rather than through slog's own JSON handler, so a logged
`error` value went straight into a map[string]any as the raw
interface. encoding/json has no Error()-aware special case, and the
concrete type behind almost every Go error (errors.errorString,
fmt.wrapError, ...) holds only unexported fields, so it marshalled to
`{}` and silently discarded the message everywhere it was logged --
including on every container/system-log line shipped to the daemon
and shown in the webui, which is where this was actually noticed: a
flood of `"err":{}}` with no way to see what had actually failed.
attrValue now resolves an error attr to its .Error() string before it
reaches the map. It also guards against the classic Go footgun where
a non-nil error interface wraps a nil concrete pointer (var e *T; var
err error = e) -- a naive err != nil check would call Error() on a
nil receiver and crash the one thing that must never crash: logging
itself.
Adds two read surfaces for per-task persisted log history so an operator
can answer "why did task N park?" without needing to watch docker ps in
real time — the deployment model documented in CLAUDE.md.
/api/tasks/{id}/logs — built on logging.Tail the same way /api/logs
already is, defaults attempt to the tasks current Attempt from the store.
task_logs chat tool — lets Archie read task logs from Telegram or webui
chat. Identity enforcement matches chatTaskListerAdapter: a model bound
to one identity cannot read another identitys logs, and forge tasks
(empty identity) are denied.
internal/logging: add TaskRegistry.Path() for read-side access without
threading baseDir separately through every caller.
Fixes from adversarial review: strict identity check (no empty-identity
bypass), non-null Entries when disabled, component doc accuracy, and
adapter tests covering the security boundary.
Ref: #455
Closes the loop #451-#453 opened. archie-agent (#453) now publishes its
own logs to agentexec.SubjectForSystem(taskID) over NATS, and a task
log sink (#452) exists to persist them, but nothing on the daemon side
subscribed, demuxed, or wrote anything -- the messages had nowhere to
land.
- agentexec: SubjectSystemWildcard + TaskIDFromSystemSubject, the
inverse of SubjectForSystem, for a daemon subscribing to every
task's system subject with one wildcard subscription.
- logging: TaskRegistry manages one open TaskSink per running task
attempt, mirroring every write onto the live dashboard Feed the same
way a locally-written log would appear. Nil-receiver-safe, matching
every other optional composition-wired Daemon field.
- daemon: Daemon.process opens a task's log sink for its own duration
(covering both the sandboxed-container and in-process dispatch
paths uniformly) and closes it on every return path.
- cmd/archied: constructs the registry, subscribes core NATS to
SubjectSystemWildcard, and wires TaskLogs into both Daemon and
webui.Server.
- webui: archiving a task now removes its log directory too, so
archived tasks' logs don't accumulate forever. Documented as a
known, accepted race: a task can reach terminal status (satisfying
ArchiveTask's precondition) before process() itself returns and
closes the sink, so a very late system-log write can race an
archive's Remove. Not corrupting -- the write lands on a
soon-to-be-unlinked inode and is discarded -- and not worth fixing
here: it would mean tying Close to the store transition instead of
to process()'s return, a larger change than this issue's scope.
Adversarial review (fresh subagent) found and fixed two real issues in
the first draft: a doc comment accidentally spliced between two
functions during an extraction meant to fix a lint regression, and a
genuine test gap on openTaskLog (added, using TaskRegistry.Write's own
true/false contract as the observable -- no Store/Forge/Trees fakes
needed, since openTaskLog only touches TaskLogs and Log).
Part of #453 (task-observability epic #450). archie-agent logged to
os.Stderr only, and its container is created with AutoRemove: true
(internal/container/pool.go), so all of it disappeared the moment a
task's container exited -- there was no way to see why, say, a gate
command failed inside one.
Adds agentexec.SystemLogHandler: a slog.Handler that tees to whatever
it wraps (kept for an operator attached to a live container) and
additionally publishes each record, JSON-encoded as a logging.Entry,
to SubjectForSystem(taskID) -- a subject that already existed,
documented for exactly this ("log dumps, health... observability"),
with zero publishers anywhere in the codebase.
cmd/archie-agent/main.go wires it in only when this container was
spawned for one specific task (bootTaskID / .git/task.json present);
shared queue-group mode serves many unrelated tasks per process, so
there's no single task subject to ship to and keeps the stderr-only
logger.
Building this surfaced a real pre-existing bug: FeedHandler's attr
flattening applied the group chain active at Handle-time to *all*
accumulated attrs, including ones added via WithAttrs before a later
WithGroup call -- nesting them under a group they were never
actually inside, contrary to slog's own documented Handler contract.
It went unnoticed because nothing in the codebase called WithGroup
until this change needed to. Extracting the shared flattening logic
into exported FlattenAttrs/PrefixAttrs (so SystemLogHandler doesn't
duplicate it) is what surfaced it; fixed in the same change since I
was the one making it public API.
Docs on the wire format's own trap for the daemon-side consumer
(#454, not yet written): the published bytes are a marshaled
logging.Entry, meant for json.Unmarshal, not logging.decode() --
decode() parses raw slog JSON *log lines* off disk, a different and
incompatible shape. See the doc comment on SystemLogHandler.
Part of #452 (task-observability epic #450). A task's real work
happens in an ephemeral archie-agent container; currently nothing
persists what it logs once the container exits (AutoRemove: true).
Adds TaskSink: a per-task, per-attempt JSONL log destination sharing
the exact on-disk format the daemon-wide logger already writes (see
Entry/Tail/Query/Components in reader.go), so a task's history is
readable with the same reader rather than a second format to keep in
sync. Backed by the existing rotatingFile, so it gets the same
size-based rotation and retention as the daemon-wide log for free.
Deliberately does not fall back to stderr on open failure the way the
daemon-wide New does: that fallback exists because losing the daemon
over a broken log path is worse than losing the durable copy, and
there is no equivalent daemon-level fallback for one task's log.
This lands only the sink primitive and its own tests. Wiring an
agent-side publisher and a daemon-side NATS consumer into it is
tracked separately in #453 and #454.
Add internal/logging.Feed, a logging-owned bounded replay buffer separate
from the persisted task-lifecycle event stream, with a live SSE endpoint.
Expose the executable workflow registry (skillbuild.Catalog) including
zero-run definitions and their origin (builtin vs. installed skill), not
just historical run statistics. Route() now returns an explicit failure
workflow for an unknown pre-assigned workflow name instead of silently
falling through to label-based routing.
Fixes#54
Under systemd, journald held the only copy of archied's output, and
running it by hand left no record at all -- so "where are the logs?" had
no good answer, and the dashboard had no history to show from before it
connected. [log] adds a file destination alongside stderr, with
size-based rotation and retention done in-package rather than pulling in
a dependency for a few hundred lines.
A file that cannot be opened is reported and the daemon continues on
stderr: losing the durable copy is not worth refusing to start over.
reader.go owns the on-disk format so nothing else has to parse it, and
bounds every read -- tailing keeps the newest entries within a fixed
scan window, so pointing it at a large file cannot exhaust memory.