cmd/archie-agent used to own worker construction, NATS lifecycle, task execution, MCP setup, and workflow wiring. It now owns process input only and invokes internal/app/agentworker, while transport-specific NATS, boot-brief, and git-safety mechanics live under infrastructure.
Closes the loop #451-#453 opened. archie-agent (#453) now publishes its
own logs to agentexec.SubjectForSystem(taskID) over NATS, and a task
log sink (#452) exists to persist them, but nothing on the daemon side
subscribed, demuxed, or wrote anything -- the messages had nowhere to
land.
- agentexec: SubjectSystemWildcard + TaskIDFromSystemSubject, the
inverse of SubjectForSystem, for a daemon subscribing to every
task's system subject with one wildcard subscription.
- logging: TaskRegistry manages one open TaskSink per running task
attempt, mirroring every write onto the live dashboard Feed the same
way a locally-written log would appear. Nil-receiver-safe, matching
every other optional composition-wired Daemon field.
- daemon: Daemon.process opens a task's log sink for its own duration
(covering both the sandboxed-container and in-process dispatch
paths uniformly) and closes it on every return path.
- cmd/archied: constructs the registry, subscribes core NATS to
SubjectSystemWildcard, and wires TaskLogs into both Daemon and
webui.Server.
- webui: archiving a task now removes its log directory too, so
archived tasks' logs don't accumulate forever. Documented as a
known, accepted race: a task can reach terminal status (satisfying
ArchiveTask's precondition) before process() itself returns and
closes the sink, so a very late system-log write can race an
archive's Remove. Not corrupting -- the write lands on a
soon-to-be-unlinked inode and is discarded -- and not worth fixing
here: it would mean tying Close to the store transition instead of
to process()'s return, a larger change than this issue's scope.
Adversarial review (fresh subagent) found and fixed two real issues in
the first draft: a doc comment accidentally spliced between two
functions during an extraction meant to fix a lint regression, and a
genuine test gap on openTaskLog (added, using TaskRegistry.Write's own
true/false contract as the observable -- no Store/Forge/Trees fakes
needed, since openTaskLog only touches TaskLogs and Log).
Part of #453 (task-observability epic #450). archie-agent logged to
os.Stderr only, and its container is created with AutoRemove: true
(internal/container/pool.go), so all of it disappeared the moment a
task's container exited -- there was no way to see why, say, a gate
command failed inside one.
Adds agentexec.SystemLogHandler: a slog.Handler that tees to whatever
it wraps (kept for an operator attached to a live container) and
additionally publishes each record, JSON-encoded as a logging.Entry,
to SubjectForSystem(taskID) -- a subject that already existed,
documented for exactly this ("log dumps, health... observability"),
with zero publishers anywhere in the codebase.
cmd/archie-agent/main.go wires it in only when this container was
spawned for one specific task (bootTaskID / .git/task.json present);
shared queue-group mode serves many unrelated tasks per process, so
there's no single task subject to ship to and keeps the stderr-only
logger.
Building this surfaced a real pre-existing bug: FeedHandler's attr
flattening applied the group chain active at Handle-time to *all*
accumulated attrs, including ones added via WithAttrs before a later
WithGroup call -- nesting them under a group they were never
actually inside, contrary to slog's own documented Handler contract.
It went unnoticed because nothing in the codebase called WithGroup
until this change needed to. Extracting the shared flattening logic
into exported FlattenAttrs/PrefixAttrs (so SystemLogHandler doesn't
duplicate it) is what surfaced it; fixed in the same change since I
was the one making it public API.
Docs on the wire format's own trap for the daemon-side consumer
(#454, not yet written): the published bytes are a marshaled
logging.Entry, meant for json.Unmarshal, not logging.decode() --
decode() parses raw slog JSON *log lines* off disk, a different and
incompatible shape. See the doc comment on SystemLogHandler.
Adds a human-consent gate for tools marked RequiresApproval, exposes
session management as LLM-invokable chat tools, and gives the operator
/delete to retire conversations permanently.
**Tool-approval gate (internal/tools, internal/agentexec)**
- ApprovalRequester contract lives in internal/tools (not gateway) so
agentexec can import it without pulling in the chat-adapter stack.
- ToolSetOptions.Approval wires the approver through to toolExecute.
- BuildToolSetFrom omits RequiresApproval tools when no approver is
configured; dispatch paths (invokeTool, tool_call bridge) refuse
them too, so the invariant holds at every indirection.
- The gate fails closed: only Approved/PermanentlyApproved pass;
an unknown decision value from a faulty approver does not execute.
- Tool-level timeout (2 min) returns a plain error, not a context
cancellation, so the model can report "approval timed out" instead
of the turn aborting.
- BuildApprovalDescription callback lets a tool surface the specific
resource being acted on ("Delete session \"abc123\"") rather than
showing a static tool description.
**Session tools (internal/gateware/session_tools.go)**
- session_list (idempotent), session_resume (mutating), session_title
(mutating), session_delete (mutating + RequiresApproval).
- Scoped to the current platform + channel: GetByChannel replaces
List() so a model in chat A cannot list or manipulate chat B.
- session_title validates the session belongs to this channel before
writing; session_resume/session_delete resolve refs against the
channel-scoped list.
- Shared limit helper extracted to tools.ListLimit.
**Telegram approval adapter (internal/channels/telegram/approval.go)**
- telegramApprover implements ApprovalRequester via the three-button
inline keyboard (Approve/Permanent/Deny). 2-min token expiry.
- Own map + mutex + "approval:" callback prefix — separate from the
dangerous-command system.
- Permanent approval uses exact-match (not prefix) lookup keyed by
action + resource description, so permanently approving "delete
session abc123" only covers that one session, not every call to
session_delete for 24h.
**Session lifecycle**
- /delete <session-id>: removes a session and its history. Refuses
prefixes shorter than 4 chars even when unique. Exact ID wins over
an ambiguous prefix. sessionTracker.forget sweeps every channel+
thread pointing at the deleted session.
- Extracted resolveSessionRef and shortSessionID for /resume and
/delete to share.
**Wiring**
- submitTurn builds the approver onto ctx before the LLM turn.
- chatGenerateOptions reads approver from ctx, passes it through
ToolSetOptions.
- SessionTools built per turn on the current message, combined with
taskTools into a single toolset.
dependencies-and-contracts.md rule 1 says domains must not import
internal/infrastructure. internal/daemon and internal/agentexec both did,
and for exactly the thing the doc carves out: "internal/eventbus is an
approved shared application contract ... it MUST remain broker-neutral."
The Publisher/Consumer/Requester interfaces existed but lived inside the
NATS package, so a caller could depend on the behaviour yet still had to
import the implementation to name it. They now live in internal/eventbus,
and nats.Client asserts against eventbus.Bus.
The larger violation was ownership of meaning. The doc says "message schemas
stay with their owning domains; the event bus transports messages, it does
not own their meaning." The bus exported TaskEnvelope and a PublishTask
method, so the transport knew what a unit of work was, and it exported the
archie.agent.* subject helpers, so it also owned the agent protocol.
- TaskEnvelope, its routing Kind, the label vocabulary and the task
subjects move to internal/domain/workintake, which encodes and decodes
its own bytes. This starts internal/domain/ per organisation.md.
- The agent request/response/system subjects move to internal/agentexec,
which defines what those messages mean.
- internal/workflow keeps only the kind-to-registry mapping; the label
table stays single and lives with the domain that reads forge issues.
- The bus now carries opaque payloads on plain-string subjects.
Consumers declare the smallest interface they need rather than taking the
whole Bus: daemon.TaskBus is publish-unique/fetch/request, agentexec.
RequestBus is publish-request/reply-inbox, and archie-agent's stageBus is
fetch/respond. None can reach what it does not use.
Two defects this surfaced:
- archie-agent declared its own JetStream stream with a second copy of the
name, subjects, retention, dedup window, MaxDeliver and TTLs that
nats.Connect already sets -- two definitions of one stream, free to
diverge. Config now takes Subjects and FilterSubject from composition,
which is what let the agent use the shared client and delete its
hand-rolled setup.
- Replying to a request needs core NATS, not the JetStream publish path:
a reply inbox is ephemeral and belongs to no stream, so a durable
publish to it fails. The old code hid this by using the raw connection.
The contract now distinguishes Respond from Publish. The agentexec
round-trip test caught it after I had made the same mistake in
archie-agent.
Also translated the SDK's ErrNoResponders into eventbus.ErrNoResponders, so
the daemon's taskrun retry no longer imports the NATS SDK to name a
transient condition. The SDK is now confined to the nats package, the *rpc
packages (themselves NATS infrastructure), and cmd/*.
The agentexec test fake previously hand-rolled a JetStream consumer; it now
drains the client's own consumer, which is both closer to production and
avoids conflicting consumers on a work-queue stream.
Verified: go build, go vet, gofmt clean; full test suite passes.
The event bus was inspecting forge issue labels ("bug", "feature",
"bootstrap") to pick a subject. That put a forge vocabulary inside
infrastructure and duplicated the label table already owned by
workflow.Route -- adding a label meant remembering both places, and
nothing would have caught it if you forgot.
The label table now lives once, in internal/workflow, which already owns
label semantics:
- workflow.Kind names the routing category, with labelKinds as the
single label vocabulary and kindWorkflows mapping kind to registry
entry. Route consumes the ordered kind list so a "bug,feature" task
still falls through to feasibility when no tdd workflow is
registered, exactly as the switch did.
- nats.TaskKind is a closed, validated set of subject tokens.
PublishTask rejects an unknown kind rather than silently delivering
it to the default queue; the zero kind means default, so messages
queued before this field decode unchanged.
- The daemon classifies at the publish site and sets TaskEnvelope.Kind.
Also fixes two defects this exposed:
- drainNATS never reached its SQLite fallback. Replacing the old
(nil, nil) no-message signal with ErrNoMessage in 492ca20 made that
case continue at the top of the loop, but no-message is precisely
when the fallback must run -- requeued tasks (waiting_human approval,
retry-parked) never go through a NATS publish, so nothing would ever
arrive to trigger them. go vet caught it as unreachable code.
- config.classifyFeatureFile silently overwrote duplicate feature
files, so config.gateway.yaml and conf.d/gateway.yaml together
applied one at random. Its own doc comment recorded the bug
("or "" on success/duplicate"). The known-feature list in the error
message is now derived from knownFeatureNames instead of being
spelled out a second time beside it.
A test also called t.Fatalf from a non-test goroutine, where it does not
stop the test.
Verified: go build, go vet, gofmt clean. Pre-existing gateway (18) and
telegram (1) failures are unchanged in count -- they are lost routing
wiring from the parallel telegram merges, tracked separately.
Relocating internal/nats as-is would have cemented its problems into the
approved layout, so this is a rewrite rather than a move.
What was wrong:
- Conn() *nats.Conn, PublishMsg(*nats.Msg) and Fetch() jetstream.Msg pushed
SDK types into all four consumers, so the broker was not replaceable --
the opposite of what dependencies-and-contracts.md requires.
- PublishTask took seven positional parameters, six of them strings, where a
transposed argument was a silent bug.
- Labels travelled as a comma-separated string and were re-split on every
publish.
- Fetch returned (nil, nil) for 'no message', which every caller had to
remember to special-case.
- The logger field was stored and never used.
- Stream name, consumer name and every timeout were hardcoded constants.
- Errors were inconsistently wrapped; several paths returned bare errors.
- Connect repeated nc.Close() on each failure branch.
What replaces it:
- One concern per file: config, errors, subjects, message, client,
publisher, consumer.
- Publisher/Consumer/Requester interfaces with compile-time assertions, so
callers depend on behaviour and can be faked without a broker.
- A Message wrapper keeps jetstream types behind the boundary; consumers no
longer import the NATS SDK at all.
- TaskEnvelope with []string labels and JSON methods preserving the
comma-separated wire form for already-queued messages.
- Sentinel errors (ErrNoMessage, ErrNotConnected, ErrNoReplyAddress,
ErrInvalidConfig) and %w wrapping with operation context throughout.
- Config with validation and defaults; a single deferred cleanup in Connect.
- CoreConn() as the one documented escape hatch, for the *rpc packages that
are themselves NATS infrastructure.
Consumers migrated: cmd/archied, cmd/archie-agent, internal/daemon,
internal/agentexec. Behaviour, subjects and wire format unchanged.
97 -> 85. Not one of the eleven nilerr findings was a bug.
I called these "real bug candidates" when I planned the phases. I was
wrong, and the reason is worth recording: every site is the same
deliberate idiom, and a linter cannot see intent.
- inprocess.go (5) and gateway.go (2) are handlers that report a
problem as TEXT to their consumer. A tool handler returning a Go
error aborts the tool call; returning "rejected: ..." lets the model
read the feedback and retry with valid arguments. A chat handler
returning a Go error surfaces as a gateway fault instead of a usage
hint to the human.
- worker.go carries runErr in the response envelope's Error field.
Returning it as a Go error too would double-report and abort the
NATS reply.
- email.go returns nil when Accept fails because context cancellation
closed the listener. That is a clean shutdown.
- codesearch.go skips an unreadable entry and continues the walk. A
best-effort index must not abort on one bad file.
- skill.go is documented in the function comment already: a
non-directory skills path means "no skills", not a fatal error.
Annotated each with //nolint:nilerr and the specific reason rather than
excluding the linter by path, so nilerr stays live everywhere else and
the next reader wondering "why nil here?" gets an answer at the site.
This is the phase I insisted on keeping single-session and out of the
subagent fan-out. A worker told to "fix nilerr" would have converted all
eleven to error returns: malformed tool arguments would hard-fail the
agent loop instead of being correctable, chat typos would read as
gateway faults, and shutdown would log spurious errors. Every test would
still have passed.
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:
- Memory manager (18.7-18.16): background sync pipeline, prefetch with
timeout, memory-write notification, threat scanning, shutdown drain
with abandoned-write tracking, frozen snapshot pattern. Wired
ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
and notification actually apply to writes, not just exist as unused
primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
internal/secret package:
- Engine interface (Name, Version, Resolve) for pluggable secret backends
- SecretRef type replacing TokenEnv/APIKeyEnv string fields
- ResolveOrEnv for backward-compatible env-var fallback
- Built-in env engine registered by default
- Registry with Register/Get/Resolve/LoadDir (yaegi-based)
- DefaultRegistry() singleton for one-off resolution
- 12 tests, race-clean
Scoped to foundation only. Provider/NATS/Telegram migration deferred
to xec.4 (separate bead — those call sites forward env var names to
subprocesses/runtimes and need eager-resolve-then-inject, not a simple
field rename).
40/40 packages pass with -race.
19 files touched. All fixes are in code — no lint configuration
weakened:
- Added unchecked error handling in defer cleanup paths
- Removed dead variables and unused functions
- Added missing imports (io in gitea.go)
- Simplified conditions (TrimSuffix, De Morgan, slices.Contains)
Also fixes JetStream TempDir teardown race (archie-core-340):
t.Cleanup(srv.Shutdown) now registers after t.TempDir() so
server shutdown (including JetStream files) completes before
temp dir removal. Added srv.WaitForShutdown() for deterministic
cleanup ordering.
Closes: archie-core-756, archie-core-340
daemon.process() branches on ContainerPool != nil: sandboxed tasks publish
a taskrun.Request to archie.taskrun.<id> and await the response via the
new runViaAgent method, instead of running workflow.Route/workflow.Run
locally. Non-container mode is unchanged. Every durable side effect
(Store, Forge, git push) already landed via the storerpc/forgerpc/
worktreerpc proxies during archie-agent's run by the time runViaAgent
returns — it only parks the task itself on transport/setup failure, never
duplicating a terminal state archie-agent's own workflow.Run already
recorded.
Also fixes a real correctness gap: with concurrent multi-task dispatch
(archie-core-u64/g1l) plus containerized archie-agent, a shared queue-group
subscription could let one container answer a sibling's taskrun message
while having a *different* task's worktree mounted. archie-agent now reads
its boot-time task.json and subscribes to its own exact per-task subject
when present, falling back to the shared queue group only in dev/pool mode
(no task.json).
Adds agentexec.ProvidersFromConfig, deduped from cmd/archied's former
executionProviders.
Closes the archie-core-5zh epic: archie-agent runs an entire task's
workflow per invocation instead of one stage per round trip, with all
credentials staying daemon-side throughout.
Add token_env to [nats] config, resolved and validated at daemon startup
(fails early if configured but the named env var is unset). Container
workers get NATS_URL/NATS_TOKEN; archie-agent honors both while CLI-supplied
URL still takes precedence. Anonymous auth remains the default for
single-node deployments.
Unblocks archie-core-5zh (promote archie-agent to per-task process).
Daemon previously processed one task at a time. Add bounded concurrent
dispatch for SQLite and NATS agent modes, reusing [containers].max_concurrency
(0 = unlimited). Same-repository tasks stay FIFO-serialized to avoid worktree
conflicts; different repositories now run concurrently.
Unblocks archie-core-l1q (NATS transport) and archie-core-g1l
(per-repo allow_concurrent override).
Skills now declare their workflow in metadata.archie.workflow.
Catalog() scans frontmatter only (lazy, Tier 1). SkillForWorkflow()
derives the skill directory from the catalog — no hardcoded maps.
Removed workflowToSkill map (agent.go) and workflowToSkillDir switch
(worker.go). Both daemon and agent use the same catalog-based lookup.
Skills renamed to archie-wf-* convention. SKILL.md files updated with
workflow metadata field.
Progressive disclosure Tier 1: Catalog() returns name+description
+workflow. Tier 2: LoadBody() loads full body on activation.
Agent skill loading: HandleMessage now loads .agents/skills/<workflow>/SKILL.md
from its own workspace before running stages. Skill body prepended to each
stage's Mission. Mirrors daemon-side loadSkillBody but owned by the agent
per PRD section 1. workflowToSkillDir mapping added.
Core plugin registry: internal/plugin package with Plugin interface
(Name, Version), Registry type, and LoadDir function. Yaegi-interprets
.go files from ~/.config/archie/plugins/ at startup. Failed plugins are
skipped. PRD section 5 Layer 2.
All 21 PRD claims now have tests. Zero failing tests.
1. Agent doesn't load its own skills:
TestHandleMessageLoadsSkillsFromWorktree — creates a worktree with
.agents/skills/tdd-bugfix/SKILL.md, sends a bare mission to
HandleMessage, verifies the agent loaded the skill body from its
workspace. Fails at runtime: HandleMessage never reads .agents/skills/,
the Mission reaches the runner unchanged.
2. Core plugin registry nonexistent:
TestPluginInterfaceExists, TestRegistryExists,
TestLoadPluginsFromConfigDir — fails to compile: Plugin interface,
Registry type, and LoadDir function are undefined. PRD section 5
Layer 2 entirely aspirational.
GracePeriod now enforced in Release(): if > 0, container stays alive
for the configured duration after task completion before being killed,
per PRD section 1.
WriteTaskJSON now called from daemon.process() before container
Acquire, writing the task payload to <workspace>/task.json per
PRD section 3 /data/ layout.
PluginSpecs populated from TaskContext.SkillPlugins into the
Request sent to the agent. PluginSpec type added to protocol.go.
pluginSpecs() converts skill.Plugin → agentexec.PluginSpec for
transport. PRD section 5 Layer 1.
Replaces field-existence checks with runtime behavior verification:
Gap 1 (per-task agent): FAILING
TestHandleMessageReadsWorkflowField verifies HandleMessage produces
different results when Workflow is set vs. unset. Currently identical
— HandleMessage ignores the Workflow field. Must run full multi-stage
workflow per PRD section 1.
Gap 3 (response/system split): PASSING
TestHandleMessageRoutesSystemMessagesSeparately verifies Channel
echoes from request to response. HandleMessage now reads input Channel
and sets it on the response envelope. Subject functions verified
distinct.
Gap 5 (daemon review): PASSING
TestReviewResultBlocksOnResultWhenRejected verifies ReviewResult
hook gates OnResult. Error blocks — OnResult never called. Nil
allows — OnResult called normally.
Agent processing extracted into agentexec.HandleMessage for testability.
cmd/archie-agent delegates to it.
Gap 1 (per-task agent): add Workflow field to Request and
AgentRequestMessage. Populated from TaskContext by AgentStage
and forwarded through NATSRunner to the agent.
Gap 3 (response/system split): add SubjectForAgentResponse and
SubjectForAgentSystem to subjects.go. Add Channel field to
AgentResponseEnvelope ("response"|"system").
Gap 5 (daemon review): add ReviewResult hook to AgentStage.
Called after status check and before OnResult. Nil = pass-through
(current behaviour). When set, errors block the stage.
All seven regression tests pass.
Seven failing tests that must pass when gaps 1, 3, and 5 are closed:
Gap 1 (per-task agent):
TestAgentRequestMessageHasWorkflowField — AgentRequestMessage needs
a Workflow field so the agent runs full multi-stage workflows, not
one stage at a time.
Gap 3 (response/system channel split):
TestAgentResponseEnvelopeHasChannelField — envelope needs a Channel
field ("response"|"system").
TestSubjectForAgentResponseExists / TestSubjectForAgentSystemExists
— subject functions for archie.agent.<id>.response and .system.
Gap 5 (daemon review step):
TestAgentStageHasReviewHook — AgentStage needs a ReviewResult hook
that gates agent output before OnResult forwards to humans.
All tests currently fail (compilation errors on missing fields).
Gap 2 (MaxUptime): container created with context deadline when
MaxUptime > 0 — Docker kills the container on expiry.
Gap 6 (/data/ volume): mount worktree at /data/worktree instead of
/workspace, matching PRD section 3 volume layout.
Gap 4 (per-task subjects): change SubjectForAgentRequest from per-stage
(archie.agent.<id>.<stage>) to per-task (archie.agent.<id>.request).
Stage name is carried in the AgentRequestMessage payload. PRD section 2.
Adds internal/skill package with YAML frontmatter parsing and skill
discovery from .agents/skills/ (agentskills.io spec convention).
- Frontmatter struct with name, description, version, metadata.archie
- Parse() extracts YAML frontmatter delimited by --- markers
- Discover() scans .agents/skills/*/SKILL.md (follows wfeval pattern)
- LoadBody() for single-skill lazy loading
Skills moved from .archie/skills/ to .agents/skills/ per the
agentskills.io specification.
Wiring: TaskContext.SkillBody field, lazy-loaded on first agent stage
from worktree. When non-empty, prepended to Mission as "Follow these
project-specific guidelines" context. Hardcoded Mission templates stay
(task-specific formatting). Backward compatible: empty when no skill
found.
Implements the full PRD (docs/prds/yaegi-plugin-system.md): repos can
now define .archie/gate.go for diff-inspecting quality rules shell
commands can't express, .archie/stages/*.go for custom pipeline steps,
and skills can ship Go scripts run via a new run_go_script agent tool —
all interpreted in-process with Yaegi, with panics recovered so a bad
script can't take down the daemon. Hot reload (phase 4) needed no code:
every loader reads from disk per call and every task gets a fresh
clone, so nothing is ever cached to go stale.