Add [containers].volume_ttl (default 72h). Persistent-storage repos get a
per-repo bare Git cache (--reference-if-able --dissociate) so task
worktrees clone fast without repeated full network clones, while staying
fully independent of the cache's lifecycle — cache expiry can never
corrupt an active or parked task's worktree. Docker repo volumes are
labeled as Archie-owned and swept non-forced (a volume still attached to a
running container survives and retries next cycle); the repo cache
directory is swept by mtime. Both cleanups run at daemon startup and each
poll cycle.
Filed archie-core-756 (repo-wide golangci-lint baseline) and archie-core-340
(intermittent JetStream TempDir teardown flake) as separate follow-ups
rather than scope-creeping them into this change.
daemon.process() branches on ContainerPool != nil: sandboxed tasks publish
a taskrun.Request to archie.taskrun.<id> and await the response via the
new runViaAgent method, instead of running workflow.Route/workflow.Run
locally. Non-container mode is unchanged. Every durable side effect
(Store, Forge, git push) already landed via the storerpc/forgerpc/
worktreerpc proxies during archie-agent's run by the time runViaAgent
returns — it only parks the task itself on transport/setup failure, never
duplicating a terminal state archie-agent's own workflow.Run already
recorded.
Also fixes a real correctness gap: with concurrent multi-task dispatch
(archie-core-u64/g1l) plus containerized archie-agent, a shared queue-group
subscription could let one container answer a sibling's taskrun message
while having a *different* task's worktree mounted. archie-agent now reads
its boot-time task.json and subscribes to its own exact per-task subject
when present, falling back to the shared queue group only in dev/pool mode
(no task.json).
Adds agentexec.ProvidersFromConfig, deduped from cmd/archied's former
executionProviders.
Closes the archie-core-5zh epic: archie-agent runs an entire task's
workflow per invocation instead of one stage per round trip, with all
credentials staying daemon-side throughout.
Add config.Repo.AllowConcurrent (allow_concurrent toml key). Repos opted in
skip the daemon's default FIFO same-repo task serialization while still
respecting [containers].max_concurrency globally. Worktrees are already keyed
per issue number, so opted-in repos don't collide on disk.
Daemon previously processed one task at a time. Add bounded concurrent
dispatch for SQLite and NATS agent modes, reusing [containers].max_concurrency
(0 = unlimited). Same-repository tasks stay FIFO-serialized to avoid worktree
conflicts; different repositories now run concurrently.
Unblocks archie-core-l1q (NATS transport) and archie-core-g1l
(per-repo allow_concurrent override).
Fixes (4 items):
S4 — Add TypeScript, Deno, Bun cache ecosystems to
cacheVolumesByEcosystem. TypeScript shares node+pnpm volumes;
Deno and Bun get dedicated cache volumes.
R4 — Share single Docker client between container.Pool and
storage.DockerBackend. NewPool now accepts an optional
DockerClient in Config; if nil, creates its own. Pool
tracks ownership (ownCli) and only closes its own client.
S2 — Wire Storage.Teardown() call in daemon.process() after
workflow.Run(). Docker backend is a no-op; this closes the
interface contract for future backends.
R5 — Emit observability event on AugmentRegistry failure.
Previously only logged a warning; now emits a
'registry_augment_failed' event with error detail.
Promoted log level to Error.
Documentation (4 items):
D1 — Document PluginRegistry as reserved for future daemon
extension points (forge resolvers, storage backends, etc.)
S1 — Fix /data/ layout comment to reflect actual task.json path
(/data/worktree/task.json, not /data/task.json)
S3 — Add TODO(PRD §3) for session.jsonl, memory.jsonl, plugins/
volume — these need agent-side implementation first
N6 — Document that VolumeInspectOptions{} requires moby v25+
(go.mod pins compatible version)
N7 — Document that _Plugin.IValue is required by Yaegi interface
bridging; warn that go generate strips nil-guards
Test: TestCacheMountsForTypeScript verifies typescript ecosystem
returns node+pnpm volumes sorted by destination.
Verification: single client.New(client.FromEnv) in main.go;
all 21 storage tests pass; full suite green.
P1 (panic): nil Storage guard in daemon.process()
C1 (correctness): check WriteTaskJSON error — don't start blind
C2 (correctness): move 'working' label after container setup
C3 (correctness): warn on duplicate workflow names in registry
C4 (correctness): test that generated _Plugin wrapper has nil-guards
C5 (correctness): .agents/skills as file (ENOTDIR) not fatal
C6 (correctness): ecosystem case-insensitive in cacheMounts
N1 (correctness): TOCTOU race — treat 'already exists' as success
N2 (correctness): recover from panicking plugin.Name() at startup
--- Details ---
P1: d.Storage.Setup() is now guarded by a nil check. In normal
operation Storage is always set when ContainerPool is set, but
the invariant is now explicit and enforced.
C1: container.WriteTaskJSON() error was discarded with _. Now it
returns early with a park transition — the container never starts
without a boot-time brief.
C2: SetStateLabel('working') was called before container setup.
If Acquire failed, the task was parked but the label stayed
'working' permanently. Now the label is set only after successful
container setup (or immediately if no containers are configured).
C3: BuildRegistry and AugmentRegistry now log a warning when two
skills declare the same workflow name and one overwrites the other.
C4: TestGeneratedWrapperHasNilGuards verifies that nil WName/
WVersion don't panic. If go generate strips the nil-guards, this
test catches it before the daemon crashes.
C5: skill.Catalog() now returns nil (no skills) instead of
propagating ENOTDIR errors when .agents/skills is a file.
C6: cacheMounts() normalizes ecosystem to lowercase. 'Python',
'NODE', 'Go' all resolve correctly now.
N1: ensureVolume() treats Docker's 'already exists' error as
success, closing the TOCTOU window between VolumeInspect and
VolumeCreate.
N2: safePluginInfo() wraps plugin.Name()/Version() calls in a
recover. A plugin with a panicking Name() no longer crashes the
daemon at startup.
Introduces the storage.Backend interface for pluggable container mount
management. The Docker implementation (DockerBackend) is the MVP —
no NFS/SMB/S3, just Docker volumes + bind mounts.
/data/ volume layout:
/data/worktree/ — bind mount of host worktree (always)
/data/repo/ — per-repo Docker volume (when persistent_storage = true)
/data/cache/go/ — shared GOMODCACHE volume
/data/cache/node/— shared npm cache volume
/data/cache/pnpm/— shared pnpm store volume
/data/cache/pip/ — shared pip cache volume
/data/cache/cargo/— shared Rust cargo cache volume
/data/task.json — boot-time brief (existing WriteTaskJSON)
Changes:
- internal/storage/: Backend interface, DockerBackend, Mount types,
ecosystem-to-cache-volume mapping, ConvertMounts helper
- container.Pool.Acquire: now accepts []storage.Mount instead of
hardcoding a single bind mount — mount list comes from the storage
backend
- daemon.process(): calls d.Storage.Setup() to build the mount list,
passes Ecosystem + PersistentStorage from repo config
- config.Repo.PersistentStorage: when true, a named Docker volume
(archie-repo-<owner>-<repo>) is mounted at /data/repo and survives
task completion
- main.go: creates DockerBackend when containers are enabled
21 tests: interface contract, ecosystem cache mounts (go/node/
python/rust/custom), worktree bind mount ordering, persistent
storage on/off/empty-owner guard, mount list sort+dedup, adversarial
(nonexistent worktree, empty ecosystem, unknown ecosystem).
Why: daemon-managed cache volumes mean warm GOMODCACHE/npm cache
across tasks. Per-repo persistent volumes preserve expensive build
artifacts. The interface leaves room for future backends without
rewiring container orchestration.
When a worktree already exists on disk (retry, waiting_human ->
implement handoff), the daemon now scans it for .agents/skills/ that
declare metadata.archie.workflow and augments the startup registry
with any newly discovered workflows. Worktree skills take precedence
over the startup registry for that task only.
Changes:
- skillbuild.AugmentRegistry(worktree, base) — scans a worktree
directory for skills, builds workflows from plugins, returns a
new registry with worktree entries merged over base. Base is never
mutated. Mirrors BuildRegistry but starts from caller's registry
instead of builtins.
- daemon.process() — computes worktree path, calls AugmentRegistry
if the directory exists, uses augmented registry for routing.
Degrades gracefully to startup registry on error.
Tests (10 new):
- Adds worktree workflows to base registry
- Worktree skills override same-named base entries
- Base registry is not mutated (copy semantics)
- Nonexistent worktree returns base unchanged
- Worktree with no .agents/skills/ returns base unchanged
- Worktree skill with no plugins does not override base
- Adversarial: concurrent augmentation is safe
- Adversarial: broken SKILL.md is skipped
- Adversarial: bare worktree (no .agents/) does not error
- Adversarial: nil base is treated as empty
Why: skills are per-project (.agents/skills/ in each repo). The
startup registry only knows about a shared skills directory. Per-
worktree augmentation means a repo can define its own workflows
without restarting the daemon.
Daemon plugins (Yaegi-interpreted .go files) are now loaded from the
configured plugin_dir at startup. Each file must export a 'Plugin'
variable using the _Plugin wrapper struct, which bridges Yaegi types
to the Go plugin.Plugin interface.
Changes:
- internal/plugin/pluginextract/ — generated Yaegi symbol table for
the plugin package (mirrors wfextract pattern), including the
_Plugin interface wrapper that lets interpreted code implement
Go interfaces
- LoadDir now accepts optional symbol tables via variadic parameter,
avoiding an import cycle between plugin and pluginextract
- Plugin tests rewritten as behavioral (not structural), verifying
actual Yaegi interpretation of valid, invalid, and missing plugins
- config.PluginDir field for specifying the plugin directory
- daemon.PluginRegistry field for holding loaded plugins
- main.go wires plugin loading: LoadDir at startup, register into
daemon, log each loaded plugin
Why: the Plugin interface and LoadDir existed but weren't connected
to the daemon. Skills define workflows; daemon plugins are the
extension point for forge resolvers, storage backends, and other
core-daemon concerns.
GracePeriod now enforced in Release(): if > 0, container stays alive
for the configured duration after task completion before being killed,
per PRD section 1.
WriteTaskJSON now called from daemon.process() before container
Acquire, writing the task payload to <workspace>/task.json per
PRD section 3 /data/ layout.
PluginSpecs populated from TaskContext.SkillPlugins into the
Request sent to the agent. PluginSpec type added to protocol.go.
pluginSpecs() converts skill.Plugin → agentexec.PluginSpec for
transport. PRD section 5 Layer 1.
Implements the full PRD (docs/prds/yaegi-plugin-system.md): repos can
now define .archie/gate.go for diff-inspecting quality rules shell
commands can't express, .archie/stages/*.go for custom pipeline steps,
and skills can ship Go scripts run via a new run_go_script agent tool —
all interpreted in-process with Yaegi, with panics recovered so a bad
script can't take down the daemon. Hot reload (phase 4) needed no code:
every loader reads from disk per call and every task gets a fresh
clone, so nothing is ever cached to go stale.
Routed via the feature label. Stages: assess (read-only agent with a
structured decide tool — verdict as a tool call, not parsed prose;
not-a-fit closes the issue as won't-do with reasons) -> PRD (read-only
agent; problem, solution, surface, acceptance criteria, non-goals) ->
deliver (PRD posted as the watched issue comment, optional [notify]
webhook POST for n8n->email, task -> waiting_human).
The daemon's cycle now watches waiting_human tasks: replies after the
watch comment (excluding the bot) are judged by an LLM (triage role,
APPROVE/REJECT/UNCLEAR + reason — never keyword matching). Approvals
requeue under implement (whose planner now treats an existing PRD as
prior context); rejections close with the reasoning acknowledged;
unclear replies keep waiting.
Supporting changes: forge.Comment returns the comment id;
forge.RepliesAfter; store watch_comment_id (additive migration),
WaitingTasks, Requeue; Route honours a pre-assigned workflow;
AgentStage.Extra for stage-specific tools.
Live: todo#7 assessed fit -> PRD delivered -> waiting_human.
Adapted from tau's eventbus/metrics designs (per Sam), slimmed for a
small daemon:
- internal/events: single-type bus with mutex fan-out and bounded
per-subscriber buffers that DROP on overflow (counted) — a stalled
dashboard can never backpressure the task engine (tau's blocking
delivery deliberately not copied)
- store: events table + queries powering the dashboards: task
timelines, per-workflow outcomes/spend, stage durations and error
counts (from stage_finish events), token spend by day
- emit points: task_queued, stage_start/finish(duration, error),
agent_finish(status, tokens, steps, model), parked, outcome,
pr_merged/pr_rejected
- internal/webui: read-only dashboard (embedded single page, no build
step) — status cards, task board with click-through timelines, live
SSE event tail with catch-up via ?since=<event id>, workflow/stage
metrics tables. [web] listen config, default 127.0.0.1:8484, no auth
by design (bind localhost/tailnet); "off" disables
- sink pipeline: bus -> SQLite insert (stamps row id) -> SSE broadcast
Verified live: dashboard serves real workflow stats from the existing
task history; bus overflow and event-log round-trips unit-tested.
- Issues are picked up by assigning them to the bot (tink-bot style)
instead of by label; labels now only influence workflow routing
- TDD: capture the failing 'go test' output after the repro stage
(parking if it unexpectedly passes) and post it on the PR as a code
block — evidence the bug was reproduced before the fix
- TDD fix stage now write-protects *_test.go via agentloop.ProtectPaths
(environmental enforcement, replacing the advisory prompt rule)
- OpenPR extracted so stages can act after the PR exists in the same
stage (the engine stops at the first stage with an outcome)
AgentStage is the reusable bridge from workflow stages to agentloop
runs: role->model selection from config, budgets, repo gate conversion,
task-row NotesStore (survives worktree cleanup, never pollutes the PR),
and park-on-anything-but-passed. Every future LLM stage (tdd,
feasibility) is an AgentStage with a different mission/gate/handler.
The implement workflow: prepare -> baseline gate (park on red base,
prevents park-storms) -> plan (read-only agentloop, plan posted to the
issue as a veto window) -> build (full toolset behind the repo's
configured gate) -> commit/push -> diff cap -> PR with the builder's
finish summary as the body.
Proven live on samcharles93/todo#3: planner 3 steps, builder 4 steps
(~31k tokens total), producing gate-verified table-driven TodoStore
tests as PR #4.