Add Forger/TaskStore/Trees interfaces in internal/workflow/workflow.go,
scoped to exactly the methods stages call mid-run: 4 Forge methods
(Comment, CloseIssue, CreatePR, SetStateLabel), 2 Store methods (Update,
Transition), 6 Trees methods (Prepare, CommitAll, Push, Diff, ChangedFiles,
ChangedLines). This lets storerpc.Client/forgerpc.Client/worktreerpc.Client
satisfy TaskContext from inside archie-agent.
Go's structural typing meant zero other changes were needed — forge.Forge,
*store.Store, and *worktree.Manager already satisfy the trimmed interfaces.
No behavior change for the daemon's existing inprocess/subprocess paths.
Also fixes a gofmt nit left over from the 5zh.2 commit.
Daemon previously processed one task at a time. Add bounded concurrent
dispatch for SQLite and NATS agent modes, reusing [containers].max_concurrency
(0 = unlimited). Same-repository tasks stay FIFO-serialized to avoid worktree
conflicts; different repositories now run concurrently.
Unblocks archie-core-l1q (NATS transport) and archie-core-g1l
(per-repo allow_concurrent override).
R2 (resilience): skip broken stage plugins, don't abort daemon startup.
BuildWorkflow now skips individual broken plugins with a warning
instead of hard-failing. BuildRegistry and AugmentRegistry skip
broken skills with a warning instead of returning an error. One
bad plugin in one skill no longer prevents the daemon from starting
or blocks other skills from registering their workflows.
R1 (resilience): log skipped daemon plugins in LoadDir. Every failure
path now emits a WARN log via slog.Default() — unreadable files,
compile errors, missing Plugin exports, wrong types. Operators can
now see WHY a plugin was skipped. PRD says 'logged and skipped';
code now matches spec.
N3 (resilience): log symbol-table load errors separately from plugin
source errors. A broken pluginextract.Symbols no longer causes
silent skip of ALL plugins — each failure is logged with the
symbol table index that failed.
N4 (resilience): centralize Yaegi interpreter setup in
newStageInterpreter(). Each plugin file still needs its own
interpreter (package main with duplicate Stage() symbols cannot
co-evaluate), but the setup is now explicit and documented.
N5 (resilience): document tuple fallback positional-field fragility.
The func() (string, func(...)) fallback in loadStage now warns
that it constructs Stage with positional fields and will silently
zero-value any future Stage fields.
R3 (resilience): document nil-client edge case in ensureVolume.
Tests: TestBuildWorkflowSkipsBrokenPlugin (broken plugin skipped,
surviving stages execute), TestBuildRegistrySkipsBrokenSkill (bad
skill doesn't block others, built-in fallbacks preserved).
P1 (panic): nil Storage guard in daemon.process()
C1 (correctness): check WriteTaskJSON error — don't start blind
C2 (correctness): move 'working' label after container setup
C3 (correctness): warn on duplicate workflow names in registry
C4 (correctness): test that generated _Plugin wrapper has nil-guards
C5 (correctness): .agents/skills as file (ENOTDIR) not fatal
C6 (correctness): ecosystem case-insensitive in cacheMounts
N1 (correctness): TOCTOU race — treat 'already exists' as success
N2 (correctness): recover from panicking plugin.Name() at startup
--- Details ---
P1: d.Storage.Setup() is now guarded by a nil check. In normal
operation Storage is always set when ContainerPool is set, but
the invariant is now explicit and enforced.
C1: container.WriteTaskJSON() error was discarded with _. Now it
returns early with a park transition — the container never starts
without a boot-time brief.
C2: SetStateLabel('working') was called before container setup.
If Acquire failed, the task was parked but the label stayed
'working' permanently. Now the label is set only after successful
container setup (or immediately if no containers are configured).
C3: BuildRegistry and AugmentRegistry now log a warning when two
skills declare the same workflow name and one overwrites the other.
C4: TestGeneratedWrapperHasNilGuards verifies that nil WName/
WVersion don't panic. If go generate strips the nil-guards, this
test catches it before the daemon crashes.
C5: skill.Catalog() now returns nil (no skills) instead of
propagating ENOTDIR errors when .agents/skills is a file.
C6: cacheMounts() normalizes ecosystem to lowercase. 'Python',
'NODE', 'Go' all resolve correctly now.
N1: ensureVolume() treats Docker's 'already exists' error as
success, closing the TOCTOU window between VolumeInspect and
VolumeCreate.
N2: safePluginInfo() wraps plugin.Name()/Version() calls in a
recover. A plugin with a panicking Name() no longer crashes the
daemon at startup.
When a worktree already exists on disk (retry, waiting_human ->
implement handoff), the daemon now scans it for .agents/skills/ that
declare metadata.archie.workflow and augments the startup registry
with any newly discovered workflows. Worktree skills take precedence
over the startup registry for that task only.
Changes:
- skillbuild.AugmentRegistry(worktree, base) — scans a worktree
directory for skills, builds workflows from plugins, returns a
new registry with worktree entries merged over base. Base is never
mutated. Mirrors BuildRegistry but starts from caller's registry
instead of builtins.
- daemon.process() — computes worktree path, calls AugmentRegistry
if the directory exists, uses augmented registry for routing.
Degrades gracefully to startup registry on error.
Tests (10 new):
- Adds worktree workflows to base registry
- Worktree skills override same-named base entries
- Base registry is not mutated (copy semantics)
- Nonexistent worktree returns base unchanged
- Worktree with no .agents/skills/ returns base unchanged
- Worktree skill with no plugins does not override base
- Adversarial: concurrent augmentation is safe
- Adversarial: broken SKILL.md is skipped
- Adversarial: bare worktree (no .agents/) does not error
- Adversarial: nil base is treated as empty
Why: skills are per-project (.agents/skills/ in each repo). The
startup registry only knows about a shared skills directory. Per-
worktree augmentation means a repo can define its own workflows
without restarting the daemon.
The daemon no longer hardcodes workflow.Registry{...} in main.go.
Instead, skillbuild.BuildRegistry() scans .agents/skills/ for catalog
entries declaring metadata.archie.workflow and builds each workflow
from its Yaegi-interpreted stage plugins.
Built-in workflows (bootstrap, implement, tdd, feasibility, default)
serve as fallbacks when no skill declares a given workflow name. A
skill that declares metadata.archie.workflow: implement overrides the
built-in — skills define workflows, plugins define stages.
Why: skills must be extension primitives. Adding a workflow should
mean creating .agents/skills/archie-wf-<name>/plugins/*.go, not
modifying Go code in main.go. The catalog is the single source of
truth for what workflows exist.
New skillbuild package: BuildWorkflow() loads Yaegi-interpreted stage
plugins from a skill's plugins/ directory and constructs a runnable
workflow.Workflow. Uses wfextract symbol tables for type access.
A skill directory with metadata.archie.workflow: greet and stage plugins
in plugins/*.go IS the workflow definition — no Go code changes needed.
Stage plugins export:
func Stage() workflow.Stage
or the fallback signature:
func Stage() (string, func(context.Context, *workflow.TaskContext) error)
Stages are loaded in filename order. Missing plugins dir is not an error
(matches wfeval.Discover pattern).
Test: skill with two plugins (hello, world) produces a 2-stage workflow
that modifies BuildSummary sequentially.
Skills now declare their workflow in metadata.archie.workflow.
Catalog() scans frontmatter only (lazy, Tier 1). SkillForWorkflow()
derives the skill directory from the catalog — no hardcoded maps.
Removed workflowToSkill map (agent.go) and workflowToSkillDir switch
(worker.go). Both daemon and agent use the same catalog-based lookup.
Skills renamed to archie-wf-* convention. SKILL.md files updated with
workflow metadata field.
Progressive disclosure Tier 1: Catalog() returns name+description
+workflow. Tier 2: LoadBody() loads full body on activation.
GracePeriod now enforced in Release(): if > 0, container stays alive
for the configured duration after task completion before being killed,
per PRD section 1.
WriteTaskJSON now called from daemon.process() before container
Acquire, writing the task payload to <workspace>/task.json per
PRD section 3 /data/ layout.
PluginSpecs populated from TaskContext.SkillPlugins into the
Request sent to the agent. PluginSpec type added to protocol.go.
pluginSpecs() converts skill.Plugin → agentexec.PluginSpec for
transport. PRD section 5 Layer 1.
loadSkillBody() now populates TaskContext.SkillPlugins alongside
SkillBody. Plugins are discovered from .agents/skills/<name>/plugins/*.go
once, lazily on the first agent stage that loads the skill body.
New field: TaskContext.SkillPlugins []skill.Plugin — populated by
loadSkillBody, nil when no skill or no plugins.
Test-first: TestSkillPluginsAvailableDuringStageExecution creates a
worktree with a skill + bundled plugin, verifies plugins are loaded
and runnable via plugin.Run() after loadSkillBody().
Replaces field-existence checks with runtime behavior verification:
Gap 1 (per-task agent): FAILING
TestHandleMessageReadsWorkflowField verifies HandleMessage produces
different results when Workflow is set vs. unset. Currently identical
— HandleMessage ignores the Workflow field. Must run full multi-stage
workflow per PRD section 1.
Gap 3 (response/system split): PASSING
TestHandleMessageRoutesSystemMessagesSeparately verifies Channel
echoes from request to response. HandleMessage now reads input Channel
and sets it on the response envelope. Subject functions verified
distinct.
Gap 5 (daemon review): PASSING
TestReviewResultBlocksOnResultWhenRejected verifies ReviewResult
hook gates OnResult. Error blocks — OnResult never called. Nil
allows — OnResult called normally.
Agent processing extracted into agentexec.HandleMessage for testability.
cmd/archie-agent delegates to it.
Gap 1 (per-task agent): add Workflow field to Request and
AgentRequestMessage. Populated from TaskContext by AgentStage
and forwarded through NATSRunner to the agent.
Gap 3 (response/system split): add SubjectForAgentResponse and
SubjectForAgentSystem to subjects.go. Add Channel field to
AgentResponseEnvelope ("response"|"system").
Gap 5 (daemon review): add ReviewResult hook to AgentStage.
Called after status check and before OnResult. Nil = pass-through
(current behaviour). When set, errors block the stage.
All seven regression tests pass.
Seven failing tests that must pass when gaps 1, 3, and 5 are closed:
Gap 1 (per-task agent):
TestAgentRequestMessageHasWorkflowField — AgentRequestMessage needs
a Workflow field so the agent runs full multi-stage workflows, not
one stage at a time.
Gap 3 (response/system channel split):
TestAgentResponseEnvelopeHasChannelField — envelope needs a Channel
field ("response"|"system").
TestSubjectForAgentResponseExists / TestSubjectForAgentSystemExists
— subject functions for archie.agent.<id>.response and .system.
Gap 5 (daemon review step):
TestAgentStageHasReviewHook — AgentStage needs a ReviewResult hook
that gates agent output before OnResult forwards to humans.
All tests currently fail (compilation errors on missing fields).
Adds internal/skill package with YAML frontmatter parsing and skill
discovery from .agents/skills/ (agentskills.io spec convention).
- Frontmatter struct with name, description, version, metadata.archie
- Parse() extracts YAML frontmatter delimited by --- markers
- Discover() scans .agents/skills/*/SKILL.md (follows wfeval pattern)
- LoadBody() for single-skill lazy loading
Skills moved from .archie/skills/ to .agents/skills/ per the
agentskills.io specification.
Wiring: TaskContext.SkillBody field, lazy-loaded on first agent stage
from worktree. When non-empty, prepended to Mission as "Follow these
project-specific guidelines" context. Hardcoded Mission templates stay
(task-specific formatting). Backward compatible: empty when no skill
found.
Implements the full PRD (docs/prds/yaegi-plugin-system.md): repos can
now define .archie/gate.go for diff-inspecting quality rules shell
commands can't express, .archie/stages/*.go for custom pipeline steps,
and skills can ship Go scripts run via a new run_go_script agent tool —
all interpreted in-process with Yaegi, with panics recovered so a bad
script can't take down the daemon. Hot reload (phase 4) needed no code:
every loader reads from disk per call and every task gets a fresh
clone, so nothing is ever cached to go stale.
The first feature run parked exactly as designed but for an avoidable
reason: todo is a templ project and the agent edited generated
_templ.go files, leaving literal template placeholders in rendered
HTML. Systemic fixes, all environmental:
- repos gain protect = ["_templ.go"]: suffixes agents can never write
(agentloop.ProtectPaths), forcing edits to the sources; the gate's
new leading ["templ","generate","./components/"] command
regenerates. AgentStage.ProtectPaths now composes with the repo list
and takes TaskContext like every other stage hook.
- archied -requeue <task-id>: the retry path for parked/waiting tasks
(keeps the task's workflow; Requeue with empty workflow preserves it,
clears the park reason).
Retry of the parked feature task drove the full loop home: PRD-guided
plan -> builder edited components.templ (generated file write-blocked),
gate regenerated + tested -> PR #8 with 109 lines of tests.
Routed via the feature label. Stages: assess (read-only agent with a
structured decide tool — verdict as a tool call, not parsed prose;
not-a-fit closes the issue as won't-do with reasons) -> PRD (read-only
agent; problem, solution, surface, acceptance criteria, non-goals) ->
deliver (PRD posted as the watched issue comment, optional [notify]
webhook POST for n8n->email, task -> waiting_human).
The daemon's cycle now watches waiting_human tasks: replies after the
watch comment (excluding the bot) are judged by an LLM (triage role,
APPROVE/REJECT/UNCLEAR + reason — never keyword matching). Approvals
requeue under implement (whose planner now treats an existing PRD as
prior context); rejections close with the reasoning acknowledged;
unclear replies keep waiting.
Supporting changes: forge.Comment returns the comment id;
forge.RepliesAfter; store watch_comment_id (additive migration),
WaitingTasks, Requeue; Route honours a pre-assigned workflow;
AgentStage.Extra for stage-specific tools.
Live: todo#7 assessed fit -> PRD delivered -> waiting_human.
Adapted from tau's eventbus/metrics designs (per Sam), slimmed for a
small daemon:
- internal/events: single-type bus with mutex fan-out and bounded
per-subscriber buffers that DROP on overflow (counted) — a stalled
dashboard can never backpressure the task engine (tau's blocking
delivery deliberately not copied)
- store: events table + queries powering the dashboards: task
timelines, per-workflow outcomes/spend, stage durations and error
counts (from stage_finish events), token spend by day
- emit points: task_queued, stage_start/finish(duration, error),
agent_finish(status, tokens, steps, model), parked, outcome,
pr_merged/pr_rejected
- internal/webui: read-only dashboard (embedded single page, no build
step) — status cards, task board with click-through timelines, live
SSE event tail with catch-up via ?since=<event id>, workflow/stage
metrics tables. [web] listen config, default 127.0.0.1:8484, no auth
by design (bind localhost/tailnet); "off" disables
- sink pipeline: bus -> SQLite insert (stamps row id) -> SSE broadcast
Verified live: dashboard serves real workflow stats from the existing
task history; bus overflow and event-log round-trips unit-tested.
- Issues are picked up by assigning them to the bot (tink-bot style)
instead of by label; labels now only influence workflow routing
- TDD: capture the failing 'go test' output after the repro stage
(parking if it unexpectedly passes) and post it on the PR as a code
block — evidence the bug was reproduced before the fix
- TDD fix stage now write-protects *_test.go via agentloop.ProtectPaths
(environmental enforcement, replacing the advisory prompt rule)
- OpenPR extracted so stages can act after the PR exists in the same
stage (the engine stops at the first stage with an outcome)
Routed via the bug label. Stages: analyse (read-only problem-surface
analysis) -> repro-tests behind the INVERTED gate (go vet must pass,
go test ./... must FAIL — GateCommand.ExpectFailure proves the repro
captures the bug) -> repro committed separately -> fix behind the
repo's full gate with the repro declared immutable in the rules ->
push, diff cap, PR. Two-commit PRs tell the story: test first, fix
second. Zero engine or step-library changes — the workflow is pure
composition, per the reusability requirement.
Proven live on samcharles93/todo#5 (empty todos bug): repro gate
pushed back once, fix stage blocked twice by the full gate before
green; PR #6 (~103k tokens total).
AgentStage is the reusable bridge from workflow stages to agentloop
runs: role->model selection from config, budgets, repo gate conversion,
task-row NotesStore (survives worktree cleanup, never pollutes the PR),
and park-on-anything-but-passed. Every future LLM stage (tdd,
feasibility) is an AgentStage with a different mission/gate/handler.
The implement workflow: prepare -> baseline gate (park on red base,
prevents park-storms) -> plan (read-only agentloop, plan posted to the
issue as a veto window) -> build (full toolset behind the repo's
configured gate) -> commit/push -> diff cap -> PR with the builder's
finish summary as the body.
Proven live on samcharles93/todo#3: planner 3 steps, builder 4 steps
(~31k tokens total), producing gate-verified table-driven TodoStore
tests as PR #4.
First live run exposed the gap: with only bootstrap registered, Route()
fell through to the definition-bug backstop. Label routing now falls
back to reg["default"], which archied maps to bootstrap until the
implement workflow exists.