3543 Commits
Author SHA1 Message Date
sam 2182cc00a5 refactor: replace hardcoded workflow→skill maps with catalog-driven lookup
Skills now declare their workflow in metadata.archie.workflow.
Catalog() scans frontmatter only (lazy, Tier 1). SkillForWorkflow()
derives the skill directory from the catalog — no hardcoded maps.

Removed workflowToSkill map (agent.go) and workflowToSkillDir switch
(worker.go). Both daemon and agent use the same catalog-based lookup.

Skills renamed to archie-wf-* convention. SKILL.md files updated with
workflow metadata field.

Progressive disclosure Tier 1: Catalog() returns name+description
+workflow. Tier 2: LoadBody() loads full body on activation.
2026-07-22 03:44:42 +10:00
sam d3c843e8f5 feat: close final two gaps — agent skill loading + core plugin registry
Agent skill loading: HandleMessage now loads .agents/skills/<workflow>/SKILL.md
from its own workspace before running stages. Skill body prepended to each
stage's Mission. Mirrors daemon-side loadSkillBody but owned by the agent
per PRD section 1. workflowToSkillDir mapping added.

Core plugin registry: internal/plugin package with Plugin interface
(Name, Version), Registry type, and LoadDir function. Yaegi-interprets
.go files from ~/.config/archie/plugins/ at startup. Failed plugins are
skipped. PRD section 5 Layer 2.

All 21 PRD claims now have tests. Zero failing tests.
2026-07-22 03:37:15 +10:00
sam 39b66fc1ef test: add failing regression tests for the two remaining gaps
1. Agent doesn't load its own skills:
   TestHandleMessageLoadsSkillsFromWorktree — creates a worktree with
   .agents/skills/tdd-bugfix/SKILL.md, sends a bare mission to
   HandleMessage, verifies the agent loaded the skill body from its
   workspace. Fails at runtime: HandleMessage never reads .agents/skills/,
   the Mission reaches the runner unchanged.

2. Core plugin registry nonexistent:
   TestPluginInterfaceExists, TestRegistryExists,
   TestLoadPluginsFromConfigDir — fails to compile: Plugin interface,
   Registry type, and LoadDir function are undefined. PRD section 5
   Layer 2 entirely aspirational.
2026-07-22 03:35:03 +10:00
sam 4edad31b66 fix: wire dead code — GracePeriod, task.json, plugin specs
GracePeriod now enforced in Release(): if > 0, container stays alive
for the configured duration after task completion before being killed,
per PRD section 1.

WriteTaskJSON now called from daemon.process() before container
Acquire, writing the task payload to <workspace>/task.json per
PRD section 3 /data/ layout.

PluginSpecs populated from TaskContext.SkillPlugins into the
Request sent to the agent. PluginSpec type added to protocol.go.
pluginSpecs() converts skill.Plugin → agentexec.PluginSpec for
transport. PRD section 5 Layer 1.
2026-07-22 03:31:27 +10:00
sam fab7a0ae6f feat: close Gaps 2, 6, and 8
Gap 8 (frontmatter plugins key): add Plugins []string to
Metadata.Archie struct. Parses the plugins list from SKILL.md
YAML frontmatter per PRD section 5.

Gap 2 (grace period): add GracePeriod field to container.Config,
distinct from MaxUptime. MaxUptime = total container lifetime cap.
GracePeriod = idle time after task completion before kill.
Documented per PRD sections 1 and 4.

Gap 6 (task.json): add TaskPayload struct and WriteTaskJSON
function to internal/container. Daemon writes the task payload
to <workspace>/task.json before container start, matching the
PRD section 3 /data/task.json boot brief.
2026-07-22 03:25:44 +10:00
sam 8630d04bbc feat: close Gap 7 — execute bundled plugins during stage runs
loadSkillBody() now populates TaskContext.SkillPlugins alongside
SkillBody. Plugins are discovered from .agents/skills/<name>/plugins/*.go
once, lazily on the first agent stage that loads the skill body.

New field: TaskContext.SkillPlugins []skill.Plugin — populated by
loadSkillBody, nil when no skill or no plugins.

Test-first: TestSkillPluginsAvailableDuringStageExecution creates a
worktree with a skill + bundled plugin, verifies plugins are loaded
and runnable via plugin.Run() after loadSkillBody().
2026-07-22 03:23:16 +10:00
sam ce9ce050ed feat: close Gap 1 — per-task agent (multi-stage batch execution)
HandleMessage now supports two modes:
- Single-stage (Request only): backward compatible, no Workflow set
- Per-task batch (Workflow + Stages): runs all stages sequentially,
  accumulates tokens/iterations/notes, sets TaskCompleted on success

New fields:
- AgentRequestMessage.Stages []Request — batch of agent stages
- AgentResponseEnvelope.TaskCompleted bool — true when batch done

RunnerFactory injection for testability — HandleMessage accepts a
factory function. DefaultRunnerFactory creates InProcessRunner.
Tests use mockRunner.

runStages() executes each stage, stops on first failure, aggregates
results. Combined summary uses "[stage] summary" format.
2026-07-22 03:20:37 +10:00
sam e8ffa20b33 test: rewrite regression tests as behavioral (not structural)
Replaces field-existence checks with runtime behavior verification:

Gap 1 (per-task agent): FAILING
  TestHandleMessageReadsWorkflowField verifies HandleMessage produces
  different results when Workflow is set vs. unset. Currently identical
  — HandleMessage ignores the Workflow field. Must run full multi-stage
  workflow per PRD section 1.

Gap 3 (response/system split): PASSING
  TestHandleMessageRoutesSystemMessagesSeparately verifies Channel
  echoes from request to response. HandleMessage now reads input Channel
  and sets it on the response envelope. Subject functions verified
  distinct.

Gap 5 (daemon review): PASSING
  TestReviewResultBlocksOnResultWhenRejected verifies ReviewResult
  hook gates OnResult. Error blocks — OnResult never called. Nil
  allows — OnResult called normally.

Agent processing extracted into agentexec.HandleMessage for testability.
cmd/archie-agent delegates to it.
2026-07-22 03:13:57 +10:00
sam 326d2265a8 feat: close PRD gaps 1, 3, and 5
Gap 1 (per-task agent): add Workflow field to Request and
AgentRequestMessage. Populated from TaskContext by AgentStage
and forwarded through NATSRunner to the agent.

Gap 3 (response/system split): add SubjectForAgentResponse and
SubjectForAgentSystem to subjects.go. Add Channel field to
AgentResponseEnvelope ("response"|"system").

Gap 5 (daemon review): add ReviewResult hook to AgentStage.
Called after status check and before OnResult. Nil = pass-through
(current behaviour). When set, errors block the stage.

All seven regression tests pass.
2026-07-22 03:07:04 +10:00
sam 13a01c4a37 test: add regression tests for remaining PRD gaps
Seven failing tests that must pass when gaps 1, 3, and 5 are closed:

Gap 1 (per-task agent):
  TestAgentRequestMessageHasWorkflowField — AgentRequestMessage needs
  a Workflow field so the agent runs full multi-stage workflows, not
  one stage at a time.

Gap 3 (response/system channel split):
  TestAgentResponseEnvelopeHasChannelField — envelope needs a Channel
  field ("response"|"system").
  TestSubjectForAgentResponseExists / TestSubjectForAgentSystemExists
  — subject functions for archie.agent.<id>.response and .system.

Gap 5 (daemon review step):
  TestAgentStageHasReviewHook — AgentStage needs a ReviewResult hook
  that gates agent output before OnResult forwards to humans.

All tests currently fail (compilation errors on missing fields).
2026-07-22 03:00:13 +10:00
sam c8b22aeb8e fix: three quick-wins to close PRD gaps
Gap 2 (MaxUptime): container created with context deadline when
MaxUptime > 0 — Docker kills the container on expiry.

Gap 6 (/data/ volume): mount worktree at /data/worktree instead of
/workspace, matching PRD section 3 volume layout.

Gap 4 (per-task subjects): change SubjectForAgentRequest from per-stage
(archie.agent.<id>.<stage>) to per-task (archie.agent.<id>.request).
Stage name is carried in the AgentRequestMessage payload. PRD section 2.
2026-07-22 02:57:41 +10:00
sam fc33e519a5 feat: bundled plugin discovery for skills
Adds plugin discovery to internal/skill — Yaegi .go files in a skill's
plugins/ directory are loaded alongside the SKILL.md body.

- Plugin type: Name, Path, Src, Run(input) (string, error) via Yaegi
- DiscoverPlugins(dir, skillName): scans .agents/skills/<name>/plugins/*.go
- Run() uses Yaegi interpreter with full stdlib access; plugins must use
  package main and export func Run(string) string
- Discover() now populates Skill.Plugins alongside frontmatter + body
- Missing plugins/ dir returns empty slice (not an error)
- Missing skill returns nil (not an error)

5 new tests: DiscoverPlugins, DiscoverPluginsNone,
DiscoverPluginsMissingSkill, PluginRun (Yaegi round-trip),
DiscoverIncludesPlugins (integration with Discover).
2026-07-22 02:55:31 +10:00
sam 121dd7d815 docs: sync PRD status table and lifecycle section with post-de57f72 implementation 2026-07-22 02:45:48 +10:00
sam de57f72cce feat: add Docker container sandbox for archie-agent
Dockerfile: multi-stage fat image (golang:1.26 builder → ubuntu:24.04)
with Go 1.26, Python 3.12/3.13 via deadsnakes, UV, Node 22, pnpm,
TypeScript, esbuild, git, gh, golangci-lint, gofumpt.

Container pool (internal/container): manages Docker container lifecycle
via moby/moby client. Per-task containers with bind-mounted workspaces.

- Acquire: create, start, mount workspace, inject NATS + API key env
- Release: stop, remove (AutoRemove as safety net)
- Orphan recovery on daemon startup (label: archie-daemon=true)
- MaxConcurrency enforcement, MaxUptime config (unused in MVP)
- Image pull with "missing"/"always" policy

Config: [containers] block (enabled, image, max_concurrency, max_uptime,
pull_policy). Validation: requires agent.mode = "nats" and [nats] url.

Daemon: acquires container before workflow.Run, releases after.
Container environment auto-populated from provider API keys.
2026-07-22 02:39:17 +10:00
sam a9641276e1 feat: parse SKILL.md frontmatter, inject skill body into agent context
Adds internal/skill package with YAML frontmatter parsing and skill
discovery from .agents/skills/ (agentskills.io spec convention).

- Frontmatter struct with name, description, version, metadata.archie
- Parse() extracts YAML frontmatter delimited by --- markers
- Discover() scans .agents/skills/*/SKILL.md (follows wfeval pattern)
- LoadBody() for single-skill lazy loading

Skills moved from .archie/skills/ to .agents/skills/ per the
agentskills.io specification.

Wiring: TaskContext.SkillBody field, lazy-loaded on first agent stage
from worktree. When non-empty, prepended to Mission as "Follow these
project-specific guidelines" context. Hardcoded Mission templates stay
(task-specific formatting). Backward compatible: empty when no skill
found.
2026-07-22 02:23:20 +10:00
sam 8c4bb1ef1d feat: promote archie-agent to NATS-connected per-task worker
Replaces stdin/stdout subprocess transport with NATS JetStream
request-reply pattern. The Runner interface and workflow orchestration
are unchanged.

New NATSRunner (implements Runner):
- Publishes AgentRequestMessage to JetStream with X-Archie-Reply header
- Blocks on core NATS inbox subscription for response
- Same Run(ctx, workspace, req) (Result, error) contract

Rewritten archie-agent (cmd/archie-agent):
- Long-running NATS-connected process (was 28-line stdin/stdout)
- Pull-consumes from ARCHIE_TASKS with archie.agent.> filter
- Builds InProcessRunner, executes agent loop, replies to inbox
- Graceful shutdown via signal handling

Extended internal/nats:
- SubjectAgentWildcard, SubjectForAgentRequest, ReplyHeader
- Conn() accessor, NewReplyInbox(), PublishMsg() for JetStream
- Stream subjects extended to ["archie.task.>", "archie.agent.>"]

Config: "nats" added as valid agent.mode alongside inprocess/subprocess
Tests: NATSRunner round-trip with embedded NATS server + handler
2026-07-22 02:13:01 +10:00
sam bb2193e5e4 feat: add optional NATS JetStream task distribution
Adds optional NATS-based task distribution alongside the existing
SQLite flow. When [nats] is absent from config, zero behaviour change.

New internal/nats package:
- Subject constants matching PRD §2 (archie.task.{bug,feature,default,bootstrap})
- Client: Connect (stream + durable pull consumer), PublishTask
  (Nats-Msg-Id dedup), Fetch, DecodeTask
- Tests with embedded NATS server (connect/publish, dedup, subject routing)

Config: NATSConfig{URL, TokenEnv} on Config, commented-out [nats] block
in config.example.toml

Store: ClaimByIssue atomic claim by owner/repo/number for NATS consumer

Daemon: Nats client field, poll() branches to pollNATS/pollSQLite,
Cycle() branches to drainNATS/drainSQLite (NATS fetch then SQLite
ClaimNext fallback for requeued tasks), processNATSTask
(decode → EnqueueIssue → ClaimByIssue → process → Ack)

Scratch: explorer and plan agent outputs from parallel-explore phase
2026-07-22 02:02:09 +10:00
sam ac44d56965 docs: finalise v2 architecture PRD
Resolve all seven open questions with concrete answers:
- Agent image versioning: tagged versions, :latest dev-only
- NATS auth: anonymous single-node, credentials for multi-node
- Worktree ownership: answered by current impl (fresh clone per task)
- Grace period: reset on human reply, cap with max_total_uptime
- Concurrent agents: blocked by default, per-repo override
- Plugin architecture: extend existing Yaegi surfaces, don't replace
- Skill loading scope: parse frontmatter + inject into prompt for v2

Status: Draft -> Final.
2026-07-22 01:38:50 +10:00
sam 334cd78471 docs: replace all unicode characters with ASCII equivalents 2026-07-22 01:30:31 +10:00
sam adcdcb4b76 docs: resolve all seven open questions with concrete answers 2026-07-22 01:26:45 +10:00
sam 0a15d62a3d chore: update gitignore 2026-07-22 01:11:14 +10:00
sam c6372ef886 docs: reconcile v2 architecture PRD with actual implementation state
Adds an implementation-status section and annotates each section
(binary split, NATS, container sandbox, /data volume, plugin layers,
skills, task lifecycle, phases) with what's actually built today vs.
aspirational. Reconciles the plugin design with the sibling
yaegi-plugin-system.md PRD, whose three phases are already
implemented but scoped narrower than this doc's Layer 1/2 split.
2026-07-22 01:07:33 +10:00
sam c0d108a5d0 feat: add Yaegi-based plugin system (gates, custom stages, skill scripts)
Implements the full PRD (docs/prds/yaegi-plugin-system.md): repos can
now define .archie/gate.go for diff-inspecting quality rules shell
commands can't express, .archie/stages/*.go for custom pipeline steps,
and skills can ship Go scripts run via a new run_go_script agent tool —
all interpreted in-process with Yaegi, with panics recovered so a bad
script can't take down the daemon. Hot reload (phase 4) needed no code:
every loader reads from disk per call and every task gets a fresh
clone, so nothing is ever cached to go stale.
2026-07-22 00:54:46 +10:00
sam 5fc61e0bad feat: redesign architecture doc with interactive SVG 2026-07-21 23:44:48 +10:00
sam 86339b4d0d feat: add subprocess agent runner 2026-07-21 23:21:24 +10:00
sam 1f3fcb8392 chore: remove outdated references 2026-07-21 23:07:23 +10:00
sam a8d005a6f6 feat: extract agent execution boundary 2026-07-21 22:16:50 +10:00
sam 1d995c587d docs: archie-core v2 end-state architecture diagram 2026-07-21 20:05:52 +10:00
sam 276f96fec3 docs: archie-core v2 architecture design doc (daemon/agent split, NATS, plugins, skills) 2026-07-21 19:48:38 +10:00
sam 3080adf87e docs: add TypeScript plugin engine PRD (goja + esbuild) 2026-07-21 17:49:50 +10:00
sam 1e73507d80 docs: add architecture docs, skills, and Yaegi PRD 2026-07-21 17:34:38 +10:00
sam 5cb4f32a8d docs: add PRD docs and move DESIGN.md to docs/ 2026-07-21 17:30:43 +10:00
sam 865a2bab9f chore: update gitignore 2026-07-21 17:28:08 +10:00
sam 251cc7022c feat: add configurable forge and workflow behavior 2026-07-21 15:03:41 +10:00
sam 748d9aa9ad feat: per-repo protected suffixes, requeue flag, templ-aware gate config
The first feature run parked exactly as designed but for an avoidable
reason: todo is a templ project and the agent edited generated
_templ.go files, leaving literal template placeholders in rendered
HTML. Systemic fixes, all environmental:

- repos gain protect = ["_templ.go"]: suffixes agents can never write
  (agentloop.ProtectPaths), forcing edits to the sources; the gate's
  new leading ["templ","generate","./components/"] command
  regenerates. AgentStage.ProtectPaths now composes with the repo list
  and takes TaskContext like every other stage hook.
- archied -requeue <task-id>: the retry path for parked/waiting tasks
  (keeps the task's workflow; Requeue with empty workflow preserves it,
  clears the park reason).

Retry of the parked feature task drove the full loop home: PRD-guided
plan -> builder edited components.templ (generated file write-blocked),
gate regenerated + tested -> PR #8 with 109 lines of tests.
2026-07-21 04:34:53 +10:00
sam 239b8eda42 feat: feasibility workflow — assess, PRD, human go/no-go loop
Routed via the feature label. Stages: assess (read-only agent with a
structured decide tool — verdict as a tool call, not parsed prose;
not-a-fit closes the issue as won't-do with reasons) -> PRD (read-only
agent; problem, solution, surface, acceptance criteria, non-goals) ->
deliver (PRD posted as the watched issue comment, optional [notify]
webhook POST for n8n->email, task -> waiting_human).

The daemon's cycle now watches waiting_human tasks: replies after the
watch comment (excluding the bot) are judged by an LLM (triage role,
APPROVE/REJECT/UNCLEAR + reason — never keyword matching). Approvals
requeue under implement (whose planner now treats an existing PRD as
prior context); rejections close with the reasoning acknowledged;
unclear replies keep waiting.

Supporting changes: forge.Comment returns the comment id;
forge.RepliesAfter; store watch_comment_id (additive migration),
WaitingTasks, Requeue; Route honours a pre-assigned workflow;
AgentStage.Extra for stage-specific tools.

Live: todo#7 assessed fit -> PRD delivered -> waiting_human.
2026-07-21 04:27:17 +10:00
sam bb9c6f89a2 docs: document [web] dashboard section in example config 2026-07-21 04:21:02 +10:00
sam 387c033b29 feat: observability — event bus, SQLite event log, live web dashboard
Adapted from tau's eventbus/metrics designs (per Sam), slimmed for a
small daemon:

- internal/events: single-type bus with mutex fan-out and bounded
  per-subscriber buffers that DROP on overflow (counted) — a stalled
  dashboard can never backpressure the task engine (tau's blocking
  delivery deliberately not copied)
- store: events table + queries powering the dashboards: task
  timelines, per-workflow outcomes/spend, stage durations and error
  counts (from stage_finish events), token spend by day
- emit points: task_queued, stage_start/finish(duration, error),
  agent_finish(status, tokens, steps, model), parked, outcome,
  pr_merged/pr_rejected
- internal/webui: read-only dashboard (embedded single page, no build
  step) — status cards, task board with click-through timelines, live
  SSE event tail with catch-up via ?since=<event id>, workflow/stage
  metrics tables. [web] listen config, default 127.0.0.1:8484, no auth
  by design (bind localhost/tailnet); "off" disables
- sink pipeline: bus -> SQLite insert (stamps row id) -> SSE broadcast

Verified live: dashboard serves real workflow stats from the existing
task history; bus overflow and event-log round-trips unit-tested.
2026-07-21 04:20:45 +10:00
sam 94a58e11b6 feat: assignee-driven pickup + repro proof on TDD PRs
- Issues are picked up by assigning them to the bot (tink-bot style)
  instead of by label; labels now only influence workflow routing
- TDD: capture the failing 'go test' output after the repro stage
  (parking if it unexpectedly passes) and post it on the PR as a code
  block — evidence the bug was reproduced before the fix
- TDD fix stage now write-protects *_test.go via agentloop.ProtectPaths
  (environmental enforcement, replacing the advisory prompt rule)
- OpenPR extracted so stages can act after the PR exists in the same
  stage (the engine stops at the first stage with an outcome)
2026-07-21 04:10:27 +10:00
sam 5465442cb3 feat: TDD workflow for bugfixes — failing repro before fix
Routed via the bug label. Stages: analyse (read-only problem-surface
analysis) -> repro-tests behind the INVERTED gate (go vet must pass,
go test ./... must FAIL — GateCommand.ExpectFailure proves the repro
captures the bug) -> repro committed separately -> fix behind the
repo's full gate with the repro declared immutable in the rules ->
push, diff cap, PR. Two-commit PRs tell the story: test first, fix
second. Zero engine or step-library changes — the workflow is pure
composition, per the reusability requirement.

Proven live on samcharles93/todo#5 (empty todos bug): repro gate
pushed back once, fix stage blocked twice by the full gate before
green; PR #6 (~103k tokens total).
2026-07-21 04:07:21 +10:00
sam 708e55a95b feat: implement workflow — planner/builder agent stages over agentloop
AgentStage is the reusable bridge from workflow stages to agentloop
runs: role->model selection from config, budgets, repo gate conversion,
task-row NotesStore (survives worktree cleanup, never pollutes the PR),
and park-on-anything-but-passed. Every future LLM stage (tdd,
feasibility) is an AgentStage with a different mission/gate/handler.

The implement workflow: prepare -> baseline gate (park on red base,
prevents park-storms) -> plan (read-only agentloop, plan posted to the
issue as a veto window) -> build (full toolset behind the repo's
configured gate) -> commit/push -> diff cap -> PR with the builder's
finish summary as the body.

Proven live on samcharles93/todo#3: planner 3 steps, builder 4 steps
(~31k tokens total), producing gate-verified table-driven TodoStore
tests as PR #4.
2026-07-21 04:03:06 +10:00
sam 61612568d6 fix: route to a registered default workflow until implement lands
First live run exposed the gap: with only bootstrap registered, Route()
fell through to the definition-bug backstop. Label routing now falls
back to reg["default"], which archied maps to bootstrap until the
implement workflow exists.
2026-07-21 03:58:46 +10:00
sam c2c8e92304 feat: archied walking skeleton — poll, store, worktree, workflow engine, bootstrap PR
Repurposes archie-core as the archie orchestrator daemon (the old
Pocketbase PM-backend plan is archived under .references/).

- cmd/archied: -config, -once (single cycle for timers/testing)
- internal/config: TOML; per-repo gate command lists; role->model refs
- internal/store: SQLite (modernc, WAL) task state machine with
  transitions audit, idempotent enqueue, crash recovery (running ->
  queued, attempt++)
- internal/forge: go-github v78 — labelled-issue polling, invitation
  auto-accept, push verification, PR create/state, comments, close
- internal/worktree: fresh clone per task, bot identity, askpass token
  handling, diff-size stats, cleanup on merge/reject
- internal/workflow: extensible engine (stages over TaskContext, shared
  step library, label routing bug->tdd feature->feasibility default->
  implement); parks always comment on the issue. Bootstrap workflow
  (deterministic marker-file PR) proves the plumbing without LLM spend.

Store and worktree round-trips covered by tests (local bare remote).
2026-07-21 03:46:06 +10:00