Commit Graph
22 Commits
Author SHA1 Message Date
sam 6ebbd3b2cf fix: add missing LinkBranch stubs to test doubles
Fix go vet/test failures by adding LinkBranch methods to stubForge, prCapturingForge, testForge, and fakeForge. Update test data from 'sam/archie' to 'acme/widget' and state label defaults from 'archie:*' to 'agent:*'. Add Taskfile.yml for project build automation. Implement container self-network detection so spawned agents join the daemon's Docker network.
2026-07-24 13:32:50 +10:00
sam d80401d595 feat: add config overlay and shared RPC/Yaegi packages
- Introduce `-config-overlay` flag and `config.LoadOverlay()` to merge a base config with an overlay file, only overriding fields the overlay sets. This enables docker-compose to mount the base config from the host and apply only deployment-specific overrides.
- Create `internal/natsrpc` package with reusable NATS request/reply primitives (Envelope, Client, Call, RegisterAll, Respond). Migrate storerpc, worktreerpc, and forgerpc to use it, eliminating duplicated marshal/unmarshal and error handling.
- Create `internal/yaegiutil` package with shared interpreter construction (stdlib + extra symbols), generic export resolution (Resolve), and panic recovery (Safe). Migrate all Yaegi-based surfaces (gate scripts, daemon plugins, skill plugins, workflow stages, skill scripts) to use it, reducing boilerplate and centralizing panic safety.
- Simplify `config.docker.toml` to contain only overlay fields (paths, container config) and remove the duplicated repos list – the base config is now the single source of truth.
- Update docker-compose.yml to pass both -config and -config-overlay flags and mount the base config from the host.
- Add tests for LoadOverlay, DockerConfigOverlaysExampleConfig, Envelope, and yaegiutil utilities.
2026-07-24 12:57:10 +10:00
sam 242762b2fb refactor: archie:* → agent:* labels, remove stale archie label 2026-07-24 12:09:38 +10:00
sam e4368f70af feat: persistent volumes with TTL for container/worktree storage (archie-core-j7v)
Add [containers].volume_ttl (default 72h). Persistent-storage repos get a
per-repo bare Git cache (--reference-if-able --dissociate) so task
worktrees clone fast without repeated full network clones, while staying
fully independent of the cache's lifecycle — cache expiry can never
corrupt an active or parked task's worktree. Docker repo volumes are
labeled as Archie-owned and swept non-forced (a volume still attached to a
running container survives and retries next cycle); the repo cache
directory is swept by mtime. Both cleanups run at daemon startup and each
poll cycle.

Filed archie-core-756 (repo-wide golangci-lint baseline) and archie-core-340
(intermittent JetStream TempDir teardown flake) as separate follow-ups
rather than scope-creeping them into this change.
2026-07-24 10:22:11 +10:00
sam 1eee983681 feat: archie-agent runs a full task's workflow itself (archie-core-5zh.7)
Add internal/taskrun (Request/Response wire types) and
cmd/archie-agent/taskrun.go: runTask builds a workflow.Registry from the
container's own bind-mounted worktree, routes and runs the task's entire
workflow via workflow.Route/workflow.Run, using storerpc.Client/
forgerpc.Client for Store/Forge and a new hybridTrees that splits
Prepare/Push (proxied over worktreerpc — network ops needing the daemon's
credential) from CommitAll/Diff/ChangedFiles/ChangedLines (local git
against the mount).

Add config.TaskConfig.ToConfig() to rebuild a usable Config from the
curated payload, and storage.WorktreeMountDir (/data/worktree) — critical
fix: hybridTrees.Prepare discards the RPC response's host-side directory
and returns the container's fixed local mount path instead, since archied
and archie-agent see the same files at different paths.

Wired a queue-grouped core-NATS subscription on archie.taskrun.> in
cmd/archie-agent/main.go alongside the existing per-stage consumer, so
this is reachable in the running binary. End-to-end test drives the real
"bootstrap" workflow (no LLM) through real storerpc/forgerpc/worktreerpc
servers and a real local git remote — registry build, route, prepare,
commit, push, PR creation, and confirms the authoritative task state
lands in archied's store via RPC, not just in-memory.

archie-core-5zh.8 (daemon.go handoff) is unblocked.
2026-07-24 09:08:19 +10:00
sam 159fbce18f feat: curated TaskConfig payload for archie-agent (archie-core-5zh.5)
Add config.TaskConfig/TaskForge and Config.ForTask(): a non-secret,
explicit-allowlist snapshot (Models, Budgets, Dispatch, DiffCapLines,
Notify, Forge.Host) of the fields workflow stages read from tc.Cfg.
Forge.TokenEnv is excluded even though it's just a var name, not a secret.
Maps are deep-copied to avoid aliasing the daemon's live config. Duration
gains JSON marshaling (string form, matching TOML); config.Repo gets full
json tags.

Groundwork for archie-core-5zh — not yet wired into workflow.TaskContext.
2026-07-24 08:53:15 +10:00
sam b6b23bbb59 feat: per-repo allow_concurrent override for same-repo dispatch (archie-core-g1l)
Add config.Repo.AllowConcurrent (allow_concurrent toml key). Repos opted in
skip the daemon's default FIFO same-repo task serialization while still
respecting [containers].max_concurrency globally. Worktrees are already keyed
per issue number, so opted-in repos don't collide on disk.
2026-07-24 07:47:10 +10:00
sam f8df051b19 feat: bounded concurrent task dispatch in daemon (archie-core-u64)
Daemon previously processed one task at a time. Add bounded concurrent
dispatch for SQLite and NATS agent modes, reusing [containers].max_concurrency
(0 = unlimited). Same-repository tasks stay FIFO-serialized to avoid worktree
conflicts; different repositories now run concurrently.

Unblocks archie-core-l1q (NATS transport) and archie-core-g1l
(per-repo allow_concurrent override).
2026-07-24 07:42:13 +10:00
archie d2b2f29096 Implement max_retries cap on maybeRetryParked to prevent infinite parked-task loops (archie)
Implements #61
2026-07-23 18:49:38 +10:00
sam edd05aea6b feat: add Gitea forge support alongside GitHub
Implements the Forge interface for Gitea instances (tested against
git.catlow.cloud, Gitea v1.26.4). The forge.New factory now dispatches
on forge type, and config validation accepts 'gitea'.

Changes:
- internal/forge/gitea.go — GiteaClient implementing all 11 Forge
  methods: issue polling (assigned + label), comments, PR creation,
  PR state reconciliation, issue closure, reactions, state labels,
  push verification. Handles edge cases: not-found PRs return
  'closed', GetUserByName errors from the Gitea SDK are mapped to
  user-friendly messages.
- internal/forge/forge.go — New() factory dispatching on forge type
  (github/gitea). GitHub constructor renamed to NewGitHub.
- internal/forge/github_test.go — updated to use New() signature
- internal/config/config.go — forge.type accepts 'gitea', BotEmail
  defaults change per forge type (gitea uses @gitea.local)
- cmd/archied/main.go — passes cfg.Forge.Type to forge.New()

Dependencies: added code.gitea.io/sdk/gitea v0.25.1

End-to-end verified against live Gitea instance: daemon starts clean,
registry built (5 workflows), web UI listening, no errors or warnings.
2026-07-22 05:15:09 +10:00
sam 7dbb17a4cb feat: pluggable storage backends with Docker volumes and /data/ layout
Introduces the storage.Backend interface for pluggable container mount
management. The Docker implementation (DockerBackend) is the MVP —
no NFS/SMB/S3, just Docker volumes + bind mounts.

/data/ volume layout:
  /data/worktree/  — bind mount of host worktree (always)
  /data/repo/      — per-repo Docker volume (when persistent_storage = true)
  /data/cache/go/  — shared GOMODCACHE volume
  /data/cache/node/— shared npm cache volume
  /data/cache/pnpm/— shared pnpm store volume
  /data/cache/pip/ — shared pip cache volume
  /data/cache/cargo/— shared Rust cargo cache volume
  /data/task.json  — boot-time brief (existing WriteTaskJSON)

Changes:
- internal/storage/: Backend interface, DockerBackend, Mount types,
  ecosystem-to-cache-volume mapping, ConvertMounts helper
- container.Pool.Acquire: now accepts []storage.Mount instead of
  hardcoding a single bind mount — mount list comes from the storage
  backend
- daemon.process(): calls d.Storage.Setup() to build the mount list,
  passes Ecosystem + PersistentStorage from repo config
- config.Repo.PersistentStorage: when true, a named Docker volume
  (archie-repo-<owner>-<repo>) is mounted at /data/repo and survives
  task completion
- main.go: creates DockerBackend when containers are enabled

21 tests: interface contract, ecosystem cache mounts (go/node/
python/rust/custom), worktree bind mount ordering, persistent
storage on/off/empty-owner guard, mount list sort+dedup, adversarial
(nonexistent worktree, empty ecosystem, unknown ecosystem).

Why: daemon-managed cache volumes mean warm GOMODCACHE/npm cache
across tasks. Per-repo persistent volumes preserve expensive build
artifacts. The interface leaves room for future backends without
rewiring container orchestration.
2026-07-22 04:28:30 +10:00
sam f38ba5d515 feat: wire Layer 2 daemon plugin loader into archied startup
Daemon plugins (Yaegi-interpreted .go files) are now loaded from the
configured plugin_dir at startup. Each file must export a 'Plugin'
variable using the _Plugin wrapper struct, which bridges Yaegi types
to the Go plugin.Plugin interface.

Changes:
- internal/plugin/pluginextract/ — generated Yaegi symbol table for
  the plugin package (mirrors wfextract pattern), including the
  _Plugin interface wrapper that lets interpreted code implement
  Go interfaces
- LoadDir now accepts optional symbol tables via variadic parameter,
  avoiding an import cycle between plugin and pluginextract
- Plugin tests rewritten as behavioral (not structural), verifying
  actual Yaegi interpretation of valid, invalid, and missing plugins
- config.PluginDir field for specifying the plugin directory
- daemon.PluginRegistry field for holding loaded plugins
- main.go wires plugin loading: LoadDir at startup, register into
  daemon, log each loaded plugin

Why: the Plugin interface and LoadDir existed but weren't connected
to the daemon. Skills define workflows; daemon plugins are the
extension point for forge resolvers, storage backends, and other
core-daemon concerns.
2026-07-22 04:11:02 +10:00
sam 8958a82233 feat: replace hardcoded workflow registry with catalog-driven skillbuild.BuildRegistry()
The daemon no longer hardcodes workflow.Registry{...} in main.go.
Instead, skillbuild.BuildRegistry() scans .agents/skills/ for catalog
entries declaring metadata.archie.workflow and builds each workflow
from its Yaegi-interpreted stage plugins.

Built-in workflows (bootstrap, implement, tdd, feasibility, default)
serve as fallbacks when no skill declares a given workflow name. A
skill that declares metadata.archie.workflow: implement overrides the
built-in — skills define workflows, plugins define stages.

Why: skills must be extension primitives. Adding a workflow should
mean creating .agents/skills/archie-wf-<name>/plugins/*.go, not
modifying Go code in main.go. The catalog is the single source of
truth for what workflows exist.
2026-07-22 04:02:58 +10:00
sam de57f72cce feat: add Docker container sandbox for archie-agent
Dockerfile: multi-stage fat image (golang:1.26 builder → ubuntu:24.04)
with Go 1.26, Python 3.12/3.13 via deadsnakes, UV, Node 22, pnpm,
TypeScript, esbuild, git, gh, golangci-lint, gofumpt.

Container pool (internal/container): manages Docker container lifecycle
via moby/moby client. Per-task containers with bind-mounted workspaces.

- Acquire: create, start, mount workspace, inject NATS + API key env
- Release: stop, remove (AutoRemove as safety net)
- Orphan recovery on daemon startup (label: archie-daemon=true)
- MaxConcurrency enforcement, MaxUptime config (unused in MVP)
- Image pull with "missing"/"always" policy

Config: [containers] block (enabled, image, max_concurrency, max_uptime,
pull_policy). Validation: requires agent.mode = "nats" and [nats] url.

Daemon: acquires container before workflow.Run, releases after.
Container environment auto-populated from provider API keys.
2026-07-22 02:39:17 +10:00
sam 8c4bb1ef1d feat: promote archie-agent to NATS-connected per-task worker
Replaces stdin/stdout subprocess transport with NATS JetStream
request-reply pattern. The Runner interface and workflow orchestration
are unchanged.

New NATSRunner (implements Runner):
- Publishes AgentRequestMessage to JetStream with X-Archie-Reply header
- Blocks on core NATS inbox subscription for response
- Same Run(ctx, workspace, req) (Result, error) contract

Rewritten archie-agent (cmd/archie-agent):
- Long-running NATS-connected process (was 28-line stdin/stdout)
- Pull-consumes from ARCHIE_TASKS with archie.agent.> filter
- Builds InProcessRunner, executes agent loop, replies to inbox
- Graceful shutdown via signal handling

Extended internal/nats:
- SubjectAgentWildcard, SubjectForAgentRequest, ReplyHeader
- Conn() accessor, NewReplyInbox(), PublishMsg() for JetStream
- Stream subjects extended to ["archie.task.>", "archie.agent.>"]

Config: "nats" added as valid agent.mode alongside inprocess/subprocess
Tests: NATSRunner round-trip with embedded NATS server + handler
2026-07-22 02:13:01 +10:00
sam bb2193e5e4 feat: add optional NATS JetStream task distribution
Adds optional NATS-based task distribution alongside the existing
SQLite flow. When [nats] is absent from config, zero behaviour change.

New internal/nats package:
- Subject constants matching PRD §2 (archie.task.{bug,feature,default,bootstrap})
- Client: Connect (stream + durable pull consumer), PublishTask
  (Nats-Msg-Id dedup), Fetch, DecodeTask
- Tests with embedded NATS server (connect/publish, dedup, subject routing)

Config: NATSConfig{URL, TokenEnv} on Config, commented-out [nats] block
in config.example.toml

Store: ClaimByIssue atomic claim by owner/repo/number for NATS consumer

Daemon: Nats client field, poll() branches to pollNATS/pollSQLite,
Cycle() branches to drainNATS/drainSQLite (NATS fetch then SQLite
ClaimNext fallback for requeued tasks), processNATSTask
(decode → EnqueueIssue → ClaimByIssue → process → Ack)

Scratch: explorer and plan agent outputs from parallel-explore phase
2026-07-22 02:02:09 +10:00
sam 86339b4d0d feat: add subprocess agent runner 2026-07-21 23:21:24 +10:00
sam 251cc7022c feat: add configurable forge and workflow behavior 2026-07-21 15:03:41 +10:00
sam 748d9aa9ad feat: per-repo protected suffixes, requeue flag, templ-aware gate config
The first feature run parked exactly as designed but for an avoidable
reason: todo is a templ project and the agent edited generated
_templ.go files, leaving literal template placeholders in rendered
HTML. Systemic fixes, all environmental:

- repos gain protect = ["_templ.go"]: suffixes agents can never write
  (agentloop.ProtectPaths), forcing edits to the sources; the gate's
  new leading ["templ","generate","./components/"] command
  regenerates. AgentStage.ProtectPaths now composes with the repo list
  and takes TaskContext like every other stage hook.
- archied -requeue <task-id>: the retry path for parked/waiting tasks
  (keeps the task's workflow; Requeue with empty workflow preserves it,
  clears the park reason).

Retry of the parked feature task drove the full loop home: PRD-guided
plan -> builder edited components.templ (generated file write-blocked),
gate regenerated + tested -> PR #8 with 109 lines of tests.
2026-07-21 04:34:53 +10:00
sam 239b8eda42 feat: feasibility workflow — assess, PRD, human go/no-go loop
Routed via the feature label. Stages: assess (read-only agent with a
structured decide tool — verdict as a tool call, not parsed prose;
not-a-fit closes the issue as won't-do with reasons) -> PRD (read-only
agent; problem, solution, surface, acceptance criteria, non-goals) ->
deliver (PRD posted as the watched issue comment, optional [notify]
webhook POST for n8n->email, task -> waiting_human).

The daemon's cycle now watches waiting_human tasks: replies after the
watch comment (excluding the bot) are judged by an LLM (triage role,
APPROVE/REJECT/UNCLEAR + reason — never keyword matching). Approvals
requeue under implement (whose planner now treats an existing PRD as
prior context); rejections close with the reasoning acknowledged;
unclear replies keep waiting.

Supporting changes: forge.Comment returns the comment id;
forge.RepliesAfter; store watch_comment_id (additive migration),
WaitingTasks, Requeue; Route honours a pre-assigned workflow;
AgentStage.Extra for stage-specific tools.

Live: todo#7 assessed fit -> PRD delivered -> waiting_human.
2026-07-21 04:27:17 +10:00
sam 387c033b29 feat: observability — event bus, SQLite event log, live web dashboard
Adapted from tau's eventbus/metrics designs (per Sam), slimmed for a
small daemon:

- internal/events: single-type bus with mutex fan-out and bounded
  per-subscriber buffers that DROP on overflow (counted) — a stalled
  dashboard can never backpressure the task engine (tau's blocking
  delivery deliberately not copied)
- store: events table + queries powering the dashboards: task
  timelines, per-workflow outcomes/spend, stage durations and error
  counts (from stage_finish events), token spend by day
- emit points: task_queued, stage_start/finish(duration, error),
  agent_finish(status, tokens, steps, model), parked, outcome,
  pr_merged/pr_rejected
- internal/webui: read-only dashboard (embedded single page, no build
  step) — status cards, task board with click-through timelines, live
  SSE event tail with catch-up via ?since=<event id>, workflow/stage
  metrics tables. [web] listen config, default 127.0.0.1:8484, no auth
  by design (bind localhost/tailnet); "off" disables
- sink pipeline: bus -> SQLite insert (stamps row id) -> SSE broadcast

Verified live: dashboard serves real workflow stats from the existing
task history; bus overflow and event-log round-trips unit-tested.
2026-07-21 04:20:45 +10:00
sam c2c8e92304 feat: archied walking skeleton — poll, store, worktree, workflow engine, bootstrap PR
Repurposes archie-core as the archie orchestrator daemon (the old
Pocketbase PM-backend plan is archived under .references/).

- cmd/archied: -config, -once (single cycle for timers/testing)
- internal/config: TOML; per-repo gate command lists; role->model refs
- internal/store: SQLite (modernc, WAL) task state machine with
  transitions audit, idempotent enqueue, crash recovery (running ->
  queued, attempt++)
- internal/forge: go-github v78 — labelled-issue polling, invitation
  auto-accept, push verification, PR create/state, comments, close
- internal/worktree: fresh clone per task, bot identity, askpass token
  handling, diff-size stats, cleanup on merge/reject
- internal/workflow: extensible engine (stages over TaskContext, shared
  step library, label routing bug->tdd feature->feasibility default->
  implement); parks always comment on the issue. Bootstrap workflow
  (deterministic marker-file PR) proves the plumbing without LLM spend.

Store and worktree round-trips covered by tests (local bare remote).
2026-07-21 03:46:06 +10:00