196 lines
9.2 KiB
Markdown
196 lines
9.2 KiB
Markdown
# archie-core Architecture
|
|
|
|
**archied** is a resident daemon that polls GitHub for issues labelled `archie`, works each one in an isolated worktree through a routed workflow, and opens pull requests for human review.
|
|
|
|
## Package Map
|
|
|
|
| Package | Role |
|
|
|---|---|
|
|
| `cmd/archied/` | Binary entry: config→forge→store→runtime→workflows→daemon |
|
|
| `cmd/archie-agent/` | One-invocation autonomous stage worker (JSON stdin/stdout) |
|
|
| `internal/config/` | TOML config, per-repo gates, budgets, providers, ecosystems |
|
|
| `internal/daemon/` | Resident loop: poll, enqueue, claim, route, process, reconcile |
|
|
| `internal/workflow/` | Engine: stage lists, routing, shared steps, workflow definitions |
|
|
| `internal/agentexec/` | Versioned agent-stage protocol, worker, and in-process/subprocess runners |
|
|
| `internal/forge/` | GitHub interface: issues, labels, PRs, comments, reactions |
|
|
| `internal/worktree/` | Git operations: clone, branch, commit, push, diff, cleanup |
|
|
| `internal/store/` | SQLite: task rows, status transitions, crash recovery |
|
|
| `internal/events/` | In-process event bus: publish/subscribe for observability |
|
|
| `internal/webui/` | SSE dashboard: live event stream, task status |
|
|
| `ai-sdk/runtime` | External: provider-agnostic LLM runtime |
|
|
| `ai-sdk/agentloop` | External: gated agent loop with budgets and tool jails |
|
|
| `ai-sdk/core` | External: tool definitions, generate options |
|
|
|
|
## Task Lifecycle
|
|
|
|
```
|
|
queued → running(workflow:stage) → pr_open → merged | rejected
|
|
↓
|
|
waiting_human → (human approves) → queued(implement)
|
|
↓
|
|
parked
|
|
```
|
|
|
|
**Crash recovery:** tasks left in `running` are re-queued on startup. Parks are never silent — every park posts a comment with the reason and gate output.
|
|
|
|
**PR reconciliation:** the daemon polls open PRs, checks GitHub state (`merged`/`closed`), transitions the task, and cleans up worktrees.
|
|
|
|
## Workflow Engine
|
|
|
|
### Core types (`workflow.go`)
|
|
|
|
- `Workflow` = named list of `Stage`s
|
|
- `Stage` = `Name` + `Run(ctx, *TaskContext) error`
|
|
- `TaskContext` = mutable bag: Task, Repo, Config, Forge, Store, Agent runner, worktree dir/branch, scratch fields
|
|
- `Outcome` = terminal decision (status + detail) — ends the workflow immediately
|
|
- `Registry` = `map[string]Workflow`
|
|
|
|
### Routing (`Route`)
|
|
|
|
Label-driven, no LLM involved. Pre-assigned workflow wins; then labels decide:
|
|
- `bug` → `tdd`
|
|
- `feature` → `feasibility`
|
|
- `bootstrap` → `bootstrap` (diagnostics, no LLM spend)
|
|
- Default → `implement`
|
|
|
|
### Execution (`Run`)
|
|
|
|
Stages execute sequentially. After each stage:
|
|
- Task is persisted to SQLite
|
|
- Error → park the task (comment on issue, keep worktree)
|
|
- Outcome set → finish (transition task, update labels)
|
|
- Neither → continue to next stage
|
|
|
|
A workflow that ends without an outcome parks — definition bugs must not vanish silently.
|
|
|
|
## Workflows
|
|
|
|
### Bootstrap (diagnostics, no LLM)
|
|
|
|
```
|
|
prepare → apply(marker file) → commit-push → diff-cap → open-pr
|
|
```
|
|
|
|
Proves the plumbing: clone, push, PR. Label any test issue `bootstrap` to verify end-to-end.
|
|
|
|
### Implement (default)
|
|
|
|
```
|
|
prepare → baseline-gate → plan(planner, read-only) → build(builder, gated) → commit-push → diff-cap → open-pr
|
|
```
|
|
|
|
- **baseline-gate:** runs the repo's gate at the base commit before anything is touched
|
|
- **plan:** read-only agent explores codebase, produces implementation plan, posts it as a comment
|
|
- **build:** full toolset agent implements against the plan behind the repo's quality gate
|
|
- **diff-cap:** parks oversized changes (>400 lines default)
|
|
- **open-pr:** builder's summary becomes the PR body
|
|
|
|
### TDD (bug label)
|
|
|
|
```
|
|
prepare → baseline-gate → analyse(planner) → repro-tests(builder, inverted gate) → capture-proof → commit-repro → fix(builder, normal gate, test files protected) → commit-push → diff-cap → open-pr + repro-evidence
|
|
```
|
|
|
|
- **repro-tests:** the quality gate is **inverted** — the test command must FAIL. Proven failure is required.
|
|
- **capture-proof:** runs the test command deterministically, captures the failing output
|
|
- **commit-repro:** commits the failing tests as the first commit
|
|
- **fix:** test files are **environmentally write-protected** (not prompt-ruled). The agent fixes code, not tests.
|
|
- **open-pr + evidence:** posts both the PR and the captured failure proof
|
|
|
|
### Feasibility (feature label)
|
|
|
|
```
|
|
prepare → assess(planner, with decide tool) → [close_if_bad_fit | prd(planner)] → deliver → waiting_human
|
|
```
|
|
|
|
- **assess:** reads AGENT.md, ROADMAP.md, README. Calls a `decide` tool (structured output, not parsed prose). Bad fit → close with reasons.
|
|
- **prd:** read-only agent writes a PRD: problem, solution, files affected, acceptance criteria, non-goals, diff estimate
|
|
- **deliver:** posts PRD as issue comment + webhook notification (n8n → email). Sets state to `waiting_human`.
|
|
- The daemon watches for human replies, judges them with a triage LLM (APPROVE/REJECT/UNCLEAR), requeues or closes.
|
|
|
|
## Forge Interface
|
|
|
|
A thin abstraction over GitHub (Gitea planned):
|
|
|
|
| Method | Purpose |
|
|
|---|---|
|
|
| `AcceptInvitations` | Auto-accept repo invites |
|
|
| `AssignedIssues` | Poll assigned issues (excluding PRs) |
|
|
| `IssuesWithLabel` | Poll labelled issues |
|
|
| `Comment` | Post issue/PR comment, return ID |
|
|
| `RepliesAfter` | Find human replies after a comment ID |
|
|
| `CreatePR` | Open pull request |
|
|
| `PRState` | Check merged/open/closed |
|
|
| `CloseIssue` | Close with final comment |
|
|
| `React` | Emoji reaction (pickup acknowledgement) |
|
|
| `SetStateLabel` | Replace old state label with new one |
|
|
| `VerifyPush` | Confirm push permission at startup |
|
|
|
|
State labels (`archie:queued`, `archie:working`, etc.) are created on demand, removed when transitioning.
|
|
|
|
## Worktree Manager
|
|
|
|
Every task gets a fresh clone — no persistent state between runs. The model never runs git.
|
|
|
|
```
|
|
~/.local/share/archie/work/<owner>-<repo>/issue-<N>/
|
|
```
|
|
|
|
Token auth via a generated `GIT_ASKPASS` helper script — the token never appears in `.git/config` or process argv.
|
|
|
|
## Config
|
|
|
|
Daemon-level TOML (`~/.config/archie/config.toml`):
|
|
|
|
- `[forge]` — type (github), host, token env var
|
|
- `[dispatch]` — trigger (assignee/label/either), labels, ack reaction
|
|
- `[[repos]]` — owner, name, base branch, gate commands, protected paths, ecosystem
|
|
- `[models]` — planner, builder, triage (provider/model refs)
|
|
- `[providers]` — LLM provider configs
|
|
- `[agent]` — execution mode (`inprocess` or `subprocess`), worker command, environment allowlist
|
|
- `[budgets]` — max steps, max tokens, wall clock, gate max failures
|
|
- `[web]` — dashboard listen address
|
|
- `[notify]` — webhook URL for notifications
|
|
|
|
Subprocess mode is a migration transport boundary, not a security sandbox: the worker still runs under the daemon UID and can access daemon-readable host resources. The "daemon never runs untrusted code" boundary requires the later container runner with a separate user, a restricted filesystem and process namespace, and explicit mounts.
|
|
|
|
Per-repo gates are command lists: `[[repos.gate]] = ["go", "vet", "./..."]`. The last command is the test runner (by convention — TDD inverts it).
|
|
|
|
## Observability
|
|
|
|
Every lifecycle event flows through an internal event bus:
|
|
- SQLite event log (every event stamped with a row ID)
|
|
- Live dashboard via SSE (server-sent events)
|
|
- Bounded subscriber buffers — slow consumers drop events, never block the daemon
|
|
|
|
## Key Design Decisions
|
|
|
|
1. **Environmental constraints over prompt rules.** The gate, test-file protection, and diff cap are code enforcement — not "please don't do that" in the system prompt.
|
|
2. **The model never runs git.** Worktree operations are deterministic steps, not agent tools.
|
|
3. **Tokens as cost, not features.** Token usage is tracked per task and emitted in observability events but never budgeted beyond the agent loop's own limits.
|
|
4. **Label-driven routing.** Workflow selection is a switch statement, not an LLM call. A triage LLM is used only for the human reply judgment in waiting_human.
|
|
5. **Shared code with tau at the module level.** `ai-sdk/agentloop` and `ai-sdk/toolkit` are extracted from tau's coordinator and promoted to shared libraries.
|
|
6. **No daemon-internal cron.** Polling is a ticker loop, not cron. The feasibility PRD notification uses an external webhook (n8n).
|
|
7. **Fresh worktree per task.** No persistent state between runs. Crash recovery just re-queues.
|
|
8. **Agent execution is a data boundary.** Autonomous stages receive a versioned request and return a fenced result; SQLite, forge operations, git, workflow sequencing, and outcome handling stay daemon-owned.
|
|
|
|
## What's Implemented vs Planned
|
|
|
|
**Implemented:**
|
|
- Daemon poll loop, enqueue, claim, process
|
|
- Bootstrap, implement, TDD, feasibility workflows
|
|
- GitHub forge (issues, PRs, labels, comments, reactions)
|
|
- Worktree isolation (clone, branch, commit, push, diff, cleanup)
|
|
- SQLite store with transition audit
|
|
- Config with ecosystem support (go, python, node, rust)
|
|
- Agent stages with gates, budgets, preflight, protected paths
|
|
- Observability (event bus, SQLite log, SSE dashboard)
|
|
- Crash recovery (re-queue stale running tasks)
|
|
- Human reply judgment (triage LLM for waiting_human)
|
|
|
|
**Planned:**
|
|
- Gitea forge implementation
|
|
- More ecosystems
|
|
- Notification integrations beyond n8n webhook
|
|
- Skills (SKILL.md support — see `.archie/skills/`)
|