docs: add architecture docs, skills, and Yaegi PRD
This commit is contained in:
1 parent
5cb4f32a8d
commit
1e73507d80
7 files changed
+825
No files matched your search
@@ -0,0 +1,85 @@
|
||||
---
|
||||
name: ecosystem-node
|
||||
description: >
|
||||
Node.js/TypeScript project conventions for archie-core: npm/pnpm, eslint,
|
||||
prettier, tsc, vitest. Use when working on Node repositories with archie-core.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [node, npm, pnpm, vitest, eslint, prettier]
|
||||
engine: any
|
||||
---
|
||||
# Node.js / TypeScript Ecosystem Conventions
|
||||
|
||||
## Preflight
|
||||
|
||||
When archie-core's `[[repos]]` entry sets `ecosystem = "node"`, the daemon runs
|
||||
`node --version` as the preflight check before any agent stage.
|
||||
|
||||
## Recommended gate configuration
|
||||
|
||||
```toml
|
||||
[[repos]]
|
||||
owner = "sam"
|
||||
name = "my-node-project"
|
||||
ecosystem = "node"
|
||||
|
||||
[[repos.gate]] = ["pnpm", "run", "lint"]
|
||||
[[repos.gate]] = ["pnpm", "run", "typecheck"]
|
||||
[[repos.gate]] = ["pnpm", "run", "test"]
|
||||
```
|
||||
|
||||
Or for npm projects:
|
||||
|
||||
```toml
|
||||
[[repos.gate]] = ["npm", "run", "lint"]
|
||||
[[repos.gate]] = ["npm", "run", "build", "--", "--noEmit"]
|
||||
[[repos.gate]] = ["npm", "test"]
|
||||
```
|
||||
|
||||
## Test glob
|
||||
|
||||
The default test glob for Node is `*.test.ts`. The TDD workflow's repro-tests
|
||||
stage write-protects files matching this glob during the fix stage.
|
||||
|
||||
Override with `test_glob = "**/*.spec.ts"` for projects using that convention,
|
||||
or `test_glob = "__tests__/**/*.ts"` for Jest-style test directories.
|
||||
|
||||
## Package management
|
||||
|
||||
arcie-core detects the lockfile:
|
||||
- `pnpm-lock.yaml` → use `pnpm`
|
||||
- `package-lock.json` → use `npm`
|
||||
- `yarn.lock` → use `yarn`
|
||||
|
||||
Install before gate:
|
||||
```bash
|
||||
pnpm install --frozen-lockfile
|
||||
```
|
||||
|
||||
## Common tools
|
||||
|
||||
| Tool | Purpose | Config file |
|
||||
|---|---|---|
|
||||
| `eslint` | Linting | `eslint.config.mjs` |
|
||||
| `prettier` | Formatting | `.prettierrc` |
|
||||
| `tsc --noEmit` | Type checking | `tsconfig.json` |
|
||||
| `vitest` | Testing | `vitest.config.ts` |
|
||||
| `jest` | Testing (legacy) | `jest.config.ts` |
|
||||
|
||||
## Common gate failures
|
||||
|
||||
### eslint
|
||||
- `no-unused-vars`: remove unused import/variable
|
||||
- `@typescript-eslint/no-explicit-any`: add proper type
|
||||
- Import order: let eslint fix it with `--fix`
|
||||
|
||||
### tsc / typecheck
|
||||
- `TS2345`: type mismatch — fix the argument or the signature
|
||||
- `TS2339`: property doesn't exist — check the type definition
|
||||
- `Cannot find module`: check the import path or install the package
|
||||
|
||||
### vitest / jest
|
||||
- Test timeout: the test is hanging — check async/await, mocks, or network calls
|
||||
- Snapshot mismatch: the output changed — update the snapshot if the change is intentional
|
||||
- Assertion failure: fix the code, not the test
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
name: ecosystem-python
|
||||
description: >
|
||||
Python project conventions for archie-core: venv, pytest, ruff, mypy, pip-audit.
|
||||
Use when working on Python repositories with archie-core.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [python, pytest, ruff, mypy, pip-audit]
|
||||
engine: any
|
||||
---
|
||||
# Python Ecosystem Conventions
|
||||
|
||||
## Preflight
|
||||
|
||||
When archie-core's `[[repos]]` entry sets `ecosystem = "python"`, the daemon runs
|
||||
`python --version` as the preflight check before any agent stage.
|
||||
|
||||
## Recommended gate configuration
|
||||
|
||||
```toml
|
||||
[[repos]]
|
||||
owner = "sam"
|
||||
name = "my-python-project"
|
||||
ecosystem = "python"
|
||||
|
||||
[[repos.gate]] = ["ruff", "check", "."]
|
||||
[[repos.gate]] = ["ruff", "format", "--check", "."]
|
||||
[[repos.gate]] = ["mypy", "."]
|
||||
[[repos.gate]] = ["pytest", "-x", "--tb=short"]
|
||||
```
|
||||
|
||||
## Test glob
|
||||
|
||||
The default test glob for Python is `test_*.py`. The TDD workflow's repro-tests
|
||||
stage write-protects files matching this glob during the fix stage.
|
||||
|
||||
Override with `test_glob = "tests/**/*.py"` in the repo config for projects
|
||||
with test files outside the package root.
|
||||
|
||||
## Package management
|
||||
|
||||
arcie-core assumes:
|
||||
- `requirements.txt` or `pyproject.toml` at the repo root
|
||||
- Tests run with `pytest` (the ecosystem default)
|
||||
- Linting with `ruff` (recommended) or `flake8`
|
||||
- Type checking with `mypy`
|
||||
- Security auditing with `pip-audit`
|
||||
|
||||
## Virtual environment
|
||||
|
||||
Before the gate runs, ensure dependencies are installed:
|
||||
```bash
|
||||
python -m venv .venv
|
||||
. .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
pip install pytest ruff mypy pip-audit
|
||||
```
|
||||
|
||||
For `pyproject.toml` projects:
|
||||
```bash
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
## Common gate failures
|
||||
|
||||
### ruff
|
||||
- `F401`: unused import — remove it
|
||||
- `E501`: line too long — break the line
|
||||
- `I001`: import order — let ruff fix it with `ruff check --fix`
|
||||
|
||||
### mypy
|
||||
- Missing type annotation on public function — add it
|
||||
- `Cannot determine type` — add explicit annotation
|
||||
- Import errors — check the package is installed in the venv
|
||||
|
||||
### pytest
|
||||
- Test discovery failed — check `__init__.py` files and test naming
|
||||
- Import errors — verify the package is installed (`pip install -e .`)
|
||||
- Assertion failures — fix the code, not the test
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
name: go-quality-gate
|
||||
description: >
|
||||
Run the full Go quality pipeline: gofumpt, go fix, golangci-lint, go test -race,
|
||||
and task deadcode. Use before any commit in a Go project. Triggered automatically
|
||||
by archie-core's implement and tdd workflows via the per-repo gate config.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [go, golangci-lint, gofumpt, task]
|
||||
engine: any
|
||||
---
|
||||
# Go Quality Gate
|
||||
|
||||
## When this runs
|
||||
|
||||
arcie-core enforces this gate after every builder agent stage. The specific commands
|
||||
come from the repository's `[[repos.gate]]` config, not this skill. This skill documents
|
||||
what a correct gate looks like and how to diagnose failures.
|
||||
|
||||
## Standard gate (Tau and similar Go projects)
|
||||
|
||||
Run in this exact order. archie-core does NOT run these — the `ai-sdk/agentloop` gate
|
||||
does. This list is what you should configure in `config.toml`:
|
||||
|
||||
```toml
|
||||
[[repos.gate]] = ["gofumpt", "-w", "."]
|
||||
[[repos.gate]] = ["go", "fix", "./..."]
|
||||
[[repos.gate]] = ["golangci-lint", "run", "./..."]
|
||||
[[repos.gate]] = ["go", "test", "-race", "-count=1", "./..."]
|
||||
```
|
||||
|
||||
Or if the project uses Task:
|
||||
|
||||
```toml
|
||||
[[repos.gate]] = ["task", "check"]
|
||||
```
|
||||
|
||||
The LAST command in the gate list is the test runner by convention. archie-core's
|
||||
TDD workflow inverts only that last command during the repro stage.
|
||||
|
||||
## Diagnosing gate failures
|
||||
|
||||
### gofumpt / go fix
|
||||
These mutate files. If they change anything, the next `git diff` will catch it.
|
||||
The agent should re-run them after making changes.
|
||||
|
||||
### golangci-lint
|
||||
Common failures and fixes:
|
||||
- `unused` — remove the unused function or add `//nolint:unused`
|
||||
- `errcheck` — handle the error or assign to `_`
|
||||
- `gosec` — review the security finding; never blindly suppress
|
||||
|
||||
### go test -race
|
||||
Data races are non-deterministic. If `-race` fails:
|
||||
1. Identify the race with `go test -race -run=TestName`
|
||||
2. Add a mutex or channel to protect the shared state
|
||||
3. Re-run with `-count=5` to confirm it's gone
|
||||
|
||||
### task deadcode
|
||||
Scoping rule: if deadcode fails on packages you didn't touch, scope it:
|
||||
```bash
|
||||
golangci-lint run --tests=false --enable-only=unused,staticcheck ./changed/pkg/...
|
||||
```
|
||||
|
||||
## When the TDD workflow runs
|
||||
|
||||
During `repro-tests`, the test command's expectations are INVERTED — it must FAIL.
|
||||
During `fix`, the full gate runs normally and test files are write-protected.
|
||||
The agent cannot modify tests in the fix stage; it must fix the code to make them pass.
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
name: security-audit
|
||||
description: >
|
||||
Scan code for security vulnerabilities: hardcoded secrets, injection vectors,
|
||||
unsafe deserialization, missing auth checks, and path traversal. Use on PRs
|
||||
touching auth, API endpoints, or data handling code.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [golangci-lint, trivy, gitleaks]
|
||||
engine: any
|
||||
---
|
||||
# Security Audit
|
||||
|
||||
## When to use
|
||||
|
||||
Run this audit when:
|
||||
- A PR touches authentication, authorization, or session code
|
||||
- New API endpoints are added
|
||||
- Database queries or file I/O paths change
|
||||
- The issue mentions "security", "vulnerability", "CVE", or "auth"
|
||||
- A dependency update changes major versions
|
||||
|
||||
## What to check
|
||||
|
||||
### 1. Hardcoded secrets
|
||||
Scan for:
|
||||
- API keys, tokens, passwords in source files
|
||||
- `.env` files committed to the repo
|
||||
- Credentials in test fixtures
|
||||
|
||||
Use `gitleaks detect --no-git` for a fast scan.
|
||||
|
||||
### 2. Injection vectors
|
||||
- **SQL:** any string concatenation in query building. Use parameterized queries.
|
||||
- **Command:** any `os/exec` with user-controlled input. Use `exec.Command` with separate args, never `sh -c`.
|
||||
- **Path:** any file path constructed from user input without validation. Use `filepath.Clean` and check the resolved path is within the expected directory.
|
||||
|
||||
### 3. Unsafe deserialization
|
||||
- `json.Unmarshal` on untrusted input into `interface{}` or `map[string]any` without validation
|
||||
- `yaml.Unmarshal` without `KnownFields` or strict mode
|
||||
- `gob.Decode` on network input
|
||||
|
||||
### 4. Missing auth checks
|
||||
- New endpoints without middleware
|
||||
- Internal/admin endpoints exposed without auth
|
||||
- Token validation that only checks signature, not expiry or revocation
|
||||
|
||||
### 5. Dependency CVEs
|
||||
Run `trivy fs --severity HIGH,CRITICAL .` on the repository.
|
||||
Report each finding with:
|
||||
- Package name and version
|
||||
- CVE ID
|
||||
- CVSS score
|
||||
- Fixed version (if available)
|
||||
|
||||
## Reporting
|
||||
|
||||
For each finding, report:
|
||||
1. **Severity:** Critical / High / Medium / Low
|
||||
2. **Location:** file, line, function
|
||||
3. **Description:** what's wrong
|
||||
4. **Fix:** specific code change needed
|
||||
5. **Prevention:** how to catch this in future (lint rule, gate addition)
|
||||
|
||||
If nothing is found, report clean. Never fabricate findings.
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
name: tdd-bugfix
|
||||
description: >
|
||||
Fix bugs using test-driven development. Write a failing repro test first, prove
|
||||
it captures the bug, then fix the code without touching the tests. archie-core
|
||||
routes issues labelled "bug" through this workflow automatically.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [go, golangci-lint]
|
||||
engine: any
|
||||
---
|
||||
# TDD Bugfix Workflow
|
||||
|
||||
## When this runs
|
||||
|
||||
arcie-core routes any GitHub issue labelled `bug` through the TDD workflow.
|
||||
The workflow has five stages: analyse → repro-tests → capture-proof → fix → deliver.
|
||||
|
||||
## Stage 1: Analyse (read-only planner)
|
||||
|
||||
The planner agent explores the codebase read-only and produces:
|
||||
- Root cause: the exact file, function, and condition
|
||||
- The expected vs actual behaviour
|
||||
- Which test cases would prove the bug exists
|
||||
|
||||
No code is written. The plan is stored but not posted — the repro proof speaks louder.
|
||||
|
||||
## Stage 2: Repro Tests (write failing tests only)
|
||||
|
||||
The agent writes tests that PROVE the bug. Critical constraints:
|
||||
- **Do not touch non-test files.** Only `*_test.go` (or the ecosystem's test glob).
|
||||
- **The gate is inverted.** The test command (last in `[[repos.gate]]`) must FAIL.
|
||||
If it passes, the repro didn't capture the bug and the stage parks.
|
||||
- **Make the failure explicit.** The test should assert the correct behaviour,
|
||||
not just `panic()` or `t.FailNow()`.
|
||||
|
||||
## Stage 3: Capture Proof (deterministic)
|
||||
|
||||
arcie-core runs the test command directly (not through the agent loop) and captures
|
||||
the output. This is deterministic proof that the tests fail before the fix.
|
||||
|
||||
## Stage 4: Fix (code only)
|
||||
|
||||
The agent fixes the bug. Critical constraints:
|
||||
- **Test files are write-protected.** The agent CANNOT modify `*_test.go`.
|
||||
This is an environmental constraint, not a prompt rule.
|
||||
- **The full gate runs normally.** All quality commands plus the test suite must pass.
|
||||
- **Make the smallest fix.** Change only what's needed to make the repro tests pass.
|
||||
|
||||
## Stage 5: Deliver
|
||||
|
||||
arcie-core:
|
||||
1. Commits both the repro and the fix as separate commits (telling the PR story)
|
||||
2. Pushes the branch
|
||||
3. Opens a PR with the builder's summary
|
||||
4. Posts the captured failure proof as a PR comment
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
### The repro tests pass unexpectedly
|
||||
This means the test doesn't actually capture the bug. Check:
|
||||
- Is the test calling the right function with the right inputs?
|
||||
- Is the assertion checking the correct behaviour (not the buggy behaviour)?
|
||||
- Are there test setup issues (missing imports, wrong package)?
|
||||
|
||||
### The fix stage modifies test files
|
||||
This shouldn't happen — test files are write-protected. If it does, the agent is
|
||||
bypassing the constraint. The worktree should be inspected.
|
||||
|
||||
### The gate is misconfigured
|
||||
If the repo has no `[[repos.gate]]` entries, TDD cannot run — it needs at least
|
||||
one gate command, with the test runner last.
|
||||
+189
@@ -0,0 +1,189 @@
|
||||
# archie-core Architecture
|
||||
|
||||
**archied** is a resident daemon that polls GitHub for issues labelled `archie`, works each one in an isolated worktree through a routed workflow, and opens pull requests for human review.
|
||||
|
||||
## Package Map
|
||||
|
||||
| Package | Role |
|
||||
|---|---|
|
||||
| `cmd/archied/` | Binary entry: config→forge→store→runtime→workflows→daemon |
|
||||
| `internal/config/` | TOML config, per-repo gates, budgets, providers, ecosystems |
|
||||
| `internal/daemon/` | Resident loop: poll, enqueue, claim, route, process, reconcile |
|
||||
| `internal/workflow/` | Engine: stage lists, routing, shared steps, workflow definitions |
|
||||
| `internal/forge/` | GitHub interface: issues, labels, PRs, comments, reactions |
|
||||
| `internal/worktree/` | Git operations: clone, branch, commit, push, diff, cleanup |
|
||||
| `internal/store/` | SQLite: task rows, status transitions, crash recovery |
|
||||
| `internal/events/` | In-process event bus: publish/subscribe for observability |
|
||||
| `internal/webui/` | SSE dashboard: live event stream, task status |
|
||||
| `ai-sdk/runtime` | External: provider-agnostic LLM runtime |
|
||||
| `ai-sdk/agentloop` | External: gated agent loop with budgets and tool jails |
|
||||
| `ai-sdk/core` | External: tool definitions, generate options |
|
||||
|
||||
## Task Lifecycle
|
||||
|
||||
```
|
||||
queued → running(workflow:stage) → pr_open → merged | rejected
|
||||
↓
|
||||
waiting_human → (human approves) → queued(implement)
|
||||
↓
|
||||
parked
|
||||
```
|
||||
|
||||
**Crash recovery:** tasks left in `running` are re-queued on startup. Parks are never silent — every park posts a comment with the reason and gate output.
|
||||
|
||||
**PR reconciliation:** the daemon polls open PRs, checks GitHub state (`merged`/`closed`), transitions the task, and cleans up worktrees.
|
||||
|
||||
## Workflow Engine
|
||||
|
||||
### Core types (`workflow.go`)
|
||||
|
||||
- `Workflow` = named list of `Stage`s
|
||||
- `Stage` = `Name` + `Run(ctx, *TaskContext) error`
|
||||
- `TaskContext` = mutable bag: Task, Repo, Config, Forge, Store, Runtime, worktree dir/branch, scratch fields
|
||||
- `Outcome` = terminal decision (status + detail) — ends the workflow immediately
|
||||
- `Registry` = `map[string]Workflow`
|
||||
|
||||
### Routing (`Route`)
|
||||
|
||||
Label-driven, no LLM involved. Pre-assigned workflow wins; then labels decide:
|
||||
- `bug` → `tdd`
|
||||
- `feature` → `feasibility`
|
||||
- `bootstrap` → `bootstrap` (diagnostics, no LLM spend)
|
||||
- Default → `implement`
|
||||
|
||||
### Execution (`Run`)
|
||||
|
||||
Stages execute sequentially. After each stage:
|
||||
- Task is persisted to SQLite
|
||||
- Error → park the task (comment on issue, keep worktree)
|
||||
- Outcome set → finish (transition task, update labels)
|
||||
- Neither → continue to next stage
|
||||
|
||||
A workflow that ends without an outcome parks — definition bugs must not vanish silently.
|
||||
|
||||
## Workflows
|
||||
|
||||
### Bootstrap (diagnostics, no LLM)
|
||||
|
||||
```
|
||||
prepare → apply(marker file) → commit-push → diff-cap → open-pr
|
||||
```
|
||||
|
||||
Proves the plumbing: clone, push, PR. Label any test issue `bootstrap` to verify end-to-end.
|
||||
|
||||
### Implement (default)
|
||||
|
||||
```
|
||||
prepare → baseline-gate → plan(planner, read-only) → build(builder, gated) → commit-push → diff-cap → open-pr
|
||||
```
|
||||
|
||||
- **baseline-gate:** runs the repo's gate at the base commit before anything is touched
|
||||
- **plan:** read-only agent explores codebase, produces implementation plan, posts it as a comment
|
||||
- **build:** full toolset agent implements against the plan behind the repo's quality gate
|
||||
- **diff-cap:** parks oversized changes (>400 lines default)
|
||||
- **open-pr:** builder's summary becomes the PR body
|
||||
|
||||
### TDD (bug label)
|
||||
|
||||
```
|
||||
prepare → baseline-gate → analyse(planner) → repro-tests(builder, inverted gate) → capture-proof → commit-repro → fix(builder, normal gate, test files protected) → commit-push → diff-cap → open-pr + repro-evidence
|
||||
```
|
||||
|
||||
- **repro-tests:** the quality gate is **inverted** — the test command must FAIL. Proven failure is required.
|
||||
- **capture-proof:** runs the test command deterministically, captures the failing output
|
||||
- **commit-repro:** commits the failing tests as the first commit
|
||||
- **fix:** test files are **environmentally write-protected** (not prompt-ruled). The agent fixes code, not tests.
|
||||
- **open-pr + evidence:** posts both the PR and the captured failure proof
|
||||
|
||||
### Feasibility (feature label)
|
||||
|
||||
```
|
||||
prepare → assess(planner, with decide tool) → [close_if_bad_fit | prd(planner)] → deliver → waiting_human
|
||||
```
|
||||
|
||||
- **assess:** reads AGENT.md, ROADMAP.md, README. Calls a `decide` tool (structured output, not parsed prose). Bad fit → close with reasons.
|
||||
- **prd:** read-only agent writes a PRD: problem, solution, files affected, acceptance criteria, non-goals, diff estimate
|
||||
- **deliver:** posts PRD as issue comment + webhook notification (n8n → email). Sets state to `waiting_human`.
|
||||
- The daemon watches for human replies, judges them with a triage LLM (APPROVE/REJECT/UNCLEAR), requeues or closes.
|
||||
|
||||
## Forge Interface
|
||||
|
||||
A thin abstraction over GitHub (Gitea planned):
|
||||
|
||||
| Method | Purpose |
|
||||
|---|---|
|
||||
| `AcceptInvitations` | Auto-accept repo invites |
|
||||
| `AssignedIssues` | Poll assigned issues (excluding PRs) |
|
||||
| `IssuesWithLabel` | Poll labelled issues |
|
||||
| `Comment` | Post issue/PR comment, return ID |
|
||||
| `RepliesAfter` | Find human replies after a comment ID |
|
||||
| `CreatePR` | Open pull request |
|
||||
| `PRState` | Check merged/open/closed |
|
||||
| `CloseIssue` | Close with final comment |
|
||||
| `React` | Emoji reaction (pickup acknowledgement) |
|
||||
| `SetStateLabel` | Replace old state label with new one |
|
||||
| `VerifyPush` | Confirm push permission at startup |
|
||||
|
||||
State labels (`archie:queued`, `archie:working`, etc.) are created on demand, removed when transitioning.
|
||||
|
||||
## Worktree Manager
|
||||
|
||||
Every task gets a fresh clone — no persistent state between runs. The model never runs git.
|
||||
|
||||
```
|
||||
~/.local/share/archie/work/<owner>-<repo>/issue-<N>/
|
||||
```
|
||||
|
||||
Token auth via a generated `GIT_ASKPASS` helper script — the token never appears in `.git/config` or process argv.
|
||||
|
||||
## Config
|
||||
|
||||
Daemon-level TOML (`~/.config/archie/config.toml`):
|
||||
|
||||
- `[forge]` — type (github), host, token env var
|
||||
- `[dispatch]` — trigger (assignee/label/either), labels, ack reaction
|
||||
- `[[repos]]` — owner, name, base branch, gate commands, protected paths, ecosystem
|
||||
- `[models]` — planner, builder, triage (provider/model refs)
|
||||
- `[providers]` — LLM provider configs
|
||||
- `[budgets]` — max steps, max tokens, wall clock, gate max failures
|
||||
- `[web]` — dashboard listen address
|
||||
- `[notify]` — webhook URL for notifications
|
||||
|
||||
Per-repo gates are command lists: `[[repos.gate]] = ["go", "vet", "./..."]`. The last command is the test runner (by convention — TDD inverts it).
|
||||
|
||||
## Observability
|
||||
|
||||
Every lifecycle event flows through an internal event bus:
|
||||
- SQLite event log (every event stamped with a row ID)
|
||||
- Live dashboard via SSE (server-sent events)
|
||||
- Bounded subscriber buffers — slow consumers drop events, never block the daemon
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
1. **Environmental constraints over prompt rules.** The gate, test-file protection, and diff cap are code enforcement — not "please don't do that" in the system prompt.
|
||||
2. **The model never runs git.** Worktree operations are deterministic steps, not agent tools.
|
||||
3. **Tokens as cost, not features.** Token usage is tracked per task and emitted in observability events but never budgeted beyond the agent loop's own limits.
|
||||
4. **Label-driven routing.** Workflow selection is a switch statement, not an LLM call. A triage LLM is used only for the human reply judgment in waiting_human.
|
||||
5. **Shared code with tau at the module level.** `ai-sdk/agentloop` and `ai-sdk/toolkit` are extracted from tau's coordinator and promoted to shared libraries.
|
||||
6. **No daemon-internal cron.** Polling is a ticker loop, not cron. The feasibility PRD notification uses an external webhook (n8n).
|
||||
7. **Fresh worktree per task.** No persistent state between runs. Crash recovery just re-queues.
|
||||
|
||||
## What's Implemented vs Planned
|
||||
|
||||
**Implemented:**
|
||||
- Daemon poll loop, enqueue, claim, process
|
||||
- Bootstrap, implement, TDD, feasibility workflows
|
||||
- GitHub forge (issues, PRs, labels, comments, reactions)
|
||||
- Worktree isolation (clone, branch, commit, push, diff, cleanup)
|
||||
- SQLite store with transition audit
|
||||
- Config with ecosystem support (go, python, node, rust)
|
||||
- Agent stages with gates, budgets, preflight, protected paths
|
||||
- Observability (event bus, SQLite log, SSE dashboard)
|
||||
- Crash recovery (re-queue stale running tasks)
|
||||
- Human reply judgment (triage LLM for waiting_human)
|
||||
|
||||
**Planned:**
|
||||
- Gitea forge implementation
|
||||
- More ecosystems
|
||||
- Notification integrations beyond n8n webhook
|
||||
- Skills (SKILL.md support — see `.archie/skills/`)
|
||||
@@ -0,0 +1,262 @@
|
||||
# archie-core Yaegi Plugin System — PRD
|
||||
|
||||
**Author:** Archie (Hermes agent)
|
||||
**Date:** 2026-07-21
|
||||
**Status:** Draft
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Embed the [Yaegi](https://github.com/traefik/yaegi) Go interpreter to let archie-core load and execute `.go` files at runtime. This enables three things that shell commands can't do: **custom gates with deep worktree inspection**, **custom workflow stages defined per-repo**, and **skill scripts written in Go instead of shell**.
|
||||
|
||||
---
|
||||
|
||||
## Why Yaegi
|
||||
|
||||
### The problem with shell-only gates
|
||||
|
||||
`[[repos.gate]]` is command lists. That's fine for standard tooling (`go vet`, `golangci-lint`), but it can't express project-specific rules:
|
||||
|
||||
- "No new `panic()` calls in the diff"
|
||||
- "No new dependencies in `go.mod` without justification"
|
||||
- "All public functions in `pkg/api/` must be documented"
|
||||
- "No `time.Sleep` in production code paths"
|
||||
|
||||
These are rules that need to read the diff, parse Go ASTs, or walk the worktree — things shell scripts can do but shouldn't have to.
|
||||
|
||||
### Why not shell scripts?
|
||||
|
||||
Shell scripts for gates are fine until they're not. They can't import the archie-core types, can't use the Go parser, and are a second language to maintain alongside the daemon. Yaegi runs Go — the same language as the daemon, the same language as the repos archie-core works on.
|
||||
|
||||
### Why not Go's native `plugin` package?
|
||||
|
||||
Go's `plugin` package (`.so` files) requires:
|
||||
- Exact same Go version and deps as the host binary
|
||||
- Linux/macOS only
|
||||
- No unload
|
||||
- Painful distribution
|
||||
|
||||
Yaegi interprets `.go` source files directly — no compilation step, portable everywhere, loads from filesystem paths.
|
||||
|
||||
### Production precedent
|
||||
|
||||
- **Traefik:** 200+ community plugins via Yaegi, loaded from plugins.traefik.io
|
||||
- **Vikunja:** Adopted Yaegi for plugin system (see PR #2178), loads plugins from `plugins/` directory
|
||||
|
||||
---
|
||||
|
||||
## Design
|
||||
|
||||
### Three extension surfaces
|
||||
|
||||
| Surface | Location | When loaded | What it does |
|
||||
|---|---|---|---|
|
||||
| **Custom gate functions** | `.archie/gate.go` | During the gate stage (after shell commands) | Returns `[]Finding` — each finding has severity, message, file, line |
|
||||
| **Custom workflow stages** | `.archie/stages/<name>.go` | When the workflow references the stage name | Returns a `workflow.Stage` implementation |
|
||||
| **Skill scripts** | `.archie/skills/<skill>/scripts/*.go` | When the parent skill is activated | Called by the skill's instructions |
|
||||
|
||||
### Gate functions (`.archie/gate.go`)
|
||||
|
||||
The most important extension surface. After the shell-based gate passes, archie-core loads and evaluates `.archie/gate.go` in the worktree directory. The script exposes a `Check` function:
|
||||
|
||||
```go
|
||||
// .archie/gate.go — per-repo gate rules interpreted at runtime
|
||||
package gate
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// GateContext carries everything the gate checker can inspect.
|
||||
type GateContext struct {
|
||||
Diff string // unified diff of all changes
|
||||
ChangedFiles []string // list of changed file paths (repo-relative)
|
||||
Dir string // absolute worktree path
|
||||
BaseRef string // base branch (e.g. "origin/main")
|
||||
Repo string // "owner/name"
|
||||
}
|
||||
|
||||
// Finding is one gate violation.
|
||||
type Finding struct {
|
||||
Level string // "error" (blocks commit) or "warn" (advisory)
|
||||
File string // optional file path
|
||||
Line int // optional line number
|
||||
Message string
|
||||
}
|
||||
|
||||
// Block defines a named rule that can prevent passing.
|
||||
type Block struct {
|
||||
Name string
|
||||
Message string
|
||||
}
|
||||
|
||||
// Check returns findings for the gate. Return nil for a clean gate.
|
||||
func Check(ctx GateContext) []Finding {
|
||||
var findings []Finding
|
||||
|
||||
// Rule: no new panic() calls in committed code
|
||||
for _, line := range strings.Split(ctx.Diff, "\n") {
|
||||
if strings.HasPrefix(line, "+") && !strings.HasPrefix(line, "+++") {
|
||||
if strings.Contains(line, "panic(") {
|
||||
findings = append(findings, Finding{
|
||||
Level: "error",
|
||||
Message: "new panic() call — use error returns instead",
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Rule: no time.Sleep in non-test files
|
||||
for _, f := range ctx.ChangedFiles {
|
||||
if strings.HasSuffix(f, "_test.go") {
|
||||
continue
|
||||
}
|
||||
path := filepath.Join(ctx.Dir, f)
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if strings.Contains(string(b), "time.Sleep") {
|
||||
findings = append(findings, Finding{
|
||||
Level: "warn",
|
||||
File: f,
|
||||
Message: "time.Sleep in non-test code — use time.Ticker or context.WithTimeout instead",
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
return findings
|
||||
}
|
||||
```
|
||||
|
||||
The daemon:
|
||||
1. Reads `.archie/gate.go` from the worktree
|
||||
2. Creates a Yaegi interpreter with `GateContext` and `Finding` types pre-loaded via `Use()`
|
||||
3. Evaluates the file
|
||||
4. Calls `Check(ctx)` via `Eval("gate.Check")` + `Interface()`
|
||||
5. Logs findings and blocks the gate if any have `Level: "error"`
|
||||
|
||||
### Custom workflow stages (`.archie/stages/`)
|
||||
|
||||
A repo can define custom pipeline stages that archie-core discovers and makes routable:
|
||||
|
||||
```go
|
||||
// .archie/stages/run-migrations.go
|
||||
package stages
|
||||
|
||||
import "github.com/samcharles93/archie-core/internal/workflow"
|
||||
|
||||
func Stage() workflow.Stage {
|
||||
return workflow.Stage{
|
||||
Name: "db-migration",
|
||||
Run: func(ctx context.Context, tc *workflow.TaskContext) error {
|
||||
// Check if go.mod contains migration tools
|
||||
// Run migration tests
|
||||
// Verify schema compatibility
|
||||
return nil
|
||||
},
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Registered stages appear alongside built-in stages. A workflow in `config.toml` can reference them by name:
|
||||
|
||||
```toml
|
||||
[[workflows.stages]]
|
||||
name = "db-migration" # matches the .go file
|
||||
source = "repo" # loaded from .archie/stages/
|
||||
```
|
||||
|
||||
### Skill scripts (`.archie/skills/<skill>/scripts/`)
|
||||
|
||||
Skills can ship Go scripts that archie-core interprets:
|
||||
|
||||
```go
|
||||
// .archie/skills/security-audit/scripts/gitleaks.go
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os/exec"
|
||||
)
|
||||
|
||||
func main() {
|
||||
cmd := exec.Command("gitleaks", "detect", "--no-git", "--verbose")
|
||||
out, _ := cmd.CombinedOutput()
|
||||
fmt.Println(string(out))
|
||||
}
|
||||
```
|
||||
|
||||
Skills reference these scripts naturally in their instructions: "Run `scripts/gitleaks.go` to scan for secrets."
|
||||
|
||||
---
|
||||
|
||||
## API surface archie-core exposes to Yaegi
|
||||
|
||||
Via `i.Use()`, the daemon pre-loads symbols that interpreted code can import:
|
||||
|
||||
| Package | What's exposed |
|
||||
|---|---|
|
||||
| `archie-core/internal/workflow` | `Stage`, `TaskContext`, `Outcome` |
|
||||
| `archie-core/internal/gate` | `GateContext`, `Finding`, `Block` |
|
||||
| `stdlib` | Full Go standard library (stdlib.Symbols) |
|
||||
|
||||
Generated with `yaegi extract` — same approach Traefik and Vikunja use. A `//go:generate yaegi extract ...` line in the source triggers symbol table generation during `go generate`.
|
||||
|
||||
---
|
||||
|
||||
## Limitations
|
||||
|
||||
From Yaegi's own docs:
|
||||
- **No assembly files** (`.s` not supported)
|
||||
- **No CGo** (can't call C code — the daemon is pure Go, so this doesn't matter)
|
||||
- **No compiler/linker directives** (no `//go:embed`, etc. — fine for gate scripts)
|
||||
- **Interfaces from pre-compiled code can't be added dynamically** — the daemon must pre-compile wrapper types for any interface interpreted code needs to implement
|
||||
- **Slower than compiled code** — gate scripts run once per commit, not in hot paths; negligible impact
|
||||
|
||||
The Vikunja PR documents a specific Yaegi quirk: **typed factory functions required.** You can't do `v.Interface().(MyInterface)` on a value returned from interpreted code. Each interface needs its own factory:
|
||||
|
||||
```go
|
||||
func NewGate() Gate { return &MyGate{} }
|
||||
func NewReportable() Reportable { return &MyGate{} }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation phases
|
||||
|
||||
### Phase 1: Gate functions
|
||||
- Add `yaegi` as a dependency (`go get github.com/traefik/yaegi`)
|
||||
- Build symbol tables for `GateContext`, `Finding` via `yaegi extract`
|
||||
- Add `internal/gate/yaegi.go`: load `.archie/gate.go`, compile, call `Check()`
|
||||
- Wire into the gate stage — runs after shell-based gate commands
|
||||
- Error findings park the task; warn findings are logged
|
||||
|
||||
### Phase 2: Custom stages
|
||||
- Build symbol tables for `workflow.Stage`, `workflow.TaskContext`
|
||||
- Add stage discovery: scan `.archie/stages/*.go` and register each `Stage()` function
|
||||
- Allow `config.toml` to reference custom stages in workflow definitions
|
||||
|
||||
### Phase 3: Skill scripts
|
||||
- Skills can ship `scripts/*.go` alongside shell scripts
|
||||
- archie-core evaluates them via Yaegi when a skill's instructions reference them
|
||||
- Stdout from the script is returned to the agent as tool output
|
||||
|
||||
### Phase 4: Hot reload
|
||||
- Watch `.archie/` for changes during daemon runtime
|
||||
- Reload gate functions without restarting the daemon
|
||||
- (Only matters for long-running daemons — archie-core currently fresh-clones per task, so phase 1 covers the common case)
|
||||
|
||||
---
|
||||
|
||||
## Open questions
|
||||
|
||||
1. **Symbol table generation:** Run `yaegi extract` as a `go generate` step, or generate on daemon startup?
|
||||
2. **Error handling:** If the interpreted gate panics or times out, park the task? Or fall back to shell-only gate?
|
||||
3. **Budget:** Interpreted gate scripts run in the gate stage — should they have their own timeout/step budget?
|
||||
4. **Template system:** Should archie-core ship with common gate templates (e.g. `go: no panic, no Sleep, no hardcoded paths`)?
|
||||
5. **Remote gates:** Could `.archie/gate.go` be pulled from a shared remote registry, like skills?
|
||||
Reference in new issue
Block a user