From 1e73507d80afa906dae7dc3dfecaa27a923049a5 Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Tue, 21 Jul 2026 17:34:38 +1000 Subject: [PATCH] docs: add architecture docs, skills, and Yaegi PRD --- .archie/skills/ecosystem-node/SKILL.md | 85 ++++++++ .archie/skills/ecosystem-python/SKILL.md | 80 +++++++ .archie/skills/go-quality-gate/SKILL.md | 70 ++++++ .archie/skills/security-audit/SKILL.md | 66 ++++++ .archie/skills/tdd-bugfix/SKILL.md | 73 +++++++ ARCHITECTURE.md | 189 ++++++++++++++++ docs/prds/yaegi-plugin-system.md | 262 +++++++++++++++++++++++ 7 files changed, 825 insertions(+) create mode 100644 .archie/skills/ecosystem-node/SKILL.md create mode 100644 .archie/skills/ecosystem-python/SKILL.md create mode 100644 .archie/skills/go-quality-gate/SKILL.md create mode 100644 .archie/skills/security-audit/SKILL.md create mode 100644 .archie/skills/tdd-bugfix/SKILL.md create mode 100644 ARCHITECTURE.md create mode 100644 docs/prds/yaegi-plugin-system.md diff --git a/.archie/skills/ecosystem-node/SKILL.md b/.archie/skills/ecosystem-node/SKILL.md new file mode 100644 index 000000000..4074d647e --- /dev/null +++ b/.archie/skills/ecosystem-node/SKILL.md @@ -0,0 +1,85 @@ +--- +name: ecosystem-node +description: > + Node.js/TypeScript project conventions for archie-core: npm/pnpm, eslint, + prettier, tsc, vitest. Use when working on Node repositories with archie-core. +version: 1.0.0 +metadata: + archie: + tools: [node, npm, pnpm, vitest, eslint, prettier] + engine: any +--- +# Node.js / TypeScript Ecosystem Conventions + +## Preflight + +When archie-core's `[[repos]]` entry sets `ecosystem = "node"`, the daemon runs +`node --version` as the preflight check before any agent stage. + +## Recommended gate configuration + +```toml +[[repos]] +owner = "sam" +name = "my-node-project" +ecosystem = "node" + +[[repos.gate]] = ["pnpm", "run", "lint"] +[[repos.gate]] = ["pnpm", "run", "typecheck"] +[[repos.gate]] = ["pnpm", "run", "test"] +``` + +Or for npm projects: + +```toml +[[repos.gate]] = ["npm", "run", "lint"] +[[repos.gate]] = ["npm", "run", "build", "--", "--noEmit"] +[[repos.gate]] = ["npm", "test"] +``` + +## Test glob + +The default test glob for Node is `*.test.ts`. The TDD workflow's repro-tests +stage write-protects files matching this glob during the fix stage. + +Override with `test_glob = "**/*.spec.ts"` for projects using that convention, +or `test_glob = "__tests__/**/*.ts"` for Jest-style test directories. + +## Package management + +arcie-core detects the lockfile: +- `pnpm-lock.yaml` → use `pnpm` +- `package-lock.json` → use `npm` +- `yarn.lock` → use `yarn` + +Install before gate: +```bash +pnpm install --frozen-lockfile +``` + +## Common tools + +| Tool | Purpose | Config file | +|---|---|---| +| `eslint` | Linting | `eslint.config.mjs` | +| `prettier` | Formatting | `.prettierrc` | +| `tsc --noEmit` | Type checking | `tsconfig.json` | +| `vitest` | Testing | `vitest.config.ts` | +| `jest` | Testing (legacy) | `jest.config.ts` | + +## Common gate failures + +### eslint +- `no-unused-vars`: remove unused import/variable +- `@typescript-eslint/no-explicit-any`: add proper type +- Import order: let eslint fix it with `--fix` + +### tsc / typecheck +- `TS2345`: type mismatch — fix the argument or the signature +- `TS2339`: property doesn't exist — check the type definition +- `Cannot find module`: check the import path or install the package + +### vitest / jest +- Test timeout: the test is hanging — check async/await, mocks, or network calls +- Snapshot mismatch: the output changed — update the snapshot if the change is intentional +- Assertion failure: fix the code, not the test diff --git a/.archie/skills/ecosystem-python/SKILL.md b/.archie/skills/ecosystem-python/SKILL.md new file mode 100644 index 000000000..c4bcda440 --- /dev/null +++ b/.archie/skills/ecosystem-python/SKILL.md @@ -0,0 +1,80 @@ +--- +name: ecosystem-python +description: > + Python project conventions for archie-core: venv, pytest, ruff, mypy, pip-audit. + Use when working on Python repositories with archie-core. +version: 1.0.0 +metadata: + archie: + tools: [python, pytest, ruff, mypy, pip-audit] + engine: any +--- +# Python Ecosystem Conventions + +## Preflight + +When archie-core's `[[repos]]` entry sets `ecosystem = "python"`, the daemon runs +`python --version` as the preflight check before any agent stage. + +## Recommended gate configuration + +```toml +[[repos]] +owner = "sam" +name = "my-python-project" +ecosystem = "python" + +[[repos.gate]] = ["ruff", "check", "."] +[[repos.gate]] = ["ruff", "format", "--check", "."] +[[repos.gate]] = ["mypy", "."] +[[repos.gate]] = ["pytest", "-x", "--tb=short"] +``` + +## Test glob + +The default test glob for Python is `test_*.py`. The TDD workflow's repro-tests +stage write-protects files matching this glob during the fix stage. + +Override with `test_glob = "tests/**/*.py"` in the repo config for projects +with test files outside the package root. + +## Package management + +arcie-core assumes: +- `requirements.txt` or `pyproject.toml` at the repo root +- Tests run with `pytest` (the ecosystem default) +- Linting with `ruff` (recommended) or `flake8` +- Type checking with `mypy` +- Security auditing with `pip-audit` + +## Virtual environment + +Before the gate runs, ensure dependencies are installed: +```bash +python -m venv .venv +. .venv/bin/activate +pip install -r requirements.txt +pip install pytest ruff mypy pip-audit +``` + +For `pyproject.toml` projects: +```bash +pip install -e ".[dev]" +``` + +## Common gate failures + +### ruff +- `F401`: unused import — remove it +- `E501`: line too long — break the line +- `I001`: import order — let ruff fix it with `ruff check --fix` + +### mypy +- Missing type annotation on public function — add it +- `Cannot determine type` — add explicit annotation +- Import errors — check the package is installed in the venv + +### pytest +- Test discovery failed — check `__init__.py` files and test naming +- Import errors — verify the package is installed (`pip install -e .`) +- Assertion failures — fix the code, not the test diff --git a/.archie/skills/go-quality-gate/SKILL.md b/.archie/skills/go-quality-gate/SKILL.md new file mode 100644 index 000000000..154c0272b --- /dev/null +++ b/.archie/skills/go-quality-gate/SKILL.md @@ -0,0 +1,70 @@ +--- +name: go-quality-gate +description: > + Run the full Go quality pipeline: gofumpt, go fix, golangci-lint, go test -race, + and task deadcode. Use before any commit in a Go project. Triggered automatically + by archie-core's implement and tdd workflows via the per-repo gate config. +version: 1.0.0 +metadata: + archie: + tools: [go, golangci-lint, gofumpt, task] + engine: any +--- +# Go Quality Gate + +## When this runs + +arcie-core enforces this gate after every builder agent stage. The specific commands +come from the repository's `[[repos.gate]]` config, not this skill. This skill documents +what a correct gate looks like and how to diagnose failures. + +## Standard gate (Tau and similar Go projects) + +Run in this exact order. archie-core does NOT run these — the `ai-sdk/agentloop` gate +does. This list is what you should configure in `config.toml`: + +```toml +[[repos.gate]] = ["gofumpt", "-w", "."] +[[repos.gate]] = ["go", "fix", "./..."] +[[repos.gate]] = ["golangci-lint", "run", "./..."] +[[repos.gate]] = ["go", "test", "-race", "-count=1", "./..."] +``` + +Or if the project uses Task: + +```toml +[[repos.gate]] = ["task", "check"] +``` + +The LAST command in the gate list is the test runner by convention. archie-core's +TDD workflow inverts only that last command during the repro stage. + +## Diagnosing gate failures + +### gofumpt / go fix +These mutate files. If they change anything, the next `git diff` will catch it. +The agent should re-run them after making changes. + +### golangci-lint +Common failures and fixes: +- `unused` — remove the unused function or add `//nolint:unused` +- `errcheck` — handle the error or assign to `_` +- `gosec` — review the security finding; never blindly suppress + +### go test -race +Data races are non-deterministic. If `-race` fails: +1. Identify the race with `go test -race -run=TestName` +2. Add a mutex or channel to protect the shared state +3. Re-run with `-count=5` to confirm it's gone + +### task deadcode +Scoping rule: if deadcode fails on packages you didn't touch, scope it: +```bash +golangci-lint run --tests=false --enable-only=unused,staticcheck ./changed/pkg/... +``` + +## When the TDD workflow runs + +During `repro-tests`, the test command's expectations are INVERTED — it must FAIL. +During `fix`, the full gate runs normally and test files are write-protected. +The agent cannot modify tests in the fix stage; it must fix the code to make them pass. diff --git a/.archie/skills/security-audit/SKILL.md b/.archie/skills/security-audit/SKILL.md new file mode 100644 index 000000000..8095625bc --- /dev/null +++ b/.archie/skills/security-audit/SKILL.md @@ -0,0 +1,66 @@ +--- +name: security-audit +description: > + Scan code for security vulnerabilities: hardcoded secrets, injection vectors, + unsafe deserialization, missing auth checks, and path traversal. Use on PRs + touching auth, API endpoints, or data handling code. +version: 1.0.0 +metadata: + archie: + tools: [golangci-lint, trivy, gitleaks] + engine: any +--- +# Security Audit + +## When to use + +Run this audit when: +- A PR touches authentication, authorization, or session code +- New API endpoints are added +- Database queries or file I/O paths change +- The issue mentions "security", "vulnerability", "CVE", or "auth" +- A dependency update changes major versions + +## What to check + +### 1. Hardcoded secrets +Scan for: +- API keys, tokens, passwords in source files +- `.env` files committed to the repo +- Credentials in test fixtures + +Use `gitleaks detect --no-git` for a fast scan. + +### 2. Injection vectors +- **SQL:** any string concatenation in query building. Use parameterized queries. +- **Command:** any `os/exec` with user-controlled input. Use `exec.Command` with separate args, never `sh -c`. +- **Path:** any file path constructed from user input without validation. Use `filepath.Clean` and check the resolved path is within the expected directory. + +### 3. Unsafe deserialization +- `json.Unmarshal` on untrusted input into `interface{}` or `map[string]any` without validation +- `yaml.Unmarshal` without `KnownFields` or strict mode +- `gob.Decode` on network input + +### 4. Missing auth checks +- New endpoints without middleware +- Internal/admin endpoints exposed without auth +- Token validation that only checks signature, not expiry or revocation + +### 5. Dependency CVEs +Run `trivy fs --severity HIGH,CRITICAL .` on the repository. +Report each finding with: +- Package name and version +- CVE ID +- CVSS score +- Fixed version (if available) + +## Reporting + +For each finding, report: +1. **Severity:** Critical / High / Medium / Low +2. **Location:** file, line, function +3. **Description:** what's wrong +4. **Fix:** specific code change needed +5. **Prevention:** how to catch this in future (lint rule, gate addition) + +If nothing is found, report clean. Never fabricate findings. diff --git a/.archie/skills/tdd-bugfix/SKILL.md b/.archie/skills/tdd-bugfix/SKILL.md new file mode 100644 index 000000000..f087b41f2 --- /dev/null +++ b/.archie/skills/tdd-bugfix/SKILL.md @@ -0,0 +1,73 @@ +--- +name: tdd-bugfix +description: > + Fix bugs using test-driven development. Write a failing repro test first, prove + it captures the bug, then fix the code without touching the tests. archie-core + routes issues labelled "bug" through this workflow automatically. +version: 1.0.0 +metadata: + archie: + tools: [go, golangci-lint] + engine: any +--- +# TDD Bugfix Workflow + +## When this runs + +arcie-core routes any GitHub issue labelled `bug` through the TDD workflow. +The workflow has five stages: analyse → repro-tests → capture-proof → fix → deliver. + +## Stage 1: Analyse (read-only planner) + +The planner agent explores the codebase read-only and produces: +- Root cause: the exact file, function, and condition +- The expected vs actual behaviour +- Which test cases would prove the bug exists + +No code is written. The plan is stored but not posted — the repro proof speaks louder. + +## Stage 2: Repro Tests (write failing tests only) + +The agent writes tests that PROVE the bug. Critical constraints: +- **Do not touch non-test files.** Only `*_test.go` (or the ecosystem's test glob). +- **The gate is inverted.** The test command (last in `[[repos.gate]]`) must FAIL. + If it passes, the repro didn't capture the bug and the stage parks. +- **Make the failure explicit.** The test should assert the correct behaviour, + not just `panic()` or `t.FailNow()`. + +## Stage 3: Capture Proof (deterministic) + +arcie-core runs the test command directly (not through the agent loop) and captures +the output. This is deterministic proof that the tests fail before the fix. + +## Stage 4: Fix (code only) + +The agent fixes the bug. Critical constraints: +- **Test files are write-protected.** The agent CANNOT modify `*_test.go`. + This is an environmental constraint, not a prompt rule. +- **The full gate runs normally.** All quality commands plus the test suite must pass. +- **Make the smallest fix.** Change only what's needed to make the repro tests pass. + +## Stage 5: Deliver + +arcie-core: +1. Commits both the repro and the fix as separate commits (telling the PR story) +2. Pushes the branch +3. Opens a PR with the builder's summary +4. Posts the captured failure proof as a PR comment + +## Common pitfalls + +### The repro tests pass unexpectedly +This means the test doesn't actually capture the bug. Check: +- Is the test calling the right function with the right inputs? +- Is the assertion checking the correct behaviour (not the buggy behaviour)? +- Are there test setup issues (missing imports, wrong package)? + +### The fix stage modifies test files +This shouldn't happen — test files are write-protected. If it does, the agent is +bypassing the constraint. The worktree should be inspected. + +### The gate is misconfigured +If the repo has no `[[repos.gate]]` entries, TDD cannot run — it needs at least +one gate command, with the test runner last. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 000000000..d2c4335c3 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,189 @@ +# archie-core Architecture + +**archied** is a resident daemon that polls GitHub for issues labelled `archie`, works each one in an isolated worktree through a routed workflow, and opens pull requests for human review. + +## Package Map + +| Package | Role | +|---|---| +| `cmd/archied/` | Binary entry: config→forge→store→runtime→workflows→daemon | +| `internal/config/` | TOML config, per-repo gates, budgets, providers, ecosystems | +| `internal/daemon/` | Resident loop: poll, enqueue, claim, route, process, reconcile | +| `internal/workflow/` | Engine: stage lists, routing, shared steps, workflow definitions | +| `internal/forge/` | GitHub interface: issues, labels, PRs, comments, reactions | +| `internal/worktree/` | Git operations: clone, branch, commit, push, diff, cleanup | +| `internal/store/` | SQLite: task rows, status transitions, crash recovery | +| `internal/events/` | In-process event bus: publish/subscribe for observability | +| `internal/webui/` | SSE dashboard: live event stream, task status | +| `ai-sdk/runtime` | External: provider-agnostic LLM runtime | +| `ai-sdk/agentloop` | External: gated agent loop with budgets and tool jails | +| `ai-sdk/core` | External: tool definitions, generate options | + +## Task Lifecycle + +``` +queued → running(workflow:stage) → pr_open → merged | rejected + ↓ + waiting_human → (human approves) → queued(implement) + ↓ + parked +``` + +**Crash recovery:** tasks left in `running` are re-queued on startup. Parks are never silent — every park posts a comment with the reason and gate output. + +**PR reconciliation:** the daemon polls open PRs, checks GitHub state (`merged`/`closed`), transitions the task, and cleans up worktrees. + +## Workflow Engine + +### Core types (`workflow.go`) + +- `Workflow` = named list of `Stage`s +- `Stage` = `Name` + `Run(ctx, *TaskContext) error` +- `TaskContext` = mutable bag: Task, Repo, Config, Forge, Store, Runtime, worktree dir/branch, scratch fields +- `Outcome` = terminal decision (status + detail) — ends the workflow immediately +- `Registry` = `map[string]Workflow` + +### Routing (`Route`) + +Label-driven, no LLM involved. Pre-assigned workflow wins; then labels decide: +- `bug` → `tdd` +- `feature` → `feasibility` +- `bootstrap` → `bootstrap` (diagnostics, no LLM spend) +- Default → `implement` + +### Execution (`Run`) + +Stages execute sequentially. After each stage: +- Task is persisted to SQLite +- Error → park the task (comment on issue, keep worktree) +- Outcome set → finish (transition task, update labels) +- Neither → continue to next stage + +A workflow that ends without an outcome parks — definition bugs must not vanish silently. + +## Workflows + +### Bootstrap (diagnostics, no LLM) + +``` +prepare → apply(marker file) → commit-push → diff-cap → open-pr +``` + +Proves the plumbing: clone, push, PR. Label any test issue `bootstrap` to verify end-to-end. + +### Implement (default) + +``` +prepare → baseline-gate → plan(planner, read-only) → build(builder, gated) → commit-push → diff-cap → open-pr +``` + +- **baseline-gate:** runs the repo's gate at the base commit before anything is touched +- **plan:** read-only agent explores codebase, produces implementation plan, posts it as a comment +- **build:** full toolset agent implements against the plan behind the repo's quality gate +- **diff-cap:** parks oversized changes (>400 lines default) +- **open-pr:** builder's summary becomes the PR body + +### TDD (bug label) + +``` +prepare → baseline-gate → analyse(planner) → repro-tests(builder, inverted gate) → capture-proof → commit-repro → fix(builder, normal gate, test files protected) → commit-push → diff-cap → open-pr + repro-evidence +``` + +- **repro-tests:** the quality gate is **inverted** — the test command must FAIL. Proven failure is required. +- **capture-proof:** runs the test command deterministically, captures the failing output +- **commit-repro:** commits the failing tests as the first commit +- **fix:** test files are **environmentally write-protected** (not prompt-ruled). The agent fixes code, not tests. +- **open-pr + evidence:** posts both the PR and the captured failure proof + +### Feasibility (feature label) + +``` +prepare → assess(planner, with decide tool) → [close_if_bad_fit | prd(planner)] → deliver → waiting_human +``` + +- **assess:** reads AGENT.md, ROADMAP.md, README. Calls a `decide` tool (structured output, not parsed prose). Bad fit → close with reasons. +- **prd:** read-only agent writes a PRD: problem, solution, files affected, acceptance criteria, non-goals, diff estimate +- **deliver:** posts PRD as issue comment + webhook notification (n8n → email). Sets state to `waiting_human`. +- The daemon watches for human replies, judges them with a triage LLM (APPROVE/REJECT/UNCLEAR), requeues or closes. + +## Forge Interface + +A thin abstraction over GitHub (Gitea planned): + +| Method | Purpose | +|---|---| +| `AcceptInvitations` | Auto-accept repo invites | +| `AssignedIssues` | Poll assigned issues (excluding PRs) | +| `IssuesWithLabel` | Poll labelled issues | +| `Comment` | Post issue/PR comment, return ID | +| `RepliesAfter` | Find human replies after a comment ID | +| `CreatePR` | Open pull request | +| `PRState` | Check merged/open/closed | +| `CloseIssue` | Close with final comment | +| `React` | Emoji reaction (pickup acknowledgement) | +| `SetStateLabel` | Replace old state label with new one | +| `VerifyPush` | Confirm push permission at startup | + +State labels (`archie:queued`, `archie:working`, etc.) are created on demand, removed when transitioning. + +## Worktree Manager + +Every task gets a fresh clone — no persistent state between runs. The model never runs git. + +``` +~/.local/share/archie/work/-/issue-/ +``` + +Token auth via a generated `GIT_ASKPASS` helper script — the token never appears in `.git/config` or process argv. + +## Config + +Daemon-level TOML (`~/.config/archie/config.toml`): + +- `[forge]` — type (github), host, token env var +- `[dispatch]` — trigger (assignee/label/either), labels, ack reaction +- `[[repos]]` — owner, name, base branch, gate commands, protected paths, ecosystem +- `[models]` — planner, builder, triage (provider/model refs) +- `[providers]` — LLM provider configs +- `[budgets]` — max steps, max tokens, wall clock, gate max failures +- `[web]` — dashboard listen address +- `[notify]` — webhook URL for notifications + +Per-repo gates are command lists: `[[repos.gate]] = ["go", "vet", "./..."]`. The last command is the test runner (by convention — TDD inverts it). + +## Observability + +Every lifecycle event flows through an internal event bus: +- SQLite event log (every event stamped with a row ID) +- Live dashboard via SSE (server-sent events) +- Bounded subscriber buffers — slow consumers drop events, never block the daemon + +## Key Design Decisions + +1. **Environmental constraints over prompt rules.** The gate, test-file protection, and diff cap are code enforcement — not "please don't do that" in the system prompt. +2. **The model never runs git.** Worktree operations are deterministic steps, not agent tools. +3. **Tokens as cost, not features.** Token usage is tracked per task and emitted in observability events but never budgeted beyond the agent loop's own limits. +4. **Label-driven routing.** Workflow selection is a switch statement, not an LLM call. A triage LLM is used only for the human reply judgment in waiting_human. +5. **Shared code with tau at the module level.** `ai-sdk/agentloop` and `ai-sdk/toolkit` are extracted from tau's coordinator and promoted to shared libraries. +6. **No daemon-internal cron.** Polling is a ticker loop, not cron. The feasibility PRD notification uses an external webhook (n8n). +7. **Fresh worktree per task.** No persistent state between runs. Crash recovery just re-queues. + +## What's Implemented vs Planned + +**Implemented:** +- Daemon poll loop, enqueue, claim, process +- Bootstrap, implement, TDD, feasibility workflows +- GitHub forge (issues, PRs, labels, comments, reactions) +- Worktree isolation (clone, branch, commit, push, diff, cleanup) +- SQLite store with transition audit +- Config with ecosystem support (go, python, node, rust) +- Agent stages with gates, budgets, preflight, protected paths +- Observability (event bus, SQLite log, SSE dashboard) +- Crash recovery (re-queue stale running tasks) +- Human reply judgment (triage LLM for waiting_human) + +**Planned:** +- Gitea forge implementation +- More ecosystems +- Notification integrations beyond n8n webhook +- Skills (SKILL.md support — see `.archie/skills/`) diff --git a/docs/prds/yaegi-plugin-system.md b/docs/prds/yaegi-plugin-system.md new file mode 100644 index 000000000..57bf8e49f --- /dev/null +++ b/docs/prds/yaegi-plugin-system.md @@ -0,0 +1,262 @@ +# archie-core Yaegi Plugin System — PRD + +**Author:** Archie (Hermes agent) +**Date:** 2026-07-21 +**Status:** Draft + +--- + +## Summary + +Embed the [Yaegi](https://github.com/traefik/yaegi) Go interpreter to let archie-core load and execute `.go` files at runtime. This enables three things that shell commands can't do: **custom gates with deep worktree inspection**, **custom workflow stages defined per-repo**, and **skill scripts written in Go instead of shell**. + +--- + +## Why Yaegi + +### The problem with shell-only gates + +`[[repos.gate]]` is command lists. That's fine for standard tooling (`go vet`, `golangci-lint`), but it can't express project-specific rules: + +- "No new `panic()` calls in the diff" +- "No new dependencies in `go.mod` without justification" +- "All public functions in `pkg/api/` must be documented" +- "No `time.Sleep` in production code paths" + +These are rules that need to read the diff, parse Go ASTs, or walk the worktree — things shell scripts can do but shouldn't have to. + +### Why not shell scripts? + +Shell scripts for gates are fine until they're not. They can't import the archie-core types, can't use the Go parser, and are a second language to maintain alongside the daemon. Yaegi runs Go — the same language as the daemon, the same language as the repos archie-core works on. + +### Why not Go's native `plugin` package? + +Go's `plugin` package (`.so` files) requires: +- Exact same Go version and deps as the host binary +- Linux/macOS only +- No unload +- Painful distribution + +Yaegi interprets `.go` source files directly — no compilation step, portable everywhere, loads from filesystem paths. + +### Production precedent + +- **Traefik:** 200+ community plugins via Yaegi, loaded from plugins.traefik.io +- **Vikunja:** Adopted Yaegi for plugin system (see PR #2178), loads plugins from `plugins/` directory + +--- + +## Design + +### Three extension surfaces + +| Surface | Location | When loaded | What it does | +|---|---|---|---| +| **Custom gate functions** | `.archie/gate.go` | During the gate stage (after shell commands) | Returns `[]Finding` — each finding has severity, message, file, line | +| **Custom workflow stages** | `.archie/stages/.go` | When the workflow references the stage name | Returns a `workflow.Stage` implementation | +| **Skill scripts** | `.archie/skills//scripts/*.go` | When the parent skill is activated | Called by the skill's instructions | + +### Gate functions (`.archie/gate.go`) + +The most important extension surface. After the shell-based gate passes, archie-core loads and evaluates `.archie/gate.go` in the worktree directory. The script exposes a `Check` function: + +```go +// .archie/gate.go — per-repo gate rules interpreted at runtime +package gate + +import ( + "fmt" + "os" + "path/filepath" + "strings" +) + +// GateContext carries everything the gate checker can inspect. +type GateContext struct { + Diff string // unified diff of all changes + ChangedFiles []string // list of changed file paths (repo-relative) + Dir string // absolute worktree path + BaseRef string // base branch (e.g. "origin/main") + Repo string // "owner/name" +} + +// Finding is one gate violation. +type Finding struct { + Level string // "error" (blocks commit) or "warn" (advisory) + File string // optional file path + Line int // optional line number + Message string +} + +// Block defines a named rule that can prevent passing. +type Block struct { + Name string + Message string +} + +// Check returns findings for the gate. Return nil for a clean gate. +func Check(ctx GateContext) []Finding { + var findings []Finding + + // Rule: no new panic() calls in committed code + for _, line := range strings.Split(ctx.Diff, "\n") { + if strings.HasPrefix(line, "+") && !strings.HasPrefix(line, "+++") { + if strings.Contains(line, "panic(") { + findings = append(findings, Finding{ + Level: "error", + Message: "new panic() call — use error returns instead", + }) + } + } + } + + // Rule: no time.Sleep in non-test files + for _, f := range ctx.ChangedFiles { + if strings.HasSuffix(f, "_test.go") { + continue + } + path := filepath.Join(ctx.Dir, f) + b, err := os.ReadFile(path) + if err != nil { + continue + } + if strings.Contains(string(b), "time.Sleep") { + findings = append(findings, Finding{ + Level: "warn", + File: f, + Message: "time.Sleep in non-test code — use time.Ticker or context.WithTimeout instead", + }) + } + } + + return findings +} +``` + +The daemon: +1. Reads `.archie/gate.go` from the worktree +2. Creates a Yaegi interpreter with `GateContext` and `Finding` types pre-loaded via `Use()` +3. Evaluates the file +4. Calls `Check(ctx)` via `Eval("gate.Check")` + `Interface()` +5. Logs findings and blocks the gate if any have `Level: "error"` + +### Custom workflow stages (`.archie/stages/`) + +A repo can define custom pipeline stages that archie-core discovers and makes routable: + +```go +// .archie/stages/run-migrations.go +package stages + +import "github.com/samcharles93/archie-core/internal/workflow" + +func Stage() workflow.Stage { + return workflow.Stage{ + Name: "db-migration", + Run: func(ctx context.Context, tc *workflow.TaskContext) error { + // Check if go.mod contains migration tools + // Run migration tests + // Verify schema compatibility + return nil + }, + } +} +``` + +Registered stages appear alongside built-in stages. A workflow in `config.toml` can reference them by name: + +```toml +[[workflows.stages]] +name = "db-migration" # matches the .go file +source = "repo" # loaded from .archie/stages/ +``` + +### Skill scripts (`.archie/skills//scripts/`) + +Skills can ship Go scripts that archie-core interprets: + +```go +// .archie/skills/security-audit/scripts/gitleaks.go +package main + +import ( + "fmt" + "os/exec" +) + +func main() { + cmd := exec.Command("gitleaks", "detect", "--no-git", "--verbose") + out, _ := cmd.CombinedOutput() + fmt.Println(string(out)) +} +``` + +Skills reference these scripts naturally in their instructions: "Run `scripts/gitleaks.go` to scan for secrets." + +--- + +## API surface archie-core exposes to Yaegi + +Via `i.Use()`, the daemon pre-loads symbols that interpreted code can import: + +| Package | What's exposed | +|---|---| +| `archie-core/internal/workflow` | `Stage`, `TaskContext`, `Outcome` | +| `archie-core/internal/gate` | `GateContext`, `Finding`, `Block` | +| `stdlib` | Full Go standard library (stdlib.Symbols) | + +Generated with `yaegi extract` — same approach Traefik and Vikunja use. A `//go:generate yaegi extract ...` line in the source triggers symbol table generation during `go generate`. + +--- + +## Limitations + +From Yaegi's own docs: +- **No assembly files** (`.s` not supported) +- **No CGo** (can't call C code — the daemon is pure Go, so this doesn't matter) +- **No compiler/linker directives** (no `//go:embed`, etc. — fine for gate scripts) +- **Interfaces from pre-compiled code can't be added dynamically** — the daemon must pre-compile wrapper types for any interface interpreted code needs to implement +- **Slower than compiled code** — gate scripts run once per commit, not in hot paths; negligible impact + +The Vikunja PR documents a specific Yaegi quirk: **typed factory functions required.** You can't do `v.Interface().(MyInterface)` on a value returned from interpreted code. Each interface needs its own factory: + +```go +func NewGate() Gate { return &MyGate{} } +func NewReportable() Reportable { return &MyGate{} } +``` + +--- + +## Implementation phases + +### Phase 1: Gate functions +- Add `yaegi` as a dependency (`go get github.com/traefik/yaegi`) +- Build symbol tables for `GateContext`, `Finding` via `yaegi extract` +- Add `internal/gate/yaegi.go`: load `.archie/gate.go`, compile, call `Check()` +- Wire into the gate stage — runs after shell-based gate commands +- Error findings park the task; warn findings are logged + +### Phase 2: Custom stages +- Build symbol tables for `workflow.Stage`, `workflow.TaskContext` +- Add stage discovery: scan `.archie/stages/*.go` and register each `Stage()` function +- Allow `config.toml` to reference custom stages in workflow definitions + +### Phase 3: Skill scripts +- Skills can ship `scripts/*.go` alongside shell scripts +- archie-core evaluates them via Yaegi when a skill's instructions reference them +- Stdout from the script is returned to the agent as tool output + +### Phase 4: Hot reload +- Watch `.archie/` for changes during daemon runtime +- Reload gate functions without restarting the daemon +- (Only matters for long-running daemons — archie-core currently fresh-clones per task, so phase 1 covers the common case) + +--- + +## Open questions + +1. **Symbol table generation:** Run `yaegi extract` as a `go generate` step, or generate on daemon startup? +2. **Error handling:** If the interpreted gate panics or times out, park the task? Or fall back to shell-only gate? +3. **Budget:** Interpreted gate scripts run in the gate stage — should they have their own timeout/step budget? +4. **Template system:** Should archie-core ship with common gate templates (e.g. `go: no panic, no Sleep, no hardcoded paths`)? +5. **Remote gates:** Could `.archie/gate.go` be pulled from a shared remote registry, like skills?