# archie-core Yaegi Plugin System -- PRD **Author:** Archie **Date:** 2026-07-21 **Status:** Draft --- ## Summary Embed the [Yaegi](https://github.com/traefik/yaegi) Go interpreter to let archie-core load and execute `.go` files at runtime. This enables three things that shell commands can't do: **custom gates with deep worktree inspection**, **custom workflow stages defined per-repo**, and **skill scripts written in Go instead of shell**. --- ## Why Yaegi ### The problem with shell-only gates `[[repos.gate]]` is command lists. That's fine for standard tooling (`go vet`, `golangci-lint`), but it can't express project-specific rules: - "No new `panic()` calls in the diff" - "No new dependencies in `go.mod` without justification" - "All public functions in `pkg/api/` must be documented" - "No `time.Sleep` in production code paths" These are rules that need to read the diff, parse Go ASTs, or walk the worktree -- things shell scripts can do but shouldn't have to. ### Why not shell scripts? Shell scripts for gates are fine until they're not. They can't import the archie-core types, can't use the Go parser, and are a second language to maintain alongside the daemon. Yaegi runs Go -- the same language as the daemon, the same language as the repos archie-core works on. ### Why not Go's native `plugin` package? Go's `plugin` package (`.so` files) requires: - Exact same Go version and deps as the host binary - Linux/macOS only - No unload - Painful distribution Yaegi interprets `.go` source files directly -- no compilation step, portable everywhere, loads from filesystem paths. ### Production precedent - **Traefik:** 200+ community plugins via Yaegi, loaded from plugins.traefik.io - **Vikunja:** Adopted Yaegi for plugin system (see PR #2178), loads plugins from `plugins/` directory --- ## Design ### Three extension surfaces | Surface | Location | When loaded | What it does | | -------------------------- | ------------------------------------- | -------------------------------------------- | --------------------------------------------------------------------- | | **Custom gate functions** | `.archie/gate.go` | During the gate stage (after shell commands) | Returns `[]Finding` -- each finding has severity, message, file, line | | **Custom workflow stages** | `.archie/stages/.go` | When the workflow references the stage name | Returns a `workflow.Stage` implementation | | **Skill scripts** | `.archie/skills//scripts/*.go` | When the parent skill is activated | Called by the skill's instructions | ### Gate functions (`.archie/gate.go`) The most important extension surface. After the shell-based gate passes, archie-core loads and evaluates `.archie/gate.go` in the worktree directory. The script exposes a `Check` function: ```go // .archie/gate.go -- per-repo gate rules interpreted at runtime package gate import ( "fmt" "os" "path/filepath" "strings" ) // GateContext carries everything the gate checker can inspect. type GateContext struct { Diff string // unified diff of all changes ChangedFiles []string // list of changed file paths (repo-relative) Dir string // absolute worktree path BaseRef string // base branch (e.g. "origin/main") Repo string // "owner/name" } // Finding is one gate violation. type Finding struct { Level string // "error" (blocks commit) or "warn" (advisory) File string // optional file path Line int // optional line number Message string } // Block defines a named rule that can prevent passing. type Block struct { Name string Message string } // Check returns findings for the gate. Return nil for a clean gate. func Check(ctx GateContext) []Finding { var findings []Finding // Rule: no new panic() calls in committed code for _, line := range strings.Split(ctx.Diff, "\n") { if strings.HasPrefix(line, "+") && !strings.HasPrefix(line, "+++") { if strings.Contains(line, "panic(") { findings = append(findings, Finding{ Level: "error", Message: "new panic() call -- use error returns instead", }) } } } // Rule: no time.Sleep in non-test files for _, f := range ctx.ChangedFiles { if strings.HasSuffix(f, "_test.go") { continue } path := filepath.Join(ctx.Dir, f) b, err := os.ReadFile(path) if err != nil { continue } if strings.Contains(string(b), "time.Sleep") { findings = append(findings, Finding{ Level: "warn", File: f, Message: "time.Sleep in non-test code -- use time.Ticker or context.WithTimeout instead", }) } } return findings } ``` The daemon: 1. Reads `.archie/gate.go` from the worktree 2. Creates a Yaegi interpreter with `GateContext` and `Finding` types pre-loaded via `Use()` 3. Evaluates the file 4. Calls `Check(ctx)` via `Eval("gate.Check")` + `Interface()` 5. Logs findings and blocks the gate if any have `Level: "error"` ### Custom workflow stages (`.archie/stages/`) A repo can define custom pipeline stages that archie-core discovers and makes routable: ```go // .archie/stages/run-migrations.go package stages import "github.com/samcharles93/archie-core/internal/workflow" func Stage() workflow.Stage { return workflow.Stage{ Name: "db-migration", Run: func(ctx context.Context, tc *workflow.TaskContext) error { // Check if go.mod contains migration tools // Run migration tests // Verify schema compatibility return nil }, } } ``` Registered stages appear alongside built-in stages. A workflow in `config.toml` can reference them by name: ```toml [[workflows.stages]] name = "db-migration" # matches the .go file source = "repo" # loaded from .archie/stages/ ``` ### Skill scripts (`.archie/skills//scripts/`) Skills can ship Go scripts that archie-core interprets: ```go // .archie/skills/security-audit/scripts/gitleaks.go package main import ( "fmt" "os/exec" "context" ) func main() { ctx := context.Background() cmd := exec.CommandContext(ctx, "gitleaks", "detect", "--no-git", "--verbose") out, _ := cmd.CombinedOutput() fmt.Println(string(out)) } ``` Skills reference these scripts naturally in their instructions: "Run `scripts/gitleaks.go` to scan for secrets." --- ## API surface archie-core exposes to Yaegi Via `i.Use()`, the daemon pre-loads symbols that interpreted code can import: | Package | What's exposed | | ------------------------------- | ----------------------------------------- | | `archie-core/internal/workflow` | `Stage`, `TaskContext`, `Outcome` | | `archie-core/internal/gate` | `GateContext`, `Finding`, `Block` | | `stdlib` | Full Go standard library (stdlib.Symbols) | Generated with `yaegi extract` -- same approach Traefik and Vikunja use. A `//go:generate yaegi extract ...` line in the source triggers symbol table generation during `go generate`. --- ## Limitations From Yaegi's own docs: - **No assembly files** (`.s` not supported) - **No CGo** (can't call C code -- the daemon is pure Go, so this doesn't matter) - **No compiler/linker directives** (no `//go:embed`, etc. -- fine for gate scripts) - **Interfaces from pre-compiled code can't be added dynamically** -- the daemon must pre-compile wrapper types for any interface interpreted code needs to implement - **Slower than compiled code** -- gate scripts run once per commit, not in hot paths; negligible impact The Vikunja PR documents a specific Yaegi quirk: **typed factory functions required.** You can't do `v.Interface().(MyInterface)` on a value returned from interpreted code. Each interface needs its own factory: ```go func NewGate() Gate { return &MyGate{} } func NewReportable() Reportable { return &MyGate{} } ``` --- ## Implementation phases ### Phase 1: Gate functions -- implemented - `yaegi` added as a dependency - Symbol table for `GateContext`, `Finding` generated via `yaegi extract` into `internal/gate/gateextract` (a separate package from `internal/gate` itself, avoiding the self-import cycle extracting a package's own symbols creates) - `internal/gate/gateeval.Evaluate`: loads `.archie/gate.go` from the worktree, interprets it, calls `Check()`; panics are recovered - Wired in as `workflow.StageYaegiGate`, run after the final commit in both the `implement` and `tdd` workflows - Error findings park the task; warn findings are logged only ### Phase 2: Custom stages -- implemented - Symbol table for `workflow.Stage`/`TaskContext`/`Outcome` generated into `internal/workflow/wfextract` - `internal/workflow/wfeval.Discover` scans `.archie/stages/*.go` in filename order and calls each file's exported `Stage()` function - `TaskContext.CustomStages` is an injected `func(dir string) ([]Stage, error)` -- the composition root (`cmd/archied`) wires it to `wfeval.Discover`, keeping `internal/workflow` decoupled from its own generated symbol table - `workflow.StageRepoStages` runs discovered stages in order right after the worktree is prepared, in both `implement` and `tdd` - Simplification vs. the original sketch: stages are auto-discovered and always run (like `.archie/gate.go`) rather than referenced by name from `config.toml` -- the workflow engine has no data-driven stage graph today, so wiring arbitrary named stages into arbitrary positions would need a larger config-driven rework. Auto-run-in-order covers the PRD's actual use case (a repo-defined pre-flight/setup hook) without it. ### Phase 3: Skill scripts -- implemented - `internal/skillscript.Run` interprets a `.go` file via `i.EvalPath`, which (like `go run`) evaluates and calls its `main()`; stdout/stderr are captured via `interp.Options` - Uses `stdlib.Symbols` + `stdlib/unrestricted.Symbols` (not just `stdlib`) -- `os/exec` lives in the unrestricted set, and the PRD's own example script shells out via `os/exec` - No archie-core-specific symbol table needed here -- skill scripts only touch the Go standard library - Exposed to the agent loop as a new `run_go_script` tool (`internal/agentexec/inprocess.go`), available in every agent stage; path is validated to stay inside the workspace - This is how "the skill's instructions reference a script" cashes out concretely: the agent calls `run_go_script` with the path instead of needing a Go toolchain in the sandbox ### Phase 4: Hot reload -- not needed, by construction - All three loaders (`gate.Evaluate`, `wfeval.Discover`, `skillscript.Run`) read their `.go` source from disk on every call -- no in-memory cache exists anywhere in the implementation - Every task additionally gets a fresh `git clone` (`worktree.Manager.Prepare` removes and re-clones the worktree directory per task) - Consequently every stage already re-reads `.archie/gate.go`, `.archie/stages/*.go`, and skill scripts fresh, every run -- there is no stale state a file watcher would ever need to invalidate - A file-watching daemon-runtime reload mechanism would add machinery with nothing to do given this architecture; revisit only if the daemon starts reusing long-lived worktrees across tasks --- ## Open questions 1. **Symbol table generation:** Run `yaegi extract` as a `go generate` step, or generate on daemon startup? 2. **Error handling:** If the interpreted gate panics or times out, park the task? Or fall back to shell-only gate? 3. **Budget:** Interpreted gate scripts run in the gate stage -- should they have their own timeout/step budget? 4. **Template system:** Should archie-core ship with common gate templates (e.g. `go: no panic, no Sleep, no hardcoded paths`)? 5. **Remote gates:** Could `.archie/gate.go` be pulled from a shared remote registry, like skills?