chore: cleanup old docs
This commit is contained in:
9 files changed
-4364
No files matched your search
@@ -1,45 +0,0 @@
|
||||
# archied -- design notes
|
||||
|
||||
Archie is a resident daemon that keeps personal projects moving: it polls GitHub for issues labelled
|
||||
`archie`, works each one in an isolated fresh-clone worktree through a routed **workflow**, and opens a
|
||||
pull request for human review. Sam reviews and merges; the machine grinds in between.
|
||||
|
||||
Guiding principle: **environmental constraints over prompt instructions.** Agents don't get asked nicely
|
||||
to write good code -- the quality gate (vet/build/test/lint, configurable per repo and per workflow stage)
|
||||
refuses to let a run succeed until it passes. The gate mechanics live in `ai-sdk/agentloop`; this repo
|
||||
owns orchestration only.
|
||||
|
||||
## Workflows, not a pipeline
|
||||
|
||||
`internal/workflow` is the engine: a Workflow is an ordered list of stages over a shared TaskContext;
|
||||
stages are deterministic steps (worktree, gate, diff cap, PR, comments -- the shared step library in
|
||||
`steps.go`) or agent stages (`agentloop.Run` with a stage-specific mission/toolset/gate). Routing is
|
||||
label-driven (`bug` → tdd, `feature` → feasibility, default → implement), refinable by an LLM triage
|
||||
stage. Adding a workflow must never require touching the engine or reimplementing steps.
|
||||
|
||||
Planned workflows:
|
||||
|
||||
- **bootstrap** (registered): deterministic marker-file PR proving invites/clone/push/PR plumbing.
|
||||
- **implement**: planner (read-only agentloop, plan posted to the issue) → builder (full toolset,
|
||||
repo gate) → diff cap → PR.
|
||||
- **tdd** (bugfixes): analyse → problem surface → write failing repro tests (gate stage with
|
||||
`ExpectFailure`) → fix (gate: suite passes) → PR.
|
||||
- **feasibility** (features): read-only analysis → roadmap fit → won't-do close with reasons, or
|
||||
plan + PRD → email Sam → `waiting_human` → LLM-judged reply → hand off to implement.
|
||||
|
||||
## Task lifecycle
|
||||
|
||||
`queued → running(workflow:stage) → waiting_human | pr_open → merged | parked | rejected | closed_wont_do`
|
||||
|
||||
SQLite (`internal/store`), WAL, one row per issue with a transitions audit table. Crash recovery:
|
||||
`running` tasks re-queue on startup with attempt++. Parks are never silent -- every park comments on the
|
||||
issue with the reason and gate output.
|
||||
|
||||
## Security posture
|
||||
|
||||
- Fine-grained PAT on the bot account only (Contents RW, PRs RW, Issues RW, Metadata R), via
|
||||
`ARCHIE_GITHUB_TOKEN`; git auth through an askpass helper so the token stays out of argv and
|
||||
`.git/config`.
|
||||
- The model never runs git: `internal/worktree` owns all repo operations deterministically.
|
||||
- Agent tools are jailed to the task worktree (toolkit path jail); intended deployment is a dedicated
|
||||
Incus container so the blast radius is environmental, not prompt-negotiated.
|
||||
@@ -1,519 +0,0 @@
|
||||
# archie-core v2 Architecture -- Design Document
|
||||
|
||||
**Author:** Archie
|
||||
**Date:** 2026-07-21 (implementation status updated 2026-07-22)
|
||||
**Status:** Final (substantially implemented -- remaining aspirational items are pluggable storage backends and Layer 2 daemon plugin loader)
|
||||
|
||||
---
|
||||
|
||||
## 0. Implementation status (as of 2026-07-22, updated 2026-07-22)
|
||||
|
||||
This document describes a target end state. The table below is the ground truth; read it before treating any section as already built.
|
||||
|
||||
| Area | Status | What actually exists today (post-e4d11b1) |
|
||||
| --------------------------------------------------------- | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Daemon/agent binary split | **Implemented** | `cmd/archied` and `cmd/archie-agent` both exist. `archie-agent` is a 189-line NATS-connected per-task worker that subscribes to task subjects, runs the full multi-stage workflow, and reports results. Workflow routing and sequencing (`workflow.Route`, `workflow.Run`) now run inside the agent, not the daemon. Three agent modes: `inprocess`, `subprocess`, `nats`. |
|
||||
| Sandbox / "daemon never runs untrusted code" | **Partial** | Default mode is still `inprocess` for development. Docker sandbox mode (`agent.mode = "nats"` + `containers.enabled = true`) runs each task in an ephemeral Docker container with workspace bind-mounted at `/workspace`. `internal/container/pool.go` manages full lifecycle: pull, create, start, release, orphan recovery. Daemon dispatch and container lifecycle both enforce `max_concurrency`; same-repo tasks remain serialised. |
|
||||
| NATS / JetStream messaging | **Implemented** | `internal/nats/client.go`: JetStream connect, stream creation (WorkQueue retention, file storage), pull consumer (durable, ack-wait 5min, max-deliver 3, dedup by Msg-Id), `PublishTask`, `Fetch`, `NewReplyInbox`, and optional token authentication. Subject hierarchy lives in `internal/nats/subjects.go`; `internal/agentexec/nats.go` implements request/reply agent execution. |
|
||||
| Docker container sandbox | **Implemented** | `internal/container/pool.go` (210 lines): Docker API client, image pull (always/missing/never), container create with workspace bind mount, start, release (stop+remove), orphan recovery on daemon restart. `Dockerfile` at repo root builds archie-agent fat image. `[containers]` config block: enabled, image, max_concurrency, max_uptime, pull_policy. `agent.mode = "nats"` required when containers enabled. |
|
||||
| `/data/` volume, `task.json` | **Partial** | `container.WriteTaskJSON()` writes task payload to the worktree before container acquire. The container pool bind-mounts the isolated task worktree at `/data/worktree`; configured repos also receive a persistent `/data/repo` Docker volume. Task payload travels over NATS as the primary channel; `/data/task.json` is a boot-time brief for the container. |
|
||||
| `[containers]` config block | **Implemented** | `ContainerConfig` struct in config.go: enabled, image, max_concurrency, max_uptime, pull_policy. Validated on load. |
|
||||
| Yaegi extensibility (gates, custom stages, skill scripts) | **Implemented** | Three surfaces: `.archie/gate.go`, `.archie/stages/*.go`, skill scripts via `run_go_script` tool. All in-process via Yaegi with generated symbol tables and panic recovery. Full detail in `docs/prds/yaegi-plugin-system.md`. |
|
||||
| Layer 1: skill-bundled plugins | **Implemented** | `skill.DiscoverPlugins()` scans `.agents/skills/<name>/plugins/*.go`. `skill.Plugin.Run()` executes them via Yaegi. `skillbuild.BuildWorkflow()` constructs `workflow.Workflow` from plugin-defined stages. `TaskContext.SkillPlugins` makes them available during stage execution. `skillbuild.BuildRegistry()` replaces the hardcoded `workflow.Registry` with catalogue-driven construction -- skills define workflows, plugins define stages. |
|
||||
| Layer 2: core daemon plugins | **Partial** | `internal/plugin/plugin.go`: `Plugin` interface (`Name()`, `Version()`), `Registry` type, `LoadDir()` using Yaegi to interpret `.go` files. No `~/.config/archie/plugins/` loader wired into `cmd/archied/main.go` yet -- the interface and registry exist but are not connected to the daemon's startup. |
|
||||
| Skills (agentskills.io spec) | **Implemented** | `internal/skill/skill.go`: parses SKILL.md YAML frontmatter, discovers skills from `.agents/skills/*/SKILL.md`, extracts body. Skills follow `archie-wf-*` naming convention. Progressive disclosure: `Catalog()` (Tier 1 -- frontmatter-only scan, ~100 tokens per entry), `LoadBody()` (Tier 2 -- full body on activation), `Discover()` (Tier 3 -- full parse with plugins). `metadata.archie.workflow` declares skill→workflow affinity. `skill.SkillForWorkflow()` is catalogue-driven, no hardcoded maps. `skillbuild.BuildRegistry()` constructs the daemon's workflow registry from the catalogue. |
|
||||
| Worktree/repo management | **Implemented** | Each issue keeps an isolated worktree. Repos with `persistent_storage = true` use a per-repo bare Git object cache to avoid repeated full network clones; caches and Archie-owned repo volumes expire via `containers.volume_ttl`. |
|
||||
| Config schema | **Closer to sec.4** | `[agent]` block: mode (inprocess/subprocess/nats) + command + env. `[nats]` block: url + token_env. `[containers]` block: enabled, image, max_concurrency, max_uptime, volume_ttl, pull_policy. Per-repo `persistent_storage` is supported as a boolean Docker-volume/Git-cache opt-in. |
|
||||
|
||||
**Net read post-1dca419:** Phases 1, 2, and 3 are substantially implemented. NATS, Docker sandbox, skill parsing, skill-bundled plugins (Layer 1), skill progressive disclosure, and catalogue-driven workflow construction (`skillbuild.BuildRegistry()`) all landed. The remaining aspirational work is Layer 2 daemon plugin loader (interface exists, not wired), pluggable storage backends, and per-worktree dynamic registry augmentation.
|
||||
|
||||
---
|
||||
|
||||
## 1. Binary split
|
||||
|
||||
### archied (daemon)
|
||||
|
||||
The resident orchestrator. Owns:
|
||||
|
||||
- **Task discovery** -- polls configured forges for issues labelled `archie` (or configured label)
|
||||
- **NATS publishing** -- publishes discovered tasks to JetStream subjects
|
||||
- **Container lifecycle** -- spawns Docker containers, kills them after grace period, cleans up volumes
|
||||
- **Message gatekeeping** -- reads agent responses from NATS, reviews them, forwards to human channels
|
||||
- **Core plugin loading** -- loads Yaegi `.go` plugins from `~/.config/archie/plugins/` at startup
|
||||
- **Web dashboard** -- SSE observability endpoint
|
||||
- **Human communication** -- issue comments, labels, PR management, notify webhook
|
||||
|
||||
The daemon never runs untrusted code. Agent containers are the sandbox boundary.
|
||||
|
||||
> **Target state, not current state.** Today, default `[agent] mode = "inprocess"` runs the agent loop (and therefore
|
||||
> all LLM-directed tool calls) directly inside `archied`. `subprocess` mode exists (`internal/agentexec/subprocess.go`)
|
||||
> but its own code comment says: _"This is not an OS security boundary: a same-UID subprocess may still access
|
||||
> daemon-readable host resources."_ This claim becomes true only once the Docker sandbox (sec.4, Phase 2) lands.
|
||||
|
||||
### archie-agent (ephemeral container)
|
||||
|
||||
Spawned per task. Lives in a Docker container. Receives its brief via `/data/task.json` (volume-mounted). Owns:
|
||||
|
||||
- **Workflow execution** -- runs the routed workflow stages (implement, tdd, feasibility, bootstrap)
|
||||
- **LLM orchestration** -- via `ai-sdk/agentloop`, the agent drives LLM calls
|
||||
- **Bundled plugin loading** -- loads Yaegi `.go` plugins from the skill's `plugins/` directory and the `/data/plugins/`
|
||||
volume
|
||||
- **Skill loading** -- reads `.agents/skills/archie-wf-*` from the worktree, follows the agentskills.io progressive
|
||||
disclosure model
|
||||
- **NATS reporting** -- publishes results to `archie.agent.<task_id>.response`, system messages to
|
||||
`archie.agent.<task_id>.system`
|
||||
|
||||
Lifecycle: spawn -> load task -> run workflow -> report -> wait for follow-ups (grace period) -> killed by daemon.
|
||||
|
||||
> **Target state, not current state.** `cmd/archie-agent/main.go` today is 28 lines: read one JSON `Invocation` from
|
||||
> stdin, run one stage via `agentexec.ServeOne`, write one `Response` to stdout, exit. It is a single-stage subprocess
|
||||
> worker invoked by `archied`'s `SubprocessRunner`, not a persistent per-task container process. Workflow routing and
|
||||
> multi-stage sequencing (`workflow.Route`, `workflow.Run`) run entirely inside `archied`
|
||||
> (`internal/daemon/daemon.go:process()`), not inside archie-agent. There is no `/data/task.json`, no bundled-plugin
|
||||
> loading, no `.agents/skills/archie-wf-*` discovery, and no NATS reporting -- see sec.0, sec.2, sec.3, sec.5. The
|
||||
> existing `agentexec.Runner` interface (`InProcessRunner`/`SubprocessRunner`) is, however, deliberately shaped so a
|
||||
> future container-backed runner can be a drop-in swap without changing workflow orchestration.
|
||||
|
||||
---
|
||||
|
||||
## 2. NATS subject hierarchy
|
||||
|
||||
> **Status: aspirational, zero code.** No NATS/JetStream dependency exists anywhere in `go.mod` or the codebase. Today,
|
||||
> task discovery is GitHub-poll-driven (`daemon.Run()` ticks on `PollInterval`, calls `pollIssues()`), distribution is a
|
||||
> SQLite `tasks` table used as a FIFO queue (`store.ClaimNext` atomically claims the oldest `queued` row), and
|
||||
> results/status flow through an in-process `internal/events.Bus` (Go channels, multi-subscriber fan-out,
|
||||
> drop-on-overflow) that already feeds both a SQLite `events` table and the SSE dashboard. `events.Bus` has roughly the
|
||||
> right shape to become the NATS publish path -- same `Event` schema, swap the transport -- which meaningfully de-risks
|
||||
> this phase, but it is currently single-process only with no network transport.
|
||||
|
||||
### Task distribution
|
||||
|
||||
```events
|
||||
archie.task.bug -- TDD workflow tasks
|
||||
archie.task.feature -- Feasibility workflow tasks
|
||||
archie.task.default -- Implement workflow tasks
|
||||
archie.task.bootstrap -- Bootstrap diagnostic tasks
|
||||
```
|
||||
|
||||
Daemon publishes to the appropriate subject based on label routing. Agents subscribe to subjects they're configured to
|
||||
handle.
|
||||
|
||||
### Agent communication (per-task)
|
||||
|
||||
```events
|
||||
archie.agent.<task_id>.response -- Agent output: summaries, PR bodies, results. Daemon reviews before human delivery.
|
||||
archie.agent.<task_id>.system -- Internal comms: log dumps, data requests, health, PII warnings. Never forwarded unprompted.
|
||||
archie.agent.<task_id>.request -- Optional: agent requests additional data from the daemon.
|
||||
```
|
||||
|
||||
The daemon gatekeeps the response channel -- no agent output reaches a human without daemon review.
|
||||
|
||||
### JetStream durability
|
||||
|
||||
Tasks are published with JetStream persistence. On agent container failure, NATS redelivers. The daemon doesn't need
|
||||
retry logic -- NATS handles it.
|
||||
|
||||
---
|
||||
|
||||
## 3. Persistent volume
|
||||
|
||||
> **Status: partially implemented.** `task.json`, `/data/worktree`, optional per-repo `/data/repo` Docker volumes,
|
||||
> ecosystem cache volumes, and TTL cleanup exist. Task worktrees remain isolated by issue, while configured repos use a
|
||||
> per-repo bare Git object cache to seed clones without repeating full network transfers. `session.jsonl`,
|
||||
> `memory.jsonl`, and staged plugin volumes remain unimplemented.
|
||||
|
||||
### /data/ layout
|
||||
|
||||
```tree
|
||||
/data/
|
||||
|---- task.json # Full task payload (owner, repo, issue, title, body, labels, workflow)
|
||||
|---- session.jsonl # Agent session transcript (appended per stage)
|
||||
|---- memory.jsonl # Optional: cross-session memory (persisted per project)
|
||||
|---- plugins/ # Bundled plugins from the skill (daemon-staged, overrides worktree)
|
||||
|---- worktree/ # Read-only mount of the cloned repo (managed by daemon)
|
||||
```
|
||||
|
||||
### task.json
|
||||
|
||||
```json
|
||||
{
|
||||
"id": 42,
|
||||
"owner": "sam",
|
||||
"repo": "example-service",
|
||||
"issue_number": 170,
|
||||
"title": "Add /copy command for messages and session transcripts",
|
||||
"body": "...",
|
||||
"labels": ["enhancement", "archie"],
|
||||
"workflow": "implement",
|
||||
"branch": "archie/issue-170",
|
||||
"plan": "",
|
||||
"config": {
|
||||
"base_branch": "main",
|
||||
"gate": [
|
||||
["gofumpt", "-w", "."],
|
||||
["go", "vet", "./..."],
|
||||
["go", "test", "-race", "./..."]
|
||||
],
|
||||
"protect": ["_templ.go"],
|
||||
"ecosystem": "go",
|
||||
"test_glob": "*_test.go"
|
||||
},
|
||||
"nats": {
|
||||
"url": "nats://archie-core:4222",
|
||||
"response_subject": "archie.agent.42.response",
|
||||
"system_subject": "archie.agent.42.system",
|
||||
"request_subject": "archie.agent.42.request"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The daemon writes `task.json` to the volume before the container starts. The agent reads it on boot.
|
||||
|
||||
### Volume lifecycle
|
||||
|
||||
- Created on task start (persistent per-repo if configured)
|
||||
- Kept until terminal outcome (`pr_open`, `merged`, `rejected`, `closed_wont_do`) OR TTL expiry
|
||||
- Parked tasks keep their volume for inspection and requeue reuse
|
||||
- Cleanup is a daemon responsibility
|
||||
|
||||
---
|
||||
|
||||
## 4. Container management
|
||||
|
||||
> **Status: implemented with a flatter schema than the sketch below.** `[containers]` configures the image,
|
||||
> concurrency, uptime, pull policy, and persistent-storage TTL. `Repo.PersistentStorage` opts a repository into the
|
||||
> per-repo Docker volume and Git object cache. The backend abstraction exists in `internal/storage`; additional storage
|
||||
> types and structured per-backend configuration remain future work.
|
||||
|
||||
### Daemon config
|
||||
|
||||
```toml
|
||||
# Container lifecycle
|
||||
[containers]
|
||||
max_concurrency = 4 # max simultaneous agent containers
|
||||
max_uptime = "30m" # grace period after task completion before kill
|
||||
volume_ttl = "72h" # max volume age before cleanup (overrides terminal wait)
|
||||
|
||||
# Image strategy
|
||||
[containers.image]
|
||||
registry = "ghcr.io/sam/archie-agent:latest" # primary: pull from registry
|
||||
registry_pull = true # allow pulls
|
||||
dockerfile = "./Dockerfile.agent" # fallback: build locally
|
||||
dockerfile_build = true # allow builds
|
||||
|
||||
# Persistent storage
|
||||
[containers.persistent_storage]
|
||||
type = "docker" # docker (volume), bind, s3, sftp, nfs
|
||||
path = "archie-data" # volume name or mount path
|
||||
```
|
||||
|
||||
### Per-repo persistent storage override
|
||||
|
||||
```toml
|
||||
[[repos]]
|
||||
owner = "sam"
|
||||
name = "example-service"
|
||||
persistent_storage = { type = "bind", path = "./archie-core-data/projects/example-service" }
|
||||
```
|
||||
|
||||
When configured, each task for this repo mounts the same bind path. Memory persists across sessions. The agent writes to
|
||||
`MEMORY.md` in this directory.
|
||||
|
||||
### Agent image
|
||||
|
||||
A fat Docker image containing:
|
||||
|
||||
- archie-agent binary
|
||||
- NATS client library
|
||||
- LLM provider SDKs (ai-sdk runtime)
|
||||
- Base tools: git, gh, go, golangci-lint, gofumpt, node, python, ruff, pytest, etc.
|
||||
- esbuild (for TypeScript plugin transpilation)
|
||||
- Yaegi interpreter (bundled in the binary)
|
||||
|
||||
Pre-built and published to a registry. The daemon pulls on startup. Users can provide their own image via config.
|
||||
|
||||
---
|
||||
|
||||
## 5. Extensibility layers
|
||||
|
||||
> **Reconciliation note.** This section describes a two-layer plugin architecture (skill-bundled plugins loaded
|
||||
> per-task, core daemon plugins loaded at startup) that does not exist in code. What _does_ exist -- and is implemented,
|
||||
> tested, and already in production -- is described in the sibling PRD `docs/prds/yaegi-plugin-system.md`, whose three
|
||||
> phases (custom gate functions, custom workflow stages, skill scripts) are marked "implemented" and match disk state
|
||||
> exactly as of commit `d45f21d`. That implementation is narrower and shaped differently than what follows:
|
||||
>
|
||||
> - It has **no skill-bundling concept** -- `.archie/gate.go` and `.archie/stages/*.go` are repo-level, not shipped
|
||||
> inside a skill's `plugins/` directory; skill scripts (`.archie/skills/<skill>/scripts/*.go`) are the closest analog
|
||||
> but none of the five current skills ship a `scripts/` directory yet.
|
||||
> - It has **no daemon-side core plugin registry** -- no `Plugin` interface, no `Register(daemon *Daemon)`, no
|
||||
> `~/.config/archie/plugins/` loader. Yaegi today only interprets small, narrowly-scoped `.go` files (gates, stages,
|
||||
> scripts) in-process inside `archied`, using generated symbol tables (`internal/gate/gateextract`,
|
||||
> `internal/workflow/wfextract`) rather than a general extension-registration interface.
|
||||
> - Custom stages are **auto-discovered and run in filename order**, not routed by name from `config.toml` -- the
|
||||
> workflow engine has no data-driven stage graph today, so the `[[workflows.stages]]` config sketch below does not
|
||||
> correspond to real config.
|
||||
>
|
||||
> Before implementation begins, decide: does Layer 1/Layer 2 as described below _replace_ the existing three Yaegi
|
||||
> surfaces, or does it _extend_ them (e.g., teach the existing loaders to also read from a skill's `plugins/` directory,
|
||||
> and add a genuinely new daemon-side `Plugin` interface as a separate, later addition)? The existing surfaces are
|
||||
> working code with real symbol-table generation and panic recovery -- treat them as the implementation substrate for
|
||||
> Layer 1, not as something to discard.
|
||||
|
||||
### Layer 1: Skills (`archie-wf-*`)
|
||||
|
||||
> **Status: cosmetic only.** Five `SKILL.md` files exist today (`ecosystem-node`, `ecosystem-python`, `go-quality-gate`,
|
||||
> `security-audit`, `tdd-bugfix`) at `.archie/skills/<name>/SKILL.md` -- a single flat repo-relative location, no
|
||||
> worktree-vs-user-level split, no `archie-wf-*` naming. Each has a `metadata.archie` block with only `tools` and
|
||||
> `engine` keys (no `plugins`, no `budget_usd`). **No Go code parses SKILL.md frontmatter, discovers skills, or does
|
||||
> progressive disclosure** -- grepping for a skill loader, catalog, or frontmatter parser across the repo returns
|
||||
> nothing. The workflow `Mission` strings that drive agent behavior today are hand-written Go string literals in
|
||||
> `internal/workflow/{implement,tdd,feasibility}.go`, not content injected from SKILL.md. The stage names described in
|
||||
> the PRD's SKILL.md prose closely mirror the hardcoded Go stage names (one drift found: `tdd-bugfix/SKILL.md` says the
|
||||
> last stage is "deliver"; the code calls it `open-pr`) -- these read as retrofitted documentation of existing v1
|
||||
> behavior wearing v2-flavored frontmatter, not a functioning specification the daemon consumes.
|
||||
|
||||
Skills are Markdown documents following the [agentskills.io](https://agentskills.io/specification) specification with
|
||||
archie-specific extensions.
|
||||
|
||||
#### Discovery
|
||||
|
||||
```tree
|
||||
.agents/skills/archie-wf-tdd/SKILL.md (worktree -- highest precedence)
|
||||
.agents/skills/archie-wf-implement/SKILL.md
|
||||
.agents/skills/archie-wf-feasibility/SKILL.md
|
||||
~/.archie/skills/archie-wf-default/SKILL.md (user-level -- fallback)
|
||||
```
|
||||
|
||||
The agent discovers skills at workflow start. The skill matching the task's workflow type is loaded first. If absent,
|
||||
falls back to `archie-wf-default`. If that's absent too, the agent works from AGENTS.md conventions and best practices.
|
||||
|
||||
#### Extended agentskills.io format
|
||||
|
||||
Skills follow the standard spec (`name`, `description`, `version`, `SKILL.md` body) with archie-specific metadata:
|
||||
|
||||
```yaml
|
||||
---
|
||||
name: archie-wf-tdd
|
||||
description: >
|
||||
Test-driven bugfix workflow for archie-core. Reproduce the bug with failing tests, prove the repro, fix the code
|
||||
without touching tests.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
plugins: # bundled Yaegi plugins shipped with this skill
|
||||
- plugins/custom-gate.go # repo-specific gate logic
|
||||
- plugins/gosec-check.go # security scanner integration
|
||||
tools: [go, golangci-lint, gofumpt, task]
|
||||
engine: any
|
||||
budget_usd: 2.00
|
||||
---
|
||||
```
|
||||
|
||||
#### Progressive disclosure
|
||||
|
||||
Per the agentskills.io spec:
|
||||
|
||||
1. **Catalogue** (~100 tokens): `name` + `description` loaded at workflow start
|
||||
2. **Instructions** (<5000 tokens): Full `SKILL.md` body when the skill is activated
|
||||
3. **Resources** (as needed): References, scripts, and bundled plugins loaded on demand
|
||||
|
||||
#### Bundled plugins
|
||||
|
||||
A skill's `plugins/` directory contains Yaegi `.go` files. These are loaded by the agent at workflow start alongside the
|
||||
skill instructions. The daemon copies them to `/data/plugins/` before the container starts.
|
||||
|
||||
If `/data/plugins/` already has a plugin with the same name (from persistent project storage), the volume version wins.
|
||||
Conflicts are logged by the daemon.
|
||||
|
||||
### Layer 2: Core plugins
|
||||
|
||||
> **Status: aspirational, zero code.** No `Plugin` interface, no plugin registry, no `~/.config/archie/plugins/` loader
|
||||
> exists anywhere in the repo (confirmed by grep for `Plugin`, plugin registry, and `Register(daemon` -- zero matches).
|
||||
> Yaegi is a real dependency (`traefik/yaegi v0.16.1`) and is genuinely wired up, but only for the three narrow surfaces
|
||||
> in sec.5's reconciliation note above, all interpreted in-process inside `archied` -- not for extending the daemon with
|
||||
> new forge/ticketing/storage/secrets/notify implementations as described below.
|
||||
|
||||
Core plugins live at `~/.config/archie/plugins/` and extend the daemon itself:
|
||||
|
||||
```tree
|
||||
~/.config/archie/plugins/
|
||||
|---- forge-gitea.go # Gitea forge implementation
|
||||
|---- forge-gitlab.go # GitLab forge implementation
|
||||
|---- ticketing-jira.go # Jira ticketing integration
|
||||
|---- ticketing-linear.go # Linear ticketing integration
|
||||
|---- storage-s3.go # S3 persistent storage backend
|
||||
|---- vault-secrets.go # HashiCorp Vault integration
|
||||
|---- notify-servicenow.go # ServiceNow integration
|
||||
```
|
||||
|
||||
#### Loading
|
||||
|
||||
Loaded by the daemon at startup via Yaegi. Each plugin registers itself with the daemon's extension registry:
|
||||
|
||||
```go
|
||||
// Plugin interface (Go, interpreted via Yaegi)
|
||||
type Plugin interface {
|
||||
Name() string
|
||||
Version() string
|
||||
Register(daemon *Daemon) error
|
||||
}
|
||||
```
|
||||
|
||||
The daemon discovers `.go` files, evaluates them, extracts the plugin symbol, and calls `Register()`. Failed plugins are
|
||||
logged and skipped -- the daemon starts with remaining plugins.
|
||||
|
||||
#### What core plugins can do
|
||||
|
||||
- **Forges** -- implement the `forge.Forge` interface. Register themselves as available forge types.
|
||||
- **Ticketing** -- implement a ticketing interface. Poll external systems, format issues into the standard `forge.Issue`
|
||||
struct.
|
||||
- **Storage backends** -- implement a storage interface. Provide volume population and cleanup.
|
||||
- **Secrets** -- implement a vault interface. Inject credentials into `/data/secrets/` before agent start.
|
||||
- **Notifications** -- implement a notify interface. Deliver PRDs, status updates beyond the built-in webhook.
|
||||
|
||||
---
|
||||
|
||||
## 6. Task lifecycle (v2)
|
||||
|
||||
> **Status: implemented.** NATS JetStream is now the task distribution path (`internal/nats/client.go`, `internal/daemon/daemon.go:drainNATS()`). Docker containers spawn per task (`internal/container/pool.go`). The flow below matches the running code post-e4d11b1.
|
||||
|
||||
```
|
||||
Discover -> Publish to JetStream -> Agent consumes -> running(workflow:stage)
|
||||
v
|
||||
pr_open -> merged | rejected
|
||||
v
|
||||
waiting_human -> (approved) -> requeue
|
||||
v
|
||||
parked
|
||||
```
|
||||
|
||||
### Detailed flow
|
||||
|
||||
1. **Discover** -- daemon polls forges for labelled/assigned issues
|
||||
2. **Publish** -- daemon publishes task to `archie.task.<workflow_type>` JetStream subject
|
||||
3. **Spawn** -- daemon creates task volume, writes `task.json`, stages bundled plugins
|
||||
4. **Start container** -- `docker run` with volume mounts, network (NATS), resource limits
|
||||
5. **Agent boots** -- reads `task.json` from `/data/`, connects to NATS, discovers skills, loads bundled plugins
|
||||
6. **Run workflow** -- agent executes routed workflow stages. Reports progress to system channel. Result to response
|
||||
channel.
|
||||
7. **Daemon reviews** -- reads agent response. Forwards to human channels (issue comment, label update, PR). System
|
||||
messages stay internal.
|
||||
8. **Grace period** -- agent stays alive for `max_uptime`. Handles follow-ups (gate re-runs, human replies forwarded by
|
||||
daemon).
|
||||
9. **Kill** -- daemon kills container. Volume cleaned up on terminal outcome or TTL expiry.
|
||||
|
||||
---
|
||||
|
||||
## 7. Agent -> daemon -> human flow
|
||||
|
||||
```flow
|
||||
,---------, response ,---------, reviewed ,----------,
|
||||
| Agent |------------ -> | Daemon |------------ -> | Human |
|
||||
| | system | | queried | channels |
|
||||
| |------------ -> | |<-------------| |
|
||||
|------------ |------------ |-------------
|
||||
|
|
||||
|---- Issue comments (parked, PR, waiting)
|
||||
|---- Label updates (archie:queued -> archie:working -> archie:pr)
|
||||
|---- Notify webhook (feasibility PRD delivered)
|
||||
|---- SSE dashboard (real-time observability)
|
||||
```
|
||||
|
||||
The agent writes whatever it wants to `response`. The daemon reviews it. No agent output reaches a human unprompted. The
|
||||
`system` channel is purely internal -- the daemon reads it for observability and debugging, never forwards it.
|
||||
|
||||
---
|
||||
|
||||
## 8. Implementation phases
|
||||
|
||||
> Updated 2026-07-24 against actual repo state (see sec.0). Items marked **done** exist in the current tree; nothing else in
|
||||
> this document is implemented beyond them. Phase ordering below is unchanged from the original sketch except Phase 3,
|
||||
> which now builds on top of existing Yaegi work instead of assuming a blank slate.
|
||||
|
||||
### Phase 1: Binary split + NATS
|
||||
|
||||
- ~~Split `archied` into daemon binary~~ -- **done**, `cmd/archied` exists
|
||||
- ~~Create `archie-agent` binary~~ -- **done, but partial**: `cmd/archie-agent` exists and executes one workflow _stage_
|
||||
per invocation over a versioned stdin/stdout JSON protocol (`internal/agentexec`, protocol v1). Still needed: promote
|
||||
it from a single-stage subprocess worker to the PRD's per-task process, and move workflow routing/sequencing
|
||||
(`workflow.Route`/`workflow.Run`) out of `archied` and into it.
|
||||
- ~~Add NATS to both (JetStream for task queue, request-reply for comms)~~ -- **done**. `[nats]` is optional, supports
|
||||
anonymous local mode and token authentication via `token_env`, and leaves `internal/events.Bus` as the local
|
||||
observability fan-out.
|
||||
- ~~Daemon publishes, agent consumes, results return via NATS~~ -- **done**. Task discovery uses JetStream work-queue
|
||||
subjects; autonomous stages use JetStream requests plus core NATS reply inboxes.
|
||||
- Current single-process mode preserved for development (`--local` flag) -- effectively already true: `inprocess`
|
||||
`[agent] mode` is the _default_, not an opt-in fallback, so this constraint is satisfied but should be revisited once
|
||||
subprocess/container modes are the default.
|
||||
|
||||
### Phase 2: Docker sandbox
|
||||
|
||||
- Docker agent image (fat, with tools) -- not started
|
||||
- Daemon container lifecycle: spawn, volume create, env inject, kill -- not started
|
||||
- ~~Persistent volumes with TTL~~ -- **done**: configured repos receive a persistent Docker volume plus a per-repo Git
|
||||
object cache; both are automatically reaped using `containers.volume_ttl`.
|
||||
- ~~`max_concurrency` enforcement~~ -- **done**: SQLite and NATS drains dispatch different repositories concurrently
|
||||
up to `[containers].max_concurrency`; same-repo tasks remain serialised by default.
|
||||
- Grace period `max_uptime` -- not started
|
||||
- **Groundwork already in place:** `agentexec.Runner` (`InProcessRunner`/`SubprocessRunner`) is an interface designed,
|
||||
per its own doc comments, to be swapped for a container-backed runner without touching workflow orchestration --
|
||||
implement `ContainerRunner` against this interface rather than redesigning the execution boundary.
|
||||
|
||||
### Phase 3: Skills + bundled plugins
|
||||
|
||||
- agentskills.io spec implementation (progressive disclosure) -- not started; current SKILL.md files are unparsed
|
||||
documentation (see sec.5)
|
||||
- `archie-wf-*` skill discovery from worktree and user paths -- not started; current skills live at a single flat
|
||||
`.archie/skills/` path with different naming
|
||||
- Bundled plugin loading via Yaegi from skill's `plugins/` directory -- not started, but **has a working substrate to
|
||||
extend**: `internal/gate/gateeval`, `internal/workflow/wfeval`, and `internal/skillscript` already interpret `.go`
|
||||
files via Yaegi with generated symbol tables and panic recovery (see `docs/prds/yaegi-plugin-system.md`, Phases 1--3,
|
||||
done). This phase should teach those loaders to also discover files from a skill's `plugins/` directory, not build a
|
||||
second Yaegi integration from scratch.
|
||||
- Volume-staged plugins with conflict resolution -- not started, and depends on Phase 2's volume work existing first
|
||||
|
||||
### Phase 4: Core plugins
|
||||
|
||||
- Yaegi plugin loader for `~/.config/archie/plugins/` -- not started (no `Plugin` interface, no registry, no loader
|
||||
exist today)
|
||||
- Plugin registry in daemon -- not started
|
||||
- Gitea forge as first core plugin (proof of concept) -- not started (current `internal/forge` only implements GitHub
|
||||
REST polling)
|
||||
- Linear ticketing as second core plugin -- not started
|
||||
|
||||
### Phase 5: Pluggable storage
|
||||
|
||||
- Docker volume backend (built-in) -- not started
|
||||
- Bind mount backend (built-in) -- not started
|
||||
- S3, SFTP, NFS backends as core plugins -- not started
|
||||
|
||||
---
|
||||
|
||||
## 9. Open questions
|
||||
|
||||
1. **Agent image versioning:** Use tagged versions (`:v1.2.3`) with a "never repoint tags" policy. `:latest` is for dev
|
||||
only, not the config default. No digest complexity needed -- archie-core is self-hosted, not a public registry.
|
||||
|
||||
2. **NATS auth:** Anonymous for single-node deployments (daemon + NATS + agents on same host). Credentials required for
|
||||
multi-node/cluster. Add an optional `[nats]` config block (default: anonymous, validated only when present).
|
||||
|
||||
3. **Worktree ownership:** ~~Answered by current implementation.~~ Fresh `git clone` per task attempt, no shared cache.
|
||||
This is fine for v2. Revisit if concurrent agents on the same repo become a bottleneck.
|
||||
|
||||
4. **Grace period interaction:** Reset `max_uptime` on human reply -- the agent just did useful work. But cap total
|
||||
lifetime with `max_total_uptime` (e.g. 4 hours) to prevent infinite hanging. `max_uptime` = grace after inactivity,
|
||||
`max_total_uptime` = absolute kill switch.
|
||||
|
||||
5. **Concurrent agents on the same repo:** ~~Resolved.~~ Serialised by default (dispatcher chains tasks per
|
||||
`owner/repo`). `[[repos]]` now supports `allow_concurrent = true` (`config.Repo.AllowConcurrent`) to opt a repo out
|
||||
of that serialisation; the global `[containers].max_concurrency` slot limit still applies. Worktrees are already
|
||||
keyed per issue number (`worktree.Manager.Dir`), so opted-in repos don't collide on disk.
|
||||
|
||||
6. **Plugin architecture reconciliation:** Extend the existing Yaegi surfaces (`internal/gate/gateeval`,
|
||||
`internal/workflow/wfeval`, `internal/skillscript`) rather than building a parallel `Plugin.Register(daemon)`
|
||||
interface. Core plugins follow the same pattern: `Eval()` a `.go` file, extract known symbols. Settled before Phase
|
||||
3/4.
|
||||
|
||||
7. **Skill loading scope:** "Parse frontmatter, inject body into system prompt" is sufficient for v2. The gap is that
|
||||
nothing reads SKILL.md today -- closing that gap is the value. Full progressive disclosure (catalogue -> activate ->
|
||||
resources) is deferred to a later phase.
|
||||
@@ -1,389 +0,0 @@
|
||||
# archie-core Skills System -- PRD
|
||||
|
||||
**Author:** Archie
|
||||
**Date:** 2026-07-18
|
||||
**Status:** Draft
|
||||
|
||||
---
|
||||
|
||||
## What archie-core is
|
||||
|
||||
arcie-core is an LLM coding agent that responds to issues, queries, and commands on connected interfaces. The current MVP runs on GitHub: it receives issue assignments, analyses scope and requirements, implements changes in isolated worktrees, runs quality gates, and opens pull requests.
|
||||
|
||||
The skills system extends this with reusable playbooks that tell archie-core **how** to work on any given project.
|
||||
|
||||
---
|
||||
|
||||
## 1. The workflow file (`archie.yaml`)
|
||||
|
||||
Every project can ship an `archie.yaml` that defines how archie-core should behave when working on that codebase. Without one, archie-core uses sensible defaults (read files, run bash, open PRs). With one, it knows the exact gates to pass, tools available, commit format, and which skills are relevant.
|
||||
|
||||
### 1.1 Discovery
|
||||
|
||||
```
|
||||
.archie.yaml (root of repo -- highest precedence)
|
||||
archie.yaml (repo root)
|
||||
.archie/archie.yaml (subdirectory, one level deep)
|
||||
```
|
||||
|
||||
### 1.2 Schema
|
||||
|
||||
```yaml
|
||||
# archie.yaml
|
||||
version: "1"
|
||||
|
||||
# Project identity
|
||||
project:
|
||||
name: my-go-service
|
||||
language: go
|
||||
repo: github.com/sam/my-go-service
|
||||
|
||||
# How archie-core connects to this project
|
||||
connection:
|
||||
interfaces:
|
||||
- github # respond to assigned issues
|
||||
- linear # respond to labelled tickets
|
||||
- webhook # respond to HTTP POSTs
|
||||
github:
|
||||
repo: sam/my-go-service
|
||||
issue_labels:
|
||||
ready: "archie:ready" # issues with this label get claimed
|
||||
claimed: "archie:claimed" # archie sets this when working
|
||||
done: "archie:done" # archie sets this after PR
|
||||
pr:
|
||||
base: main
|
||||
draft: false
|
||||
|
||||
# Skills -- playbooks archie-core loads to know how to work
|
||||
skills:
|
||||
paths:
|
||||
- .archie/skills/ # project-local skills
|
||||
- ~/.archie/skills/ # per-user skills
|
||||
remotes:
|
||||
- github.com/sam/archie-skills@v1 # shared skill library
|
||||
|
||||
# Sandbox -- what's available in the container
|
||||
sandbox:
|
||||
image: ghcr.io/sam/archie-sandbox:latest
|
||||
tools:
|
||||
- go
|
||||
- golangci-lint
|
||||
- gofumpt
|
||||
- task
|
||||
- gh
|
||||
- docker
|
||||
env:
|
||||
GOFLAGS: "-mod=readonly"
|
||||
|
||||
# Gates -- archie-core must pass these before committing
|
||||
gates:
|
||||
test:
|
||||
run: go test -race -count=1 ./...
|
||||
timeout: 300
|
||||
lint:
|
||||
run: golangci-lint run ./...
|
||||
timeout: 120
|
||||
format:
|
||||
run: gofumpt -w . && git diff --exit-code
|
||||
deadcode:
|
||||
run: task deadcode
|
||||
optional: true
|
||||
|
||||
# Quality pipeline -- which gates run when
|
||||
quality:
|
||||
pre_commit: [test, lint, format]
|
||||
pre_push: [deadcode]
|
||||
|
||||
# Commit conventions
|
||||
commit:
|
||||
format: conventional
|
||||
signoff: false
|
||||
|
||||
# Branch conventions
|
||||
branch:
|
||||
format: "{type}/cat-{number}-{kebab-description}"
|
||||
types:
|
||||
feat: "Feature work"
|
||||
fix: "Bug fixes"
|
||||
test: "Test additions"
|
||||
chore: "Maintenance"
|
||||
refactor: "Code restructuring"
|
||||
docs: "Documentation"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Skills (playbooks)
|
||||
|
||||
Skills are Markdown documents that tell archie-core **how** to perform specific tasks. They follow the [agentskills.io](https://agentskills.io/specification) format, making them portable across compatible LLM coding agents.
|
||||
|
||||
### 2.1 Format
|
||||
|
||||
```
|
||||
go-testing/
|
||||
├── SKILL.md # Required: frontmatter + instructions
|
||||
├── references/ # Optional: supplementary docs
|
||||
│ └── coverage.md
|
||||
├── scripts/ # Optional: executable helpers
|
||||
│ └── bench.sh
|
||||
└── assets/ # Optional: templates
|
||||
└── test.vscode.code-snippets
|
||||
```
|
||||
|
||||
### 2.2 SKILL.md
|
||||
|
||||
```markdown
|
||||
---
|
||||
name: go-quality-gate
|
||||
description: >
|
||||
Run the full Go quality pipeline:
|
||||
gofumpt, go fix, golangci-lint, go test -race, and task deadcode.
|
||||
Use before any commit in a Go project.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
# Which gates this skill covers
|
||||
gates: [test, lint, format, deadcode]
|
||||
# Required sandbox tools
|
||||
tools: [go, golangci-lint, gofumpt, task]
|
||||
# Preferred engine (if multiple available)
|
||||
engine: any
|
||||
# Max spend for this skill
|
||||
budget_usd: 2.00
|
||||
---
|
||||
|
||||
# Go Quality Gate
|
||||
|
||||
## When to use
|
||||
|
||||
Before committing any Go change. After implementing a feature or fix.
|
||||
|
||||
## Procedure
|
||||
|
||||
Run in this exact order. If any step fails, fix the issue before continuing.
|
||||
|
||||
1. `gofumpt -w .`
|
||||
2. `go fix ./...`
|
||||
3. `golangci-lint run ./...`
|
||||
4. `go test -race -count=1 ./...`
|
||||
5. `task deadcode`
|
||||
|
||||
## Verification
|
||||
|
||||
All five steps must exit 0. If `task deadcode` fails on unrelated packages,
|
||||
scope it to only changed packages: `golangci-lint run --tests=false --enable-only=unused,staticcheck ./changed/pkg/...`
|
||||
|
||||
## Common failures
|
||||
|
||||
- `golangci-lint`: unused function -- remove it or add `//nolint:unused` comment
|
||||
- `go test -race`: data race -- protect with mutex or channel
|
||||
- `task deadcode`: pre-existing dead code in unrelated package -- scope to changed packages
|
||||
```
|
||||
|
||||
### 2.3 Skill metadata (arcie-specific)
|
||||
|
||||
Skills carry archie-specific metadata under `metadata.archie`:
|
||||
|
||||
| Field | Purpose |
|
||||
| ------------ | ------------------------------------------------ |
|
||||
| `gates` | Which gates from `archie.yaml` this skill covers |
|
||||
| `tools` | Sandbox tools this skill needs |
|
||||
| `engine` | Preferred execution engine |
|
||||
| `model` | Preferred model for this skill |
|
||||
| `budget_usd` | Max spend ceiling |
|
||||
| `timeout` | Max seconds |
|
||||
| `worktree` | Whether to create isolated worktree |
|
||||
|
||||
### 2.4 Progressive disclosure
|
||||
|
||||
Following the skills.sh model to keep context lean:
|
||||
|
||||
1. **Advertise** (~100 tokens per skill): `name` + `description` loaded at startup
|
||||
2. **Activate** (<5000 tokens): Full SKILL.md body loaded when skill is relevant
|
||||
3. **Resources** (as needed): References, scripts, assets loaded on demand
|
||||
|
||||
---
|
||||
|
||||
## 3. How archie-core processes an issue
|
||||
|
||||
This is the core loop. When an issue lands with the configured label:
|
||||
|
||||
### 3.1 Analyse
|
||||
|
||||
1. Read issue title, body, labels, comments
|
||||
2. Load relevant skills from the project's `archie.yaml`
|
||||
3. Determine scope: which packages, files, and gates are in play
|
||||
4. Determine branch type (`feat/`, `fix/`, `test/`, etc.) from issue labels or conventional commit prefix
|
||||
|
||||
### 3.2 Claim
|
||||
|
||||
1. Set issue label to `archie:claimed`
|
||||
2. Comment: "Archie claimed this. Branch: `<branch>`"
|
||||
3. Create isolated worktree from `main`
|
||||
|
||||
### 3.3 Implement
|
||||
|
||||
1. Load full skill bodies for each relevant skill
|
||||
2. Read project source and tests in the affected area
|
||||
3. Implement changes, writing tests first where applicable
|
||||
4. Run targeted tests iteratively
|
||||
|
||||
### 3.4 Gate
|
||||
|
||||
1. Run all gates from the quality pipeline in order
|
||||
2. Fix any failures, re-running from the failing gate
|
||||
3. Audit diff for secrets, unrelated changes, attribution
|
||||
|
||||
### 3.5 Deliver
|
||||
|
||||
1. Commit with conventional format: `feat(scope): description`
|
||||
2. Push branch
|
||||
3. Open PR to `main` with:
|
||||
- Summary of changes
|
||||
- Acceptance criteria addressed
|
||||
- Gate results
|
||||
- Link back to source issue/Linear ticket
|
||||
4. Set issue label to `archie:done`
|
||||
5. Comment with PR URL
|
||||
|
||||
### 3.6 On failure
|
||||
|
||||
If any gate fails irrecoverably:
|
||||
|
||||
1. Leave issue as `archie:claimed`
|
||||
2. Comment with exact blocker, failed command, and worktree path
|
||||
3. Preserve worktree for inspection
|
||||
4. Do not open a broken PR
|
||||
|
||||
---
|
||||
|
||||
## 4. Engine abstraction
|
||||
|
||||
arcie-core can delegate implementation to different LLM engines. The workflow file declares preference order; archie-core resolves the first available engine at runtime.
|
||||
|
||||
```yaml
|
||||
engines:
|
||||
- type: claude # Claude Code CLI -- full agent loop
|
||||
model: sonnet
|
||||
budget_usd: 5.00
|
||||
- type: codex # OpenAI Codex CLI
|
||||
model: gpt-5.1-codex-max
|
||||
- type: raw # Direct LLM -- no agent loop, single prompt
|
||||
model: gpt-4o-mini
|
||||
```
|
||||
|
||||
Each engine provides the same interface:
|
||||
|
||||
```
|
||||
Engine:
|
||||
- Run(prompt, tools, budget, timeout) -> Result
|
||||
- Skills() -> []SkillName
|
||||
- Capabilities() -> {streaming, reasoning, vision, agentic}
|
||||
```
|
||||
|
||||
For skills that need a specific engine, the skill metadata overrides:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
archie:
|
||||
engine: claude # this skill only works with Claude
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Skills in practice
|
||||
|
||||
### 5.1 Shared skill library
|
||||
|
||||
```yaml
|
||||
# archie.yaml
|
||||
skills:
|
||||
remotes:
|
||||
- github.com/sam/archie-skills@v1
|
||||
```
|
||||
|
||||
Skills from the shared library are available alongside project-local skills. The agent loads them on demand when an issue references relevant concepts.
|
||||
|
||||
### 5.3 Skill that wraps an external tool
|
||||
|
||||
```yaml
|
||||
---
|
||||
name: trivy-scan
|
||||
description: >
|
||||
Scan container images and filesystems for vulnerabilities using Trivy.
|
||||
Use when asked about security scanning, CVEs, or container hardening.
|
||||
metadata:
|
||||
archie:
|
||||
tools: [docker, trivy]
|
||||
engine: any
|
||||
---
|
||||
## Procedure
|
||||
1. `trivy image --severity HIGH,CRITICAL <image>`
|
||||
2. If findings exist, report each CVE with package, version, and fixed version
|
||||
3. If no findings, report clean
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Comparison: archie-core vs existing systems
|
||||
|
||||
| | archie-core | Claude Code | Codex CLI | GitHub Actions | External Agent |
|
||||
| ------------------------ | ---------------------------------------------- | ------------------------------- | ------------------------------- | ---------------------------------- | ------------------------------- |
|
||||
| **How it receives work** | GitHub issue assignment, Linear label, webhook | Chat, slash commands | Chat, slash commands | YAML triggers (push, PR, schedule) | Chat, cron, webhooks |
|
||||
| **What it does** | Analyse → implement → gate → PR | Analyse → implement → gate → PR | Analyse → implement → gate → PR | Run arbitrary shell commands | Analyse → implement → gate → PR |
|
||||
| **Skills** | SKILL.md (agentskills.io) | SKILL.md (native) | SKILL.md (agentskills.io) | Composite actions | SKILL.md (native) |
|
||||
| **Gates** | Named checkpoints in archie.yaml | Ad-hoc (CLAUDE.md + lint) | Ad-hoc | Steps in jobs | Skills with gates |
|
||||
| **Engine pluggable** | Yes | No (fixed to Claude) | No (fixed to Codex) | No (fixed to GitHub) | Yes (provider config) |
|
||||
| **Worktree isolation** | Yes | Yes (`--worktree`) | No | No (ephemeral runner) | No |
|
||||
| **Blueprint/cron** | Export externally | No | No | Schedule trigger | Native |
|
||||
| **Where it runs** | Docker sandbox | Host | Host or sandbox | GitHub runner | Host, Docker, Modal, SSH |
|
||||
|
||||
---
|
||||
|
||||
## 7. Implementation phases
|
||||
|
||||
### Phase 1: Workflow file and gates (current MVP)
|
||||
|
||||
- `archie.yaml` parser
|
||||
- Gate runner executing shell commands and checking exit codes
|
||||
- Skill discovery from project and user paths
|
||||
- GitHub interface: claim issues, create worktrees, open PRs
|
||||
|
||||
### Phase 2: Skills
|
||||
|
||||
- SKILL.md progressive disclosure (advertise → activate → resources)
|
||||
- archie-specific metadata parsing
|
||||
- Skill chaining: skills that depend on other skills
|
||||
- `archie skills list/install` CLI
|
||||
|
||||
### Phase 3: Engine abstraction
|
||||
|
||||
- Engine interface definition
|
||||
- Claude Code adapter
|
||||
- Raw LLM adapter
|
||||
- Engine fallback chain
|
||||
- Per-skill engine preferences
|
||||
|
||||
### Phase 4: Advanced features
|
||||
|
||||
- Gate overrides per skill
|
||||
- Skill templates for project scaffolding (`archie init --skill go-service`)
|
||||
- Remote skill libraries (git repos as skill sources)
|
||||
- Blueprint export to an external scheduler
|
||||
|
||||
### Phase 5: Additional interfaces
|
||||
|
||||
- Linear webhook integration
|
||||
- Generic webhook interface for arbitrary sources
|
||||
- CLI command interface (`archie run <command>`)
|
||||
|
||||
---
|
||||
|
||||
## 8. Open questions
|
||||
|
||||
1. **Claim protocol:** Single-claim (arcie takes one issue at a time) or queue-based (claims N issues, works through them sequentially)?
|
||||
2. **Concurrent worktrees:** Allow multiple archie-core instances working on different issues simultaneously?
|
||||
3. **PR merging:** Should archie-core ever auto-merge, or always require human review?
|
||||
4. **Skill registry:** Self-host on git.catlow.cloud, use agentskills.io, or both?
|
||||
5. **Engine budget enforcement:** Per-issue, per-skill, or monthly cap?
|
||||
6. **Sandbox building:** Pre-built Docker images vs. project-defined Dockerfiles vs. Nix shells?
|
||||
@@ -1,400 +0,0 @@
|
||||
# archie-core Workflows & Skills -- Product Requirements Document
|
||||
|
||||
**Author:** Archie
|
||||
**Date:** 2026-07-18
|
||||
**Status:** Draft
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This PRD defines a workflow and skills system for archie-core. Workflow files declare gates, tool availability, execution engines, and skill references. Skills are reusable playbook documents the agent loads on demand. Together they enable any project to define how agents work within their sandbox and select an available execution engine.
|
||||
|
||||
---
|
||||
|
||||
## 1. Workflow Files (`archie.yaml`)
|
||||
|
||||
### 1.1 Discovery
|
||||
|
||||
A project declares its workflow in `archie.yaml` at the project root. archie-core discovers it the same way Task (Taskfile) and devcontainers do:
|
||||
|
||||
```
|
||||
.archie.yaml (root of repo -- highest precedence)
|
||||
archie.yaml (repo root)
|
||||
.archie/archie.yaml (subdirectory, one level deep)
|
||||
```
|
||||
|
||||
If no file is found, archie-core uses built-in defaults (read, write, bash).
|
||||
|
||||
### 1.2 Schema
|
||||
|
||||
```yaml
|
||||
# archie.yaml -- complete schema
|
||||
version: "1"
|
||||
|
||||
# Project identity
|
||||
project:
|
||||
name: my-go-service
|
||||
language: go
|
||||
|
||||
# Available skills (directories or individual files)
|
||||
skills:
|
||||
paths:
|
||||
- .archie/skills/ # project-local skills
|
||||
- ~/.archie/skills/ # per-user skills
|
||||
- github.com/org/shared-skills@v1 # remote skill repo
|
||||
skills:
|
||||
- go-testing # named skill from any path
|
||||
- security-audit
|
||||
|
||||
# Execution engines in preference order
|
||||
engines:
|
||||
- type: claude # Claude Code CLI
|
||||
model: sonnet
|
||||
- type: codex # OpenAI Codex
|
||||
- type: pi # Pi agent harness
|
||||
- type: raw # Direct LLM without agent wrapper
|
||||
|
||||
# Sandbox definition (what the Docker container provides)
|
||||
sandbox:
|
||||
image: ghcr.io/samcharles93/archie-sandbox:latest
|
||||
tools:
|
||||
- go
|
||||
- golangci-lint
|
||||
- gofumpt
|
||||
- task
|
||||
- gh
|
||||
- docker
|
||||
mounts:
|
||||
- source: /var/run/docker.sock
|
||||
target: /var/run/docker.sock
|
||||
env:
|
||||
GOFLAGS: "-mod=readonly"
|
||||
CGO_ENABLED: "0"
|
||||
|
||||
# Gates -- ordered checkpoints that must pass
|
||||
gates:
|
||||
test:
|
||||
run: go test -race -count=1 ./...
|
||||
timeout: 300
|
||||
description: "All tests must pass"
|
||||
lint:
|
||||
run: golangci-lint run ./...
|
||||
timeout: 120
|
||||
description: "Zero lint issues"
|
||||
format:
|
||||
run: gofumpt -w . && git diff --exit-code
|
||||
description: "No formatting drift"
|
||||
vet:
|
||||
run: go vet ./...
|
||||
description: "No vet warnings"
|
||||
deadcode:
|
||||
run: task deadcode
|
||||
optional: true # non-blocking gate
|
||||
description: "Dead code check (advisory)"
|
||||
|
||||
# Quality pipeline -- ordered sequence of gate groups
|
||||
quality:
|
||||
pre_commit:
|
||||
- test
|
||||
- lint
|
||||
- format
|
||||
- vet
|
||||
pre_push:
|
||||
- deadcode
|
||||
|
||||
# Commit conventions
|
||||
commit:
|
||||
format: conventional # conventional commits
|
||||
signoff: false
|
||||
template: "feat({scope}): {description}"
|
||||
|
||||
# PR conventions
|
||||
pr:
|
||||
template: .github/PULL_REQUEST_TEMPLATE.md
|
||||
checks:
|
||||
- test
|
||||
- lint
|
||||
|
||||
# Project-specific commands (what `archie run <name>` executes)
|
||||
commands:
|
||||
build:
|
||||
run: go build -o bin/app ./cmd/server
|
||||
description: "Build the binary"
|
||||
serve:
|
||||
run: go run ./cmd/server
|
||||
description: "Start dev server"
|
||||
test:
|
||||
run: go test ./...
|
||||
description: "Run all tests"
|
||||
lint:
|
||||
run: golangci-lint run ./...
|
||||
description: "Run linter"
|
||||
all:
|
||||
run:
|
||||
- gofumpt -w .
|
||||
- go fix ./...
|
||||
- golangci-lint run ./...
|
||||
- go test -race ./...
|
||||
- task deadcode
|
||||
description: "Full quality pipeline"
|
||||
```
|
||||
|
||||
### 1.3 Design decisions (with rationale)
|
||||
|
||||
| Decision | Rationale |
|
||||
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
||||
| YAML over TOML/JSON | Already dominant in CI (GitHub Actions, GitLab CI, Taskfile). Human-readable with comments. |
|
||||
| Single file over multi-file | Task discovery (like `Taskfile.yml`), devcontainer conventions. Avoids directory sprawl. |
|
||||
| Gates as named checkpoints | GitHub Actions job model. Gates can be composed into quality pipelines, selected individually, or skipped with `--skip`. |
|
||||
| Engines as ordered preference list | Supports deterministic fallback between available engines. Each engine declares its own tool surface. |
|
||||
| Skills as paths + references | Matches devcontainer features (paths resolve to directories) and skills.sh (namespaced references). |
|
||||
| Commands section for direct invocation | Arnie can `archie run test` without an agent turn. Thin wrapper over the sandbox. |
|
||||
| Quality pipeline groups gates | Pre-commit vs pre-push vs pre-merge distinction. Some gates are required, some advisory. |
|
||||
|
||||
---
|
||||
|
||||
## 2. Skills
|
||||
|
||||
### 2.1 Format
|
||||
|
||||
Skills follow the [agentskills.io](https://agentskills.io/specification) specification:
|
||||
|
||||
```
|
||||
skill-name/
|
||||
├── SKILL.md # Required: YAML frontmatter + Markdown instructions
|
||||
├── scripts/ # Optional: executable helpers
|
||||
├── references/ # Optional: longer docs, checklists
|
||||
└── assets/ # Optional: templates, configs
|
||||
```
|
||||
|
||||
**SKILL.md frontmatter:**
|
||||
|
||||
```yaml
|
||||
---
|
||||
name: go-testing
|
||||
description: >
|
||||
Run Go tests with race detection, coverage, and benchmark analysis.
|
||||
Use when asked to test Go code, run the test suite, or check coverage.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
gates: [test] # gates this skill requires
|
||||
tools: [go, golangci-lint] # sandbox tools it needs
|
||||
engine: any # engine preference
|
||||
compatibility: Requires Go 1.22+
|
||||
---
|
||||
```
|
||||
|
||||
### 2.2 Progressive disclosure
|
||||
|
||||
Following the skills.sh and Claude Code patterns:
|
||||
|
||||
1. **Advertise** (~100 tokens): `name` + `description` loaded at session start
|
||||
2. **Load** (<5000 tokens recommended): Full SKILL.md body when skill activates
|
||||
3. **Resources** (as needed): Scripts, references, assets loaded on demand
|
||||
|
||||
### 2.3 archie-specific metadata
|
||||
|
||||
Skills can carry archie-specific frontmatter under `metadata.archie`:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
archie:
|
||||
gates: [test, lint] # must-pass gates before skill runs
|
||||
tools: [go, docker] # sandbox tools required
|
||||
engine: any # preferred engine
|
||||
model: gpt-4o # preferred model
|
||||
budget_usd: 2.00 # max spend ceiling
|
||||
timeout: 600 # max seconds
|
||||
worktree: true # create isolated worktree
|
||||
```
|
||||
|
||||
### 2.4 Skill sources (inspired by devcontainer features + Dagger modules)
|
||||
|
||||
```
|
||||
skills:
|
||||
paths:
|
||||
- .archie/skills/ # project-local (committed)
|
||||
- ~/.archie/skills/ # per-user (personal)
|
||||
skills:
|
||||
- go-testing # by name from any path
|
||||
remotes:
|
||||
- github.com/samcharles93/archie-skills@v1 # tagged git repo
|
||||
- https://agentskills.io/registry # community registry
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Execution Engine Model
|
||||
|
||||
### 3.1 Engine abstraction
|
||||
|
||||
Each engine provides the same interface. The workflow file declares preference order; archie-core resolves the first available engine at runtime.
|
||||
|
||||
```
|
||||
Engine interface:
|
||||
- Run(prompt, tools[], budget, timeout) -> Result
|
||||
- Tools() -> []ToolSchema
|
||||
- Capabilities() -> {streaming, reasoning, vision, agentic}
|
||||
```
|
||||
|
||||
### 3.2 Supported engines
|
||||
|
||||
| Engine | Type | What it provides |
|
||||
| -------- | ---------------- | ------------------------------------------------------ |
|
||||
| `claude` | Claude Code CLI | Full agent loop, tool use, worktrees, subagents |
|
||||
| `codex` | OpenAI Codex CLI | Agentic coding with sandbox |
|
||||
| `pi` | Pi agent harness | Minimalist coding agent, stdio protocol |
|
||||
| `raw` | Direct LLM call | Single-turn prompt, no agent loop. For simple scripts. |
|
||||
|
||||
### 3.3 Engine selection logic
|
||||
|
||||
```yaml
|
||||
engines:
|
||||
- type: claude
|
||||
model: sonnet
|
||||
budget_usd: 5.00
|
||||
- type: raw
|
||||
model: gpt-4o-mini
|
||||
```
|
||||
|
||||
arcie-core tries engines in order and selects the first available,
|
||||
authenticated engine. Each engine declares its budget ceiling.
|
||||
|
||||
---
|
||||
|
||||
## 4. Additional Features
|
||||
|
||||
### 4.1 Worktree isolation (`worktree: true`)
|
||||
|
||||
Inspired by Claude Code's `--worktree` flag. When a skill or command sets `worktree: true`, archie-core:
|
||||
|
||||
1. Creates an isolated git worktree at `.archie/worktrees/<skill-name>-<timestamp>/`
|
||||
2. Runs all gates and implementation in that worktree
|
||||
3. Cleans up on success, preserves on failure for inspection
|
||||
|
||||
This prevents concurrent agent sessions from conflicting and enables parallel agent execution.
|
||||
|
||||
### 4.2 Skill chaining and composition
|
||||
|
||||
Skills can declare dependencies, forming a DAG:
|
||||
|
||||
```yaml
|
||||
---
|
||||
name: full-ci
|
||||
description: Complete CI pipeline including security audit
|
||||
metadata:
|
||||
archie:
|
||||
requires: [go-testing, security-audit]
|
||||
run_sequential: true
|
||||
---
|
||||
```
|
||||
|
||||
arcie-core resolves the dependency graph and runs skills in order (parallel where possible, respecting `run_sequential`). This is conceptually similar to GitHub Actions `needs` and Taskfile `deps`.
|
||||
|
||||
### 4.3 Gate overrides per skill
|
||||
|
||||
A skill can override project-level gates:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
archie:
|
||||
gates:
|
||||
test:
|
||||
run: go test -short ./... # override: faster test for this skill
|
||||
lint: skip # skip lint for this skill
|
||||
```
|
||||
|
||||
Three override modes: `override` (replace command), `skip` (omit gate), `add` (add new gate requirement).
|
||||
|
||||
### 4.4 Skill templates (scaffolding)
|
||||
|
||||
Skills can carry an `assets/` directory with templates. When archie-core initialises a new project, it can apply skill templates:
|
||||
|
||||
```
|
||||
skill-name/
|
||||
├── SKILL.md
|
||||
└── assets/
|
||||
├── .archie.yaml.tmpl # Template workflow file
|
||||
├── Taskfile.yml.tmpl # Template task runner
|
||||
└── .golangci.yml.tmpl # Template linter config
|
||||
```
|
||||
|
||||
`archie init --skill go-service` scaffolds a new project with the go-service skill's templates applied.
|
||||
|
||||
### 4.5 Blueprint export (external cron integration)
|
||||
|
||||
Any archie-core workflow can be exported as an automation blueprint:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
archie:
|
||||
blueprint:
|
||||
schedule: "every 6h"
|
||||
deliver: telegram
|
||||
name: "CI monitor"
|
||||
```
|
||||
|
||||
The export produces a `SKILL.md` with scheduler metadata. The blueprint's prompt is the archie workflow's quality pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 5. Comparison to Existing Systems
|
||||
|
||||
| Feature | archie-core | GitHub Actions | Dagger | Taskfile | devcontainer | External Skills |
|
||||
| ---------------------- | ------------------- | ----------------- | -------------- | ---------- | ----------------- | --------------- |
|
||||
| Language | YAML | YAML | Go/Python/TS | YAML | JSON | Markdown |
|
||||
| Gates | Named + composable | Steps in jobs | Functions | Tasks | Lifecycle scripts | N/A |
|
||||
| Engine abstraction | First-class | GitHub only | Dagger Engine | Shell only | None | Provider |
|
||||
| Skills | agentskills.io spec | Composite actions | Dagger Modules | Includes | Features | SKILL.md |
|
||||
| Worktree isolation | Built-in | No | Containers | No | Container | No |
|
||||
| Blueprint export | To external cron | No | No | No | No | Native |
|
||||
| Multi-engine fallback | Yes (ordered list) | No | No | No | No | Provider config |
|
||||
| Progressive disclosure | Yes (skill spec) | No | No | No | Feature metadata | Yes |
|
||||
|
||||
---
|
||||
|
||||
## 6. Implementation Phases
|
||||
|
||||
### Phase 1: Core workflow file
|
||||
|
||||
- `archie.yaml` parser and schema validation
|
||||
- Gate runner (test, lint, format, vet)
|
||||
- `archie run <command>` CLI
|
||||
- Sandbox Docker integration
|
||||
|
||||
### Phase 2: Skills
|
||||
|
||||
- SKILL.md discovery and progressive disclosure
|
||||
- Skill metadata with gate/tool/engine requirements
|
||||
- `archie skills list/install/search`
|
||||
|
||||
### Phase 3: Engine abstraction
|
||||
|
||||
- Engine interface definition
|
||||
- Claude Code adapter
|
||||
- Raw LLM adapter
|
||||
- Engine fallback chain
|
||||
|
||||
### Phase 4: Advanced features
|
||||
|
||||
- Worktree isolation
|
||||
- Skill chaining (dependency DAG)
|
||||
- Gate overrides
|
||||
- Blueprint export
|
||||
|
||||
### Phase 5: Ecosystem
|
||||
|
||||
- Public skill registry
|
||||
- `archie init --skill <name>` scaffolding
|
||||
- Community skill contributions
|
||||
|
||||
---
|
||||
|
||||
## 7. Open Questions
|
||||
|
||||
1. **Skill registry hosting:** Self-hosted on git.catlow.cloud, or use agentskills.io?
|
||||
2. **External integration:** Should archie-core be exposed as a tool, a skill, or standalone?
|
||||
3. **Budget enforcement:** Per-skill budget vs. per-workflow budget vs. monthly cap?
|
||||
4. **Sandbox image building:** Pre-built images vs. Dockerfile in each skill?
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,220 +0,0 @@
|
||||
# Archie Design Decisions
|
||||
|
||||
Record of architectural choices, reversals, and lessons learned during
|
||||
development. Each entry has a date, a decision, the reasoning, and the
|
||||
context that led to it. This document is the first thing to read before
|
||||
changing anything that contradicts a prior decision.
|
||||
|
||||
---
|
||||
|
||||
## 2026-08-24 -- Turn continuation is never output-volume or token gated
|
||||
|
||||
**Decision**: A turn is never stopped because an aggregate token count or
|
||||
aggregate tool-result character count was reached. `max_tokens` and
|
||||
`turn_budget_chars` are not runtime continuation controls. Tool output is
|
||||
bounded only per invocation with `max_result_chars`; oversized individual
|
||||
results are spilled when a workspace-readable spill directory is configured,
|
||||
or explicitly truncated otherwise.
|
||||
|
||||
**Reasoning**: Aggregate caps discard useful work and caused the model to emit
|
||||
repeated `turn budget exceeded (200000 chars)` tool failures instead of
|
||||
finishing. Model context/output accounting and tool-result volume are distinct:
|
||||
provider output limits may describe one model response, but they must never be
|
||||
used to terminate the multi-step turn.
|
||||
|
||||
**Context**: GitHub #604 showed full file and module listings followed by four
|
||||
identical read failures and a shell failure at the old 200,000-character cap.
|
||||
The renderer now summarises those legacy events for replay, while current
|
||||
dispatch never creates them. Find and grep are confined to the active
|
||||
workspace/worktree; the Go module cache remains readable only by explicit
|
||||
path, not by discovery.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-23 -- Forge-agnostic issue creation via CreateIssue
|
||||
|
||||
**Decision**: Add `CreateIssue(ctx, owner, repo, title, body, labels)
|
||||
(int, error)` to the `forge.Forge` interface. Every forge backend
|
||||
(Gitea, GitHub, future beads) implements it.
|
||||
|
||||
**Reasoning**: Agents discover out-of-scope findings during planning
|
||||
and building. These should be filed as tickets immediately rather than
|
||||
lost. Making `CreateIssue` part of the Forge interface means the agent
|
||||
calls one method regardless of the underlying ticketing system.
|
||||
|
||||
**Future**: A `beads` forge backend will implement `CreateIssue` by
|
||||
calling `bd create`. This makes beads the internal/default issue tracker
|
||||
while Gitea/GitHub are external forges. The agent doesn't know which
|
||||
it's talking to -- the daemon resolves the forge from repo config.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-23 -- Dolt and Beads for agent-native issue tracking
|
||||
|
||||
**Decision**: Install dolt (version-controlled SQL) and beads (`bd`,
|
||||
a graph-based issue tracker for AI agents) as the foundation for
|
||||
multi-repo scaling and multi-agent coordination.
|
||||
|
||||
**Reasoning**: Beads provides persistent memory across sessions, first-
|
||||
class dependency tracking, JSON output for programmatic use, and hash-
|
||||
based IDs that prevent merge conflicts in multi-agent workflows. It's
|
||||
built on Dolt which gives cell-level merge, native branching, and
|
||||
remote sync -- the same primitives git gives code but applied to issues.
|
||||
|
||||
**Context**: Gas Town (`~/gt/`) is cloned for reference -- it implements
|
||||
a multi-agent workspace on top of beads. The archie-core scaling
|
||||
strategy will follow a similar model: a coordinator daemon with per-repo
|
||||
agents, all sharing a beads-backed task graph.
|
||||
|
||||
**Current state**: dolt 2.2.2, beads 1.1.0 installed. Beads initialised
|
||||
in archie-core (`.beads/`). Six issues seeded for the scaling roadmap.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-23 -- Token budgets must be generous, not predictive
|
||||
|
||||
> **Superseded by the 2026-08-24 decision above.** There is no configured
|
||||
> aggregate token cap; model token usage remains accounting only.
|
||||
|
||||
**Decision**: Set `max_tokens` to 100,000,000 (100M). Do not attempt to
|
||||
predict what a task needs without data. Lower limits later based on
|
||||
actual per-workflow/per-repo metrics.
|
||||
|
||||
**Reasoning**: An agent killed mid-task at 1M tokens wastes every token
|
||||
it already consumed. There is no compaction or checkpointing yet, so a
|
||||
killed agent is a total loss. Long-running agents regularly use 160+ steps and
|
||||
hundreds of thousands of tokens per request. Large repositories need
|
||||
proportionally more.
|
||||
|
||||
**Context**: The planner burned 357K tokens just to discover a fix
|
||||
already existed. The builder hit 1M at step 25 of 90. Neither task was
|
||||
unreasonable -- they were just exploring a large codebase.
|
||||
|
||||
**Future**: Track actual token consumption per workflow/stage/repo. Use
|
||||
that data to set informed limits. Build compaction or checkpointing so
|
||||
a killed agent can resume rather than restart.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-23 -- baseline-fix: auto-repair pre-existing gate failures
|
||||
|
||||
**Decision**: When `StageBaselineGate` detects a red gate (gofumpt, go
|
||||
vet, etc. failing on clean main), launch the builder to fix the
|
||||
failures via TDD rather than parking. Only park if the builder cannot
|
||||
fix them.
|
||||
|
||||
**Reasoning**: A red baseline means the repo was broken before archie
|
||||
touched it. Parking creates an infinite loop -- the next retry hits the
|
||||
same failure. The agent should fix pre-existing issues, not fail on them.
|
||||
|
||||
**Implementation**: `StageBaselineGate` now calls `tc.Agent.Run()` with
|
||||
a TDD mission when the gate fails. Uses the same builder agent. Note:
|
||||
this required wiring `tc.Agent` (the runner) through the stage context.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-23 -- No hardcoded MaxSteps in workflow stages
|
||||
|
||||
**Decision**: Remove all hardcoded `MaxSteps` values from workflow
|
||||
stages. Every stage inherits from `[budgets].max_steps` in the config.
|
||||
|
||||
**Reasoning**: Hardcoded values (15 for plan, 15 for TDD analyse, 15
|
||||
for feasibility assess, 20 for PRD) were arbitrary and broke on large
|
||||
repos. The config should be the single source of truth.
|
||||
|
||||
**Files affected**: `implement.go`, `tdd.go`, `feasibility.go`.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-23 -- No-op detection: close issues when build has 0 changes
|
||||
|
||||
**Decision**: When the builder completes with `StatusPassed` and 0 file
|
||||
changes, close the issue with a comment rather than parking on "worktree
|
||||
has no changes to commit".
|
||||
|
||||
**Reasoning**: A builder that passes with 0 changes means the fix
|
||||
already exists or the issue is a no-op. The old behaviour parked in an
|
||||
infinite loop (commit-push fails because nothing changed, task re-queues,
|
||||
builder runs again, same result).
|
||||
|
||||
**Implementation**: `TaskContext.BuildNoChanges` is set in the build
|
||||
stage's `OnResult` when `Status == Passed && len(Changes) == 0`.
|
||||
`StageCommitPush` checks this flag and calls `Forge.CloseIssue()` +
|
||||
sets `Outcome = Merged` to stop the workflow.
|
||||
|
||||
**Test**: `TestStageCommitPushClosesIssueWhenBuildNoChanges` in
|
||||
`implement_test.go`.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-23 -- assignee trigger over label trigger
|
||||
|
||||
**Decision**: Default to `trigger = "assignee"` instead of `trigger =
|
||||
"label"`. The daemon only claims issues explicitly assigned to the bot
|
||||
user.
|
||||
|
||||
**Reasoning**: Codex created 80+ issues with the `archie` label, and
|
||||
the daemon auto-claimed every one. Label-based triggering is too
|
||||
promiscuous for repos shared between humans and agents.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-23 -- DeepSeek v4 Pro as primary model
|
||||
|
||||
**Decision**: Use `deepseek/deepseek-v4-pro` as the planner and builder
|
||||
model. Qwythos-9B (local llama.cpp) is the fallback for offline/cheap
|
||||
tasks.
|
||||
|
||||
**Reasoning**: Qwythos couldn't complete a representative large-repository
|
||||
plan stage before timing out. DeepSeek completed it in 21 iterations. Large
|
||||
repositories need a larger model.
|
||||
|
||||
**Provider setup**: `class = "deepseek"`, API key via `DEEPSEEK_API_KEY`
|
||||
env var from Bitwarden Secrets Manager (bws).
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-22 -- v1.0.0 tagged
|
||||
|
||||
**Decision**: Tagged `v1.0.0` at commit incorporating the daemon/agent
|
||||
split, NATS JetStream, Docker sandbox, Yaegi plugins, agentskills.io
|
||||
skills, and Gitea forge support.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-22 -- Docker Compose stack with NATS + containers
|
||||
|
||||
**Decision**: The production deployment uses `docker-compose.yml` with
|
||||
NATS, archied, and archie-agent containers. Config at `config.docker.toml`.
|
||||
|
||||
**Key fixes**:
|
||||
|
||||
- NATS healthcheck uses `wget localhost:8222/healthz` (not `nats-server check`)
|
||||
- `host.docker.internal:host-gateway` extra_hosts for Linux
|
||||
- Config mount path: `config.docker.toml` → `/etc/archie/config.toml`
|
||||
- No stale `image:` tag on archied service (forces local build)
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-22 -- Worktree Prepare before Container Acquire
|
||||
|
||||
**Decision**: Clone the worktree (`Trees.Prepare()`) before acquiring
|
||||
a Docker container. Made `Prepare()` idempotent -- skips if `.git`
|
||||
already exists.
|
||||
|
||||
**Reasoning**: Docker bind-mounts fail with "bind source path does not
|
||||
exist" if the worktree hasn't been cloned yet. The old code acquired
|
||||
the container first, then the prepare stage cloned into a path that
|
||||
didn't exist on the host.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-22 -- Gofumpt and Go tools must be on PATH
|
||||
|
||||
**Decision**: The daemon's `exec.Command` for gate checks inherits the
|
||||
process PATH. `~/go/bin` (where `gofumpt`, `golangci-lint`, etc. live)
|
||||
must be in the PATH before starting the daemon.
|
||||
|
||||
**Symptom**: `baseline red -- gofumpt -w . fails` but manual run passes.
|
||||
Root cause: `which gofumpt` returns nothing because `~/go/bin` isn't on
|
||||
PATH in external agent sessions.
|
||||
@@ -1,235 +0,0 @@
|
||||
# Multi-Agent Collaboration -- PRD
|
||||
|
||||
**Status:** Draft (discovery interview complete, 2026-07-25)
|
||||
**Author:** Discovery interview (archie-core-abg.1), synthesised from Sam's answers
|
||||
**Parent epic:** archie-core-abg
|
||||
|
||||
---
|
||||
|
||||
## 0. Framing
|
||||
|
||||
Today, "running multiple agents" means running multiple independent `archied`
|
||||
processes on the same host (`carina`): `archie` (bot_user=archie, multiple
|
||||
repositories, deepseek models) and `winter` (bot_user=winter, a restricted
|
||||
repository set, local llama.cpp models). Confirmed by diffing
|
||||
`~/.config/archie/config.toml` and `~/.config/archie/winter.toml` on carina:
|
||||
they differ in `bot_user`, `[[repos]]` list, and `[models]`/`[providers]` --
|
||||
not just identity. `Config.BotUser` is a single global field, so one process
|
||||
= one identity today.
|
||||
|
||||
The goal stated for archie-core overall is to become a fully extensible
|
||||
agentic workhorse. This PRD
|
||||
scopes the "multi-agent" slice of that: what it takes to go from N processes
|
||||
to one daemon that can do what N processes do, plus the surrounding
|
||||
capabilities (chat channels, plugins, memory) needed to reach that bar.
|
||||
|
||||
## 1. Scope (confirmed in interview)
|
||||
|
||||
All of the following are **in scope** for the epic (not necessarily the first
|
||||
implementation slice -- see section 6):
|
||||
|
||||
1. **Daemon-side concurrency** -- one daemon process running multiple bot
|
||||
identities concurrently, replacing N separate `archied` processes.
|
||||
2. **Cross-repo / cross-config flexibility** -- identities are not statically
|
||||
bound to one repo list at process startup; repo/model/provider
|
||||
configuration varies per identity within a single daemon.
|
||||
3. **Agent-to-agent collaboration on a single task** -- one agent's output
|
||||
feeding another (e.g. plan -> implement -> review handoff), not just N
|
||||
identities running in parallel unaware of each other.
|
||||
4. **Conversational access** -- chat channels (Telegram, web UI, more via
|
||||
plugins) where Sam talks to archie directly, outside the Gitea
|
||||
issue-driven flow. Confirmed requirement: **full parity** with the
|
||||
issue-driven flow -- chat is an alternate entry point into the same
|
||||
triage/planner/builder dispatch pipeline, not a separate toy Q&A surface.
|
||||
Chat must be able to: answer questions read-only, spawn/control real work
|
||||
(start an agent, check status, approve/reject a PR, cancel a run), and
|
||||
drive the same pipeline Gitea assignment drives today.
|
||||
5. **General extensibility** -- plugins/channels/integrations should be
|
||||
addable, not hardcoded:
|
||||
- Additional forges beyond GitHub/Gitea (e.g. GitLab) via the existing
|
||||
`forge.Forge` interface, as Layer 2 core plugins (see
|
||||
`docs/prds/archie-core-v2-architecture.md` sec.5).
|
||||
- MCP server connections (archie as an MCP client, to consume external
|
||||
tools/data sources).
|
||||
- SSH / remote execution as an agent capability.
|
||||
- Arbitrary API integrations, callable by agents as tools.
|
||||
- Additional chat channels beyond Telegram/web UI, as plugins.
|
||||
6. **Memory systems for multi-session chat** -- conversations with archie
|
||||
need to persist across sessions (distinct from the existing per-task
|
||||
`/data/memory.jsonl` sketch in the v2 PRD, which is task/project scoped,
|
||||
not conversation scoped).
|
||||
7. **Slash-command capabilities** in chat channels (e.g. `/status`, `/spawn`,
|
||||
`/approve`) for direct channel interaction.
|
||||
8. **Multi-user support**, as a capability the architecture should not
|
||||
foreclose -- "if I want to extend my agent to multiple people, I should be
|
||||
able to." Not required to ship in the first slice (see section 5), but
|
||||
the identity/session model must not assume single-operator forever.
|
||||
|
||||
## 2. Explicitly out of scope (for now)
|
||||
|
||||
- **Voice/audio interfaces.** Text-based chat only.
|
||||
- **Building GitHub support** -- it already exists (`internal/forge`
|
||||
implements GitHub REST polling per `ARCHITECTURE.md`).
|
||||
- Anything not listed in section 1 that isn't a natural extension of the
|
||||
plugin/forge/channel interfaces below.
|
||||
|
||||
Note: multi-user auth and GitLab support are **in scope as capabilities the
|
||||
architecture must support**, but are not committed to the first
|
||||
implementation slice -- see section 5.
|
||||
|
||||
## 3. Constraints from existing architecture
|
||||
|
||||
From `ARCHITECTURE.md` and `docs/prds/archie-core-v2-architecture.md`
|
||||
(read in full before implementation; summarised here):
|
||||
|
||||
- **`Config.BotUser` is a single global field** (`internal/config`) -- the
|
||||
root blocker for daemon-side multi-identity. Becomes a per-identity field
|
||||
in a new config shape (see section 4).
|
||||
- **NATS/JetStream task distribution is implemented**: `internal/nats`
|
||||
publishes tasks to `archie.task.<workflow_type>` subjects; agents
|
||||
(in-process, subprocess, or Docker containers per `internal/container`)
|
||||
consume and report back over `archie.agent.<task_id>.{response,system,request}`.
|
||||
This subject hierarchy is a natural fit for routing by identity too
|
||||
(e.g. `archie.<identity>.task.<type>`) but that's a design decision, not
|
||||
yet made.
|
||||
- **Dispatch is `daemon.Run()` polling per configured forge/repo list**,
|
||||
serialised per `owner/repo` by default (`AllowConcurrent` opts out).
|
||||
Multi-identity daemon means multiple poll loops (or one loop iterating
|
||||
multiple identity configs) feeding the same NATS/container dispatch
|
||||
machinery.
|
||||
- **Layer 2 core plugins (`~/.config/archie/plugins/`, Yaegi-loaded,
|
||||
`Plugin.Register(daemon)`) are designed but not wired** -- interface
|
||||
exists (`internal/plugin/plugin.go`), no loader in `cmd/archied/main.go`.
|
||||
This is the natural extension point for new forges, MCP client
|
||||
connections, and chat channel plugins. Building the multi-agent epic on
|
||||
top of an unwired plugin layer means **wiring Layer 2 is a prerequisite**,
|
||||
not parallel work.
|
||||
- **No chat/messaging code exists today** -- no Telegram, no web UI beyond
|
||||
the read-only SSE dashboard (`internal/webui`). This is greenfield.
|
||||
- **The daemon gatekeeps all agent output** -- "no agent output reaches a
|
||||
human without daemon review" (`archie-core-v2-architecture.md` sec.7).
|
||||
Chat channels must respect this: a chat message that triggers a spawn goes
|
||||
through the same review/labelling machinery PRs and comments do today, not
|
||||
a side channel that bypasses it.
|
||||
- **Environmental constraints over prompt rules** (`ARCHITECTURE.md` design
|
||||
decision #1) -- chat-triggered actions (spawn, approve, cancel) should be
|
||||
enforced by code (permission checks, rate limits), not by asking the LLM
|
||||
nicely.
|
||||
|
||||
## 4. Identity model (design direction, not yet finalized)
|
||||
|
||||
Confirmed requirement: within one daemon, identities differ by bot identity,
|
||||
repo list, _and_ model/provider config (per the archie/winter diff). Proposed
|
||||
shape -- **not yet approved, needs a follow-up design pass before
|
||||
implementation**:
|
||||
|
||||
```toml
|
||||
# ~/.config/archie/config.toml (sketch)
|
||||
[[identities]]
|
||||
name = "archie"
|
||||
bot_user = "archie"
|
||||
forge = { type = "gitea", host = "...", token_env = "..." }
|
||||
models = { triage = "...", planner = "...", builder = "..." }
|
||||
repos = [ ... ]
|
||||
|
||||
[[identities]]
|
||||
name = "winter"
|
||||
bot_user = "winter"
|
||||
forge = { type = "gitea", host = "...", token_env = "..." }
|
||||
models = { triage = "...", planner = "...", builder = "..." }
|
||||
repos = [ ... ]
|
||||
```
|
||||
|
||||
```
|
||||
|
||||
**Design answer (2026-07-25): per-identity goroutine, shared infrastructure.**
|
||||
|
||||
Each identity gets its own goroutine running an independent poll loop with
|
||||
its own forge client, worktree manager, and NATS subject namespace
|
||||
(`archie.<identity>.task.<type>`). They share only the infrastructure
|
||||
pieces where isolation adds cost without benefit:
|
||||
|
||||
| Shared (one instance for all identities) | Per-identity (one per goroutine) |
|
||||
|---|---|
|
||||
| SQLite store (`internal/store`) | Forge client (different token per identity) |
|
||||
| NATS connection (`internal/nats`) | Worktree manager (different `BotUser` per identity) |
|
||||
| Container pool (`internal/container`) | Config subset: forge, repos, models, providers, dispatch, poll_interval, bot_user, bot_email |
|
||||
| Event bus (`internal/events`) | NATS subject namespace: `archie.<identity>.task.<type>` |
|
||||
| Web UI (`internal/webui`) | Poll loop goroutine |
|
||||
| Plugin registry (`internal/plugin`) | |
|
||||
| Gateway router (`internal/gateway`) | |
|
||||
|
||||
Decision rationale: per-identity goroutines guarantee failure isolation (one
|
||||
identity's forge outage cannot block another's poll tick), each identity
|
||||
gets its own forge rate-limit domain, and the existing `Daemon` struct maps
|
||||
cleanly -- each goroutine holds a thin `Identity` value (forge client + config
|
||||
subset + worktree manager + NATS subject prefix) and dispatches tasks with
|
||||
an `identity_name` column on the task row. The shared NATS connection
|
||||
multiplexes subjects by identity prefix; the shared store already handles
|
||||
concurrent ClaimNext via SQLite's serialized writes. NATS subjects carry
|
||||
the identity name so agent containers (which consume from NATS) route
|
||||
responses back to the correct daemon identity.
|
||||
|
||||
This design resolves the open question in section 4 of the original PRD
|
||||
draft (per-identity goroutine vs single loop iterating identities -- the
|
||||
former, for failure isolation).
|
||||
|
||||
## 5. Priority / phasing
|
||||
|
||||
Sam deferred ordering to this PRD (interview answer: "let the interview
|
||||
settle scope, I'll decide after seeing it written up"). Proposed
|
||||
dependency-ordered phases, for Sam to confirm or reorder:
|
||||
|
||||
**Phase A -- Foundation (blocks everything else)**
|
||||
- Wire Layer 2 core plugin loading (`~/.config/archie/plugins/`,
|
||||
`Plugin.Register(daemon)`) into `cmd/archied/main.go`. Currently designed,
|
||||
unwired -- both chat channels and new forges depend on this existing.
|
||||
|
||||
**Phase B -- Multi-identity daemon core**
|
||||
- Replace single `Config.BotUser` with the `[[identities]]` shape (section
|
||||
4, after design confirmation).
|
||||
- Multi-identity poll/dispatch, still issue-driven (Gitea only, no chat yet).
|
||||
- Migrate the two carina processes (archie, winter) onto one daemon as a
|
||||
real-world validation.
|
||||
|
||||
**Phase C -- Chat channel foundation**
|
||||
- Chat channel plugin interface (parallel to `forge.Forge`).
|
||||
- Telegram channel plugin: read-only Q&A against task/daemon state first,
|
||||
then spawn/control actions gated through the same review pipeline as
|
||||
issue-driven dispatch.
|
||||
- Web UI: extend `internal/webui` from read-only SSE dashboard to
|
||||
bidirectional chat.
|
||||
- Multi-session conversation memory (distinct persistence from per-task
|
||||
`/data/memory.jsonl`).
|
||||
- Slash commands (`/status`, `/spawn`, `/approve`, `/cancel`).
|
||||
|
||||
**Phase D -- Agent-to-agent collaboration**
|
||||
- Plan -> implement -> review handoff within a single task's workflow
|
||||
(builds on existing `workflow.Stage` sequencing -- this may already be
|
||||
close given the `implement` workflow's plan/build stage split; needs
|
||||
investigation into how much is genuinely new vs. reframing).
|
||||
|
||||
**Phase E -- Extended integrations (as needed, each is an independent
|
||||
plugin, no fixed order)**
|
||||
- GitLab forge plugin.
|
||||
- MCP client capability (archie consuming external MCP servers as tools).
|
||||
- SSH / remote execution tool.
|
||||
- Additional chat channels.
|
||||
- Multi-user support (identity/session model extended beyond
|
||||
single-operator -- needs its own design pass; not blocking Phase A-D).
|
||||
|
||||
Sam's explicit instruction was to identify next requirements **and begin
|
||||
work**, not just deliver a document. Phase A (wiring the Layer 2 plugin
|
||||
loader) is the correct starting point: it's small, well-specified already in
|
||||
`docs/prds/archie-core-v2-architecture.md` sec.5, and unblocks both the chat
|
||||
channel work and new forge plugins that everything else in this PRD depends
|
||||
on.
|
||||
|
||||
## 6. Acceptance criteria for this PRD
|
||||
|
||||
- [x] Reflects Sam's actual interview answers, not assumptions.
|
||||
- [x] Confirms the archie/winter config diff (repo lists differ, not just
|
||||
identity).
|
||||
- [ ] Broken into scoped, dependency-ordered beads issues under
|
||||
`archie-core-abg`, the way the 5zh epic was. (Next step, this session.)
|
||||
```
|
||||
@@ -1,461 +0,0 @@
|
||||
# archie-core TypeScript Plugin Engine -- PRD
|
||||
|
||||
**Author:** Archie
|
||||
**Date:** 2026-07-21
|
||||
**Status:** Draft
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
A TypeScript-based plugin engine that lets repositories ship interpreted, dynamically-loaded workflow extensions. Plugins are `.ts` files placed in `.archie/workflows/`. archie-core discovers them on its next poll cycle, transpiles them to JavaScript via esbuild, and executes them in an embedded [goja](https://github.com/dop251/goja) runtime -- a pure-Go ECMAScript interpreter with zero CGO dependencies.
|
||||
|
||||
---
|
||||
|
||||
## Why TypeScript + goja
|
||||
|
||||
### TypeScript as the plugin language
|
||||
|
||||
TypeScript is the most widely understood language among developers writing automation, CI, and workflow scripts. It already ships in most projects. Gate authors already know it. The ecosystem for linting, formatting, and type-checking is mature and free.
|
||||
|
||||
Having a TypeScript-native plugin engine means:
|
||||
|
||||
- Plugin authors get familiar syntax, type safety, and IDE support
|
||||
- No separate toolchain -- just `.ts` files in a folder
|
||||
- Skills, gates, and workflow stages share the same language
|
||||
|
||||
### Why goja
|
||||
|
||||
| Competitor | Problem |
|
||||
| ------------------ | -------------------------------------------------------------------- |
|
||||
| V8 CGO bindings | Requires CGO, ~50MB static libraries, platform-specific builds |
|
||||
| QuickJS CGO | Requires CGO, smaller than V8 but still a C dependency |
|
||||
| QJS (Wazero) | Pure Go WASM, but sandboxed -- can't expose Go functions efficiently |
|
||||
| Ramune | Heavy dependency, bundles an entire TypeScript 7 compiler |
|
||||
| Node.js subprocess | Process overhead per execution, IPC serialisation cost |
|
||||
|
||||
goja is a pure-Go ECMAScript 5.1+ interpreter at ~20k lines. It:
|
||||
|
||||
- Has **zero CGO** -- cross-compiles everywhere Go does
|
||||
- Passes nearly all TC39 Test262 tests for ES5.1 + significant ES6 coverage
|
||||
- Can run Babel and the TypeScript compiler itself (proven capability)
|
||||
- Provides direct Go↔JS interop via `Set()`/`Get()`/`Runtime.RunString()`
|
||||
- Has a companion `goja_nodejs` package for `require()`, `setTimeout`, and `console`
|
||||
- Is used in production by k6 (Grafana's load testing tool, 30k+ stars)
|
||||
|
||||
### How TypeScript becomes JavaScript
|
||||
|
||||
arcie-core bundles esbuild via `go:embed`. On plugin load, TypeScript source is transpiled to ES2017 JavaScript in-process:
|
||||
|
||||
```
|
||||
.archie/workflows/security-gate.ts → esbuild.Transform() → goja.Runtime.RunString()
|
||||
```
|
||||
|
||||
esbuild is pure Go, takes ~3ms for typical gate scripts, and strips type annotations without type-checking (the plugin author does that in their editor). If the transpilation fails, the plugin is skipped with a diagnostic comment on the issue.
|
||||
|
||||
---
|
||||
|
||||
## Design
|
||||
|
||||
### Plugin discovery
|
||||
|
||||
During the poll cycle, archie-core scans each repo's worktree for `.archie/workflows/*.ts`:
|
||||
|
||||
```
|
||||
.archie/workflows/
|
||||
├── custom-gate.ts # gate functions: exported `check()`
|
||||
├── db-migration.ts # custom stage: exported `stage()`
|
||||
└── notify-slack.ts # skill script: exported `run()`
|
||||
```
|
||||
|
||||
Each file is a self-contained module. No `package.json`, no `node_modules`, no external imports. If you need external deps, they need to be exposed via archie-core's Go→JS bridge.
|
||||
|
||||
### Plugin types
|
||||
|
||||
A plugin declares its role via an exported symbol:
|
||||
|
||||
| Export | Role | When called |
|
||||
| ---------------------------------------------------- | -------------- | --------------------------------------------- |
|
||||
| `export function check(ctx: GateContext): Finding[]` | Gate function | After shell-based gate, before commit |
|
||||
| `export function stage(): Stage` | Workflow stage | When a workflow references the stage name |
|
||||
| `export function run(ctx: ScriptContext): void` | Skill script | When a skill's instructions invoke the script |
|
||||
|
||||
arcie-core discovers the role by checking which exports exist. A file can export multiple roles.
|
||||
|
||||
### Gate functions
|
||||
|
||||
```typescript
|
||||
// .archie/workflows/custom-gate.ts
|
||||
|
||||
interface GateContext {
|
||||
diff: string; // unified diff of changes
|
||||
changedFiles: string[]; // list of changed file paths
|
||||
dir: string; // absolute worktree path
|
||||
repo: string; // "owner/name"
|
||||
baseRef: string; // "origin/main"
|
||||
}
|
||||
|
||||
interface Finding {
|
||||
level: "error" | "warn";
|
||||
file?: string;
|
||||
line?: number;
|
||||
message: string;
|
||||
}
|
||||
|
||||
interface Block {
|
||||
name: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
export function check(ctx: GateContext): Finding[] {
|
||||
const findings: Finding[] = [];
|
||||
|
||||
// Rule: no new panic() calls
|
||||
for (const line of ctx.diff.split("\n")) {
|
||||
if (line.startsWith("+") && !line.startsWith("+++")) {
|
||||
if (line.includes("panic(")) {
|
||||
findings.push({
|
||||
level: "error",
|
||||
message: "new panic() call -- use error returns instead",
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Rule: no time.Sleep in non-test files
|
||||
for (const file of ctx.changedFiles) {
|
||||
if (file.endsWith("_test.go")) continue;
|
||||
const content = archie.readFile(`${ctx.dir}/${file}`);
|
||||
if (content.includes("time.Sleep")) {
|
||||
findings.push({
|
||||
level: "warn",
|
||||
file: file,
|
||||
message: "time.Sleep in non-test code -- use time.Ticker instead",
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return findings;
|
||||
}
|
||||
```
|
||||
|
||||
After transpilation and execution, archie-core calls `check(ctx)` via goja. Error-level findings park the task. Warn-level findings are logged.
|
||||
|
||||
### Custom workflow stages
|
||||
|
||||
```typescript
|
||||
// .archie/workflows/run-migrations.ts
|
||||
|
||||
export function stage(): Stage {
|
||||
return {
|
||||
name: "db-migration",
|
||||
run: (ctx: TaskContext): string | null => {
|
||||
// Check if migrations directory exists
|
||||
const hasMigrations = archie.fileExists(`${ctx.dir}/migrations`);
|
||||
if (!hasMigrations) return null; // nothing to do
|
||||
|
||||
// Run migration tests
|
||||
const result = archie.exec("go", ["test", "./migrations/..."]);
|
||||
if (result.exitCode !== 0) {
|
||||
return `migration tests failed: ${result.stderr}`;
|
||||
}
|
||||
return null; // success
|
||||
},
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Skill scripts
|
||||
|
||||
```typescript
|
||||
// .archie/workflows/security-audit.ts
|
||||
|
||||
export function run(ctx: ScriptContext): void {
|
||||
const result = archie.exec("gitleaks", ["detect", "--no-git"]);
|
||||
if (result.exitCode !== 0) {
|
||||
archie.log("warn", `gitleaks found secrets:\n${result.stdout}`);
|
||||
}
|
||||
archie.log("info", "security audit complete");
|
||||
}
|
||||
```
|
||||
|
||||
### archie Go→JS bridge
|
||||
|
||||
arcie-core exposes a `archie` global in the goja runtime:
|
||||
|
||||
```typescript
|
||||
interface Archie {
|
||||
// File operations (scoped to worktree)
|
||||
readFile(path: string): string;
|
||||
fileExists(path: string): boolean;
|
||||
listDir(path: string): string[];
|
||||
walkDir(path: string): string[];
|
||||
|
||||
// Shell execution
|
||||
exec(command: string, args?: string[]): ExecResult;
|
||||
|
||||
// Logging
|
||||
log(level: string, message: string): void;
|
||||
|
||||
// Path utilities
|
||||
resolve(...parts: string[]): string;
|
||||
ext(path: string): string;
|
||||
|
||||
// Environment
|
||||
env(key: string): string;
|
||||
|
||||
// HTTP (optional, gated)
|
||||
fetch(url: string, options?: FetchOptions): FetchResult;
|
||||
}
|
||||
|
||||
interface ExecResult {
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
exitCode: number;
|
||||
}
|
||||
```
|
||||
|
||||
All file operations are **jailed to the worktree**. `archie.readFile("/etc/passwd")` fails. The bridge enforces this in Go before passing the path to the OS.
|
||||
|
||||
### Reload cycle
|
||||
|
||||
Plugins are loaded fresh on every poll cycle -- archie-core already fresh-clones per task. No hot-reload complexity, no cache invalidation. The worktree is the source of truth.
|
||||
|
||||
For the rare case where a plugin errors at parse/transpile time, archie-core comments on the issue with the diagnostic and parks the task. The plugin author fixes the `.ts` file and requeues.
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
internal/plugins/
|
||||
├── loader.go # scan .archie/workflows/, transpile, evaluate
|
||||
├── runtime.go # goja runtime pool management
|
||||
├── bridge.go # archie global (file I/O, exec, log, fetch)
|
||||
├── types.go # GateContext, Finding, Stage, TaskContext → JS types
|
||||
├── loader_test.go
|
||||
└── testdata/
|
||||
└── workflows/
|
||||
├── gate.ts # test gate plugin
|
||||
└── stage.ts # test stage plugin
|
||||
```
|
||||
|
||||
### goja runtime pool
|
||||
|
||||
Workflow stages and gates need a clean goja runtime per execution. Creating a new runtime per call is slow (~50ms for stdlib setup). A pool of pre-warmed runtimes solves this:
|
||||
|
||||
```go
|
||||
type Pool struct {
|
||||
warm chan *goja.Runtime
|
||||
}
|
||||
|
||||
func NewPool(size int) *Pool {
|
||||
p := &Pool{warm: make(chan *goja.Runtime, size)}
|
||||
for i := 0; i < size; i++ {
|
||||
rt := goja.New()
|
||||
installBridge(rt) // register the `archie` global
|
||||
installStdlib(rt) // console, setTimeout, etc.
|
||||
p.warm <- rt
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func (p *Pool) Run(ctx context.Context, src string, globals map[string]any) (goja.Value, error) {
|
||||
rt := <-p.warm
|
||||
defer func() { p.warm <- rt }()
|
||||
// Set per-execution globals
|
||||
for k, v := range globals {
|
||||
rt.Set(k, v)
|
||||
}
|
||||
return rt.RunString(src)
|
||||
}
|
||||
```
|
||||
|
||||
### Transpile pipeline
|
||||
|
||||
```go
|
||||
func (l *Loader) transpile(filename string, ts []byte) (string, error) {
|
||||
result := l.esbuild.Transform(string(ts), api.TransformOptions{
|
||||
Loader: api.LoaderTS,
|
||||
Target: api.ES2017,
|
||||
Sourcefile: filename,
|
||||
})
|
||||
if len(result.Errors) > 0 {
|
||||
return "", fmt.Errorf("transpile %s: %v", filename, result.Errors)
|
||||
}
|
||||
return string(result.Code), nil
|
||||
}
|
||||
```
|
||||
|
||||
### load pipeline
|
||||
|
||||
```go
|
||||
func (l *Loader) LoadPlugins(worktreeDir string) ([]Plugin, error) {
|
||||
dir := filepath.Join(worktreeDir, ".archie", "workflows")
|
||||
entries, err := os.ReadDir(dir)
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil // no plugins -- not an error
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var plugins []Plugin
|
||||
for _, e := range entries {
|
||||
if !strings.HasSuffix(e.Name(), ".ts") {
|
||||
continue
|
||||
}
|
||||
path := filepath.Join(dir, e.Name())
|
||||
src, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
js, err := l.transpile(e.Name(), src)
|
||||
if err != nil {
|
||||
l.log.Warn("plugin transpile failed", "file", e.Name(), "err", err)
|
||||
continue
|
||||
}
|
||||
rt := l.pool.Acquire()
|
||||
_, err = rt.RunString(js)
|
||||
l.pool.Release(rt)
|
||||
if err != nil {
|
||||
l.log.Warn("plugin eval failed", "file", e.Name(), "err", err)
|
||||
continue
|
||||
}
|
||||
plugins = append(plugins, discover(rt, e.Name())...)
|
||||
}
|
||||
return plugins, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Export discovery
|
||||
|
||||
```go
|
||||
func discover(rt *goja.Runtime, filename string) []Plugin {
|
||||
var out []Plugin
|
||||
if check := rt.Get("check"); check != nil {
|
||||
out = append(out, Plugin{Kind: "gate", Name: filename, Func: check})
|
||||
}
|
||||
if stage := rt.Get("stage"); stage != nil {
|
||||
out = append(out, Plugin{Kind: "stage", Name: filename, Func: stage})
|
||||
}
|
||||
if run := rt.Get("run"); run != nil {
|
||||
out = append(out, Plugin{Kind: "script", Name: filename, Func: run})
|
||||
}
|
||||
return out
|
||||
}
|
||||
```
|
||||
|
||||
### Wiring into the daemon
|
||||
|
||||
In `daemon.process()`:
|
||||
|
||||
```go
|
||||
// After worktree prepare, before gate stage:
|
||||
plugins, err := d.Plugins.LoadPlugins(tc.Dir)
|
||||
if err != nil {
|
||||
d.Log.Warn("plugin load failed", "err", err)
|
||||
}
|
||||
tc.Plugins = plugins // available to all stages via TaskContext
|
||||
|
||||
// In the gate stage:
|
||||
if gatePlugin := tc.Plugins.Gate(); gatePlugin != nil {
|
||||
findings := callGatePlugin(gatePlugin, gateContext(tc))
|
||||
for _, f := range findings {
|
||||
if f.Level == "error" {
|
||||
// park the task
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Limitations
|
||||
|
||||
### goja's ECMAScript coverage
|
||||
|
||||
goja natively supports ES5.1 with significant ES6 features. Modern syntax like `async/await`, `??`, `?.`, or `class` fields may not parse. The transpile step (esbuild targeting ES2017) handles most of this, but there will be edge cases:
|
||||
|
||||
| Feature | Status |
|
||||
| ----------------- | -------------------------------------- |
|
||||
| Arrow functions | ✅ via transpile |
|
||||
| Template literals | ✅ via transpile |
|
||||
| Destructuring | ✅ via transpile |
|
||||
| `const`/`let` | ✅ via transpile |
|
||||
| `async/await` | ❌ not supported (goja is synchronous) |
|
||||
| `Promise` | ❌ requires `goja_nodejs` event loop |
|
||||
| Generators | ❌ not supported |
|
||||
| `Proxy` | ❌ not supported |
|
||||
|
||||
For gate scripts, this is fine -- gates are synchronous by nature. If async is genuinely needed later, the Bun backend (via `tsgo`) is the escape hatch.
|
||||
|
||||
### Performance
|
||||
|
||||
Gate scripts run once per task. Even a 100ms transpile+eval overhead is negligible compared to the LLM calls in the workflow. If a plugin becomes hot-path, it should be promoted to a compiled Go stage.
|
||||
|
||||
### No external imports
|
||||
|
||||
Plugins can't `import` from npm or other files. They're self-contained modules. Complexity that needs external deps should be a compiled Go stage, not a plugin. The `archie` global provides filesystem and shell access, which covers the common cases.
|
||||
|
||||
---
|
||||
|
||||
## Comparison: Yaegi vs goja
|
||||
|
||||
| | Yaegi (Go) | goja + esbuild (TypeScript) |
|
||||
| -------------------- | ----------------------------------------------------------- | --------------------------------------------------------- |
|
||||
| **Language** | Go | TypeScript |
|
||||
| **Pure Go** | ✅ | ✅ (goja + esbuild both pure Go) |
|
||||
| **CGO** | None | None |
|
||||
| **Transpile step** | None (interpreted directly) | esbuild TS→JS (~3ms) |
|
||||
| **Author audience** | Go developers | Anyone who knows TypeScript |
|
||||
| **Type annotations** | N/A (Go is typed) | Stripped by esbuild (no type-check at runtime) |
|
||||
| **Interop** | Direct Go↔Go via `Use()` | Bridge via `archie` global |
|
||||
| **Performance** | Interpreted Go (~10x slower than compiled) | Interpreted JS (~50x slower than V8, fine for gates) |
|
||||
| **Maturity** | 8k stars, used by Traefik for plugins | 7k stars, used by k6, goja is battle-tested |
|
||||
| **File extension** | `.go` | `.ts` |
|
||||
| **Plugin folder** | `.archie/workflows/` | `.archie/workflows/` |
|
||||
| **Best for** | Custom stages needing Go's stdlib, tight daemon integration | Custom gates and scripts where author familiarity matters |
|
||||
|
||||
**Recommendation: support both.** They solve different problems. Yaegi for repo owners who want full Go power in their custom stages. goja for repo owners who want to write gates and scripts in TypeScript. Both load from `.archie/workflows/` -- archie-core discovers which engine to use by file extension (`.go` → Yaegi, `.ts` → goja+esbuild).
|
||||
|
||||
---
|
||||
|
||||
## Implementation phases
|
||||
|
||||
### Phase 1: goja runtime + esbuild
|
||||
|
||||
- Add `github.com/dop251/goja` and `github.com/evanw/esbuild` as dependencies
|
||||
- Build `internal/plugins/` with loader, runtime pool, bridge
|
||||
- Wire gate plugin discovery into the implement and TDD workflows
|
||||
- Gate plugins run after shell-based gate, before commit
|
||||
|
||||
### Phase 2: Custom stages
|
||||
|
||||
- Stage plugins discovered and registered alongside built-in stages
|
||||
- `config.toml` can reference plugin stages by name
|
||||
- TaskContext exposed to JS plugins
|
||||
|
||||
### Phase 3: Skill scripts
|
||||
|
||||
- Skill SKILL.md can reference `scripts/security-audit.ts`
|
||||
- archie-core loads, transpiles, and evaluates when the skill activates
|
||||
- Script stdout captured and returned to the agent as tool output
|
||||
|
||||
### Phase 4: Yaegi side-by-side
|
||||
|
||||
- Go plugin files (`.go`) loaded via Yaegi from the same `.archie/workflows/` directory
|
||||
- Same discovery pattern, different interpreter
|
||||
- archie-core chooses engine by file extension
|
||||
|
||||
---
|
||||
|
||||
## Open questions
|
||||
|
||||
1. **Jailing:** How strict should the `archie` bridge be? File I/O jailed to worktree, no network access by default? Or trust the repo author?
|
||||
2. **Error UX:** When a plugin fails to transpile, should archie-core comment on the issue with the specific error, or just park silently?
|
||||
3. **Runtime pool sizing:** 4 warmed runtimes? Dynamic?
|
||||
4. **Plugin versioning:** Do plugins get a `version` export? Or is the git commit the version?
|
||||
5. **TypeScript type definitions:** Ship a `.d.ts` file for the `archie` global so plugin authors get IntelliSense?
|
||||
@@ -1,337 +0,0 @@
|
||||
# archie-core Yaegi Plugin System -- PRD
|
||||
|
||||
**Author:** Archie
|
||||
**Date:** 2026-07-21
|
||||
**Status:** Draft
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Embed the [Yaegi](https://github.com/traefik/yaegi) Go interpreter to let
|
||||
archie-core load and execute `.go` files at runtime. This enables three things
|
||||
that shell commands can't do: **custom gates with deep worktree inspection**,
|
||||
**custom workflow stages defined per-repo**, and **skill scripts written in Go
|
||||
instead of shell**.
|
||||
|
||||
---
|
||||
|
||||
## Why Yaegi
|
||||
|
||||
### The problem with shell-only gates
|
||||
|
||||
`[[repos.gate]]` is command lists. That's fine for standard tooling (`go vet`,
|
||||
`golangci-lint`), but it can't express project-specific rules:
|
||||
|
||||
- "No new `panic()` calls in the diff"
|
||||
- "No new dependencies in `go.mod` without justification"
|
||||
- "All public functions in `pkg/api/` must be documented"
|
||||
- "No `time.Sleep` in production code paths"
|
||||
|
||||
These are rules that need to read the diff, parse Go ASTs, or walk the worktree
|
||||
-- things shell scripts can do but shouldn't have to.
|
||||
|
||||
### Why not shell scripts?
|
||||
|
||||
Shell scripts for gates are fine until they're not. They can't import the
|
||||
archie-core types, can't use the Go parser, and are a second language to
|
||||
maintain alongside the daemon. Yaegi runs Go -- the same language as the daemon,
|
||||
the same language as the repos archie-core works on.
|
||||
|
||||
### Why not Go's native `plugin` package?
|
||||
|
||||
Go's `plugin` package (`.so` files) requires:
|
||||
|
||||
- Exact same Go version and deps as the host binary
|
||||
- Linux/macOS only
|
||||
- No unload
|
||||
- Painful distribution
|
||||
|
||||
Yaegi interprets `.go` source files directly -- no compilation step, portable
|
||||
everywhere, loads from filesystem paths.
|
||||
|
||||
### Production precedent
|
||||
|
||||
- **Traefik:** 200+ community plugins via Yaegi, loaded from plugins.traefik.io
|
||||
- **Vikunja:** Adopted Yaegi for plugin system (see PR #2178), loads plugins
|
||||
from `plugins/` directory
|
||||
|
||||
---
|
||||
|
||||
## Design
|
||||
|
||||
### Three extension surfaces
|
||||
|
||||
| Surface | Location | When loaded | What it does |
|
||||
| -------------------------- | ------------------------------------- | -------------------------------------------- | --------------------------------------------------------------------- |
|
||||
| **Custom gate functions** | `.archie/gate.go` | During the gate stage (after shell commands) | Returns `[]Finding` -- each finding has severity, message, file, line |
|
||||
| **Custom workflow stages** | `.archie/stages/<name>.go` | When the workflow references the stage name | Returns a `workflow.Stage` implementation |
|
||||
| **Skill scripts** | `.archie/skills/<skill>/scripts/*.go` | When the parent skill is activated | Called by the skill's instructions |
|
||||
|
||||
### Gate functions (`.archie/gate.go`)
|
||||
|
||||
The most important extension surface. After the shell-based gate passes,
|
||||
archie-core loads and evaluates `.archie/gate.go` in the worktree directory. The
|
||||
script exposes a `Check` function:
|
||||
|
||||
```go
|
||||
// .archie/gate.go -- per-repo gate rules interpreted at runtime
|
||||
package gate
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// GateContext carries everything the gate checker can inspect.
|
||||
type GateContext struct {
|
||||
Diff string // unified diff of all changes
|
||||
ChangedFiles []string // list of changed file paths (repo-relative)
|
||||
Dir string // absolute worktree path
|
||||
BaseRef string // base branch (e.g. "origin/main")
|
||||
Repo string // "owner/name"
|
||||
}
|
||||
|
||||
// Finding is one gate violation.
|
||||
type Finding struct {
|
||||
Level string // "error" (blocks commit) or "warn" (advisory)
|
||||
File string // optional file path
|
||||
Line int // optional line number
|
||||
Message string
|
||||
}
|
||||
|
||||
// Block defines a named rule that can prevent passing.
|
||||
type Block struct {
|
||||
Name string
|
||||
Message string
|
||||
}
|
||||
|
||||
// Check returns findings for the gate. Return nil for a clean gate.
|
||||
func Check(ctx GateContext) []Finding {
|
||||
var findings []Finding
|
||||
|
||||
// Rule: no new panic() calls in committed code
|
||||
for _, line := range strings.Split(ctx.Diff, "\n") {
|
||||
if strings.HasPrefix(line, "+") && !strings.HasPrefix(line, "+++") {
|
||||
if strings.Contains(line, "panic(") {
|
||||
findings = append(findings, Finding{
|
||||
Level: "error",
|
||||
Message: "new panic() call -- use error returns instead",
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Rule: no time.Sleep in non-test files
|
||||
for _, f := range ctx.ChangedFiles {
|
||||
if strings.HasSuffix(f, "_test.go") {
|
||||
continue
|
||||
}
|
||||
path := filepath.Join(ctx.Dir, f)
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if strings.Contains(string(b), "time.Sleep") {
|
||||
findings = append(findings, Finding{
|
||||
Level: "warn",
|
||||
File: f,
|
||||
Message: "time.Sleep in non-test code -- use time.Ticker or context.WithTimeout instead",
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
return findings
|
||||
}
|
||||
```
|
||||
|
||||
The daemon:
|
||||
|
||||
1. Reads `.archie/gate.go` from the worktree
|
||||
2. Creates a Yaegi interpreter with `GateContext` and `Finding` types pre-loaded
|
||||
via `Use()`
|
||||
3. Evaluates the file
|
||||
4. Calls `Check(ctx)` via `Eval("gate.Check")` + `Interface()`
|
||||
5. Logs findings and blocks the gate if any have `Level: "error"`
|
||||
|
||||
### Custom workflow stages (`.archie/stages/`)
|
||||
|
||||
A repo can define custom pipeline stages that archie-core discovers and makes
|
||||
routable:
|
||||
|
||||
```go
|
||||
// .archie/stages/run-migrations.go
|
||||
package stages
|
||||
|
||||
import "github.com/samcharles93/archie-core/internal/workflow"
|
||||
|
||||
func Stage() workflow.Stage {
|
||||
return workflow.Stage{
|
||||
Name: "db-migration",
|
||||
Run: func(ctx context.Context, tc *workflow.TaskContext) error {
|
||||
// Check if go.mod contains migration tools
|
||||
// Run migration tests
|
||||
// Verify schema compatibility
|
||||
return nil
|
||||
},
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Registered stages appear alongside built-in stages. A workflow in `config.toml`
|
||||
can reference them by name:
|
||||
|
||||
```toml
|
||||
[[workflows.stages]]
|
||||
name = "db-migration" # matches the .go file
|
||||
source = "repo" # loaded from .archie/stages/
|
||||
```
|
||||
|
||||
### Skill scripts (`.archie/skills/<skill>/scripts/`)
|
||||
|
||||
Skills can ship Go scripts that archie-core interprets:
|
||||
|
||||
```go
|
||||
// .archie/skills/security-audit/scripts/gitleaks.go
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"context"
|
||||
)
|
||||
|
||||
func main() {
|
||||
ctx := context.Background()
|
||||
cmd := exec.CommandContext(ctx, "gitleaks", "detect", "--no-git", "--verbose")
|
||||
out, _ := cmd.CombinedOutput()
|
||||
fmt.Println(string(out))
|
||||
}
|
||||
```
|
||||
|
||||
Skills reference these scripts naturally in their instructions: "Run
|
||||
`scripts/gitleaks.go` to scan for secrets."
|
||||
|
||||
---
|
||||
|
||||
## API surface archie-core exposes to Yaegi
|
||||
|
||||
Via `i.Use()`, the daemon pre-loads symbols that interpreted code can import:
|
||||
|
||||
| Package | What's exposed |
|
||||
| ------------------------------- | ----------------------------------------- |
|
||||
| `archie-core/internal/workflow` | `Stage`, `TaskContext`, `Outcome` |
|
||||
| `archie-core/internal/gate` | `GateContext`, `Finding`, `Block` |
|
||||
| `stdlib` | Full Go standard library (stdlib.Symbols) |
|
||||
|
||||
Generated with `yaegi extract` -- same approach Traefik and Vikunja use. A
|
||||
`//go:generate yaegi extract ...` line in the source triggers symbol table
|
||||
generation during `go generate`.
|
||||
|
||||
---
|
||||
|
||||
## Limitations
|
||||
|
||||
From Yaegi's own docs:
|
||||
|
||||
- **No assembly files** (`.s` not supported)
|
||||
- **No CGo** (can't call C code -- the daemon is pure Go, so this doesn't
|
||||
matter)
|
||||
- **No compiler/linker directives** (no `//go:embed`, etc. -- fine for gate
|
||||
scripts)
|
||||
- **Interfaces from pre-compiled code can't be added dynamically** -- the daemon
|
||||
must pre-compile wrapper types for any interface interpreted code needs to
|
||||
implement
|
||||
- **Slower than compiled code** -- gate scripts run once per commit, not in hot
|
||||
paths; negligible impact
|
||||
|
||||
The Vikunja PR documents a specific Yaegi quirk: **typed factory functions
|
||||
required.** You can't do `v.Interface().(MyInterface)` on a value returned from
|
||||
interpreted code. Each interface needs its own factory:
|
||||
|
||||
```go
|
||||
func NewGate() Gate { return &MyGate{} }
|
||||
func NewReportable() Reportable { return &MyGate{} }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation phases
|
||||
|
||||
### Phase 1: Gate functions -- implemented
|
||||
|
||||
- `yaegi` added as a dependency
|
||||
- Symbol table for `GateContext`, `Finding` generated via `yaegi extract` into
|
||||
`internal/gate/gateextract` (a separate package from `internal/gate` itself,
|
||||
avoiding the self-import cycle extracting a package's own symbols creates)
|
||||
- `internal/gate/gateeval.Evaluate`: loads `.archie/gate.go` from the worktree,
|
||||
interprets it, calls `Check()`; panics are recovered
|
||||
- Wired in as `workflow.StageYaegiGate`, run after the final commit in both the
|
||||
`implement` and `tdd` workflows
|
||||
- Error findings park the task; warn findings are logged only
|
||||
|
||||
### Phase 2: Custom stages -- implemented
|
||||
|
||||
- Symbol table for `workflow.Stage`/`TaskContext`/`Outcome` generated into
|
||||
`internal/workflow/wfextract`
|
||||
- `internal/workflow/wfeval.Discover` scans `.archie/stages/*.go` in filename
|
||||
order and calls each file's exported `Stage()` function
|
||||
- `TaskContext.CustomStages` is an injected `func(dir string) ([]Stage, error)`
|
||||
-- the composition root (`cmd/archied`) wires it to `wfeval.Discover`, keeping
|
||||
`internal/workflow` decoupled from its own generated symbol table
|
||||
- `workflow.StageRepoStages` runs discovered stages in order right after the
|
||||
worktree is prepared, in both `implement` and `tdd`
|
||||
- Simplification vs. the original sketch: stages are auto-discovered and always
|
||||
run (like `.archie/gate.go`) rather than referenced by name from `config.toml`
|
||||
-- the workflow engine has no data-driven stage graph today, so wiring
|
||||
arbitrary named stages into arbitrary positions would need a larger
|
||||
config-driven rework. Auto-run-in-order covers the PRD's actual use case (a
|
||||
repo-defined pre-flight/setup hook) without it.
|
||||
|
||||
### Phase 3: Skill scripts -- implemented
|
||||
|
||||
- `internal/skillscript.Run` interprets a `.go` file via `i.EvalPath`, which
|
||||
(like `go run`) evaluates and calls its `main()`; stdout/stderr are captured
|
||||
via `interp.Options`
|
||||
- Uses `stdlib.Symbols` + `stdlib/unrestricted.Symbols` (not just `stdlib`) --
|
||||
`os/exec` lives in the unrestricted set, and the PRD's own example script
|
||||
shells out via `os/exec`
|
||||
- No archie-core-specific symbol table needed here -- skill scripts only touch
|
||||
the Go standard library
|
||||
- Exposed to the agent loop as a new `run_go_script` tool
|
||||
(`internal/agentexec/inprocess.go`), available in every agent stage; path is
|
||||
validated to stay inside the workspace
|
||||
- This is how "the skill's instructions reference a script" cashes out
|
||||
concretely: the agent calls `run_go_script` with the path instead of needing a
|
||||
Go toolchain in the sandbox
|
||||
|
||||
### Phase 4: Hot reload -- not needed, by construction
|
||||
|
||||
- All three loaders (`gate.Evaluate`, `wfeval.Discover`, `skillscript.Run`) read
|
||||
their `.go` source from disk on every call -- no in-memory cache exists
|
||||
anywhere in the implementation
|
||||
- Every task additionally gets a fresh `git clone` (`worktree.Manager.Prepare`
|
||||
removes and re-clones the worktree directory per task)
|
||||
- Consequently every stage already re-reads `.archie/gate.go`,
|
||||
`.archie/stages/*.go`, and skill scripts fresh, every run -- there is no stale
|
||||
state a file watcher would ever need to invalidate
|
||||
- A file-watching daemon-runtime reload mechanism would add machinery with
|
||||
nothing to do given this architecture; revisit only if the daemon starts
|
||||
reusing long-lived worktrees across tasks
|
||||
|
||||
---
|
||||
|
||||
## Open questions
|
||||
|
||||
1. **Symbol table generation:** Run `yaegi extract` as a `go generate` step, or
|
||||
generate on daemon startup?
|
||||
2. **Error handling:** If the interpreted gate panics or times out, park the
|
||||
task? Or fall back to shell-only gate?
|
||||
3. **Budget:** Interpreted gate scripts run in the gate stage -- should they
|
||||
have their own timeout/step budget?
|
||||
4. **Template system:** Should archie-core ship with common gate templates (e.g.
|
||||
`go: no panic, no Sleep, no hardcoded paths`)?
|
||||
5. **Remote gates:** Could `.archie/gate.go` be pulled from a shared remote
|
||||
registry, like skills?
|
||||
Reference in new issue
Block a user