chore: cleanup
This commit is contained in:
1 parent
6ee605a4fc
commit
65aa5b462d
311 files changed
+6115
-3973
No files matched your search
@@ -18,7 +18,7 @@ arcie-core enforces this gate after every builder agent stage. The specific comm
|
||||
come from the repository's `[[repos.gate]]` config, not this skill. This skill documents
|
||||
what a correct gate looks like and how to diagnose failures.
|
||||
|
||||
## Standard gate (Tau and similar Go projects)
|
||||
## Standard gate
|
||||
|
||||
Run in this exact order. archie-core does NOT run these -- the `ai-sdk/agentloop` gate
|
||||
does. This list is what you should configure in `config.toml`:
|
||||
|
||||
@@ -156,3 +156,4 @@
|
||||
{"id":"int-0515875f328056cbb0888785d35497c9","kind":"field_change","created_at":"2026-07-27T12:11:14.316202683Z","actor":"Sam Catlow","issue_id":"archie-core-5dn","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Investigated local Hermes CLI and current Archie implementation; added the requested parity TODO checklist."}}
|
||||
{"id":"int-e2b07b5c81dd19cabd9e2a18ff94f7f7","kind":"field_change","created_at":"2026-07-27T12:20:51.572157777Z","actor":"Sam Catlow","issue_id":"archie-core-sc5","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Implemented gateway.BuildSystemPrompt rendering internal/gateway/templates/archie.md.tpl: instruction precedence, core rules, communication rules, <tools> and <env> blocks, with PersonaRegistry kept as the style layer on top. Tool inventory is derived from the same core.ToolSet passed to the model (cmd/archied/toolSummaries) so prompt and reality cannot drift; empty toolset renders an explicit 'no tools are available'. Tool metadata is markup-escaped and whitespace-flattened against forged tags and forged list entries. task check clean: vet, full go test ./... , golangci-lint 0 issues on changed packages, gofumpt clean. Commit 7fde716. NOT yet deployed - not pushed."}}
|
||||
{"id":"int-c52b1ab5add5f30c8a63b5f03e5faff7","kind":"field_change","created_at":"2026-07-27T13:47:42.515040687Z","actor":"Sam Catlow","issue_id":"archie-core-4we","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Ported tau internal/indexing/codesearch.go @ 4893818d0c76 into archie-core internal/indexing. Mutations: global config path helpers replaced by explicit indexing.Config + DefaultConfig(workDir) (two archied instances share carina and must not share an index); child-process helper is archied's own 'workspace-codesearch' subcommand in cmd/archied/codesearch.go, dispatched from main() before flag parsing so it touches no config/store/NATS; deferred db.Close explicitly discarded for errcheck. Added dep github.com/google/codesearch v1.3.0-rc.1 (uuid and modernc sqlite already present). Verified E2E against the real repo: 2.2MB indexed in 0.05s, correct hits, [] for no match, corrupt index -> clean error exit 1 rather than process death. task check clean; golangci-lint 0 issues on both packages. Commit c924281. NOT pushed. Follow-up: nothing calls indexing.NewManager yet -- it is ported and proven but not wired into the agent's grep path."}}
|
||||
{"id":"int-eccef7ef7597bda3a8162e06651e8ea2","kind":"field_change","created_at":"2026-07-28T08:35:54.89529236Z","actor":"Sam Catlow","issue_id":"archie-core-6pp","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Removed all unauthorized source-project references and accompanying rationale; validation and independent review complete."}}
|
||||
@@ -53,6 +53,8 @@
|
||||
{"_type":"issue","id":"archie-core-abg.18","title":"[EPIC] Memory: pluggable provider architecture with file-backed persistence","description":"Pluggable memory system to match Hermes parity:\\n\\n1. **MemoryProvider interface**: Abstract interface with methods for initialize(), get_tool_schemas(), prefetch(), sync_turn(), and optional lifecycle hooks (on_session_end, on_session_switch, on_pre_compress, on_memory_write).\\n\\n2. **Built-in file-backed memory**: MEMORY.md and USER.md stored per-session/profile. LLM-callable memory tool with add/replace/remove actions. Frozen snapshot at session start (preserves prefix cache), immediate disk write on updates. Section-sign delimited entries with substring matching.\\n\\n3. **MemoryManager**: Orchestrates built-in + at most one external provider. Background serialized write executor. Prefetch with timeout (8s). Streaming context scrubber that strips injected memory spans from model output.\\n\\nRef: docs/prds/hermes-gap-analysis.md section 4","acceptance_criteria":"An external memory provider (e.g. Honcho, Mem0) can be plugged in via config. The built-in memory tool persists across sessions. Conversation context is recalled on session resume.","status":"open","priority":0,"issue_type":"epic","owner":"sam@catlow.tech","created_at":"2026-07-25T14:02:22Z","created_by":"Sam Catlow","updated_at":"2026-07-25T14:02:22Z","dependencies":[{"issue_id":"archie-core-abg.18","depends_on_id":"archie-core-abg","type":"parent-child","created_at":"2026-07-26T00:02:22Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-abg.16","title":"[EPIC] Gateway: rich message model, adapter lifecycle, session management","description":"Foundational gateway infrastructure to match Hermes parity. Three interdependent pieces:\\n\\n1. **Message model**: Replace the 3-field Message struct with a rich MessageEvent (MessageType enum: text/photo/video/audio/voice/document/sticker/command, media_urls, reply_to context, metadata dict, source). This is the wire type every adapter produces.\\n\\n2. **Adapter lifecycle**: Replace the 3-method Gateway interface with a BasePlatformAdapter interface: connect()/disconnect()/send() as required methods, plus optional overrides for typing indicators, edit/delete, image/voice/video/document send, interactive UX (clarify buttons, approval dialogs). Capability flags (supports_code_blocks, splits_long_messages, supports_async_delivery).\\n\\n3. **Session management**: SessionSource struct (platform, chat_id, chat_type, user_id, thread_id, guild_id, profile), AsyncSessionStore for persistent session storage, SessionContext for system prompt injection.\\n\\nAll three blocks together enable adding new platforms as plugins without touching core gateway code.\\n\\nRef: docs/prds/hermes-gap-analysis.md sections 1-2","acceptance_criteria":"New adapters can be added by implementing a Go interface with connect/disconnect/send + optional media methods. Messages carry full metadata through the pipeline. Sessions persist across gateway restarts.","status":"open","priority":0,"issue_type":"epic","owner":"sam@catlow.tech","created_at":"2026-07-25T14:02:21Z","created_by":"Sam Catlow","updated_at":"2026-07-25T14:02:21Z","dependencies":[{"issue_id":"archie-core-abg.16","depends_on_id":"archie-core-abg","type":"parent-child","created_at":"2026-07-26T00:02:21Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-jfg","title":"Spawned archie-agent containers not on daemon's docker network — can't reach NATS","description":"Root cause of 'agent still failing on the server': archied's container.Pool.Acquire creates archie-agent containers via the Docker SDK with no NetworkMode set, so Docker attaches them to the default bridge network. archied and nats run on the compose-created 'archie-core_default' network instead, so the spawned container's hostname resolution for 'nats' (from NATS_URL=nats://nats:4222) fails and archie-agent can't connect. Confirmed on carina host logs: archied logs show repeating 'taskrun request failed err=\"nats: no responders available for request\"' for every task, containers created and destroyed within ~1.5s (consistent with archie-agent's NATS client failing fast). Fixed by detecting the daemon's own container's network in NewPool (selfNetwork, via ContainerInspect on os.Hostname()) and setting HostConfig.NetworkMode on spawned containers to match.","status":"closed","priority":0,"issue_type":"bug","owner":"sam@catlow.tech","created_at":"2026-07-24T03:10:26Z","created_by":"Sam Catlow","updated_at":"2026-07-25T13:04:34Z","closed_at":"2026-07-25T13:04:34Z","close_reason":"Fix already implemented and verified: selfNetwork() in internal/container/pool.go detects the daemon's own container network via ContainerInspect(os.Hostname()) and Acquire() sets HostConfig.NetworkMode to match, so spawned archie-agent containers join the same compose network as nats. Build and internal/container tests pass.","dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-6pp","title":"Remove source-project references from Archie","description":"Remove every tau reference outside the explicitly preserved Beads data and docs-2/README.md. Delete copied product-specific documentation and accompanied rationale rather than mechanically renaming it. Retain reusable documentation-generation and VitePress scaffolding only after removing source-specific coupling. Update code, tests, archives, and architecture documents so archie-core contains no unauthorized tau references.","acceptance_criteria":"rg 'tau' -ow excluding .beads/** and docs-2/README.md returns zero; retained tools and documentation scaffolding contain no source-project imports, paths, product claims, or rationale; relevant validation passes.","notes":"Complete. Exact case-insensitive whole-word scan excluding .git/**, .beads/**, and docs-2/README.md returns 0. Source-specific provenance and rationale removed, neutral fixtures preserve behavior, docs workflow/site build from docs-2, retained README excluded from publication, and Archie-owned docsgen generates current agent contracts without process-global cwd mutation. tools tests/vet pass; affected package tests pass; VitePress production build passes. Fresh adversarial review reports zero surviving findings. Full task check passes fmt/go-fix/vet/build and all packages except unrelated pre-existing internal/skillscript TestRunWrapsExternalCommand (Run returned newline).","status":"in_progress","priority":1,"issue_type":"task","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-07-28T08:00:32Z","created_by":"Sam Catlow","updated_at":"2026-07-28T08:35:54Z","started_at":"2026-07-28T08:00:37Z","dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-5d7","title":"Define target project organisation and restructuring rules","description":"Run a package-by-package architecture grilling session, establish the target DDD and semi-microservice organisation, record strict restructuring requirements and package destinations, and produce docs/prds/01-project-management.md as the durable handoff.","acceptance_criteria":"The PRD records agreed project-wide organisation rules, bounded-context and service/plugin boundaries, dependency constraints, migration principles, and the destination of each reviewed package. Decisions are confirmed one at a time with the user before the document is finalized.","notes":"Corrected documentation-generation wording: removed repeated source-project provenance from the target architecture. The copied generator/site are now described only as an implementation baseline with concrete adaptation gaps; the architecture is expressed entirely in Archie terms. Updated generated-documentation.md and its migration-decisions.md reference.","status":"in_progress","priority":1,"issue_type":"task","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-07-28T00:00:36Z","created_by":"Sam Catlow","updated_at":"2026-07-28T07:27:47Z","started_at":"2026-07-28T00:00:42Z","dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-qa4.4","title":"Phase 4: structural refactor (~28 issues)","description":"funlen, gocognit, cyclop, nestif, maintidx, interfacebloat. This IS the internal/config restructure already discussed (finalize at cognitive complexity 106, feature_config loadDirInto at 74) plus daemon process/checkWaiting. MUST BE LAST: refactoring moves line numbers and invalidates every other phase in flight. Fold into the config file-per-responsibility split rather than doing the same work twice.","status":"open","priority":1,"issue_type":"task","owner":"sam@catlow.tech","created_at":"2026-07-27T15:23:01Z","created_by":"Sam Catlow","updated_at":"2026-07-27T15:23:01Z","dependencies":[{"issue_id":"archie-core-qa4.4","depends_on_id":"archie-core-qa4","type":"parent-child","created_at":"2026-07-28T01:23:00Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-qa4.3","title":"Phase 3: correctness findings needing judgement (~62 issues)","description":"nilerr (11: 6 in internal/agentexec inprocess.go/worker.go, 2 gateway), bodyclose (1), contextcheck (7), containedctx (2), fatcontext. SINGLE SESSION, sequential, no subagents: each nilerr site needs a read of whether the swallow is deliberate degradation or a real bug, and a context-free worker will silently convert deliberate behaviour into a change. Do nilerr FIRST -- highest bug probability in the whole set. BLOCKED on a policy decision for noctx (41 sites, mostly webui/worktree/store): fixing it means threading context through public APIs, which is an API change, not a lint fix.","status":"open","priority":1,"issue_type":"task","owner":"sam@catlow.tech","created_at":"2026-07-27T15:23:00Z","created_by":"Sam Catlow","updated_at":"2026-07-27T15:23:00Z","dependencies":[{"issue_id":"archie-core-qa4.3","depends_on_id":"archie-core-qa4","type":"parent-child","created_at":"2026-07-28T01:23:00Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-qa4.1","title":"Phase 1: autofix pass (~35 issues)","description":"golangci-lint run --fix plus the formatters. Covers errorlint, fatcontext, gocritic, sloglint, usestdlibvars, usetesting, gci, gofumpt, goimports. Zero judgement. MUST run first: it touches formatting across every file and would conflict with anything in flight. Single command, single commit, verify with task check.","status":"open","priority":1,"issue_type":"task","owner":"sam@catlow.tech","created_at":"2026-07-27T15:22:59Z","created_by":"Sam Catlow","updated_at":"2026-07-27T15:22:59Z","dependencies":[{"issue_id":"archie-core-qa4.1","depends_on_id":"archie-core-qa4","type":"parent-child","created_at":"2026-07-28T01:22:58Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
name: Deploy docs
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "docs-2/**"
|
||||
- "tools/**"
|
||||
- ".github/workflows/docs.yml"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: pages
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0 # full history - VitePress lastUpdated needs it
|
||||
|
||||
- uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 10
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: latest
|
||||
cache: pnpm
|
||||
cache-dependency-path: docs-2/pnpm-lock.yaml
|
||||
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: docs-2/.vitepress/cache
|
||||
key: ${{ runner.os }}-vitepress-${{ hashFiles('docs-2/**', 'docs-2/pnpm-lock.yaml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-vitepress-
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: docs-2
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
- name: Build docs site
|
||||
working-directory: docs-2
|
||||
run: pnpm build
|
||||
|
||||
- uses: actions/configure-pages@v5
|
||||
|
||||
- uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: docs-2/.vitepress/dist
|
||||
|
||||
deploy:
|
||||
needs: build
|
||||
runs-on: ubuntu-latest
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
steps:
|
||||
- id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
+8
-29
@@ -17,16 +17,9 @@ run:
|
||||
go: "1.26"
|
||||
|
||||
linters:
|
||||
# The standard set (errcheck, govet, ineffassign, staticcheck, unused)
|
||||
# plus the curated additions below. Deliberately not `all`: the
|
||||
# formatting-opinion linters (wsl_v5, nlreturn, varnamelen) and the
|
||||
# absolutist ones (exhaustruct, err113, wrapcheck) produce more noise
|
||||
# than signal here.
|
||||
default: standard
|
||||
enable:
|
||||
# -- Structure and complexity ------------------------------------
|
||||
# Their absence is why a 180-line, cognitive-complexity-106
|
||||
# finalize() in a 745-line file reported clean.
|
||||
# Structure and complexity
|
||||
- cyclop
|
||||
- dupl
|
||||
- funlen
|
||||
@@ -35,12 +28,11 @@ linters:
|
||||
- maintidx
|
||||
- nestif
|
||||
|
||||
# -- Architecture -------------------------------------------------
|
||||
- depguard
|
||||
# Architecture
|
||||
- importas
|
||||
- interfacebloat
|
||||
|
||||
# -- Correctness --------------------------------------------------
|
||||
# Correctness
|
||||
- bodyclose
|
||||
- containedctx
|
||||
- contextcheck
|
||||
@@ -58,12 +50,12 @@ linters:
|
||||
- rowserrcheck
|
||||
- sqlclosecheck
|
||||
|
||||
# -- Idiom and hygiene --------------------------------------------
|
||||
- gocritic # also the host for ruleguard dynamic AST rules
|
||||
# Idiom and hygiene
|
||||
- gocritic
|
||||
- godox
|
||||
- misspell
|
||||
- predeclared
|
||||
- sloglint # this daemon logs with log/slog throughout
|
||||
- sloglint
|
||||
- unconvert
|
||||
- usestdlibvars
|
||||
- usetesting
|
||||
@@ -86,11 +78,10 @@ linters:
|
||||
min-complexity: 5
|
||||
|
||||
godox:
|
||||
# TODO is deliberately absent: tracked work lives in beads, and
|
||||
# flagging every TODO trains people to delete the marker rather
|
||||
# than the debt. FIXME/HACK/BUG should not survive review.
|
||||
# tracked work lives in beads - use it: `bd create [title] [flags]`
|
||||
keywords:
|
||||
- FIXME
|
||||
- TODO
|
||||
- HACK
|
||||
- BUG
|
||||
|
||||
@@ -101,18 +92,6 @@ linters:
|
||||
no-mixed-args: true
|
||||
key-naming-case: snake
|
||||
|
||||
depguard:
|
||||
rules:
|
||||
# Code ported from tau must stay a straight diff against its
|
||||
# upstream original, so it may not reach into archie-core's own
|
||||
# configuration layer. It takes config as constructor params.
|
||||
ported-stays-portable:
|
||||
files:
|
||||
- "**/internal/indexing/**"
|
||||
deny:
|
||||
- pkg: github.com/samcharles93/archie-core/internal/config
|
||||
desc: ported from tau and must remain refreshable by diff; take configuration as constructor parameters
|
||||
|
||||
exclusions:
|
||||
generated: lax
|
||||
warn-unused: true
|
||||
|
||||
@@ -1,23 +1,38 @@
|
||||
We would need to implement the agentskills spec, I'm not sure if it needs to be implemented in the Frontend app or the backend, let me know. here's the client implementation details:
|
||||
We would need to implement the agentskills spec, I'm not sure if it needs to be
|
||||
implemented in the Frontend app or the backend, let me know. here's the client
|
||||
implementation details:
|
||||
|
||||
> ## Documentation Index
|
||||
> Fetch the complete documentation index at: https://agentskills.io/llms.txt
|
||||
> Use this file to discover all available pages before exploring further.
|
||||
>
|
||||
> Fetch the complete documentation index at: https://agentskills.io/llms.txt Use
|
||||
> this file to discover all available pages before exploring further.
|
||||
|
||||
# How to add skills support to your agent
|
||||
|
||||
> A guide for adding Agent Skills support to an AI agent or development tool.
|
||||
|
||||
This guide walks through how to add Agent Skills support to an AI agent or development tool. It covers the full lifecycle: discovering skills, telling the model about them, loading their content into context, and keeping that content effective over time.
|
||||
This guide walks through how to add Agent Skills support to an AI agent or
|
||||
development tool. It covers the full lifecycle: discovering skills, telling the
|
||||
model about them, loading their content into context, and keeping that content
|
||||
effective over time.
|
||||
|
||||
The core integration is the same regardless of your agent's architecture. The implementation details vary based on two factors:
|
||||
The core integration is the same regardless of your agent's architecture. The
|
||||
implementation details vary based on two factors:
|
||||
|
||||
* **Where do skills live?** A locally-running agent can scan the user's filesystem for skill directories. A cloud-hosted or sandboxed agent will need an alternative discovery mechanism — an API, a remote registry, or bundled assets.
|
||||
* **How does the model access skill content?** If the model has file-reading capabilities, it can read `SKILL.md` files directly. Otherwise, you'll provide a dedicated tool or inject skill content into the prompt programmatically.
|
||||
- **Where do skills live?** A locally-running agent can scan the user's
|
||||
filesystem for skill directories. A cloud-hosted or sandboxed agent will need
|
||||
an alternative discovery mechanism — an API, a remote registry, or bundled
|
||||
assets.
|
||||
- **How does the model access skill content?** If the model has file-reading
|
||||
capabilities, it can read `SKILL.md` files directly. Otherwise, you'll provide
|
||||
a dedicated tool or inject skill content into the prompt programmatically.
|
||||
|
||||
The guide notes where these differences matter. You don't need to support every scenario — follow the path that fits your agent.
|
||||
The guide notes where these differences matter. You don't need to support every
|
||||
scenario — follow the path that fits your agent.
|
||||
|
||||
**Prerequisites**: Familiarity with the [Agent Skills specification](/specification), which defines the `SKILL.md` file format, frontmatter fields, and directory conventions.
|
||||
**Prerequisites**: Familiarity with the
|
||||
[Agent Skills specification](/specification), which defines the `SKILL.md` file
|
||||
format, frontmatter fields, and directory conventions.
|
||||
|
||||
## The core principle: progressive disclosure
|
||||
|
||||
@@ -29,9 +44,15 @@ Every skills-compatible agent follows the same three-tier loading strategy:
|
||||
| 2. Instructions | Full `SKILL.md` body | When the skill is activated | \<5000 tokens (recommended) |
|
||||
| 3. Resources | Scripts, references, assets | When the instructions reference them | Varies |
|
||||
|
||||
The model sees the catalog from the start, so it knows what skills are available. When it decides a skill is relevant, it loads the full instructions. If those instructions reference supporting files, the model loads them individually as needed.
|
||||
The model sees the catalog from the start, so it knows what skills are
|
||||
available. When it decides a skill is relevant, it loads the full instructions.
|
||||
If those instructions reference supporting files, the model loads them
|
||||
individually as needed.
|
||||
|
||||
This keeps the base context small while giving the model access to specialized knowledge on demand. An agent with 20 installed skills doesn't pay the token cost of 20 full instruction sets upfront — only the ones actually used in a given conversation.
|
||||
This keeps the base context small while giving the model access to specialized
|
||||
knowledge on demand. An agent with 20 installed skills doesn't pay the token
|
||||
cost of 20 full instruction sets upfront — only the ones actually used in a
|
||||
given conversation.
|
||||
|
||||
## Step 1: Discover skills
|
||||
|
||||
@@ -39,14 +60,20 @@ At session startup, find all available skills and load their metadata.
|
||||
|
||||
### Where to scan
|
||||
|
||||
Which directories you scan depends on your agent's environment. Most locally-running agents scan at least two scopes:
|
||||
Which directories you scan depends on your agent's environment. Most
|
||||
locally-running agents scan at least two scopes:
|
||||
|
||||
* **Project-level** (relative to the working directory): Skills specific to a project or repository.
|
||||
* **User-level** (relative to the home directory): Skills available across all projects for a given user.
|
||||
- **Project-level** (relative to the working directory): Skills specific to a
|
||||
project or repository.
|
||||
- **User-level** (relative to the home directory): Skills available across all
|
||||
projects for a given user.
|
||||
|
||||
Other scopes are possible too — for example, organization-wide skills deployed by an admin, or skills bundled with the agent itself. The right set of scopes depends on your agent's deployment model.
|
||||
Other scopes are possible too — for example, organization-wide skills deployed
|
||||
by an admin, or skills bundled with the agent itself. The right set of scopes
|
||||
depends on your agent's deployment model.
|
||||
|
||||
Within each scope, consider scanning both a **client-specific directory** and the **`.agents/skills/` convention**:
|
||||
Within each scope, consider scanning both a **client-specific directory** and
|
||||
the **`.agents/skills/` convention**:
|
||||
|
||||
| Scope | Path | Purpose |
|
||||
| ------- | ---------------------------------- | ----------------------------- |
|
||||
@@ -55,7 +82,11 @@ Within each scope, consider scanning both a **client-specific directory** and th
|
||||
| User | `~/.<your-client>/skills/` | Your client's native location |
|
||||
| User | `~/.agents/skills/` | Cross-client interoperability |
|
||||
|
||||
The `.agents/skills/` paths have emerged as a widely-adopted convention for cross-client skill sharing. While the Agent Skills specification does not mandate where skill directories live (it only defines what goes inside them), scanning `.agents/skills/` means skills installed by other compliant clients are automatically visible to yours, and vice versa.
|
||||
The `.agents/skills/` paths have emerged as a widely-adopted convention for
|
||||
cross-client skill sharing. While the Agent Skills specification does not
|
||||
mandate where skill directories live (it only defines what goes inside them),
|
||||
scanning `.agents/skills/` means skills installed by other compliant clients are
|
||||
automatically visible to yours, and vice versa.
|
||||
|
||||
<Note>
|
||||
Some implementations also scan `.claude/skills/` (both project-level and user-level) for pragmatic compatibility, since many existing skills are installed there. Other additional locations include ancestor directories up to the git root (useful for monorepos), [XDG](https://specifications.freedesktop.org/basedir-spec/latest/) config directories, and user-configured paths.
|
||||
@@ -63,7 +94,8 @@ The `.agents/skills/` paths have emerged as a widely-adopted convention for cros
|
||||
|
||||
### What to scan for
|
||||
|
||||
Within each skills directory, look for **subdirectories containing a file named exactly `SKILL.md`**:
|
||||
Within each skills directory, look for **subdirectories containing a file named
|
||||
exactly `SKILL.md`**:
|
||||
|
||||
```
|
||||
~/.agents/skills/
|
||||
@@ -78,31 +110,51 @@ Within each skills directory, look for **subdirectories containing a file named
|
||||
|
||||
Practical scanning rules:
|
||||
|
||||
* Skip directories that won't contain skills, such as `.git/` and `node_modules/`
|
||||
* Optionally respect `.gitignore` to avoid scanning build artifacts
|
||||
* Set reasonable bounds (e.g., max depth of 4-6 levels, max 2000 directories) to prevent runaway scanning in large directory trees
|
||||
- Skip directories that won't contain skills, such as `.git/` and
|
||||
`node_modules/`
|
||||
- Optionally respect `.gitignore` to avoid scanning build artifacts
|
||||
- Set reasonable bounds (e.g., max depth of 4-6 levels, max 2000 directories) to
|
||||
prevent runaway scanning in large directory trees
|
||||
|
||||
### Handling name collisions
|
||||
|
||||
When two skills share the same `name`, apply a deterministic precedence rule.
|
||||
|
||||
The universal convention across existing implementations: **project-level skills override user-level skills.**
|
||||
The universal convention across existing implementations: **project-level skills
|
||||
override user-level skills.**
|
||||
|
||||
Within the same scope (e.g., two skills named `code-review` found under both `<project>/.agents/skills/` and `<project>/.<your-client>/skills/`), either first-found or last-found is acceptable — pick one and be consistent. Log a warning when a collision occurs so the user knows a skill was shadowed.
|
||||
Within the same scope (e.g., two skills named `code-review` found under both
|
||||
`<project>/.agents/skills/` and `<project>/.<your-client>/skills/`), either
|
||||
first-found or last-found is acceptable — pick one and be consistent. Log a
|
||||
warning when a collision occurs so the user knows a skill was shadowed.
|
||||
|
||||
### Trust considerations
|
||||
|
||||
Project-level skills come from the repository being worked on, which may be untrusted (e.g., a freshly cloned open-source project). Consider gating project-level skill loading on a trust check — only load them if the user has marked the project folder as trusted. This prevents untrusted repositories from silently injecting instructions into the agent's context.
|
||||
Project-level skills come from the repository being worked on, which may be
|
||||
untrusted (e.g., a freshly cloned open-source project). Consider gating
|
||||
project-level skill loading on a trust check — only load them if the user has
|
||||
marked the project folder as trusted. This prevents untrusted repositories from
|
||||
silently injecting instructions into the agent's context.
|
||||
|
||||
### Cloud-hosted and sandboxed agents
|
||||
|
||||
If your agent runs in a container or on a remote server, it won't have access to the user's local filesystem. Discovery needs to work differently depending on the skill scope:
|
||||
If your agent runs in a container or on a remote server, it won't have access to
|
||||
the user's local filesystem. Discovery needs to work differently depending on
|
||||
the skill scope:
|
||||
|
||||
* **Project-level skills** are often the easiest case. If the agent operates on a cloned repository (even inside a sandbox), project-level skills travel with the code and can be scanned from the repo's directory tree.
|
||||
* **User-level and organization-level skills** don't exist in the sandbox. You'll need to provision them from an external source — for example, cloning a configuration repository, accepting skill URLs or packages through your agent's settings, or letting users upload skill directories through a web UI.
|
||||
* **Built-in skills** can be packaged as static assets within the agent's deployment artifact, making them available in every session without external fetching.
|
||||
- **Project-level skills** are often the easiest case. If the agent operates on
|
||||
a cloned repository (even inside a sandbox), project-level skills travel with
|
||||
the code and can be scanned from the repo's directory tree.
|
||||
- **User-level and organization-level skills** don't exist in the sandbox.
|
||||
You'll need to provision them from an external source — for example, cloning a
|
||||
configuration repository, accepting skill URLs or packages through your
|
||||
agent's settings, or letting users upload skill directories through a web UI.
|
||||
- **Built-in skills** can be packaged as static assets within the agent's
|
||||
deployment artifact, making them available in every session without external
|
||||
fetching.
|
||||
|
||||
Once skills are available to the agent, the rest of the lifecycle — parsing, disclosure, activation — works the same.
|
||||
Once skills are available to the agent, the rest of the lifecycle — parsing,
|
||||
disclosure, activation — works the same.
|
||||
|
||||
## Step 2: Parse `SKILL.md` files
|
||||
|
||||
@@ -110,35 +162,45 @@ For each discovered `SKILL.md`, extract the metadata and body content.
|
||||
|
||||
### Frontmatter extraction
|
||||
|
||||
A `SKILL.md` file has two parts: YAML frontmatter between `---` delimiters, and a markdown body after the closing delimiter. To parse:
|
||||
A `SKILL.md` file has two parts: YAML frontmatter between `---` delimiters, and
|
||||
a markdown body after the closing delimiter. To parse:
|
||||
|
||||
1. Find the opening `---` at the start of the file and the closing `---` after it.
|
||||
2. Parse the YAML block between them. Extract `name` and `description` (required), plus any optional fields.
|
||||
1. Find the opening `---` at the start of the file and the closing `---` after
|
||||
it.
|
||||
2. Parse the YAML block between them. Extract `name` and `description`
|
||||
(required), plus any optional fields.
|
||||
3. Everything after the closing `---`, trimmed, is the skill's body content.
|
||||
|
||||
See the [specification](/specification) for the full set of frontmatter fields and their constraints.
|
||||
See the [specification](/specification) for the full set of frontmatter fields
|
||||
and their constraints.
|
||||
|
||||
### Handling malformed YAML
|
||||
|
||||
Skill files authored for other clients may contain technically invalid YAML that their parsers happen to accept. The most common issue is unquoted values containing colons:
|
||||
Skill files authored for other clients may contain technically invalid YAML that
|
||||
their parsers happen to accept. The most common issue is unquoted values
|
||||
containing colons:
|
||||
|
||||
```yaml theme={null}
|
||||
# Technically invalid YAML — the colon breaks parsing
|
||||
description: Use this skill when: the user asks about PDFs
|
||||
```
|
||||
|
||||
Consider a fallback that wraps such values in quotes or converts them to YAML block scalars before retrying. This improves cross-client compatibility at minimal cost.
|
||||
Consider a fallback that wraps such values in quotes or converts them to YAML
|
||||
block scalars before retrying. This improves cross-client compatibility at
|
||||
minimal cost.
|
||||
|
||||
### Lenient validation
|
||||
|
||||
Warn on issues but still load the skill when possible:
|
||||
|
||||
* Name doesn't match the parent directory name → warn, load anyway
|
||||
* Name exceeds 64 characters → warn, load anyway
|
||||
* Description is missing or empty → skip the skill (a description is essential for disclosure), log the error
|
||||
* YAML is completely unparseable → skip the skill, log the error
|
||||
- Name doesn't match the parent directory name → warn, load anyway
|
||||
- Name exceeds 64 characters → warn, load anyway
|
||||
- Description is missing or empty → skip the skill (a description is essential
|
||||
for disclosure), log the error
|
||||
- YAML is completely unparseable → skip the skill, log the error
|
||||
|
||||
Record diagnostics so they can be surfaced to the user (in a debug command, log file, or UI), but don't block skill loading on cosmetic issues.
|
||||
Record diagnostics so they can be surfaced to the user (in a debug command, log
|
||||
file, or UI), but don't block skill loading on cosmetic issues.
|
||||
|
||||
<Note>
|
||||
The [specification](/specification) defines strict constraints on the `name` field (matching the parent directory, character set, max length). The lenient approach above deliberately relaxes these to improve compatibility with skills authored for other clients.
|
||||
@@ -154,19 +216,28 @@ At minimum, each skill record needs three fields:
|
||||
| `description` | From frontmatter |
|
||||
| `location` | Absolute path to the `SKILL.md` file |
|
||||
|
||||
Store these in an in-memory map keyed by `name` for fast lookup during activation.
|
||||
Store these in an in-memory map keyed by `name` for fast lookup during
|
||||
activation.
|
||||
|
||||
You can also store the **body** (the markdown content after the frontmatter) at discovery time, or read it from `location` at activation time. Storing it makes activation faster; reading it at activation time uses less memory in aggregate and picks up changes to skill files between activations.
|
||||
You can also store the **body** (the markdown content after the frontmatter) at
|
||||
discovery time, or read it from `location` at activation time. Storing it makes
|
||||
activation faster; reading it at activation time uses less memory in aggregate
|
||||
and picks up changes to skill files between activations.
|
||||
|
||||
The skill's **base directory** (the parent directory of `location`) is needed later to resolve relative paths and enumerate bundled resources — derive it from `location` when needed.
|
||||
The skill's **base directory** (the parent directory of `location`) is needed
|
||||
later to resolve relative paths and enumerate bundled resources — derive it from
|
||||
`location` when needed.
|
||||
|
||||
## Step 3: Disclose available skills to the model
|
||||
|
||||
Tell the model what skills exist without loading their full content. This is [tier 1 of progressive disclosure](#the-core-principle-progressive-disclosure).
|
||||
Tell the model what skills exist without loading their full content. This is
|
||||
[tier 1 of progressive disclosure](#the-core-principle-progressive-disclosure).
|
||||
|
||||
### Building the skill catalog
|
||||
|
||||
For each discovered skill, include `name`, `description`, and optionally `location` (the path to the `SKILL.md` file) in whatever structured format suits your stack — XML, JSON, or a bulleted list all work:
|
||||
For each discovered skill, include `name`, `description`, and optionally
|
||||
`location` (the path to the `SKILL.md` file) in whatever structured format suits
|
||||
your stack — XML, JSON, or a bulleted list all work:
|
||||
|
||||
```xml theme={null}
|
||||
<available_skills>
|
||||
@@ -183,23 +254,37 @@ For each discovered skill, include `name`, `description`, and optionally `locati
|
||||
</available_skills>
|
||||
```
|
||||
|
||||
The `location` field serves two purposes: it enables file-read activation (see [Step 4](#step-4-activate-skills)), and it gives the model a base path for resolving relative references in the skill body (like `scripts/evaluate.py`). If your dedicated activation tool provides the skill directory path in its result (see [Structured wrapping](#structured-wrapping) in Step 4), you can omit `location` from the catalog. Otherwise, include it.
|
||||
The `location` field serves two purposes: it enables file-read activation (see
|
||||
[Step 4](#step-4-activate-skills)), and it gives the model a base path for
|
||||
resolving relative references in the skill body (like `scripts/evaluate.py`). If
|
||||
your dedicated activation tool provides the skill directory path in its result
|
||||
(see [Structured wrapping](#structured-wrapping) in Step 4), you can omit
|
||||
`location` from the catalog. Otherwise, include it.
|
||||
|
||||
Each skill adds roughly 50-100 tokens to the catalog. Even with dozens of skills installed, the catalog remains compact.
|
||||
Each skill adds roughly 50-100 tokens to the catalog. Even with dozens of skills
|
||||
installed, the catalog remains compact.
|
||||
|
||||
### Where to place the catalog
|
||||
|
||||
Two approaches are common:
|
||||
|
||||
**System prompt section**: Add the catalog as a labeled section in the system prompt, preceded by brief instructions on how to use skills. This is the simplest approach and works with any model that has access to a file-reading tool.
|
||||
**System prompt section**: Add the catalog as a labeled section in the system
|
||||
prompt, preceded by brief instructions on how to use skills. This is the
|
||||
simplest approach and works with any model that has access to a file-reading
|
||||
tool.
|
||||
|
||||
**Tool description**: Embed the catalog in the description of a dedicated skill-activation tool (see [Step 4](#step-4-activate-skills)). This keeps the system prompt clean and naturally couples discovery with activation.
|
||||
**Tool description**: Embed the catalog in the description of a dedicated
|
||||
skill-activation tool (see [Step 4](#step-4-activate-skills)). This keeps the
|
||||
system prompt clean and naturally couples discovery with activation.
|
||||
|
||||
Both work. System prompt placement is simpler and more broadly compatible; tool description embedding is cleaner when you have a dedicated activation tool.
|
||||
Both work. System prompt placement is simpler and more broadly compatible; tool
|
||||
description embedding is cleaner when you have a dedicated activation tool.
|
||||
|
||||
### Behavioral instructions
|
||||
|
||||
Include a short instruction block alongside the catalog telling the model how and when to use skills. The wording depends on which activation mechanism you support (see [Step 4](#step-4-activate-skills)):
|
||||
Include a short instruction block alongside the catalog telling the model how
|
||||
and when to use skills. The wording depends on which activation mechanism you
|
||||
support (see [Step 4](#step-4-activate-skills)):
|
||||
|
||||
**If the model activates skills by reading files:**
|
||||
|
||||
@@ -219,41 +304,62 @@ When a task matches a skill's description, call the activate_skill tool
|
||||
with the skill's name to load its full instructions.
|
||||
```
|
||||
|
||||
Keep these instructions concise. The goal is to tell the model that skills exist and how to load them — the skill content itself provides the detailed instructions once loaded.
|
||||
Keep these instructions concise. The goal is to tell the model that skills exist
|
||||
and how to load them — the skill content itself provides the detailed
|
||||
instructions once loaded.
|
||||
|
||||
### Filtering
|
||||
|
||||
Some skills should be excluded from the catalog. Common reasons:
|
||||
|
||||
* The user has disabled the skill in settings
|
||||
* A permission system denies access to the skill
|
||||
* The skill has opted out of model-driven activation (e.g., via a `disable-model-invocation` flag)
|
||||
- The user has disabled the skill in settings
|
||||
- A permission system denies access to the skill
|
||||
- The skill has opted out of model-driven activation (e.g., via a
|
||||
`disable-model-invocation` flag)
|
||||
|
||||
**Hide filtered skills entirely** from the catalog rather than listing them and blocking at activation time. This prevents the model from wasting turns attempting to load skills it can't use.
|
||||
**Hide filtered skills entirely** from the catalog rather than listing them and
|
||||
blocking at activation time. This prevents the model from wasting turns
|
||||
attempting to load skills it can't use.
|
||||
|
||||
### When no skills are available
|
||||
|
||||
If no skills are discovered, omit the catalog and behavioral instructions entirely. Don't show an empty `<available_skills/>` block or register a skill tool with no valid options — this would confuse the model.
|
||||
If no skills are discovered, omit the catalog and behavioral instructions
|
||||
entirely. Don't show an empty `<available_skills/>` block or register a skill
|
||||
tool with no valid options — this would confuse the model.
|
||||
|
||||
## Step 4: Activate skills
|
||||
|
||||
When the model or user selects a skill, deliver the full instructions into the conversation context. This is [tier 2 of progressive disclosure](#the-core-principle-progressive-disclosure).
|
||||
When the model or user selects a skill, deliver the full instructions into the
|
||||
conversation context. This is
|
||||
[tier 2 of progressive disclosure](#the-core-principle-progressive-disclosure).
|
||||
|
||||
### Model-driven activation
|
||||
|
||||
Most implementations rely on the model's own judgment as the activation mechanism, rather than implementing harness-side trigger matching or keyword detection. The model reads the catalog (from [Step 3](#step-3-disclose-available-skills-to-the-model)), decides a skill is relevant to the current task, and loads it.
|
||||
Most implementations rely on the model's own judgment as the activation
|
||||
mechanism, rather than implementing harness-side trigger matching or keyword
|
||||
detection. The model reads the catalog (from
|
||||
[Step 3](#step-3-disclose-available-skills-to-the-model)), decides a skill is
|
||||
relevant to the current task, and loads it.
|
||||
|
||||
Two implementation patterns:
|
||||
|
||||
**File-read activation**: The model calls its standard file-read tool with the `SKILL.md` path from the catalog. No special infrastructure needed — the agent's existing file-reading capability is sufficient. The model receives the file content as a tool result. This is the simplest approach when the model has file access.
|
||||
**File-read activation**: The model calls its standard file-read tool with the
|
||||
`SKILL.md` path from the catalog. No special infrastructure needed — the agent's
|
||||
existing file-reading capability is sufficient. The model receives the file
|
||||
content as a tool result. This is the simplest approach when the model has file
|
||||
access.
|
||||
|
||||
**Dedicated tool activation**: Register a tool (e.g., `activate_skill`) that takes a skill name and returns the content. This is required when the model can't read files directly, and optional (but useful) even when it can. Advantages over raw file reads:
|
||||
**Dedicated tool activation**: Register a tool (e.g., `activate_skill`) that
|
||||
takes a skill name and returns the content. This is required when the model
|
||||
can't read files directly, and optional (but useful) even when it can.
|
||||
Advantages over raw file reads:
|
||||
|
||||
* Control what content is returned — e.g., strip YAML frontmatter or preserve it (see [What the model receives](#what-the-model-receives) below)
|
||||
* Wrap content in structured tags for identification during context management
|
||||
* List bundled resources (e.g., `references/*`) alongside the instructions
|
||||
* Enforce permissions or prompt for user consent
|
||||
* Track activation for analytics
|
||||
- Control what content is returned — e.g., strip YAML frontmatter or preserve it
|
||||
(see [What the model receives](#what-the-model-receives) below)
|
||||
- Wrap content in structured tags for identification during context management
|
||||
- List bundled resources (e.g., `references/*`) alongside the instructions
|
||||
- Enforce permissions or prompt for user consent
|
||||
- Track activation for analytics
|
||||
|
||||
<Tip>
|
||||
If you use a dedicated activation tool, constrain the `name` parameter to the set of valid skill names (e.g., as an enum in the tool schema). This prevents the model from hallucinating nonexistent skill names. If no skills are available, don't register the tool at all.
|
||||
@@ -261,23 +367,40 @@ Two implementation patterns:
|
||||
|
||||
### User-explicit activation
|
||||
|
||||
Users should also be able to activate skills directly, without waiting for the model to decide. The most common pattern is a **slash command or mention syntax** (`/skill-name` or `$skill-name`) that the harness intercepts. The specific syntax is up to you — the key idea is that the harness handles the lookup and injection, so the model receives skill content without needing to take an activation action itself.
|
||||
Users should also be able to activate skills directly, without waiting for the
|
||||
model to decide. The most common pattern is a **slash command or mention
|
||||
syntax** (`/skill-name` or `$skill-name`) that the harness intercepts. The
|
||||
specific syntax is up to you — the key idea is that the harness handles the
|
||||
lookup and injection, so the model receives skill content without needing to
|
||||
take an activation action itself.
|
||||
|
||||
An autocomplete widget (listing available skills as the user types) can also make this discoverable.
|
||||
An autocomplete widget (listing available skills as the user types) can also
|
||||
make this discoverable.
|
||||
|
||||
### What the model receives
|
||||
|
||||
When a skill is activated, the model receives the skill's instructions. Two options for what exactly that content looks like:
|
||||
When a skill is activated, the model receives the skill's instructions. Two
|
||||
options for what exactly that content looks like:
|
||||
|
||||
**Full file**: The model sees the entire `SKILL.md` including YAML frontmatter. This is the natural outcome with file-read activation, where the model reads the raw file. It's also a valid choice for dedicated tools. The frontmatter may contain fields useful at activation time — for example, [`compatibility`](/specification#compatibility-field) notes environment requirements that could inform how the model executes the skill's instructions.
|
||||
**Full file**: The model sees the entire `SKILL.md` including YAML frontmatter.
|
||||
This is the natural outcome with file-read activation, where the model reads the
|
||||
raw file. It's also a valid choice for dedicated tools. The frontmatter may
|
||||
contain fields useful at activation time — for example,
|
||||
[`compatibility`](/specification#compatibility-field) notes environment
|
||||
requirements that could inform how the model executes the skill's instructions.
|
||||
|
||||
**Body only (frontmatter stripped)**: The harness parses and removes the YAML frontmatter, returning only the markdown instructions. Among existing implementations with dedicated activation tools, most take this approach — stripping the frontmatter after extracting `name` and `description` during discovery.
|
||||
**Body only (frontmatter stripped)**: The harness parses and removes the YAML
|
||||
frontmatter, returning only the markdown instructions. Among existing
|
||||
implementations with dedicated activation tools, most take this approach —
|
||||
stripping the frontmatter after extracting `name` and `description` during
|
||||
discovery.
|
||||
|
||||
Both approaches work in practice.
|
||||
|
||||
### Structured wrapping
|
||||
|
||||
If you use a dedicated activation tool, consider wrapping skill content in identifying tags. For example:
|
||||
If you use a dedicated activation tool, consider wrapping skill content in
|
||||
identifying tags. For example:
|
||||
|
||||
```xml theme={null}
|
||||
<skill_content name="pdf-processing">
|
||||
@@ -301,67 +424,122 @@ Relative paths in this skill are relative to the skill directory.
|
||||
|
||||
This has practical benefits:
|
||||
|
||||
* The model can clearly distinguish skill instructions from other conversation content
|
||||
* The harness can identify skill content during context compaction ([Step 5](#step-5-manage-skill-context-over-time))
|
||||
* Bundled resources are surfaced to the model without being eagerly loaded
|
||||
- The model can clearly distinguish skill instructions from other conversation
|
||||
content
|
||||
- The harness can identify skill content during context compaction
|
||||
([Step 5](#step-5-manage-skill-context-over-time))
|
||||
- Bundled resources are surfaced to the model without being eagerly loaded
|
||||
|
||||
### Listing bundled resources
|
||||
|
||||
When a dedicated activation tool returns skill content, it can also enumerate supporting files (scripts, references, assets) in the skill directory — but it should **not eagerly read them**. The model loads specific files on demand using its file-read tools when the skill's instructions reference them.
|
||||
When a dedicated activation tool returns skill content, it can also enumerate
|
||||
supporting files (scripts, references, assets) in the skill directory — but it
|
||||
should **not eagerly read them**. The model loads specific files on demand using
|
||||
its file-read tools when the skill's instructions reference them.
|
||||
|
||||
For large skill directories, consider capping the listing and noting that it may be incomplete.
|
||||
For large skill directories, consider capping the listing and noting that it may
|
||||
be incomplete.
|
||||
|
||||
### Permission allowlisting
|
||||
|
||||
If your agent has a permission system that gates file access, **allowlist skill directories** so the model can read bundled resources without triggering user confirmation prompts. Without this, every reference to a bundled script or reference file results in a permission dialog, breaking the flow for skills that include resources beyond the `SKILL.md` itself.
|
||||
If your agent has a permission system that gates file access, **allowlist skill
|
||||
directories** so the model can read bundled resources without triggering user
|
||||
confirmation prompts. Without this, every reference to a bundled script or
|
||||
reference file results in a permission dialog, breaking the flow for skills that
|
||||
include resources beyond the `SKILL.md` itself.
|
||||
|
||||
## Step 5: Manage skill context over time
|
||||
|
||||
Once skill instructions are in the conversation context, keep them effective for the duration of the session.
|
||||
Once skill instructions are in the conversation context, keep them effective for
|
||||
the duration of the session.
|
||||
|
||||
### Protect skill content from context compaction
|
||||
|
||||
If your agent truncates or summarizes older messages when the context window fills up, **exempt skill content from pruning**. Skill instructions are durable behavioral guidance — losing them mid-conversation silently degrades the agent's performance without any visible error. The model continues operating but without the specialized instructions the skill provided.
|
||||
If your agent truncates or summarizes older messages when the context window
|
||||
fills up, **exempt skill content from pruning**. Skill instructions are durable
|
||||
behavioral guidance — losing them mid-conversation silently degrades the agent's
|
||||
performance without any visible error. The model continues operating but without
|
||||
the specialized instructions the skill provided.
|
||||
|
||||
Common approaches:
|
||||
|
||||
* Flag skill tool outputs as protected so the pruning algorithm skips them
|
||||
* Use the [structured tags](#structured-wrapping) from Step 4 to identify skill content and preserve it during compaction
|
||||
- Flag skill tool outputs as protected so the pruning algorithm skips them
|
||||
- Use the [structured tags](#structured-wrapping) from Step 4 to identify skill
|
||||
content and preserve it during compaction
|
||||
|
||||
### Deduplicate activations
|
||||
|
||||
Consider tracking which skills have been activated in the current session. If the model (or user) attempts to load a skill that's already in context, you can skip the re-injection to avoid the same instructions appearing multiple times in the conversation.
|
||||
Consider tracking which skills have been activated in the current session. If
|
||||
the model (or user) attempts to load a skill that's already in context, you can
|
||||
skip the re-injection to avoid the same instructions appearing multiple times in
|
||||
the conversation.
|
||||
|
||||
### Subagent delegation (optional)
|
||||
|
||||
This is an advanced pattern only supported by some clients. Instead of injecting skill instructions into the main conversation, the skill is run in a **separate subagent session**. The subagent receives the skill instructions, performs the task, and returns a summary of its work to the main conversation.
|
||||
This is an advanced pattern only supported by some clients. Instead of injecting
|
||||
skill instructions into the main conversation, the skill is run in a **separate
|
||||
subagent session**. The subagent receives the skill instructions, performs the
|
||||
task, and returns a summary of its work to the main conversation.
|
||||
|
||||
This pattern is useful when a skill's workflow is complex enough to benefit from a dedicated, focused session.
|
||||
This pattern is useful when a skill's workflow is complex enough to benefit from
|
||||
a dedicated, focused session.
|
||||
|
||||
---
|
||||
|
||||
Based on the specification provided and Archie's architecture (a mobile/web Flutter app with a backend profile), the **Agent Skills implementation needs to be split, with the core engine living in the Backend**.
|
||||
Based on the specification provided and Archie's architecture (a mobile/web
|
||||
Flutter app with a backend profile), the **Agent Skills implementation needs to
|
||||
be split, with the core engine living in the Backend**.
|
||||
|
||||
Here is how the responsibilities break down for Archie:
|
||||
|
||||
### 1. The Backend (The Agent Engine)
|
||||
The backend is where the actual LLM lives (or is orchestrated) and where the "cloud-hosted or sandboxed agent" rules apply.
|
||||
* **Storage & Discovery:** It stores the user's custom skills (which we create in the app) either in a database or a secure cloud bucket. When the user starts a chat session, the backend gathers the metadata (`name`, `description`) for all skills the user has enabled.
|
||||
* **Disclosure (Tier 1):** The backend injects the `catalog` of available skills into the LLM's system prompt or tool descriptions.
|
||||
* **Activation (Tier 2 & 3):** The backend provides an `activate_skill` tool to the LLM. When the LLM calls it, the backend fetches the `SKILL.md` content from the user's profile and returns it to the LLM, tracking the context so it isn't lost during conversation compaction.
|
||||
* **Execution:** If a skill has a `scripts/` folder, the backend needs a secure sandbox to execute those scripts if the agent attempts to run them.
|
||||
|
||||
The backend is where the actual LLM lives (or is orchestrated) and where the
|
||||
"cloud-hosted or sandboxed agent" rules apply.
|
||||
|
||||
- **Storage & Discovery:** It stores the user's custom skills (which we create
|
||||
in the app) either in a database or a secure cloud bucket. When the user
|
||||
starts a chat session, the backend gathers the metadata (`name`,
|
||||
`description`) for all skills the user has enabled.
|
||||
- **Disclosure (Tier 1):** The backend injects the `catalog` of available skills
|
||||
into the LLM's system prompt or tool descriptions.
|
||||
- **Activation (Tier 2 & 3):** The backend provides an `activate_skill` tool to
|
||||
the LLM. When the LLM calls it, the backend fetches the `SKILL.md` content
|
||||
from the user's profile and returns it to the LLM, tracking the context so it
|
||||
isn't lost during conversation compaction.
|
||||
- **Execution:** If a skill has a `scripts/` folder, the backend needs a secure
|
||||
sandbox to execute those scripts if the agent attempts to run them.
|
||||
|
||||
### 2. The Frontend App (What we are building now)
|
||||
Since Archie is a client app and doesn't run the LLM locally on the device (and lacks access to arbitrary system folders like `~/.agents/skills/`), its job is purely **Management and Presentation**:
|
||||
* **Skill Creation (The Library Tab):** The UI we just planned. It lets users construct the `SKILL.md` and associated `scripts/` or assets via the visual file tree. The app then bundles this into a JSON payload and sends it to the backend to be saved to their profile.
|
||||
* **Skill Toggling:** The toggles in the Library screen enable or disable which skills the backend should inject into the session catalog for that user.
|
||||
* **Visibility:** Optionally showing a visual indicator in the Chat screen when the backend reports that the agent has activated a specific skill.
|
||||
|
||||
Since Archie is a client app and doesn't run the LLM locally on the device (and
|
||||
lacks access to arbitrary system folders like `~/.agents/skills/`), its job is
|
||||
purely **Management and Presentation**:
|
||||
|
||||
- **Skill Creation (The Library Tab):** The UI we just planned. It lets users
|
||||
construct the `SKILL.md` and associated `scripts/` or assets via the visual
|
||||
file tree. The app then bundles this into a JSON payload and sends it to the
|
||||
backend to be saved to their profile.
|
||||
- **Skill Toggling:** The toggles in the Library screen enable or disable which
|
||||
skills the backend should inject into the session catalog for that user.
|
||||
- **Visibility:** Optionally showing a visual indicator in the Chat screen when
|
||||
the backend reports that the agent has activated a specific skill.
|
||||
|
||||
### How this impacts our UI plan
|
||||
The plan we just made is **perfect** for this architecture. By creating a `NewSkillEntry` widget that builds a visual file tree, we are simply constructing the package that the backend will eventually consume.
|
||||
|
||||
The plan we just made is **perfect** for this architecture. By creating a
|
||||
`NewSkillEntry` widget that builds a visual file tree, we are simply
|
||||
constructing the package that the backend will eventually consume.
|
||||
|
||||
To answer the two questions from my previous plan based on this context:
|
||||
1. **For Knowledge Entry:** We should use a simple multi-line Material `TextField` (or basic Markdown editor) for now, as it just saves raw text knowledge to the backend database.
|
||||
2. **For Skill Entry:** We don't need real device file uploads. We just need to give users text editors to define `SKILL.md` and any script strings directly in the app. The app constructs a structured JSON representing the folder and sends it to the backend.
|
||||
|
||||
Should we proceed with building out the `NewSkillEntry` file tree UI and `NewKnowledgeEntry` modules in the `LibraryScreen` as planned?
|
||||
1. **For Knowledge Entry:** We should use a simple multi-line Material
|
||||
`TextField` (or basic Markdown editor) for now, as it just saves raw text
|
||||
knowledge to the backend database.
|
||||
2. **For Skill Entry:** We don't need real device file uploads. We just need to
|
||||
give users text editors to define `SKILL.md` and any script strings directly
|
||||
in the app. The app constructs a structured JSON representing the folder and
|
||||
sends it to the backend.
|
||||
|
||||
Should we proceed with building out the `NewSkillEntry` file tree UI and
|
||||
`NewKnowledgeEntry` modules in the `LibraryScreen` as planned?
|
||||
@@ -1,219 +0,0 @@
|
||||
# agentexec-protocol
|
||||
|
||||
我现在掌握了所有需要的信息。以下是完整的报告。
|
||||
|
||||
---
|
||||
|
||||
## 报告:Archie 核心中的代理执行协议和 Runner 架构
|
||||
|
||||
### 1. Runner 接口
|
||||
|
||||
定义位置:`/work/apps/archie-core/internal/agentexec/inprocess.go`,第 20-22 行
|
||||
|
||||
```go
|
||||
type Runner interface {
|
||||
Run(ctx context.Context, workspace string, req Request) (Result, error)
|
||||
}
|
||||
```
|
||||
|
||||
**一个单一方法接口:** `Run` 接受一个上下文、一个工作目录路径(字符串)和一个 `Request`,返回一个 `Result` 和一个错误。工作目录路径与请求分开传递,供容器运行器将主机路径映射到内部文件系统。
|
||||
|
||||
该接口有两个具体实现:
|
||||
|
||||
- **`InProcessRunner`** (`/work/apps/archie-core/internal/agentexec/inprocess.go`,第 29-33 行) -- 拥有一个 `*runtime.Runtime`、一个 `*slog.Logger` 和一个 `loopFunc` 字段(默认为 `agentloop.Run`)。它在与 `archied` 相同的进程空间中运行代理循环。
|
||||
|
||||
- **`SubprocessRunner`** (`/work/apps/archie-core/internal/agentexec/subprocess.go`,第 17-27 行) -- 拥有 `Command`、`Args`、`Environ`、`AdditionalEnv`、`Diagnostics io.Writer` 和 `Providers`。它将完整的 `Invocation` 序列化到 stdin,启动一个 `archie-agent` 二进制文件,并从 stdout 解码 `Response`。
|
||||
|
||||
### 2. 线路协议
|
||||
|
||||
所有消息类型都定义在 `/work/apps/archie-core/internal/agentexec/protocol.go` 中。
|
||||
|
||||
**Invocation** (第 139-144 行) -- 被 `archie-agent` 接受的完整请求:
|
||||
```go
|
||||
type Invocation struct {
|
||||
Version int `json:"version"`
|
||||
Workspace string `json:"workspace"`
|
||||
Request Request `json:"request"`
|
||||
Providers map[string]Provider `json:"providers"`
|
||||
}
|
||||
```
|
||||
|
||||
**Request** (第 59-74 行) -- 代理阶段的有效载荷:
|
||||
```go
|
||||
type Request struct {
|
||||
Version int `json:"version"`
|
||||
TaskID int64 `json:"task_id"`
|
||||
Attempt int `json:"attempt"`
|
||||
Stage string `json:"stage"`
|
||||
Model string `json:"model"`
|
||||
Mission string `json:"mission"`
|
||||
ExtraRules string `json:"extra_rules,omitempty"`
|
||||
ReadOnly bool `json:"read_only,omitempty"`
|
||||
Budget Budget `json:"budget"`
|
||||
Gate Gate `json:"gate"`
|
||||
Preflight []Command `json:"preflight,omitempty"`
|
||||
Protection Protection `json:"protection"`
|
||||
Notes string `json:"notes,omitempty"`
|
||||
CaptureTools []CaptureTool `json:"capture_tools,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
**Result** (第 97-111 行) -- 输出:
|
||||
```go
|
||||
type Result struct {
|
||||
Version int `json:"version"`
|
||||
TaskID int64 `json:"task_id"`
|
||||
Attempt int `json:"attempt"`
|
||||
Stage string `json:"stage"`
|
||||
Status string `json:"status"`
|
||||
StopReason string `json:"stop_reason,omitempty"`
|
||||
Changes []string `json:"changes,omitempty"`
|
||||
Iterations int `json:"iterations,omitempty"`
|
||||
TokensUsed int `json:"tokens_used,omitempty"`
|
||||
Summary string `json:"summary,omitempty"`
|
||||
Detail string `json:"detail,omitempty"`
|
||||
AppendedNotes []string `json:"appended_notes,omitempty"`
|
||||
Captures map[string][]json.RawMessage `json:"captures,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
**Response** (第 168-172 行) -- 从 `archie-agent` 返回的顶层信封:
|
||||
```go
|
||||
type Response struct {
|
||||
Version int `json:"version"`
|
||||
Result Result `json:"result"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
**辅助类型:**
|
||||
- `Command` (第 18-22 行):`Name`、`Argv`、`ExpectFailure`
|
||||
- `Gate` (第 25-28 行):`Commands`、`MaxConsecutiveFailures`
|
||||
- `Budget` (第 31-35 行):`MaxSteps`、`MaxTokens`、`WallClock`
|
||||
- `Protection` (第 38-41 行):`Suffixes`、`Globs`
|
||||
- `CaptureTool` (第 45-53 行):`Name`、`Description`、`Parameters`、`RequiredFields`、`NonEmptyStrings`、`BooleanFields`、`MaxCalls`
|
||||
- `Provider` (第 115-119 行):`Class`、`APIKeyEnv`、`BaseURL`
|
||||
|
||||
协议通过 JSON 在 stdin/stdout 上使用。`ServeOne` (`worker.go`,第 16-43 行) 解码一个 `Invocation`,验证它,构建一个 `InProcessRunner`,运行它,然后将单个 `Response` 写入 stdout。`ServeOne` 不会循环——它处理一个调用然后返回。
|
||||
|
||||
### 3. cmd/archied 如何连接 Runner
|
||||
|
||||
在 `/work/apps/archie-core/cmd/archied/main.go` 中,第 109-125 行:
|
||||
|
||||
```go
|
||||
providers := executionProviders(cfg)
|
||||
llm := agentexec.NewRuntime(providers)
|
||||
var agentRunner agentexec.Runner
|
||||
if llm != nil {
|
||||
switch cfg.Agent.Mode {
|
||||
case "subprocess":
|
||||
agentRunner = &agentexec.SubprocessRunner{
|
||||
Command: cfg.Agent.Command,
|
||||
Environ: os.Environ(),
|
||||
AdditionalEnv: cfg.Agent.Env,
|
||||
Diagnostics: os.Stderr,
|
||||
Providers: providers,
|
||||
}
|
||||
case "inprocess":
|
||||
agentRunner = agentexec.NewInProcessRunner(llm, log)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
运行器模式由 `cfg.Agent.Mode` 控制,它来自配置 TOML。有两个 case:
|
||||
|
||||
- **`"subprocess"`**:创建一个 `SubprocessRunner`,将完整的环境、一个可执行文件命令(来自 `cfg.Agent.Command`,通常是 `archie-agent` 二进制文件的路径)以及提供者映射传递给它。
|
||||
- **`"inprocess"`**:通过 `NewInProcessRunner(llm, log)` 创建一个 `InProcessRunner`,其中 `llm` 是 `runtime.NewRuntime(providers)` 的结果。
|
||||
|
||||
然后,`agentRunner` 被放入 `Daemon` 结构体 (`/work/apps/archie-core/internal/daemon/daemon.go`,第 33 行) 作为 `Agent agentexec.Runner`。守护进程的 `process` 方法 (第 356-377 行) 将其传递到 `TaskContext`:
|
||||
|
||||
```go
|
||||
workflow.Run(ctx, wf, &workflow.TaskContext{
|
||||
Task: task,
|
||||
Repo: repo,
|
||||
...
|
||||
Agent: d.Agent,
|
||||
...
|
||||
})
|
||||
```
|
||||
|
||||
如果 `cfg.Agent.Mode` 既不是 `"subprocess"` 也不是 `"inprocess"`,或者如果 `llm` 是 nil(没有提供者),则 `agentRunner` 保持为 nil。当 `Agent` 为 nil 时,`AgentStage.Stage()` 方法 (`agent.go`,第 49-50 行) 返回一个合适的错误。
|
||||
|
||||
### 4. TaskContext 和 Agent 字段
|
||||
|
||||
定义位置:`/work/apps/archie-core/internal/workflow/workflow.go`,第 26-57 行。
|
||||
|
||||
```go
|
||||
type TaskContext struct {
|
||||
Task *store.Task
|
||||
Repo config.Repo
|
||||
Cfg config.Config
|
||||
Forge forge.Forge
|
||||
Store *store.Store
|
||||
Trees *worktree.Manager
|
||||
Agent agentexec.Runner // <-- 第 33 行
|
||||
Bus *events.Bus
|
||||
Log *slog.Logger
|
||||
CustomStages func(dir string) ([]Stage, error)
|
||||
Dir string
|
||||
Branch string
|
||||
BuildSummary string
|
||||
ReproProof string
|
||||
decision *decision
|
||||
Outcome Outcome
|
||||
}
|
||||
```
|
||||
|
||||
`Agent` 字段是 `agentexec.Runner`。工作流通过 `AgentStage.Stage()` (`agent.go`,第 47-148 行) 调用它。核心调用顺序是:
|
||||
|
||||
1. `AgentStage.Stage()` 从上下文字段 (`agent.go`,第 94-109 行) 构建一个 `agentexec.Request`。
|
||||
2. 调用 `tc.Agent.Run(ctx, tc.Dir, req)` (第 110 行)。
|
||||
3. 然后处理结果:验证 (`ValidateFor`,第 114 行)、追加笔记 (第 117-124 行)、累积代币/迭代次数 (第 129-130 行)、检查非 `"passed"` 状态 (第 136-141 行),以及分派给 `OnResult` 回调 (第 143-145 行)。
|
||||
|
||||
只有 `InProcessRunner` 使用 `runtime.Runtime` 直接调用 `agentloop.Run`。`SubprocessRunner` 构建一个 `Invocation`,通过提供者名称映射将完整的请求序列化,并通过 `exec.CommandContext` 启动一个子进程。
|
||||
|
||||
### 5. 使 `archie-agent` 成为长期运行的消息总线进程所需的变化
|
||||
|
||||
以下是关键的变化领域:
|
||||
|
||||
**A. 入口点 (`cmd/archie-agent/main.go`)**
|
||||
|
||||
- 目前它调用 `agentexec.ServeOne`,后者解码一个 JSON 调用,执行,并将一个 JSON 响应写入 stdout。这严格是一次性的。
|
||||
- 要变为长期运行,`main()` 需要初始化一个消息总线客户端(例如,NATS、RabbitMQ、AMQP 甚至是简单的 gRPC 流),订阅一个主题/队列,并在每个传入的 `Invocation` 上循环调用 `ServeOne`(或者更确切地说,其核心逻辑)。
|
||||
- 信号处理需要优雅地等待进行中的工作完成,而不是在 `SIGTERM` 时直接退出。
|
||||
|
||||
**B. worker.go 中的 `ServeOne` 函数**
|
||||
|
||||
- 目前它在全局范围内构建一个 `InProcessRunner`(将 `NewRuntime(invocation.Providers)` 传递给 `NewInProcessRunner`)。
|
||||
- 如果进程是长期运行的,则 `runtime.Runtime` 应该在启动时构造一次(可能来自配置或环境变量),而不是每个调用都动态构建。提供者配置会在进程的生命周期内保持不变,因此没有理由每次从 JSON 重新创建运行时。
|
||||
- `protocol.go` 中的 `Invocation` 类型包含 `Providers`——如果一个长期运行的 worker 已经在启动时知道其提供者,这可能就不需要了。工作区路径也有问题:对于长期运行的进程,工作区可能由消息路由到(或使用单独的 RPC 调用来准备),而不是嵌入在调用请求中。
|
||||
|
||||
**C. 线路协议适配**
|
||||
|
||||
- 当前协议是 request/reply(1 个 JSON 值用于输入,1 个用于输出)。对于总线,`Response` 会被发布到单独的回复主题/队列,而不是写入 stdout。
|
||||
- 运行器接口本身 (`Runner.Run(ctx, workspace, req)`) 是同步的,这很适合总线——每个消息都可以通过 `Run` 处理,结果被发布出去。
|
||||
- 调用/响应帧中的 `Version`、`TaskID`、`Attempt`、`Stage` 字段提供了足够的路由/关联上下文——`Response` 中的 `TaskID` 和 `Stage` 使得在回复主题上正确路由变得简单。
|
||||
|
||||
**D. 环境架构**
|
||||
|
||||
- 目前,`SubprocessRunner` 通过继承的环境变量传递 API 密钥,并使用 `WorkerEnvironment` 进行过滤(`subprocess.go`,第 155-162 行)。一个长期运行的进程已经在进行中,但需要能够更新/重新加载提供者凭证而不重新启动(例如,密钥轮换)。这可能需要一个控制通道或一个动态重新加载配置的方法。
|
||||
|
||||
**E. 配置和工作目录**
|
||||
|
||||
- 目前,`Invocation` 承载 `Workspace`,并且每个路径都是请求的一部分。一个基于总线的 worker 可能希望要么:
|
||||
- 在工作区已经准备就绪时接收带外消息(例如,一个单独的“准备”RPC),或者
|
||||
- 除了运行代理之外,还执行一些较小的设置步骤(克隆 repo)。
|
||||
- 守护进程当前通过 `worktree.Manager` (`steps.go`,第 22-30 行) 处理工作目录准备。如果 `archie-agent` 处理其自己的工作目录设置,这会创建一个职责划分问题。
|
||||
|
||||
**F. `SubprocessRunner` 对 `agentexec` 包的依赖**
|
||||
|
||||
- `SubprocessRunner` 是 `archied` 端的替代品——它从 `archied` 启动 `archie-agent`。如果 `archie-agent` 是一个长期运行的进程,`archied` 不会直接启动它;连接成为出站消息总线连接。`SubprocessRunner` 可能会变得过时或需要重构为一个“远程”运行器,该运行器通过总线发送消息而不是生成子进程。
|
||||
|
||||
**G. 错误处理和死信队列**
|
||||
|
||||
- 目前,`cmd.Run()` 中的进程退出代码表示协议失败。对于长期运行的进程,反序列化故障或无效调用应发布到死信队列主题。
|
||||
|
||||
总而言之,`Runner` 接口 (`Run(ctx, workspace, req)`) 本身不需要改变。然而,它的调用方式会改变:
|
||||
- `cmd/archie-agent/main.go` 将从一次性 stdin/stdout 变为总线主题订阅循环。
|
||||
- `cmd/archied/main.go` 将使用一个基于总线的远程运行器,而不是 `SubprocessRunner`。
|
||||
- 提供者查找和运行时生命周期管理将从每次调用(`worker.go` 中的 `ServeOne`)移动到进程启动时。
|
||||
@@ -1,329 +0,0 @@
|
||||
# agentexec
|
||||
|
||||
Now I have all the information needed. Here is the comprehensive report.
|
||||
|
||||
---
|
||||
|
||||
## 1. All Fields of Invocation, Request, Result, Response
|
||||
|
||||
All types are defined in `/work/apps/archie-core/internal/agentexec/protocol.go`.
|
||||
|
||||
**Invocation** (lines 139-144):
|
||||
```go
|
||||
Version int `json:"version"`
|
||||
Workspace string `json:"workspace"`
|
||||
Request Request `json:"request"`
|
||||
Providers map[string]Provider `json:"providers"`
|
||||
```
|
||||
|
||||
**Request** (lines 59-74):
|
||||
```go
|
||||
Version int `json:"version"`
|
||||
TaskID int64 `json:"task_id"`
|
||||
Attempt int `json:"attempt"`
|
||||
Stage string `json:"stage"`
|
||||
Model string `json:"model"`
|
||||
Mission string `json:"mission"`
|
||||
ExtraRules string `json:"extra_rules,omitempty"`
|
||||
ReadOnly bool `json:"read_only,omitempty"`
|
||||
Budget Budget `json:"budget"`
|
||||
Gate Gate `json:"gate"`
|
||||
Preflight []Command `json:"preflight,omitempty"`
|
||||
Protection Protection `json:"protection"`
|
||||
Notes string `json:"notes,omitempty"`
|
||||
CaptureTools []CaptureTool `json:"capture_tools,omitempty"`
|
||||
```
|
||||
|
||||
**Result** (lines 97-111):
|
||||
```go
|
||||
Version int `json:"version"`
|
||||
TaskID int64 `json:"task_id"`
|
||||
Attempt int `json:"attempt"`
|
||||
Stage string `json:"stage"`
|
||||
Status string `json:"status"`
|
||||
StopReason string `json:"stop_reason,omitempty"`
|
||||
Changes []string `json:"changes,omitempty"`
|
||||
Iterations int `json:"iterations,omitempty"`
|
||||
TokensUsed int `json:"tokens_used,omitempty"`
|
||||
Summary string `json:"summary,omitempty"`
|
||||
Detail string `json:"detail,omitempty"`
|
||||
AppendedNotes []string `json:"appended_notes,omitempty"`
|
||||
Captures map[string][]json.RawMessage `json:"captures,omitempty"`
|
||||
```
|
||||
|
||||
**Response** (lines 168-172):
|
||||
```go
|
||||
Version int `json:"version"`
|
||||
Result Result `json:"result"`
|
||||
Error string `json:"error,omitempty"`
|
||||
```
|
||||
|
||||
Supporting types referenced above:
|
||||
|
||||
- **Budget** (lines 31-35): `MaxSteps int`, `MaxTokens int`, `WallClock time.Duration`
|
||||
- **Gate** (lines 25-28): `Commands []Command`, `MaxConsecutiveFailures int`
|
||||
- **Command** (lines 18-22): `Name string`, `Argv []string`, `ExpectFailure bool`
|
||||
- **Protection** (lines 38-41): `Suffixes []string`, `Globs []string`
|
||||
- **CaptureTool** (lines 45-53): `Name string`, `Description string`, `Parameters json.RawMessage`, `RequiredFields []string`, `NonEmptyStrings []string`, `BooleanFields []string`, `MaxCalls int`
|
||||
- **Provider** (lines 115-119): `Class string`, `APIKeyEnv string`, `BaseURL string`
|
||||
|
||||
---
|
||||
|
||||
## 2. ServeOne Step by Step
|
||||
|
||||
Defined in `/work/apps/archie-core/internal/agentexec/worker.go`.
|
||||
|
||||
**`ServeOne` (lines 16-20)** is the public entry point. It delegates to `serveOne` with a factory closure that, given an `Invocation`, returns a runner:
|
||||
|
||||
```go
|
||||
func ServeOne(ctx context.Context, in io.Reader, out io.Writer, log *slog.Logger) error {
|
||||
return serveOne(ctx, in, out, func(invocation Invocation) Runner {
|
||||
return NewInProcessRunner(NewRuntime(invocation.Providers), log)
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
**`serveOne` (lines 22-43)** does:
|
||||
|
||||
1. **Decode invocation** (line 24): Calls `decodeOne(in, &invocation)` which reads up to `maxProtocolBytes+1` bytes (8 MB + 1) from `in`. Rejects if the value exceeds 8 MB. JSON-decodes into the value (here, `Invocation`). Also checks there is exactly one JSON value (rejects trailing JSON values, line 58-61).
|
||||
|
||||
2. **Validate invocation** (line 27): Calls `invocation.Validate()` which checks:
|
||||
- `Version == ProtocolVersion` (must be 1)
|
||||
- `Workspace` is non-empty
|
||||
- `Providers` map is non-empty, and every entry's `Class` is non-empty and `BaseURL` is parseable without userinfo/query/fragment
|
||||
- The inner `Request.Validate()`: `Version` must be 1, `TaskID` > 0, `Attempt` > 0, `Stage` non-empty, `Model` non-empty
|
||||
|
||||
3. **Create runner** (line 30): Calls `newRunner(invocation)` -- in ServeOne's case, `NewInProcessRunner(NewRuntime(invocation.Providers), log)`.
|
||||
|
||||
4. **Run agent** (line 34): `runner.Run(ctx, invocation.Workspace, invocation.Request)` returns `result, runErr`.
|
||||
|
||||
5. **Build response** (lines 35-38): Constructs `Response{Version: ProtocolVersion, Result: result}`. If `runErr != nil`, sets `response.Error = runErr.Error()`.
|
||||
|
||||
6. **Encode response to stdout** (lines 39-41): `json.NewEncoder(out).Encode(response)`. Returns error if encoding fails.
|
||||
|
||||
7. **Return nil** on success.
|
||||
|
||||
Key design detail: The `serveOne` generic function separates the wire protocol (stdin/stdout JSON framing) from the runner creation strategy. The factory `func(Invocation) Runner` allows different strategies (in-process vs. subprocess vs. future NATS). Currently `ServeOne` hardcodes the in-process factory, but a future NATS-based archie-agent could call `serveOne` with a different factory or skip it entirely.
|
||||
|
||||
---
|
||||
|
||||
## 3. SubprocessRunner.Run Step by Step
|
||||
|
||||
Defined in `/work/apps/archie-core/internal/agentexec/subprocess.go`.
|
||||
|
||||
The `SubprocessRunner` struct (lines 17-27):
|
||||
```go
|
||||
type SubprocessRunner struct {
|
||||
Command string
|
||||
Args []string
|
||||
Environ []string // daemon's os.Environ()
|
||||
AdditionalEnv []string // operator-approved env var names
|
||||
Diagnostics io.Writer // typically os.Stderr
|
||||
Providers map[string]Provider
|
||||
}
|
||||
```
|
||||
|
||||
**`Run` (lines 29-94)**:
|
||||
|
||||
1. **Extract provider name from model** (lines 30-32): Splits `req.Model` at the first "/". E.g. `"anthropic/claude-sonnet-4"` yields providerName=`"anthropic"`. If no "/" or empty, returns error.
|
||||
|
||||
2. **Look up provider** (lines 34-37): Indexes `r.Providers[providerName]`. Missing returns error.
|
||||
|
||||
3. **Build and validate invocation** (lines 38-42): Creates `Invocation{Version, Workspace, Request, Providers}` with just that one provider entry. Validates.
|
||||
|
||||
4. **Check command** (lines 43-45): `strings.TrimSpace(r.Command)` must be non-empty.
|
||||
|
||||
5. **Marshal invocation** (lines 46-50): `json.Marshal(invocation)` + appends newline.
|
||||
|
||||
6. **Create OS command** (line 52): `exec.CommandContext(ctx, r.Command, r.Args...)`.
|
||||
|
||||
7. **Configure process cancellation** (line 53): `configureProcessCancellation(cmd)` -- on Unix this sets `Setpgid: true` and a `Cancel` function that does `syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)` (see `/work/apps/archie-core/internal/agentexec/subprocess_process_unix.go` lines 15-27). This ensures the worker process AND all its grandchildren die on context cancellation.
|
||||
|
||||
8. **Set stdin** (line 54): `cmd.Stdin = bytes.NewReader(payload)`.
|
||||
|
||||
9. **Set environment** (line 55): `WorkerEnvironment(r.Environ, r.AdditionalEnv, providers)` which (lines 155-162):
|
||||
- Starts with `DefaultEnvironmentNames()`: `PATH, HOME, TMPDIR, TMP, TEMP, LANG, LC_ALL, SSL_CERT_FILE, SSL_CERT_DIR`
|
||||
- Appends `r.AdditionalEnv`
|
||||
- Appends each provider's `APIKeyEnv`
|
||||
- Calls `SelectEnvironment` which copies only those named variables from `r.Environ`, preserving order.
|
||||
|
||||
10. **Capture stdout and stderr** (lines 56-62):
|
||||
- Stdout: `cappedBuffer{limit: maxProtocolBytes}` (8 MB). If `r.Diagnostics != nil`, stderr goes to `io.MultiWriter(r.Diagnostics, stderr)` (tee to both the diagnostics writer and the capped buffer). Otherwise just the capped buffer (64 KB limit via `maxDiagnosticBytes`).
|
||||
|
||||
11. **Run the process** (lines 63-71): `cmd.Run()`. On error:
|
||||
- If `context.Cause(ctx)` is set (cancellation), returns the cause wrapped with any stderr diagnostic.
|
||||
- Otherwise returns the exec error with stderr output.
|
||||
|
||||
12. **Check stdout truncation** (lines 72-74): If `stdout.truncated` is true, return error.
|
||||
|
||||
13. **Decode response** (lines 75-78): `decodeOne(bytes.NewReader(stdout.Bytes()), &response)` -- same framing as the reader side.
|
||||
|
||||
14. **Validate response version** (lines 79-81): Must match `ProtocolVersion` (1).
|
||||
|
||||
15. **Validate result against request** (lines 82-91):
|
||||
- If `response.Result.Version != 0`, calls `response.Result.ValidateFor(req)` -- checks Version==1, TaskID/Attempt/Stage identity match, Status non-empty.
|
||||
- If `response.Error != ""`, returns result with `errors.New(response.Error)`.
|
||||
- Validates one more time at line 90 for the non-error case.
|
||||
|
||||
16. **Return** (line 93): `response.Result, nil`.
|
||||
|
||||
---
|
||||
|
||||
## 4. NewRuntime and InProcessRunner
|
||||
|
||||
**`NewRuntime`** in `/work/apps/archie-core/internal/agentexec/runtime.go` (lines 7-23):
|
||||
- Takes `providers map[string]Provider` (agentexec.Provider type).
|
||||
- Returns `*runtime.Runtime` (from `github.com/samcharles93/ai-sdk/runtime`).
|
||||
- If providers is empty, returns `nil`.
|
||||
- Calls `runtime.RegisterBuiltinClasses()` (global, side-effecting).
|
||||
- For each entry: creates `runtime.ProviderConfig{ID: name, Class: provider.Class, BaseURL: provider.BaseURL}`.
|
||||
- If `provider.APIKeyEnv == ""`, sets `Auth = AuthConfig{Type: AuthTypeNone}`, otherwise `Auth = AuthConfig{APIKeyEnv: provider.APIKeyEnv}`.
|
||||
- Calls `runtime.NewRuntime(runtime.Config{Providers: catalog})` and returns it.
|
||||
|
||||
**`InProcessRunner`** in `/work/apps/archie-core/internal/agentexec/inprocess.go` (lines 29-33):
|
||||
```go
|
||||
type InProcessRunner struct {
|
||||
runtime *runtime.Runtime
|
||||
log *slog.Logger
|
||||
run loopFunc // func(context.Context, agentloop.Config) (agentloop.Result, error)
|
||||
}
|
||||
```
|
||||
|
||||
**`NewInProcessRunner`** (lines 35-37): Stores runtime, log, and sets `run = agentloop.Run` (the exported function from the `agentloop` package).
|
||||
|
||||
**`InProcessRunner.Run`** (lines 39-86):
|
||||
1. Validates `req` via `req.Validate()`.
|
||||
2. Checks `r.runtime != nil` and `r.run != nil`.
|
||||
3. Creates `memoryNotes` struct (lines 219-228 of inprocess.go) -- implements a simple `Load`/`Append` interface. Initialized with `req.Notes`, collects `appended` entries.
|
||||
4. Creates `captures` map: `map[string][]json.RawMessage`.
|
||||
5. Calls `r.run(ctx, agentloop.Config{...})` with all fields mapped from `req`:
|
||||
- `Runtime: r.runtime`
|
||||
- `ModelRef: req.Model`
|
||||
- `WorkDir: workspace`
|
||||
- `Mission: req.Mission`, `ExtraRules: req.ExtraRules`
|
||||
- `Notes: notes` (the memoryNotes)
|
||||
- `Gate: toAgentGate(req.Gate)` (converts agentexec.Gate -> agentloop.GateConfig)
|
||||
- `Preflight: toAgentCommands(req.Preflight)`
|
||||
- `Budget: agentloop.Budget(req.Budget)` (direct cast)
|
||||
- `ReadOnly: req.ReadOnly`
|
||||
- `ProtectPaths: protectionMatcher(...)` (closure that checks suffixes/globs)
|
||||
- `Extra: mergeToolSets(captureToolSet(req.CaptureTools, captures), scriptToolSet(workspace))`
|
||||
- `Logger: r.logger(req)` (structured with task/attempt/stage/model)
|
||||
6. Builds `Result` from the returned `agentloop.Result` fields.
|
||||
7. Checks `context.Cause(ctx)` for cancellation -- if cancelled, returns result with cancellation cause as error.
|
||||
8. Returns `result, err`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Provider Types
|
||||
|
||||
There are **two separate `Provider` structs** with identical fields but different packages and tags:
|
||||
|
||||
**`agentexec.Provider`** (`/work/apps/archie-core/internal/agentexec/protocol.go`, lines 115-119):
|
||||
```go
|
||||
type Provider struct {
|
||||
Class string `json:"class"`
|
||||
APIKeyEnv string `json:"api_key_env,omitempty"`
|
||||
BaseURL string `json:"base_url,omitempty"`
|
||||
}
|
||||
```
|
||||
Used for: JSON serialization to the subprocess worker, and as the type consumed by `NewRuntime` and `SubprocessRunner.Providers`.
|
||||
|
||||
**`config.Provider`** (`/work/apps/archie-core/internal/config/config.go`, lines 125-129):
|
||||
```go
|
||||
type Provider struct {
|
||||
Class string `toml:"class"`
|
||||
APIKeyEnv string `toml:"api_key_env"`
|
||||
BaseURL string `toml:"base_url"`
|
||||
}
|
||||
```
|
||||
Used for: TOML deserialization from the config file.
|
||||
|
||||
**Bridge** (`/work/apps/archie-core/cmd/archied/main.go`, lines 182-188):
|
||||
```go
|
||||
func executionProviders(cfg config.Config) map[string]agentexec.Provider {
|
||||
providers := make(map[string]agentexec.Provider, len(cfg.Providers))
|
||||
for name, p := range cfg.Providers {
|
||||
providers[name] = agentexec.Provider{Class: p.Class, APIKeyEnv: p.APIKeyEnv, BaseURL: p.BaseURL}
|
||||
}
|
||||
return providers
|
||||
}
|
||||
```
|
||||
The daemon composition root converts config.Provider to agentexec.Provider by field-wise copy and then passes them to both `NewRuntime` and `SubprocessRunner`.
|
||||
|
||||
---
|
||||
|
||||
## 6. How drainNATS Interacts with SubprocessRunner
|
||||
|
||||
**Current state: drainNATS only affects task distribution, not agent execution.**
|
||||
|
||||
The flow:
|
||||
|
||||
1. **Daemon.Cycle** (daemon.go lines 94-103): If `d.Nats != nil`, calls `d.drainNATS(ctx)`, else `d.drainSQLite(ctx)`.
|
||||
|
||||
2. **drainNATS** (lines 126-151): Calls `d.Nats.Fetch(ctx)` to get a JetStream message. If found, calls `processNATSTask`. Falls back to `drainSQLite` logic (SQLite `ClaimNext` then `d.process`) if no NATS message.
|
||||
|
||||
3. **processNATSTask** (lines 219-253): Decodes `TaskMessage`, enqueues to SQLite, claims the task, then calls `d.process(ctx, task)` -- the **same** `process` function used by the SQLite path.
|
||||
|
||||
4. **d.process** (lines 472-493): Routes the task to the correct workflow, then calls:
|
||||
```go
|
||||
workflow.Run(ctx, wf, &workflow.TaskContext{
|
||||
...
|
||||
Agent: d.Agent, // line 488 -- the agentexec.Runner interface
|
||||
...
|
||||
})
|
||||
```
|
||||
|
||||
5. **workflow.Run** (workflow.go line 145): Iterates stages. For LLM-driven stages, calls `AgentStage.Stage()` (agent.go line 110):
|
||||
```go
|
||||
res, err := tc.Agent.Run(ctx, tc.Dir, req)
|
||||
```
|
||||
This `Agent` field is the `agentexec.Runner` interface. It was wired in `cmd/archied/main.go` (lines 124-138):
|
||||
- If `cfg.Agent.Mode == "subprocess"`: creates `&agentexec.SubprocessRunner{...}` with `Command`, `Environ`, `AdditionalEnv`, `Diagnostics`, `Providers`.
|
||||
- If `cfg.Agent.Mode == "inprocess"`: creates `agentexec.NewInProcessRunner(llm, log)`.
|
||||
|
||||
6. **SubprocessRunner.Run** spawns a fresh `archie-agent` binary per stage, sends JSON on stdin, reads JSON from stdout.
|
||||
|
||||
7. **The daemon's NATS `processNATSTask` does NOT interact with SubprocessRunner directly** -- it only fetches the task message from NATS, enqueues it to SQLite, and claims it. The actual agent execution happens through `d.Agent.Run` which remains either in-process or subprocess regardless of the NATS path.
|
||||
|
||||
---
|
||||
|
||||
## Design Implications for NATSRunner and NATS-based archie-agent
|
||||
|
||||
**Runner interface** (`/work/apps/archie-core/internal/agentexec/inprocess.go`, line 20-22):
|
||||
```go
|
||||
type Runner interface {
|
||||
Run(ctx context.Context, workspace string, req Request) (Result, error)
|
||||
}
|
||||
```
|
||||
|
||||
A new `NATSRunner` would need to implement this same interface. It would:
|
||||
|
||||
1. Accept a NATS connection, a subject prefix, and the provider map (or the whole Invocation fields).
|
||||
2. In `Run(ctx, workspace, req)`:
|
||||
- Build an `Invocation` (same as `SubprocessRunner.Run` does on lines 38-42).
|
||||
- Publish it to a NATS subject (e.g. `archie.stage.request.<taskID>`).
|
||||
- Include a reply subject in the message headers or as part of the payload.
|
||||
- Wait for the response on the reply subject (with the context's deadline).
|
||||
- Decode the `Response`, validate, return `Result`.
|
||||
3. No need for stdin/stdout marshalling or process management.
|
||||
|
||||
**New archie-agent** (`cmd/archie-agent/main.go`): Instead of `ServeOne(ctx, os.Stdin, os.Stdout, log)`, it would:
|
||||
1. Connect to NATS.
|
||||
2. Subscribe to a subject like `archie.stage.request.>`.
|
||||
3. For each message: decode `Invocation` from the message body, call `serveOne`-style logic (validate, build runner, run, build response), publish `Response` back to the reply subject, ack.
|
||||
4. Could be a long-running daemon with a pool of subscribers.
|
||||
|
||||
**Key integration point**: The daemon composition root (`cmd/archied/main.go` lines 124-138) would gain a third branch:
|
||||
```go
|
||||
case "nats":
|
||||
agentRunner = &agentexec.NATSRunner{
|
||||
Conn: natsClient.Conn(), // or a new NATS connection
|
||||
Subject: "archie.stage.request",
|
||||
Providers: providers,
|
||||
}
|
||||
```
|
||||
|
||||
And in `config.go` the `Agent.Mode` validation (lines 298-305) would accept `"nats"` in addition to `"inprocess"` and `"subprocess"`.
|
||||
|
||||
**The `serveOne` generic function** in `worker.go` (lines 22-43) is already well-factored: it separates wire protocol from runner creation. A NATS-based archie-agent could reuse `serveOne` by providing a factory that creates an in-process runner, or it could have its own message handler that calls the same `Validate` -> `Runner.Run` -> `Response` logic.
|
||||
@@ -1,322 +0,0 @@
|
||||
# nats-reply
|
||||
|
||||
Now I have a complete understanding. Here is the comprehensive report.
|
||||
|
||||
---
|
||||
|
||||
## NATS Agent Communication Reply Pattern -- Design Report
|
||||
|
||||
### Sources Read
|
||||
|
||||
- `/work/apps/archie-core/internal/nats/client.go` -- existing JetStream Client, PublishTask, Fetch, Connect
|
||||
- `/work/apps/archie-core/internal/nats/subjects.go` -- existing subject constants
|
||||
- `/work/apps/archie-core/internal/daemon/daemon.go` -- current NATS integration in the daemon
|
||||
- `/work/apps/archie-core/internal/config/config.go` -- NATSConfig struct
|
||||
- `/work/apps/archie-core/go.mod` -- nats.go v1.52.0, nats-server/v2 v2.14.3
|
||||
- `/work/apps/archie-core/internal/agentexec/inprocess.go` -- Runner interface definition (line 20)
|
||||
- `/work/apps/archie-core/internal/agentexec/subprocess.go` -- SubprocessRunner.Run (blocks on exec.Command)
|
||||
- `/work/apps/archie-core/internal/agentexec/protocol.go` -- Request, Result, Invocation, Response types
|
||||
- `/work/apps/archie-core/internal/workflow/agent.go` -- how AgentStage calls `tc.Agent.Run(ctx, tc.Dir, req)` (line 110)
|
||||
- `/home/sam/go/pkg/mod/github.com/nats-io/nats.go@v1.52.0/jetstream/publish.go` -- PublishMsg internals
|
||||
- `/home/sam/go/pkg/mod/github.com/nats-io/nats.go@v1.52.0/jetstream/message.go` -- jetstream.Msg interface (Reply(), Headers(), etc.)
|
||||
- `/home/sam/go/pkg/mod/github.com/nats-io/nats.go@v1.52.0/context.go` -- RequestMsgWithContext, requestWithContext
|
||||
- `/home/sam/go/pkg/mod/github.com/nats-io/nats.go@v1.52.0/nats.go` -- Msg struct, NewInbox, createNewRequestAndSend
|
||||
- `/work/apps/archie-core/.scratch/plan-nats-design.md` -- the original Phase 1 NATS task distribution design document
|
||||
|
||||
---
|
||||
|
||||
### Key Finding: `PublishMsg` Eats the Reply Field
|
||||
|
||||
The most critical technical finding is in `jetstream/publish.go` (line 229-232) and `context.go` (line 40-84). When you call:
|
||||
|
||||
```go
|
||||
js.PublishMsg(ctx, &nats.Msg{Subject: subj, Reply: "my.reply", Data: data})
|
||||
```
|
||||
|
||||
The `PublishMsg` implementation calls `js.conn.RequestMsgWithContext(ctx, m)`, which internally calls `createNewRequestAndSend(subj, hdr, data)` (new style) or `oldRequestWithContext` (old style). Both paths create a **fresh inbox** for the JetStream PubAck response and call `nc.publish(subj, respInbox, ...)` -- the reply subject you set on the `nats.Msg` is **completely ignored**. The wire-level `Reply` on the published NATS message is the PubAck inbox, not your custom reply subject.
|
||||
|
||||
There is a second issue: even if the Reply did pass through, it would need to point at an inbox the JetStream server could reach. And `PublishMsgAsync` explicitly rejects messages with `Reply` set (`ErrAsyncPublishReplySubjectSet`, line 336-337 of publish.go).
|
||||
|
||||
**This means: the reply-to address must be carried outside the `nats.Msg.Reply` field -- in custom headers or the message body.**
|
||||
|
||||
---
|
||||
|
||||
### 1. Subject Naming Pattern
|
||||
|
||||
**Requests** (published by the daemon to JetStream):
|
||||
```
|
||||
archie.agent.<task_id>.<stage>
|
||||
```
|
||||
Examples: `archie.agent.42.planner`, `archie.agent.42.builder`, `archie.agent.17.analyst`
|
||||
|
||||
**Responses** (published by the agent worker back to the daemon):
|
||||
Use core NATS reply inboxes, generated by `nc.NewInbox()`:
|
||||
```
|
||||
_INBOX.<random-hex-string>
|
||||
```
|
||||
These are ephemeral, one-shot subjects. No JetStream stream is involved on the response path.
|
||||
|
||||
**Why not a deterministic subject for responses?** The daemon needs to wait for exactly one response from a specific agent execution. An inbox subscription is the NATS-idiomatic way: it creates a unique subject per request, the daemon subscribes to it before publishing, and no other subscriber can accidentally consume the response.
|
||||
|
||||
**New subject constants to add to `internal/nats/subjects.go`:**
|
||||
```go
|
||||
const SubjectAgentRequest = "archie.agent." // prefix, full subject is built with task_id and stage
|
||||
|
||||
func SubjectForAgentRequest(taskID int64, stage string) string {
|
||||
return fmt.Sprintf("archie.agent.%d.%s", taskID, stage)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. JetStream or Core NATS?
|
||||
|
||||
**Hybrid: JetStream for requests, core NATS for responses.**
|
||||
|
||||
| Aspect | Requests | Responses |
|
||||
|--------|----------|-----------|
|
||||
| Transport | JetStream (`js.PublishMsg`) | Core NATS (`nc.Publish`) |
|
||||
| Persistence | Yes (survives daemon crash) | No (ephemeral) |
|
||||
| Dedup | Yes (Nats-Msg-Id header) | Not needed |
|
||||
| Delivery guarantee | At-least-once (consumer retries) | Best-effort (daemon subscribed) |
|
||||
|
||||
**Rationale:**
|
||||
- The **request** (agentexec.Request) is a large, important payload. If the daemon crashes after publishing but before the response arrives, JetStream retains the message. A restarted daemon (or a different daemon on the same stream) can pick it up.
|
||||
- The **response** is an ephemeral, one-shot delivery. If the daemon disconnects while waiting (crash, network blip), the response is lost -- but that is acceptable because the agent will eventually time out waiting for a result confirmation and the daemon can re-issue the request.
|
||||
- Using JetStream for responses would require a second pull consumer and polling -- incompatible with the blocking `Runner.Run` contract.
|
||||
- The existing `nc.Conn` is already available via `js.Conn()`, so no extra connection is needed.
|
||||
|
||||
---
|
||||
|
||||
### 3. Correlating Requests with Responses
|
||||
|
||||
Use **reply inbox** correlation -- the standard NATS request-reply pattern:
|
||||
|
||||
1. Daemon calls `nc.NewInbox()` to get a unique reply subject
|
||||
2. Daemon subscribes to that inbox via `nc.SubscribeSync(inbox)` with `AutoUnsubscribe(1)`
|
||||
3. Daemon sets the inbox address in a **custom header** `X-Archie-Reply` on the JetStream message
|
||||
4. Daemon publishes the agent request via `js.PublishMsg`
|
||||
5. Daemon blocks on `sub.NextMsg(timeout)` waiting for the response
|
||||
6. Agent worker receives the JetStream message, reads the `X-Archie-Reply` header
|
||||
7. Agent worker processes the request (may take minutes)
|
||||
8. Agent worker publishes the result to the reply inbox via `nc.Publish(replySubject, jsonResult)`
|
||||
9. Daemon receives the response and returns `agentexec.Result`
|
||||
|
||||
**No separate correlation ID is needed.** The reply inbox IS the correlation mechanism. The `agentexec.Response` already carries `TaskID`, `Attempt`, and `Stage` for validation (`ValidateFor` in `agentexec/protocol.go:176-190`), so we get double-checking on the identity of the response without any new identifiers.
|
||||
|
||||
---
|
||||
|
||||
### 4. Changes to `internal/nats`
|
||||
|
||||
#### 4a. New message types
|
||||
|
||||
Add to `client.go` (or a new `agent.go`):
|
||||
```go
|
||||
// AgentRequestMessage is the NATS payload for an agent stage execution request.
|
||||
type AgentRequestMessage struct {
|
||||
TaskID int64 `json:"task_id"`
|
||||
Attempt int `json:"attempt"`
|
||||
Stage string `json:"stage"`
|
||||
Workspace string `json:"workspace"`
|
||||
Request agentexec.Request `json:"request"`
|
||||
Providers map[string]agentexec.Provider `json:"providers,omitempty"`
|
||||
}
|
||||
|
||||
// AgentResponseEnvelope is the response payload published to the reply inbox.
|
||||
type AgentResponseEnvelope struct {
|
||||
Version int `json:"version"`
|
||||
Result agentexec.Result `json:"result"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
#### 4b. New methods on `Client`
|
||||
|
||||
```go
|
||||
// ReplyHeader is the custom header carrying the daemon's reply inbox.
|
||||
const ReplyHeader = "X-Archie-Reply"
|
||||
|
||||
// NewReplyInbox creates a unique inbox and returns a sync subscription.
|
||||
func (c *Client) NewReplyInbox() (*nats.Subscription, error) {
|
||||
sub, err := c.conn.SubscribeSync(c.conn.NewInbox())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sub.AutoUnsubscribe(1) // exactly one response expected
|
||||
return sub, nil
|
||||
}
|
||||
|
||||
// PublishAgentRequest publishes an agent execution request to JetStream
|
||||
// with the reply inbox set in the X-Archie-Reply header.
|
||||
func (c *Client) PublishAgentRequest(ctx context.Context, req agentexec.Request, providers map[string]agentexec.Provider, workspace string, replyInbox string) error {
|
||||
msg := AgentRequestMessage{
|
||||
TaskID: req.TaskID,
|
||||
Attempt: req.Attempt,
|
||||
Stage: req.Stage,
|
||||
Workspace: workspace,
|
||||
Request: req,
|
||||
Providers: providers,
|
||||
}
|
||||
data, err := json.Marshal(msg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
subject := SubjectForAgentRequest(req.TaskID, req.Stage)
|
||||
headers := nats.Header{}
|
||||
headers.Set(ReplyHeader, replyInbox)
|
||||
msgID := fmt.Sprintf("%s%d/%d/%s", msgPrefix, req.TaskID, req.Attempt, req.Stage)
|
||||
headers.Set("Nats-Msg-Id", msgID)
|
||||
|
||||
_, err = c.js.PublishMsg(ctx, &nats.Msg{
|
||||
Subject: subject,
|
||||
Data: data,
|
||||
Header: headers,
|
||||
})
|
||||
return err
|
||||
}
|
||||
```
|
||||
|
||||
#### 4c. New stream or extend existing
|
||||
|
||||
The existing `ARCHIE_TASKS` stream has subjects `["archie.task.>"]`. Agent requests use `archie.agent.<task_id>.<stage>`, which does **not** match `archie.task.>`.
|
||||
|
||||
**Recommendation: Extend the existing stream's subjects** by adding `"archie.agent.>"` to `StreamConfig.Subjects`. This is simpler than creating a second stream and avoids managing two stream lifecycles:
|
||||
|
||||
```go
|
||||
stream, err := js.CreateOrUpdateStream(ctx, jetstream.StreamConfig{
|
||||
Name: streamName,
|
||||
Subjects: []string{"archie.task.>", "archie.agent.>"},
|
||||
Storage: jetstream.FileStorage,
|
||||
Retention: jetstream.WorkQueuePolicy,
|
||||
Duplicates: dedupWindow,
|
||||
})
|
||||
```
|
||||
|
||||
The same WorkQueuePolicy works: when an agent receives and acks the request, it is removed from the stream. If the daemon crashes, the message stays until the agent acks or `MaxDeliver` is reached.
|
||||
|
||||
**Alternative: Create a dedicated `ARCHIE_AGENT` stream.** This would allow different retention/deduplication settings (e.g., a `LimitsPolicy` with a `MaxAge` of 1 hour vs the current WorkQueue). But this adds complexity. Start with the extension approach.
|
||||
|
||||
#### 4d. NATSRunner: bridge between Runner interface and NATS
|
||||
|
||||
Create `/work/apps/archie-core/internal/agentexec/nats.go`:
|
||||
|
||||
```go
|
||||
type NATSRunner struct {
|
||||
Nats *nats.Client
|
||||
Providers map[string]agentexec.Provider
|
||||
Log *slog.Logger
|
||||
}
|
||||
|
||||
func (r *NATSRunner) Run(ctx context.Context, workspace string, req agentexec.Request) (agentexec.Result, error) {
|
||||
if err := req.Validate(); err != nil {
|
||||
return agentexec.Result{}, err
|
||||
}
|
||||
|
||||
// 1. Create reply inbox
|
||||
replySub, err := r.Nats.NewReplyInbox()
|
||||
if err != nil {
|
||||
return agentexec.Result{}, fmt.Errorf("nats reply inbox: %w", err)
|
||||
}
|
||||
defer replySub.Unsubscribe()
|
||||
|
||||
// 2. Compute timeout from budget
|
||||
timeout := defaultAgentTimeout
|
||||
if req.Budget.WallClock > 0 {
|
||||
timeout = req.Budget.WallClock
|
||||
}
|
||||
|
||||
// 3. Publish request with reply header
|
||||
if err := r.Nats.PublishAgentRequest(ctx, req, r.Providers, workspace, replySub.Subject); err != nil {
|
||||
return agentexec.Result{}, fmt.Errorf("publish agent request: %w", err)
|
||||
}
|
||||
|
||||
// 4. Wait for reply (blocking, matching SubprocessRunner.Run behavior)
|
||||
reply, err := replySub.NextMsg(timeout)
|
||||
if err != nil {
|
||||
return agentexec.Result{}, fmt.Errorf("agent reply timeout: %w", err)
|
||||
}
|
||||
|
||||
// 5. Decode response
|
||||
var envelope AgentResponseEnvelope
|
||||
if err := json.Unmarshal(reply.Data, &envelope); err != nil {
|
||||
return agentexec.Result{}, fmt.Errorf("decode agent response: %w", err)
|
||||
}
|
||||
if envelope.Error != "" {
|
||||
return envelope.Result, errors.New(envelope.Error)
|
||||
}
|
||||
if err := envelope.Result.ValidateFor(req); err != nil {
|
||||
return agentexec.Result{}, err
|
||||
}
|
||||
return envelope.Result, nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Simplest Possible Reply Pattern
|
||||
|
||||
**Respondent's note: the paragraphs below are the core answer.**
|
||||
|
||||
The simplest pattern that works with the existing JetStream setup:
|
||||
|
||||
1. **Daemon side** (NATSRunner, blocking like SubprocessRunner):
|
||||
- `sub, _ := nc.SubscribeSync(nc.NewInbox())` -- creates a unique reply subject as a sync subscription
|
||||
- `sub.AutoUnsubscribe(1)` -- exactly one reply expected
|
||||
- Publish to JetStream: `js.PublishMsg(ctx, &nats.Msg{Subject: "archie.agent.<taskID>.<stage>", Header: {"X-Archie-Reply": [inbox]}, Data: payload})`
|
||||
- Block: `msg, err := sub.NextMsg(budget.WallClock)` -- returns `agentexec.Result` on success, error on timeout
|
||||
- Unsubscribe the inbox
|
||||
|
||||
2. **Agent side** (archie-agent worker, or any consumer):
|
||||
- Receive via JetStream pull consumer
|
||||
- `replyTo := msg.Headers().Get("X-Archie-Reply")`
|
||||
- Process the request (the existing agentloop)
|
||||
- Result: `nc.Publish(replyTo, jsonBytes)` -- a core NATS publish, no JetStream involved
|
||||
|
||||
3. **No additional JetStream consumers, streams, or streams needed.** The existing `nc.Conn` (exposed as `js.Conn()`) handles both directions.
|
||||
|
||||
**NATS API calls involved:**
|
||||
|
||||
| Operation | API Call | Package |
|
||||
|-----------|----------|---------|
|
||||
| Create reply inbox | `nc.NewInbox()` / `nc.SubscribeSync(inbox)` | `nats.go` core |
|
||||
| Publish request | `js.PublishMsg(ctx, &nats.Msg{...})` | `jetstream` |
|
||||
| Wait for reply | `sub.NextMsg(timeout)` | `nats.go` core |
|
||||
| Send reply (agent) | `nc.Publish(reply, data)` | `nats.go` core |
|
||||
|
||||
**Total new lines of code in `internal/nats`:** approximately 100-120 lines (two message structs, two new helper functions, one new subject function, and the header constant).
|
||||
|
||||
**Total new lines for `NATSRunner`:** approximately 50-60 lines.
|
||||
|
||||
---
|
||||
|
||||
### Configuration and Wiring
|
||||
|
||||
The daemon already supports `agent.mode` with values `"inprocess"` and `"subprocess"`. Add `"nats"` as a third mode.
|
||||
|
||||
In `cmd/archied/main.go`, when `cfg.Agent.Mode == "nats"`:
|
||||
```go
|
||||
runner := &agentexec.NATSRunner{
|
||||
Nats: natsClient,
|
||||
Providers: agentexec.ProviderConfig(cfg.Providers),
|
||||
Log: log.With("runner", "nats"),
|
||||
}
|
||||
```
|
||||
|
||||
No changes are needed to the `workflow` package or the daemon's `process` method -- they already consume `tc.Agent.Run(ctx, tc.Dir, req)` via the `agentexec.Runner` interface.
|
||||
|
||||
---
|
||||
|
||||
### Critical Technical Decision: Reply Subject in Headers
|
||||
|
||||
The `X-Archie-Reply` header approach was chosen over alternatives:
|
||||
|
||||
- **Reply in `nats.Msg.Reply` field**: Does not work -- `PublishMsg` internally uses the Reply field for the JetStream PubAck response and overwrites whatever you set.
|
||||
- **Reply in message body**: Would require the agent to parse the JSON body before knowing where to respond -- a chicken-and-egg problem if the body is corrupted or the format changes.
|
||||
- **Correlation ID in headers + deterministic response subject**: Would require provisioning a consumer on the response subject and polling -- breaks the blocking `Runner.Run` contract.
|
||||
|
||||
The reply inbox approach gives us:
|
||||
- No polling (daemon blocks on sync subscription)
|
||||
- Automatic correlation (each request gets a unique inbox)
|
||||
- No consumer lifecycle management for responses
|
||||
- Matching the existing `SubprocessRunner` blocking contract exactly
|
||||
@@ -1,188 +0,0 @@
|
||||
# workflow
|
||||
|
||||
Here is the full analysis across all five files.
|
||||
|
||||
---
|
||||
|
||||
## 1. Fields AgentStage reads from TaskContext to build the Request
|
||||
|
||||
The `AgentStage.Stage()` method (lines 47-148 of `internal/workflow/agent.go`) constructs a `agentexec.Request` and reads the following from `TaskContext`:
|
||||
|
||||
| Request field (line) | TaskContext source | Notes |
|
||||
|---|---|---|
|
||||
| `Version` (94) | Constant `agentexec.ProtocolVersion` | Not from TaskContext |
|
||||
| `TaskID` (96) | `tc.Task.ID` | |
|
||||
| `Attempt` (97) | `tc.Task.Attempt` | |
|
||||
| `Stage` (98) | `a.Name` (the AgentStage field) | Not from TaskContext |
|
||||
| `Model` (99) | `tc.Cfg.Models[a.Role]` | Falls back to `tc.Cfg.Models["builder"]` at line 54 if the role key is empty; returns an error at line 57 if still empty |
|
||||
| `Mission` (100) | `a.Mission(tc)` | The Mission func receives the full `*TaskContext` — implement's plan/build missions read `tc.Task.Title`, `tc.Task.Body`, `tc.Task.Plan`, `tc.Task.IssueNumber`, `tc.Repo.FullName()` |
|
||||
| `ExtraRules` (101) | `a.ExtraRules` | Static field on AgentStage, not read from TaskContext per se |
|
||||
| `ReadOnly` (102) | `a.ReadOnly` | Static field |
|
||||
| `Budget` (103) | `tc.Cfg.Budgets.MaxSteps`, `.MaxTokens`, `.WallClock.Std()` | Overridden by `a.MaxSteps` if > 0 (line 65-67) |
|
||||
| `Gate` (104) | `a.Gate(tc)` | The Gate function receives TaskContext; implement's build stage calls `GateFromRepo(tc.Repo, tc.Cfg.Budgets)` (agent.go line 152-161) |
|
||||
| `Preflight` (105) | `tc.Repo.ResolvedPreflight()` | Split into `agentexec.Command` structs (lines 86-92) |
|
||||
| `Protection` (106) | `tc.Repo.Protect` + `a.ProtectGlobs(tc)` | Combined; set to zero Protection if `a.ReadOnly` is true (line 82-84) |
|
||||
| `Notes` (107) | `tc.Task.Notes` | |
|
||||
| `CaptureTools` (108) | `a.CaptureTools(tc)` | The CaptureTools function receives TaskContext |
|
||||
|
||||
At line 110, `tc.Agent.Run(ctx, tc.Dir, req)` is called, which also reads `tc.Dir` (the worktree directory path) and `tc.Agent` (the Runner interface).
|
||||
|
||||
The Mission function for the **plan** stage (implement.go lines 48-64) reads `tc.Task.Plan`, `tc.Task.IssueNumber`, `tc.Task.Title`, `tc.Task.Body`, `tc.Repo.FullName()`.
|
||||
|
||||
The Mission function for the **build** stage (implement.go lines 83-93) reads `tc.Task.IssueNumber`, `tc.Task.Title`, `tc.Task.Body`, `tc.Task.Plan`, `tc.Repo.FullName()`.
|
||||
|
||||
---
|
||||
|
||||
## 2. What happens after AgentStage.Run returns — Result processing
|
||||
|
||||
The full post-Run sequence is in `agent.go` lines 110-146:
|
||||
|
||||
1. **Line 111-113: Early-return on hard transport error.** If `err != nil && res.Version == 0`, the error is wrapped and returned immediately — the result is unusable (version 0 signals no valid response).
|
||||
|
||||
2. **Line 114-116: Validate.** `res.ValidateFor(req)` checks that the result is consistent with the request (e.g., tool call parameters match what was requested). Returns validation error.
|
||||
|
||||
3. **Lines 117-124: Persist appended notes.** Any entries in `res.AppendedNotes` are appended to `tc.Task.Notes` (prefixed with "- ") and saved via `tc.Store.Update(ctx, tc.Task)`. If the store write fails, the error is returned.
|
||||
|
||||
4. **Lines 125-127: Return on agent-level error.** If `err != nil` (and we've already passed the version-zero check), wraps and returns it as `"agent run: ..."`. This catches cases where the agent loop ran but produced an error.
|
||||
|
||||
5. **Lines 129-130: Accumulate usage.** `tc.Task.TokensUsed += res.TokensUsed; tc.Task.Iterations += res.Iterations`.
|
||||
|
||||
6. **Lines 131-134: Emit event.** `tc.Emit("agent_finish", a.Name, res.Summary, ...)` with status, stop_reason, tokens, iterations, model.
|
||||
|
||||
7. **Lines 136-142: Check status.** If `res.Status != agentexec.StatusPassed`, it returns an error of the form `"agent {status} ({stop_reason}): {detail or summary clipped to 2000 chars}"`. This handles "blocked", "idle", "parked", etc. — any non-passing status causes the stage to fail.
|
||||
|
||||
8. **Lines 143-145: OnResult callback.** If the stage's `OnResult` func is non-nil, it is called with `(tc, res)`. The implement workflow's plan stage OnResult (lines 65-72 of implement.go) sets `tc.Task.Plan = res.Summary` and posts it as a forge comment. The build stage OnResult (lines 94-97 of implement.go) sets `tc.BuildSummary = res.Summary`.
|
||||
|
||||
9. **Line 146: Return nil** if no OnResult or OnResult succeeded.
|
||||
|
||||
---
|
||||
|
||||
## 3. Full list of stage names in the implement workflow; which call AgentStage
|
||||
|
||||
From `implement.go` lines 38-109, the stages in order:
|
||||
|
||||
| Order | Stage name | Calls AgentStage? | Implementation |
|
||||
|---|---|---|---|
|
||||
| 1 | `"prepare"` | No | `StagePrepareWorktree()` — clones repo, checks out branch (steps.go:21-31) |
|
||||
| 2 | `"repo-stages"` | No | `StageRepoStages()` — runs Yaegi custom stages from `.archie/stages/*.go` (steps.go:84-104) |
|
||||
| 3 | `"baseline"` | No | `StageBaselineGate()` — verifies the gate commands pass at base commit before any agent work (implement.go:16-30) |
|
||||
| 4 | `"plan"` | **Yes** | `AgentStage{...}` — read-only planner, max 15 steps, posts the plan as a comment on the issue (implement.go:43-73) |
|
||||
| 5 | `"build"` | **Yes** | `AgentStage{...}` — builder with gate, no ReadOnly, mission includes the plan, OnResult captures `tc.BuildSummary` (implement.go:75-98) |
|
||||
| 6 | `"commit-push"` | No | `StageCommitPush(...)` — commits all worktree changes and pushes the branch (steps.go:51-58) |
|
||||
| 7 | `"custom-gate"` | No | `StageYaegiGate()` — evaluates `.archie/gate.go` for project-specific rules (steps.go:111-151) |
|
||||
| 8 | `"diff-cap"` | No | `StageDiffCap()` — parks if the diff exceeds configured line cap (steps.go:63-76) |
|
||||
| 9 | `"open-pr"` | No | `StageOpenPR(...)` — opens the pull request with BuildSummary as body (steps.go:170-174) |
|
||||
|
||||
Two stages call AgentStage: **"plan"** (read-only, no gate, posts plan comment) and **"build"** (read-write, has gate, captures summary).
|
||||
|
||||
---
|
||||
|
||||
## 4. Every TaskContext.Forge call in steps.go and implement.go
|
||||
|
||||
All invocations of `tc.Forge.*` or `tc.TaskContext.Forge.*`:
|
||||
|
||||
**In `implement.go`:**
|
||||
|
||||
- **Line 68** — `tc.Forge.Comment(context.Background(), tc.Task.Owner, tc.Task.Repo, tc.Task.IssueNumber, body)` — Post the plan as an issue comment (inside the plan stage's `OnResult`). Uses `context.Background()` rather than the stage's context.
|
||||
|
||||
**In `steps.go`:**
|
||||
|
||||
- **Line 160** — `tc.Forge.CreatePR(ctx, t.Owner, t.Repo, title, tc.Branch, tc.Repo.BaseBranch(), body)` — Create the pull request (inside `OpenPR()` helper, called by `StageOpenPR`).
|
||||
|
||||
**In `workflow.go` (not steps.go or implement.go, but part of the workflow engine that uses `tc.Forge`):**
|
||||
|
||||
- **Line 195** — `tc.Forge.SetStateLabel(...)` — Set the state label on the issue when a workflow finishes with an outcome (inside `finish()`).
|
||||
- **Line 221** — `tc.Forge.SetStateLabel(...)` — Set the "parked" label on the issue (inside `park()`).
|
||||
- **Line 225** — `tc.Forge.Comment(ctx, t.Owner, t.Repo, t.IssueNumber, body)` — Post the park reason as an issue comment (inside `park()`).
|
||||
|
||||
Total distinct forge operations during workflow execution: **Comment** (plan posting, park notification), **CreatePR** (opening PRs), **SetStateLabel** (three call sites: finish outcome, park, and park again for the label).
|
||||
|
||||
Note: Forge is also used in `daemon.go` outside the workflow engine — `AcceptInvitations`, `VerifyPush`, `React`, `IssuesWithLabel`, `AssignedIssues`, `PRState`, `RepliesAfter`, `CloseIssue`, `SetStateLabel` (for queue/working labels). But in the workflow stages themselves, only the three operations above.
|
||||
|
||||
---
|
||||
|
||||
## 5. If agent execution moves to NATS: what stays in archied vs. moves to archie-agent
|
||||
|
||||
The central boundary is `tc.Agent.Run(ctx, tc.Dir, req)` at `agent.go:110`. This is the single point where workflow execution crosses from deterministic orchestration into LLM-driven agent execution.
|
||||
|
||||
### Stays in archied (all non-agent orchestration)
|
||||
|
||||
- **Workflow engine** (`workflow.go`): the `Run()` loop that iterates stages, persists task state after each stage, handles outcomes/parking/finish.
|
||||
- **All deterministic stages** (steps.go, implement.go):
|
||||
- `StagePrepareWorktree` — git clone and branch checkout via `tc.Trees`
|
||||
- `StageRepoStages` — Yaegi custom stage discovery and execution
|
||||
- `StageBaselineGate` — runs shell gate commands at base commit
|
||||
- `StageCommitPush` — git commit and push via `tc.Trees`
|
||||
- `StageYaegiGate` — Yaegi gate evaluation
|
||||
- `StageDiffCap` — diff line counting
|
||||
- `StageOpenPR` — forge PR creation
|
||||
- **AgentStage Result processing** (`agent.go` lines 110-146) — validation, note appending, token/iteration accumulation, event emission, OnResult callbacks (plan posting to forge, BuildSummary capture).
|
||||
- **TaskContext construction** (`daemon.go:481-492`) — the daemon builds TaskContext from store, config, trees, forge, etc.
|
||||
- **All forge interactions** (`tc.Forge.*`) — commenting on issues, setting labels, creating PRs.
|
||||
- **Store operations** (`tc.Store.*`) — task persistence, transitions, claims.
|
||||
- **Task queueing and polling** (`daemon.go`): `poll`, `pollNATS`, `pollSQLite`, `checkWaiting`, `reconcilePRs`, `drainSQLite`, `drainNATS`.
|
||||
|
||||
### Moves to archie-agent (the LLM execution layer)
|
||||
|
||||
- **The actual `tc.Agent.Run(ctx, tc.Dir, req)` call** — the entire agent loop: LLM chat completion, tool execution (read/write files in the worktree, run shell commands, execute gates), budget tracking, stop reason detection.
|
||||
- **The worktree directory (`tc.Dir`)** and all file I/O within it. The agent's file read/write/edit tools operate on `tc.Dir`. If archie-agent is a separate process, it needs access to the same worktree (either via shared filesystem, or by serializing file operations back to archied, or by archied exposing a filesystem server).
|
||||
- **Preflight command execution** (lines 86-92 of agent.go) — `tc.Repo.ResolvedPreflight()` commands are baked into the `Request.Preflight` field and executed by the Runner; they would run in archie-agent.
|
||||
- **Gate execution** (the gate commands in `Request.Gate`) — run by the Runner during the agent loop.
|
||||
|
||||
### What the NATS bridge layer must carry
|
||||
|
||||
The NATS message would need to convey:
|
||||
|
||||
**From archied to archie-agent (the Request):**
|
||||
- TaskID, Attempt, Stage name, Model reference
|
||||
- Mission string (already a string — the produced mission text)
|
||||
- ExtraRules, ReadOnly flag
|
||||
- Budget (MaxSteps, MaxTokens, WallClock)
|
||||
- Gate (command list, max failures)
|
||||
- Preflight (command list)
|
||||
- Protection (suffixes + globs)
|
||||
- Notes
|
||||
- CaptureTools definitions
|
||||
- **Worktree access** — either the path (if shared filesystem) or the repo coordinates + branch + commit SHA for the agent to clone locally
|
||||
|
||||
**From archie-agent to archied (the Result):**
|
||||
- Version, Status, StopReason
|
||||
- Summary, Detail
|
||||
- TokensUsed, Iterations
|
||||
- AppendedNotes (string list)
|
||||
- Full tool call records (for CaptureTools)
|
||||
- File modifications list (if archied needs to track what changed)
|
||||
|
||||
### Architectural observation about worktree access
|
||||
|
||||
The critical coupling is `tc.Dir` being passed directly to `tc.Agent.Run` (line 110). Currently the agent loop reads and writes files inside that directory. If archie-agent runs as a separate process:
|
||||
- **Option A (shared filesystem)**: Both archied and archie-agent mount the same worktree directory. ARCHied passes the path, archie-agent writes directly. Simple but requires shared storage.
|
||||
- **Option B (archied as filesystem proxy)**: Archie-agent sends file read/write/stat requests back to archied over NATS (or a separate channel). Looser coupling but higher latency and complexity.
|
||||
- **Option C (archied streams the Request, agent clones independently)**: Archie-agent receives the repo/branch/commit via NATS and clones its own worktree copy. This avoids file-routing latency but requires git credentials in the agent process.
|
||||
|
||||
### Summary diagram
|
||||
|
||||
```
|
||||
archied (stays) NATS bridge archie-agent (moves)
|
||||
─────────────────────── ────────── ──────────────────────
|
||||
Poll → Enqueue → Claim
|
||||
Build TaskContext
|
||||
↓
|
||||
workflow.Run()
|
||||
prepare (git clone)
|
||||
repo-stages
|
||||
baseline (gate verify)
|
||||
plan (AgentStage) ───── Request ────→ ──────→ agent loop (LLM + file I/O)
|
||||
←──── Result ──── ←── gate/preflight execution
|
||||
OnResult: post plan comment
|
||||
build (AgentStage) ───── Request ────→ ──────→ agent loop (LLM + file I/O)
|
||||
←──── Result ──── ←── gate/preflight execution
|
||||
OnResult: capture BuildSummary
|
||||
commit-push (git)
|
||||
custom-gate (yaegi)
|
||||
diff-cap
|
||||
open-pr (forge CreatePR)
|
||||
finish/outcome
|
||||
```
|
||||
|
||||
The workflow engine, stage orchestration, all forge/store interactions, and AgentStage result processing stay in archied. Only the `tc.Agent.Run()` call — the LLM agent loop with its file and command execution — moves to archie-agent.
|
||||
@@ -1,345 +0,0 @@
|
||||
# missions
|
||||
|
||||
Here is the complete report.
|
||||
|
||||
---
|
||||
|
||||
## 1. The `AgentStage` Struct (from `/work/apps/archie-core/internal/workflow/agent.go`)
|
||||
|
||||
**Lines 16-44.** The struct has 10 fields:
|
||||
|
||||
| Field | Type | Line | Purpose |
|
||||
|---|---|---|---|
|
||||
| `Name` | `string` | 17 | Stage name (e.g. `"plan"`, `"build"`) |
|
||||
| `Role` | `string` | 20 | Model role key, looked up in `cfg.Models[Role]` — falls back to `cfg.Models["builder"]` |
|
||||
| `ReadOnly` | `bool` | 21 | When true, disables write protection entirely |
|
||||
| `Mission` | `func(*TaskContext) string` | 23 | Produces the task statement string sent to the agent |
|
||||
| `Gate` | `func(*TaskContext) agentexec.Gate` | 26 | Returns the quality gate; nil = ungated |
|
||||
| `ExtraRules` | `string` | 28 | Literal string appended to the agent's system prompt |
|
||||
| `MaxSteps` | `int` | 31 | Overrides budget when > 0 |
|
||||
| `ProtectGlobs` | `func(*TaskContext) []string` | 37 | Globs blocked from writes, combined with repo-configured suffix protection |
|
||||
| `CaptureTools` | `func(*TaskContext) []agentexec.CaptureTool` | 40 | Structured-output tools whose calls return as data |
|
||||
| `OnResult` | `func(*TaskContext, agentexec.Result) error` | 43 | Consumes a passed result |
|
||||
|
||||
---
|
||||
|
||||
## 2. How `AgentStage.Stage()` Calls `Mission` and Builds the `Request` (from `/work/apps/archie-core/internal/workflow/agent.go`)
|
||||
|
||||
**Lines 47-148.** The `Stage()` method is the adapter that converts an `AgentStage` into the engine's `Stage` type.
|
||||
|
||||
### The Request Construction (lines 94-109)
|
||||
|
||||
```go
|
||||
req := agentexec.Request{
|
||||
Version: agentexec.ProtocolVersion, // 1
|
||||
TaskID: tc.Task.ID,
|
||||
Attempt: tc.Task.Attempt,
|
||||
Stage: a.Name,
|
||||
Model: modelRef,
|
||||
Mission: a.Mission(tc), // <-- Mission called HERE
|
||||
ExtraRules: a.ExtraRules, // <-- ExtraRules passed directly
|
||||
ReadOnly: a.ReadOnly,
|
||||
Budget: budget,
|
||||
Gate: gate,
|
||||
Preflight: preflight,
|
||||
Protection: protection,
|
||||
Notes: tc.Task.Notes,
|
||||
CaptureTools: captureTools,
|
||||
}
|
||||
```
|
||||
|
||||
Key observations:
|
||||
|
||||
- **`a.Mission(tc)` is called at line 100** to produce the `Mission` string. It receives the full `*TaskContext`, giving it access to `tc.Task.Title`, `tc.Task.Body`, `tc.Task.Plan`, `tc.Repo.FullName()`, `tc.Task.IssueNumber`, and any other field on `TaskContext`.
|
||||
- **`a.ExtraRules` is used directly as a static string** (line 101). It is never processed, templated, or constructed — it is literally the `string` field set at AgentStage construction time.
|
||||
- The result string from `a.Mission(tc)` is placed verbatim into `req.Mission`. It is not combined with ExtraRules there. The agent runtime receives both as separate fields and presumably concatenates or places them into the system prompt.
|
||||
|
||||
### The agent run (line 110)
|
||||
|
||||
```go
|
||||
res, err := tc.Agent.Run(ctx, tc.Dir, req)
|
||||
```
|
||||
|
||||
The `Request` is passed to the agent runner. The response is validated (lines 114-116), persisted notes are saved (lines 117-124), and if the result status is not `"passed"`, it returns an error (lines 136-142). Otherwise `a.OnResult` is called if set (lines 143-145).
|
||||
|
||||
---
|
||||
|
||||
## 3. Every `AgentStage` in Implement, TDD, and Feasibility
|
||||
|
||||
### 3a. Implement (`/work/apps/archie-core/internal/workflow/implement.go`)
|
||||
|
||||
**Stage: `"plan"` (lines 43-73)**
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| `Name` | `"plan"` |
|
||||
| `Role` | `"planner"` |
|
||||
| `ReadOnly` | `true` |
|
||||
| `MaxSteps` | `15` |
|
||||
| `Mission` | Lines 48-64. Produces a `fmt.Sprintf` string: |
|
||||
| | `"Produce a concrete implementation plan for this GitHub issue on the repository %s.\n"+prd+"\n<issue number=%d>\n# %s\n\n%s\n</issue>\n\nExplore the codebase with your read-only tools, then call finish with status \"passed\" and the plan as the summary: files to touch, the approach, acceptance criteria, and what tests should prove it. Keep the plan tightly scoped to the issue — call finish with status \"blocked\" if the issue is too vague or too large for one PR."` |
|
||||
| | If `tc.Task.Plan != ""` (from a feasibility PRD), it prefixes: `"\n<approved_prd>\n" + tc.Task.Plan + "\n</approved_prd>\n"` |
|
||||
| `Gate` | `nil` (no gate — read-only) |
|
||||
| `ExtraRules` | `""` (empty) |
|
||||
| `ProtectGlobs` | `nil` |
|
||||
| `CaptureTools` | `nil` |
|
||||
| `OnResult` | Line 65-72: Sets `tc.Task.Plan = res.Summary`, posts a comment to the issue |
|
||||
| *Full mission example when no PRD exists:* `"Produce a concrete implementation plan for this GitHub issue on the repository owner/repo.\n\n<issue number=42>\n# Issue Title\n\nIssue body text\n</issue>\n\nExplore the codebase..."` |
|
||||
|
||||
**Stage: `"build"` (lines 75-98)**
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| `Name` | `"build"` |
|
||||
| `Role` | `"builder"` |
|
||||
| `ReadOnly` | `false` |
|
||||
| `MaxSteps` | `0` (uses default budget) |
|
||||
| `Mission` | Lines 83-93. Template: |
|
||||
| | `"Implement this GitHub issue on the repository %s, following the plan below.\n\n<issue number=%d>\n# %s\n\n%s\n</issue>\n\n<plan>\n%s\n</plan>\n\nMake the smallest change that satisfies the issue and the plan's acceptance criteria. Do not run git — the orchestrator commits and pushes for you. When done, call finish with status \"passed\" and a summary written for the human who will review the pull request: what changed, why, and how it was verified."` |
|
||||
| `Gate` | Line 78-79: `GateFromRepo(tc.Repo, tc.Cfg.Budgets)` — full quality gate |
|
||||
| `ExtraRules` | Lines 81-82: `"Files matching the repository's protected suffixes (e.g. generated code) are write-blocked — edit their sources instead; the gate regenerates them."` |
|
||||
| `ProtectGlobs` | `nil` |
|
||||
| `CaptureTools` | `nil` |
|
||||
| `OnResult` | Line 94-97: Sets `tc.BuildSummary = res.Summary` |
|
||||
|
||||
### 3b. TDD (`/work/apps/archie-core/internal/workflow/tdd.go`)
|
||||
|
||||
**Stage: `"analyse"` (lines 26-46)**
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| `Name` | `"analyse"` |
|
||||
| `Role` | `"planner"` |
|
||||
| `ReadOnly` | `true` |
|
||||
| `MaxSteps` | `15` |
|
||||
| `Mission` | Lines 31-41. Template: |
|
||||
| | `"Analyse this bug report for the repository %s and determine the problem surface.\n\n<issue number=%d>\n# %s\n\n%s\n</issue>\n\nExplore the code read-only and call finish with status \"passed\" and a summary containing: the root cause (file and function), the exact conditions that trigger it, the expected vs actual behaviour, and which test cases would prove the bug. Call finish with status \"blocked\" if you cannot locate a plausible cause."` |
|
||||
| `Gate` | `nil` |
|
||||
| `ExtraRules` | `""` |
|
||||
| `ProtectGlobs` | `nil` |
|
||||
| `CaptureTools` | `nil` |
|
||||
| `OnResult` | Lines 42-45: Sets `tc.Task.Plan = res.Summary` |
|
||||
|
||||
**Stage: `"repro-tests"` (lines 48-69)**
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| `Name` | `"repro-tests"` |
|
||||
| `Role` | `"builder"` |
|
||||
| `ReadOnly` | `false` |
|
||||
| `MaxSteps` | `0` |
|
||||
| `Mission` | Lines 57-68. Template: |
|
||||
| | `"Write tests that REPRODUCE this bug in the repository %s. Do NOT fix the bug yet.\n\n<issue number=%d>\n# %s\n\n%s\n</issue>\n\n<analysis>\n%s\n</analysis>\n\nAdd tests that pass once the bug is fixed but FAIL today because of it. The gate is inverted for this stage: it requires the full gate to pass EXCEPT %s which is required to FAIL. Do not touch non-test files. Call finish with status \"passed\" once the failing repro is in place, summarising which tests capture the bug."` |
|
||||
| | Note: `%s` at the end is `testCommand(tc.Repo)` — the last gate command (e.g. `go test ./...`). |
|
||||
| `Gate` | Lines 54-56: `tddReproGate(tc.Repo, tc.Cfg.Budgets)` — inverted gate where the last command expects failure |
|
||||
| `ExtraRules` | `""` |
|
||||
| `ProtectGlobs` | `nil` |
|
||||
| `CaptureTools` | `nil` |
|
||||
| `OnResult` | `nil` |
|
||||
|
||||
**Stage: `"fix"` (lines 92-123)**
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| `Name` | `"fix"` |
|
||||
| `Role` | `"builder"` |
|
||||
| `ReadOnly` | `false` |
|
||||
| `MaxSteps` | `0` |
|
||||
| `Mission` | Lines 109-118. Template: |
|
||||
| | `"Fix the bug in the repository %s. Failing repro tests are already committed; make them pass.\n\n<issue number=%d>\n# %s\n\n%s\n</issue>\n\n<analysis>\n%s\n</analysis>\n\nThe full quality gate (including 'go test ./...') must pass. Make the smallest fix that makes the repro tests pass without changing them. Call finish with status \"passed\" and a summary for the PR reviewer: root cause, the fix, and verification."` |
|
||||
| `Gate` | Lines 95-97: `GateFromRepo(tc.Repo, tc.Cfg.Budgets)` — normal gate |
|
||||
| `ExtraRules` | Lines 107-108: `"The repro tests written in the previous stage are the bug's specification. Test files are write-protected in this stage — make them pass by fixing the code they exercise."` |
|
||||
| `ProtectGlobs` | Lines 100-105: `tc.Repo.ResolvedTestGlob()` — protects test files |
|
||||
| `CaptureTools` | `nil` |
|
||||
| `OnResult` | Lines 119-122: Sets `tc.BuildSummary = res.Summary` |
|
||||
|
||||
### 3c. Feasibility (`/work/apps/archie-core/internal/workflow/feasibility.go`)
|
||||
|
||||
**Stage: `"assess"` (lines 28-73)**
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| `Name` | `"assess"` |
|
||||
| `Role` | `"planner"` |
|
||||
| `ReadOnly` | `true` |
|
||||
| `MaxSteps` | `15` |
|
||||
| `Mission` | Lines 34-44. Template: |
|
||||
| | `"Assess whether this feature request fits the project %s.\n\n<issue number=%d>\n# %s\n\n%s\n</issue>\n\nRead the repository's AGENT.md, ROADMAP.md, and README (whichever exist) plus enough code to judge scope and architectural fit. Then call the decide tool EXACTLY ONCE with fit=true or fit=false and your reasons, and afterwards call finish with status \"passed\" summarising the assessment."` |
|
||||
| `Gate` | `nil` |
|
||||
| `ExtraRules` | `""` |
|
||||
| `ProtectGlobs` | `nil` |
|
||||
| `CaptureTools` | Lines 33, 123-136: `decideCaptureTools` — provides a `decide` structured-output tool with boolean `fit` and string `reasons` |
|
||||
| `OnResult` | Lines 45-72: Decodes the `decide` capture, sets `tc.decision`. If not a fit, closes the issue with `Outcome{Status: store.StatusClosedWontDo}` |
|
||||
|
||||
**Stage: `"prd"` (lines 75-95)**
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| `Name` | `"prd"` |
|
||||
| `Role` | `"planner"` |
|
||||
| `ReadOnly` | `true` |
|
||||
| `MaxSteps` | `20` |
|
||||
| `Mission` | Lines 80-90. Template: |
|
||||
| | `"Write a PRD for this accepted feature request on %s.\n\n<issue number=%d>\n# %s\n\n%s\n</issue>\n\n<assessment>\n%s\n</assessment>\n\nExplore the affected code surface, then call finish with status \"passed\" and the PRD as the summary: problem, proposed solution, files/components affected, acceptance criteria, explicit non-goals, and estimated diff size. It will be read by a human deciding whether to green-light implementation."` |
|
||||
| | The last `%s` is `tc.decision.Reasons` |
|
||||
| `Gate` | `nil` |
|
||||
| `ExtraRules` | `""` |
|
||||
| `ProtectGlobs` | `nil` |
|
||||
| `CaptureTools` | `nil` |
|
||||
| `OnResult` | Lines 91-94: Sets `tc.Task.Plan = res.Summary` |
|
||||
|
||||
---
|
||||
|
||||
## 4. The `Workflow` Struct, `Stage` Struct, `Registry` Type, and `Route()`
|
||||
|
||||
### `Stage` (from `/work/apps/archie-core/internal/workflow/workflow.go`, lines 86-89)
|
||||
|
||||
```go
|
||||
type Stage struct {
|
||||
Name string
|
||||
Run func(ctx context.Context, tc *TaskContext) error
|
||||
}
|
||||
```
|
||||
|
||||
### `Workflow` (lines 92-95)
|
||||
|
||||
```go
|
||||
type Workflow struct {
|
||||
Name string
|
||||
Stages []Stage
|
||||
}
|
||||
```
|
||||
|
||||
### `Registry` (line 98)
|
||||
|
||||
```go
|
||||
type Registry map[string]Workflow
|
||||
```
|
||||
|
||||
### `Route()` (lines 103-141)
|
||||
|
||||
`Route(t *store.Task, reg Registry) Workflow`:
|
||||
|
||||
1. If `t.Workflow` is pre-assigned (non-empty) and exists in the registry, return it immediately (line 104-108).
|
||||
2. Otherwise, iterate over `t.Labels` (comma-separated):
|
||||
- `"bug"` -> workflow `"tdd"`
|
||||
- `"feature"` -> workflow `"feasibility"`
|
||||
- `"bootstrap"` -> workflow `"bootstrap"`
|
||||
3. No label match -> fall back to `"implement"`, then `"default"`.
|
||||
4. If nothing found, returns a no-op "fail" workflow (lines 135-141).
|
||||
|
||||
---
|
||||
|
||||
## 5. What Would Need to Change to Replace Hardcoded Mission Strings with SKILL.md Content
|
||||
|
||||
This is a read-only analysis of the changes required. Here is the full scope:
|
||||
|
||||
### 5a. Where the strings live today
|
||||
|
||||
Every Mission is an inline `func(*TaskContext) string` closure that calls `fmt.Sprintf` with a hardcoded format string. The seven unique templates are:
|
||||
|
||||
| Workflow | Stage | File | Lines |
|
||||
|---|---|---|---|
|
||||
| implement | plan | `implement.go` | 48-64 |
|
||||
| implement | build | `implement.go` | 83-93 |
|
||||
| tdd | analyse | `tdd.go` | 31-41 |
|
||||
| tdd | repro-tests | `tdd.go` | 57-68 |
|
||||
| tdd | fix | `tdd.go` | 109-118 |
|
||||
| feasibility | assess | `feasibility.go` | 34-44 |
|
||||
| feasibility | prd | `feasibility.go` | 80-90 |
|
||||
|
||||
### 5b. The `ExtraRules` field is also hardcoded
|
||||
|
||||
| Workflow | Stage | String | File:Line |
|
||||
|---|---|---|---|
|
||||
| implement | build | `"Files matching the repository's protected suffixes..."` | `implement.go:81-82` |
|
||||
| tdd | fix | `"The repro tests written in the previous stage are the bug's specification..."` | `tdd.go:107-108` |
|
||||
|
||||
### 5c. All the pieces that must change
|
||||
|
||||
**Layer 1: `AgentStage` struct definition** (`agent.go`, lines 16-44)
|
||||
|
||||
- The `Mission` field is currently `func(*TaskContext) string`. To source missions from a file, you have three options:
|
||||
- **A)** Keep the same signature but have the function read a file at runtime (not great — makes the function impure and harder to test).
|
||||
- **B)** Add a new field like `MissionSource string` (path to SKILL.md) and modify `Stage()` to load content from that path when `Mission` is nil. This cleanly separates inline missions from file-sourced ones.
|
||||
- **C)** Replace `Mission` entirely with a `func(*TaskContext) string` that loads and templates from SKILL.md. But you still need per-stage sections within the file.
|
||||
|
||||
**Layer 2: `Stage()` method** (`agent.go`, lines 94-109)
|
||||
|
||||
- Currently line 100: `Mission: a.Mission(tc)` — the mission is called synchronously.
|
||||
- If files are loaded at construction time or stage-run time, this call site needs to either:
|
||||
- Use a pre-loaded template string (loaded during `Implement()`, `TDD()`, `Feasibility()` construction), or
|
||||
- Load the file at runtime inside the closure (less efficient but avoids an init-time dependency on the filesystem).
|
||||
|
||||
**Layer 3: `ExtraRules` sourcing**
|
||||
|
||||
- Currently a plain `string` field. If it should also come from SKILL.md, you need the same treatment: either load it at construction time or add a load-on-use path.
|
||||
|
||||
**Layer 4: How the template variables get injected**
|
||||
|
||||
The current missions use Go `fmt.Sprintf` with arguments like:
|
||||
- `tc.Repo.FullName()` (passed as `%s`)
|
||||
- `tc.Task.IssueNumber` (passed as `%d`)
|
||||
- `tc.Task.Title` (passed as `%s`)
|
||||
- `tc.Task.Body` (passed as `%s`)
|
||||
- `tc.Task.Plan` (passed as `%s` — for implement build and tdd fix)
|
||||
- `tc.decision.Reasons` (passed as `%s` — for feasibility prd)
|
||||
- `testCommand(tc.Repo)` (passed as `%s` — for tdd repro-tests)
|
||||
|
||||
If you move to a SKILL.md file, you need a template language. Options:
|
||||
- **Go `text/template`** — already in the stdlib, no dependency. You would pass `*TaskContext` as the data context. Templates like `{{.Repo.FullName}}`, `{{.Task.Title}}` replace the `%s` slots.
|
||||
- **Keep `fmt.Sprintf`** with numbered placeholders — but then the SKILL.md author must know argument ordering, which is fragile.
|
||||
|
||||
**Layer 5: The SKILL.md file format**
|
||||
|
||||
You need a way to express multiple missions in one file. Options:
|
||||
- **One SKILL.md per workflow** (e.g., `SKILL_implement.md`, `SKILL_tdd.md`, `SKILL_feasibility.md`) with YAML front-matter for per-stage sections.
|
||||
- **One section per stage** delimited by headings (`## plan`, `## build`, etc.) and a convention for ExtraRules.
|
||||
- **YAML front matter** with a `stages` map: each key is a stage name with `mission` and `extra_rules` fields.
|
||||
|
||||
Example YAML-front-matter approach:
|
||||
```yaml
|
||||
---
|
||||
stages:
|
||||
plan:
|
||||
mission: |
|
||||
Produce a concrete implementation plan for this GitHub issue on the repository {{.Repo.FullName}}.
|
||||
...
|
||||
extra_rules: ""
|
||||
build:
|
||||
mission: |
|
||||
Implement this GitHub issue on the repository {{.Repo.FullName}}...
|
||||
extra_rules: "Files matching the protected suffixes..."
|
||||
---
|
||||
```
|
||||
|
||||
**Layer 6: Where SKILL.md lives**
|
||||
|
||||
You need a lookup strategy. Likely candidates:
|
||||
- Repository root (e.g., `.archie/SKILL.md`) — repo-specific missions
|
||||
- Config-defined path (`cfg.SkillFile` or similar)
|
||||
- Embedded in the binary as a fallback
|
||||
|
||||
**Layer 7: Loading and caching**
|
||||
|
||||
- When does the file get read? Per `Run()` call (every stage loads it), once at workflow construction in `Implement()`/`TDD()`/`Feasibility()`, or on daemon startup?
|
||||
- If per-workflow construction, add a `loadSkillMissions(dir string)` helper called at the top of each workflow factory.
|
||||
- Caching: the file is small, so a simple `sync.Once` per path or an `fsnotify` watch is overkill unless performance is critical.
|
||||
|
||||
**Layer 8: Fallback behavior**
|
||||
|
||||
- If no SKILL.md exists (which is the default for all repos today), the workflow must fall back to the current hardcoded missions. The simplest approach: keep the existing `Mission` closures as the zero-config default, and only apply SKILL.md content when the file is present.
|
||||
|
||||
### 5d. Concrete minimal change list
|
||||
|
||||
1. **`internal/workflow/agent.go`**: Optionally add a `MissionLoader func(*TaskContext) (string, error)` field or modify `Stage()` to detect and load from a path. Keep backward compat with inline `Mission`.
|
||||
|
||||
2. **`internal/workflow/workflow.go`**: Add a `SkillPath string` (or similar) to `TaskContext` so stage missions can locate the skill file from the worktree directory `tc.Dir`.
|
||||
|
||||
3. **Each workflow factory** (`implement.go`, `tdd.go`, `feasibility.go`): At the top of `Implement()`/`TDD()`/`Feasibility()`, attempt to load `tc.Repo.Dir + "/.archie/SKILL.md"` (or wherever configured). Parse per-stage templates. If the file doesn't exist, keep the existing inline closures as-is.
|
||||
|
||||
4. **Template parsing**: Introduce a package-level helper, e.g. `renderMission(tmpl string, tc *TaskContext) string`, that uses `text/template` against `*TaskContext`.
|
||||
|
||||
5. **`ExtraRules`**: Same treatment — source from the SKILL.md per-stage front matter when present.
|
||||
|
||||
6. **Test the fallback**: Every existing test must continue passing without a SKILL.md file present, proving backward compatibility.
|
||||
@@ -1,230 +0,0 @@
|
||||
# patterns
|
||||
|
||||
Here is the comprehensive report.
|
||||
|
||||
---
|
||||
|
||||
## 1. Existing Skill-Loading and Frontmatter-Parsing Code
|
||||
|
||||
**No existing frontmatter-parsing or SKILL.md-loading code exists in the Go codebase.** My searches confirm:
|
||||
|
||||
- `grep -rn 'SKILL\.md' --include='*.go'` -- zero results.
|
||||
- `grep -rn 'frontmatter\|front.matter\|front_matter' --include='*.go'` -- zero results.
|
||||
- `grep -rn 'yaml\.' --include='*.go'` -- zero results. No YAML parsing of any kind exists yet.
|
||||
- `grep -rn 'metadata\.archie\|archie-wf' --include='*.go'` -- zero results.
|
||||
- `grep -rn 'ReadFile.*\.md\|os\.ReadFile.*skill' --include='*.go'` -- zero results.
|
||||
|
||||
**What *does* reference "skill" in Go code are two packages, neither of which loads or parses SKILL.md:**
|
||||
|
||||
- **`internal/skillscript/yaegi.go`** -- An interpreter helper that runs a Go script at a given filesystem path via Yaegi. It has no discovery logic, no directory scanning, no idea what a SKILL.md is. Its `Run(path string)` function just interprets a single `.go` file.
|
||||
- **`internal/agentexec/inprocess.go`** -- Uses `internal/skillscript` to register a `run_go_script` tool that agents can call. It discovers nothing -- it takes a script path from the LLM's tool call arguments and delegates to `skillscript.Run()`.
|
||||
|
||||
**The SKILL.md files that exist on disk** (5 total) are written by hand in the skills directory:
|
||||
|
||||
| File | path |
|
||||
|---|---|
|
||||
| go-quality-gate | `.archie/skills/go-quality-gate/SKILL.md` |
|
||||
| ecosystem-node | `.archie/skills/ecosystem-node/SKILL.md` |
|
||||
| ecosystem-python | `.archie/skills/ecosystem-python/SKILL.md` |
|
||||
| security-audit | `.archie/skills/security-audit/SKILL.md` |
|
||||
| tdd-bugfix | `.archie/skills/tdd-bugfix/SKILL.md` |
|
||||
|
||||
All five have identical frontmatter structure:
|
||||
```yaml
|
||||
---
|
||||
name: <hyphenated-name>
|
||||
description: >
|
||||
<multi-line description of when to use>
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [<tool-list>]
|
||||
engine: any
|
||||
---
|
||||
```
|
||||
|
||||
This frontmatter is currently **dead content** -- no Go code reads or parses it.
|
||||
|
||||
---
|
||||
|
||||
## 2. The Existing Directory-Discovery Pattern (wfeval.Discover)
|
||||
|
||||
File: `/work/apps/archie-core/internal/workflow/wfeval/yaegi.go`
|
||||
|
||||
This is the pattern to follow for any new skill-discovery code. Here is exactly how it works:
|
||||
|
||||
### The constant
|
||||
```go
|
||||
const stagesDir = ".archie/stages"
|
||||
```
|
||||
|
||||
### The function signature
|
||||
```go
|
||||
func Discover(dir string) ([]workflow.Stage, error)
|
||||
```
|
||||
|
||||
### How it scans
|
||||
1. `os.ReadDir(filepath.Join(dir, stagesDir))` -- reads the directory
|
||||
2. If `os.IsNotExist(err)`, returns `(nil, nil)` -- **missing directory is not an error**
|
||||
3. Any other read error is returned as a wrapped error
|
||||
4. Iterates entries, filtering for `!e.IsDir() && filepath.Ext(e.Name()) == ".go"`
|
||||
5. `sort.Strings(names)` -- **alphabetical order** is deterministic ordering
|
||||
6. Calls `loadStage()` for each file, wrapping errors with the relative path
|
||||
7. Returns the slice (or an empty slice if no `.go` files found)
|
||||
|
||||
### How it loads a single stage
|
||||
1. `os.ReadFile(path)` -- reads the entire Go source
|
||||
2. Creates a Yaegi interpreter with `stdlib.Symbols` + `wfextract.Symbols` (the generated symbol table for `internal/workflow`)
|
||||
3. `i.Eval(string(src))` -- interprets the Go source
|
||||
4. `i.Eval("stages.Stage")` -- looks up the exported `Stage` function
|
||||
5. Type-asserts it as `func() workflow.Stage`
|
||||
6. Calls the factory function to get the `workflow.Stage` value
|
||||
|
||||
### Error handling philosophy
|
||||
- Missing directory: silent no-op (nil, nil)
|
||||
- File-system errors: wrapped with the relative path context
|
||||
- Panics in interpreted code: recovered via `defer`/`recover()`, returned as error
|
||||
- Wrong function signature: clear error message
|
||||
- A single bad file blocks the entire Discover call (fail-fast)
|
||||
|
||||
### How it's wired in
|
||||
In `internal/workflow/workflow.go`, the `TaskContext` struct has:
|
||||
```go
|
||||
CustomStages func(dir string) ([]Stage, error)
|
||||
```
|
||||
This is injected by the composition root (`cmd/archied/`) to avoid a direct import cycle between `workflow` and its own generated Yaegi symbol table.
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommended YAML Dependency and Parsing Approach
|
||||
|
||||
### Current state
|
||||
- `go.mod` at `/work/apps/archie-core/go.mod` has **no YAML library**.
|
||||
- The only markup parser is `github.com/BurntSushi/toml v1.6.0` for TOML config files.
|
||||
- `gopkg.in/yaml.v3` is the standard Go YAML library and is **not** present.
|
||||
|
||||
### Recommendation
|
||||
**Add `gopkg.in/yaml.v3`** -- this is the de facto standard Go YAML library, maintained by the Go team, supports the full YAML 1.2 spec, and is what every major Go project uses.
|
||||
|
||||
### Parsing approach for SKILL.md frontmatter
|
||||
The SKILL.md files use standard YAML frontmatter between `---` delimiters. The parsing pattern would be:
|
||||
|
||||
```go
|
||||
import "gopkg.in/yaml.v3"
|
||||
|
||||
type SkillFrontmatter struct {
|
||||
Name string `yaml:"name"`
|
||||
Description string `yaml:"description"`
|
||||
Version string `yaml:"version"`
|
||||
Metadata struct {
|
||||
Archie *struct {
|
||||
Tools []string `yaml:"tools"`
|
||||
Engine string `yaml:"engine"`
|
||||
// future: plugins, budget_usd, etc.
|
||||
} `yaml:"archie"`
|
||||
} `yaml:"metadata"`
|
||||
}
|
||||
|
||||
// ParseFrontmatter extracts YAML frontmatter from a SKILL.md byte slice.
|
||||
// Returns the frontmatter struct and the body content after the closing ---.
|
||||
func ParseFrontmatter(src []byte) (*SkillFrontmatter, string, error) {
|
||||
// 1. Split on "---\n" — first block is YAML, rest is body
|
||||
// 2. yaml.Unmarshal() the YAML block into SkillFrontmatter
|
||||
// 3. Return parsed metadata + body string
|
||||
}
|
||||
```
|
||||
|
||||
The TOML parsing pattern already in the codebase (file `/work/apps/archie-core/internal/config/config.go`) uses struct tags and straightforward `toml.DecodeFile()` -- the YAML equivalent would use `yaml.Unmarshal()` instead.
|
||||
|
||||
---
|
||||
|
||||
## 4. What the agentskills.io Specification Says
|
||||
|
||||
Source: https://agentskills.io/specification (fetched successfully)
|
||||
|
||||
### Naming convention
|
||||
- The spec defines the `name` field: 1-64 chars, lowercase alphanumeric + hyphens, must match the parent directory name.
|
||||
- There is **no "archie-wf-" prefix** in the public agentskills.io spec. That prefix appears to be archie-core's own convention (possibly for workflow-specific skills), not a spec requirement.
|
||||
|
||||
### Metadata block
|
||||
- The `metadata` field is an **optional** key-value mapping (string to string).
|
||||
- The spec does **not** define any `metadata.archie` sub-block. That block (`metadata.archie.tools`, `metadata.archie.engine`) is archie-core's own extension.
|
||||
|
||||
### Full frontmatter fields defined by the spec
|
||||
|
||||
| Field | Required | Description |
|
||||
|---|---|---|
|
||||
| `name` | Yes | 1-64 chars, lowercase+hyphens, matches dir name |
|
||||
| `description` | Yes | 1-1024 chars, what + when to use |
|
||||
| `license` | No | License name or reference |
|
||||
| `compatibility` | No | Environment requirements (500 char max) |
|
||||
| `metadata` | No | Arbitrary string-to-string map |
|
||||
| `allowed-tools` | No | Space-separated pre-approved tools (experimental) |
|
||||
|
||||
### SKILL.md structure
|
||||
```
|
||||
---
|
||||
<YAML frontmatter>
|
||||
---
|
||||
<Markdown body>
|
||||
```
|
||||
|
||||
### Directory structure
|
||||
```
|
||||
skill-name/
|
||||
SKILL.md (required)
|
||||
scripts/ (optional)
|
||||
references/ (optional)
|
||||
assets/ (optional)
|
||||
```
|
||||
|
||||
### Progressive disclosure tiers
|
||||
1. **Catalog** (~100 tokens/skill): `name` and `description` loaded at startup
|
||||
2. **Instructions** (<5000 tokens): Full SKILL.md body when skill is activated
|
||||
3. **Resources**: Scripts/references/assets loaded on demand
|
||||
|
||||
### Scanning locations
|
||||
- Project: `<project>/.<client>/skills/` and `<project>/.agents/skills/`
|
||||
- User: `~/.<client>/skills/` and `~/.agents/skills/`
|
||||
- `.claude/skills/` is mentioned as a pragmatic compatibility convention
|
||||
|
||||
---
|
||||
|
||||
## 5. The Full .archie/ Directory Tree
|
||||
|
||||
```
|
||||
.archie/
|
||||
skills/
|
||||
ecosystem-node/
|
||||
SKILL.md
|
||||
ecosystem-python/
|
||||
SKILL.md
|
||||
go-quality-gate/
|
||||
SKILL.md
|
||||
security-audit/
|
||||
SKILL.md
|
||||
tdd-bugfix/
|
||||
SKILL.md
|
||||
```
|
||||
|
||||
**What does NOT exist yet:**
|
||||
- No `.archie/stages/` directory (stage scripts are in the spec but none exist yet)
|
||||
- No `.archie/config` or `.archie/config.toml` (configuration is in a top-level `config.toml`, not `.archie/`)
|
||||
- No `scripts/` subdirectories inside any skill (all 5 skills have only `SKILL.md`)
|
||||
- No `references/` or `assets/` subdirectories inside any skill
|
||||
|
||||
---
|
||||
|
||||
### Summary of Key File Paths
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `/work/apps/archie-core/internal/workflow/wfeval/yaegi.go` | **The discovery pattern to follow** -- scans `.archie/stages/*.go`, sorts, loads via Yaegi |
|
||||
| `/work/apps/archie-core/internal/workflow/wfeval/yaegi_test.go` | Tests showing the discovery pattern |
|
||||
| `/work/apps/archie-core/internal/workflow/workflow.go` | Workflow engine, Stage struct, CustomStages injection pattern |
|
||||
| `/work/apps/archie-core/internal/skillscript/yaegi.go` | Yaegi runner for skill Go scripts (no discovery, no SKILL.md parsing) |
|
||||
| `/work/apps/archie-core/internal/agentexec/inprocess.go` | Registers `run_go_script` tool using skillscript.Run() (line 181-207) |
|
||||
| `/work/apps/archie-core/internal/config/config.go` | Existing TOML config parsing pattern (reference for adding YAML) |
|
||||
| `/work/apps/archie-core/go.mod` | Dependencies -- no YAML lib yet, BurntSushi/toml v1.6.0 present |
|
||||
| `/work/apps/archie-core/.references/skills-implementation.md` | Research notes on implementing agentskills.io support in archie |
|
||||
| `/work/apps/archie-core/.archie/skills/*/SKILL.md` | 5 hand-written skill files with YAML frontmatter (currently dead content) |
|
||||
@@ -1,342 +0,0 @@
|
||||
# skills
|
||||
|
||||
Here is the complete analysis.
|
||||
|
||||
---
|
||||
|
||||
## Complete Report: SKILL.md Files in archie-core
|
||||
|
||||
### 1. Tally of files
|
||||
|
||||
There are exactly **5 SKILL.md files** in the repository, all under `/work/apps/archie-core/.archie/skills/`. There are no others anywhere in the repo.
|
||||
|
||||
---
|
||||
|
||||
### 2. Supporting directories
|
||||
|
||||
There are **no `plugins/` or `scripts/` subdirectories** inside any skill directory. The `scripts/` feature described in the agentskills.io spec and the Yaoi interpreter (`internal/skillscript/yaegi.go`) exist in the Go code but have no content in the actual skill directories. The `run_go_script` tool in `internal/agentexec/inprocess.go` (lines 181-207) references scripts at `.archie/skills/<skill>/scripts/*.go`, but no such files exist.
|
||||
|
||||
---
|
||||
|
||||
### 3. Per-file analysis
|
||||
|
||||
---
|
||||
|
||||
#### 3.1 tdd-bugfix/SKILL.md
|
||||
|
||||
**File:** `/work/apps/archie-core/.archie/skills/tdd-bugfix/SKILL.md`
|
||||
|
||||
**Complete YAML frontmatter:**
|
||||
|
||||
```yaml
|
||||
name: tdd-bugfix
|
||||
description: >
|
||||
Fix bugs using test-driven development. Write a failing repro test first, prove
|
||||
it captures the bug, then fix the code without touching the tests. archie-core
|
||||
routes issues labelled "bug" through this workflow automatically.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [go, golangci-lint]
|
||||
engine: any
|
||||
```
|
||||
|
||||
Key-value pairs extracted:
|
||||
- `name`: "tdd-bugfix"
|
||||
- `description`: Multiline string (about TDD bugfix)
|
||||
- `version`: "1.0.0"
|
||||
- `metadata.archie.tools`: [go, golangci-lint]
|
||||
- `metadata.archie.engine`: "any"
|
||||
|
||||
**Body content structure:**
|
||||
|
||||
The body has 6 sections:
|
||||
1. `## When this runs` (1 paragraph: describes that archie-core routes "bug" labelled issues through this workflow; mentions the 5 stages)
|
||||
2. `## Stage 1: Analyse (read-only planner)` (prose: what the planner produces, no-code constraint)
|
||||
3. `## Stage 2: Repro Tests (write failing tests only)` (prose with bullet constraints: don't touch non-test files, gate inverted, make failure explicit)
|
||||
4. `## Stage 3: Capture Proof (deterministic)` (prose: archie-core runs the test command directly)
|
||||
5. `## Stage 4: Fix (code only)` (prose with bullet constraints: test files write-protected, full gate runs, smallest fix)
|
||||
6. `## Stage 5: Deliver` (numbered list: commits, push, PR, comment with proof)
|
||||
7. `## Common pitfalls` with 3 sub-pitfalls (each with prose and checklist)
|
||||
|
||||
**Mapping to hardcoded Go code:**
|
||||
|
||||
This skill documents the exact workflow that is hardcoded in `internal/workflow/tdd.go` lines 18-149. The 5 stages correspond directly:
|
||||
- "analyse" -> `AgentStage{Name: "analyse", ...}` (line 27)
|
||||
- "repro-tests" -> `AgentStage{Name: "repro-tests", ...}` (line 48)
|
||||
- "capture-proof" -> `Stage{Name: "capture-proof", Run: ...}` (line 73)
|
||||
- "fix" -> `AgentStage{Name: "fix", ...}` (line 92)
|
||||
- "deliver" -> Stage `open-pr` (line 133) plus the two commit stages (lines 88, 125)
|
||||
|
||||
Every constraint in the SKILL.md body (inverted gate, test file write-protection, smallest fix, two-commit history) is reflected as code in `tdd.go` and `agent.go`.
|
||||
|
||||
---
|
||||
|
||||
#### 3.2 go-quality-gate/SKILL.md
|
||||
|
||||
**File:** `/work/apps/archie-core/.archie/skills/go-quality-gate/SKILL.md`
|
||||
|
||||
**Complete YAML frontmatter:**
|
||||
|
||||
```yaml
|
||||
name: go-quality-gate
|
||||
description: >
|
||||
Run the full Go quality pipeline: gofumpt, go fix, golangci-lint, go test -race,
|
||||
and task deadcode. Use before any commit in a Go project. Triggered automatically
|
||||
by archie-core's implement and tdd workflows via the per-repo gate config.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [go, golangci-lint, gofumpt, task]
|
||||
engine: any
|
||||
```
|
||||
|
||||
Key-value pairs extracted:
|
||||
- `name`: "go-quality-gate"
|
||||
- `description`: Multiline string (about Go quality pipeline)
|
||||
- `version`: "1.0.0"
|
||||
- `metadata.archie.tools`: [go, golangci-lint, gofumpt, task]
|
||||
- `metadata.archie.engine`: "any"
|
||||
|
||||
**Body content structure:**
|
||||
|
||||
5 sections with mixed content:
|
||||
1. `## When this runs` (prose: gate is enforced after every builder stage; commands come from repo config)
|
||||
2. `## Standard gate (Tau and similar Go projects)` (TOML examples showing gate commands, plus convention about last command being test runner)
|
||||
3. `## Diagnosing gate failures` with 4 subsections (each with prose and common fixes):
|
||||
- `### gofumpt / go fix`
|
||||
- `### golangci-lint` (bullet list of common linters and fixes)
|
||||
- `### go test -race` (numbered diagnostic steps)
|
||||
- `### task deadcode` (bash example for scoping)
|
||||
4. `## When the TDD workflow runs` (prose about inverted gate during repro-tests)
|
||||
|
||||
**Mapping to hardcoded Go code:**
|
||||
|
||||
- The gate mechanism is implemented in `internal/workflow/agent.go` lines 152-161 (`GateFromRepo()`) and `internal/workflow/tdd.go` lines 155-174 (`tddReproGate()`).
|
||||
- The `tools` list maps conceptually to `config.Repo.Gate` in `internal/config/config.go` lines 37-43 (a `[][]string` of commands).
|
||||
- The skill documents conventions, not code that loads the SKILL.md.
|
||||
|
||||
---
|
||||
|
||||
#### 3.3 security-audit/SKILL.md
|
||||
|
||||
**File:** `/work/apps/archie-core/.archie/skills/security-audit/SKILL.md`
|
||||
|
||||
**Complete YAML frontmatter:**
|
||||
|
||||
```yaml
|
||||
name: security-audit
|
||||
description: >
|
||||
Scan code for security vulnerabilities: hardcoded secrets, injection vectors,
|
||||
unsafe deserialization, missing auth checks, and path traversal. Use on PRs
|
||||
touching auth, API endpoints, or data handling code.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [golangci-lint, trivy, gitleaks]
|
||||
engine: any
|
||||
```
|
||||
|
||||
Key-value pairs extracted:
|
||||
- `name`: "security-audit"
|
||||
- `description`: Multiline string (about security scanning triggers)
|
||||
- `version`: "1.0.0"
|
||||
- `metadata.archie.tools`: [golangci-lint, trivy, gitleaks]
|
||||
- `metadata.archie.engine`: "any"
|
||||
|
||||
**Body content structure:**
|
||||
|
||||
4 top-level sections:
|
||||
1. `## When to use` (bullet list of trigger conditions: PR touches auth, new API endpoints, db queries, dependency updates)
|
||||
2. `## What to check` with 5 numbered subsections:
|
||||
- `### 1. Hardcoded secrets` (bullet list of scan targets + `gitleaks` example)
|
||||
- `### 2. Injection vectors` (SQL, Command, Path sub-bullets with specific Go tools)
|
||||
- `### 3. Unsafe deserialization` (bullet list of Go-specific patterns)
|
||||
- `### 4. Missing auth checks` (bullet list of endpoint/token issues)
|
||||
- `### 5. Dependency CVEs` (`trivy` command example + report format)
|
||||
3. `## Reporting` (numbered list of 5 report fields: Severity, Location, Description, Fix, Prevention)
|
||||
4. Closing line: "If nothing is found, report clean. Never fabricate findings."
|
||||
|
||||
**Mapping to hardcoded Go code:**
|
||||
|
||||
- **No hardcoded workflow exists** for security-audit in the Go codebase. This skill is purely informational/documentation.
|
||||
- It is not registered as a workflow in `cmd/archied/main.go` lines 163-169.
|
||||
- There is no label routing for security tasks in `internal/workflow/workflow.go` lines 100-141.
|
||||
- It exists as a skill that could be invoked manually or through future orchestration.
|
||||
|
||||
---
|
||||
|
||||
#### 3.4 ecosystem-node/SKILL.md
|
||||
|
||||
**File:** `/work/apps/archie-core/.archie/skills/ecosystem-node/SKILL.md`
|
||||
|
||||
**Complete YAML frontmatter:**
|
||||
|
||||
```yaml
|
||||
name: ecosystem-node
|
||||
description: >
|
||||
Node.js/TypeScript project conventions for archie-core: npm/pnpm, eslint,
|
||||
prettier, tsc, vitest. Use when working on Node repositories with archie-core.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [node, npm, pnpm, vitest, eslint, prettier]
|
||||
engine: any
|
||||
```
|
||||
|
||||
Key-value pairs extracted:
|
||||
- `name`: "ecosystem-node"
|
||||
- `description`: Multiline string (about Node/TS project conventions)
|
||||
- `version`: "1.0.0"
|
||||
- `metadata.archie.tools`: [node, npm, pnpm, vitest, eslint, prettier]
|
||||
- `metadata.archie.engine`: "any"
|
||||
|
||||
**Body content structure:**
|
||||
|
||||
7 sections:
|
||||
1. `## Preflight` (prose + mention of `node --version`)
|
||||
2. `## Recommended gate configuration` (TOML example blocks for pnpm and npm)
|
||||
3. `## Test glob` (prose: default `*.test.ts`, override config examples)
|
||||
4. `## Package management` (prose + bash: lockfile detection, install command)
|
||||
5. `## Common tools` (markdown table: Tool, Purpose, Config file — 5 rows)
|
||||
6. `## Common gate failures` with 3 subsections:
|
||||
- `### eslint` (bullet list)
|
||||
- `### tsc / typecheck` (bullet list)
|
||||
- `### vitest / jest` (bullet list)
|
||||
|
||||
**Mapping to hardcoded Go code:**
|
||||
|
||||
The content maps directly to `internal/config/ecosystem.go` lines 22-25:
|
||||
```go
|
||||
"node": {
|
||||
Preflight: [][]string{{"node", "--version"}},
|
||||
TestGlob: "*.test.ts",
|
||||
},
|
||||
```
|
||||
The preflight check and default test glob are hardcoded. Lockfile detection logic is referenced in prose but not implemented in the Go code shown.
|
||||
|
||||
---
|
||||
|
||||
#### 3.5 ecosystem-python/SKILL.md
|
||||
|
||||
**File:** `/work/apps/archie-core/.archie/skills/ecosystem-python/SKILL.md`
|
||||
|
||||
**Complete YAML frontmatter:**
|
||||
|
||||
```yaml
|
||||
name: ecosystem-python
|
||||
description: >
|
||||
Python project conventions for archie-core: venv, pytest, ruff, mypy, pip-audit.
|
||||
Use when working on Python repositories with archie-core.
|
||||
version: 1.0.0
|
||||
metadata:
|
||||
archie:
|
||||
tools: [python, pytest, ruff, mypy, pip-audit]
|
||||
engine: any
|
||||
```
|
||||
|
||||
Key-value pairs extracted:
|
||||
- `name`: "ecosystem-python"
|
||||
- `description`: Multiline string (about Python project conventions)
|
||||
- `version`: "1.0.0"
|
||||
- `metadata.archie.tools`: [python, pytest, ruff, mypy, pip-audit]
|
||||
- `metadata.archie.engine`: "any"
|
||||
|
||||
**Body content structure:**
|
||||
|
||||
7 sections:
|
||||
1. `## Preflight` (prose + `python --version`)
|
||||
2. `## Recommended gate configuration` (TOML example)
|
||||
3. `## Test glob` (prose: default `test_*.py`, override example)
|
||||
4. `## Package management` (bullet list of assumptions)
|
||||
5. `## Virtual environment` (bash examples for venv setup)
|
||||
6. `## Common gate failures` with 3 subsections:
|
||||
- `### ruff` (bullet list)
|
||||
- `### mypy` (bullet list)
|
||||
- `### pytest` (bullet list)
|
||||
|
||||
**Mapping to hardcoded Go code:**
|
||||
|
||||
Maps directly to `internal/config/ecosystem.go` lines 18-21:
|
||||
```go
|
||||
"python": {
|
||||
Preflight: [][]string{{"python", "--version"}},
|
||||
TestGlob: "test_*.py",
|
||||
},
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. The agentskills.io Specification
|
||||
|
||||
The specification at `https://agentskills.io/specification` defines these fields:
|
||||
|
||||
| Field | Required | Notes |
|
||||
|---|---|---|
|
||||
| `name` | Yes | 1-64 chars, lowercase + hyphens, matches directory name |
|
||||
| `description` | Yes | 1-1024 chars, describes what and when |
|
||||
| `license` | No | Short license name or reference |
|
||||
| `compatibility` | No | Environment requirements (product, packages, network) |
|
||||
| `metadata` | No | Arbitrary key-value map for additional properties |
|
||||
| `allowed-tools` | No | Space-separated string of pre-approved tools (experimental) |
|
||||
|
||||
The spec also defines optional subdirectories: `scripts/`, `references/`, `assets/`.
|
||||
|
||||
**Contrast with current archie-core format:**
|
||||
|
||||
- `version: 1.0.0` exists in all 5 skills but is **not an agentskills.io field** -- it is custom archie-core metadata inside `metadata.archie`. The spec would expect `metadata.version` or simply omitting version from the spec-level fields.
|
||||
- `metadata.archie.tools` is a custom archie extension. The spec has `allowed-tools` (string format, experimental) which serves a similar purpose but with different semantics (pre-approval for tool use vs. listing required tools).
|
||||
- `metadata.archie.engine: any` is custom archie metadata with no spec equivalent.
|
||||
- None of the current skills use `license`, `compatibility`, or `allowed-tools` from the spec.
|
||||
|
||||
---
|
||||
|
||||
### 5. Analysis: Which fields would be useful as machine-readable skill configuration
|
||||
|
||||
Based on how the Go code currently consumes configuration and how the workflow engine works (`internal/workflow/workflow.go`, `internal/workflow/agent.go`, `internal/config/config.go`), the following would benefit from machine parsing:
|
||||
|
||||
**A. `tools` / `allowed-tools` (pre-approval list)**
|
||||
- Currently hardcoded per skill in `metadata.archie.tools` but not parsed by any Go code.
|
||||
- Would enable the agent runtime to pre-authorize external tool calls (golangci-lint, trivy, gitleaks, etc.) without prompting, matching the agentskills.io experimental `allowed-tools` concept.
|
||||
- The TDD workflow's gate mechanism already runs these tools via `config.Repo.Gate` -- a parsed tools list could validate the gate commands against the skill's declared toolset.
|
||||
|
||||
**B. `ecosystem` (language/runtime constraint)**
|
||||
- Currently configured per-repo in TOML as `config.Repo.Ecosystem` (via `internal/config/ecosystem.go`), not per-skill.
|
||||
- The ecosystem-node and ecosystem-python skills document conventions that overlap with the Go hardcodes. A machine-readable ecosystem field on a skill would let the skill declare its language affinity.
|
||||
- No skill currently has this as a top-level field.
|
||||
|
||||
**C. `engine` constraint (e.g., any, archied v1, ...)**
|
||||
- Currently `metadata.archie.engine: any` in every skill, not parsed.
|
||||
- Could gate skill activation to specific engine versions (`archie-core >= 0.5`, `claude-code >= 1.0`).
|
||||
|
||||
**D. `budget` (max steps, max tokens, wall clock)**
|
||||
- Currently hardcoded per workflow in Go: `internal/workflow/agent.go` lines 60-67 reads from `config.Budgets` with per-AgentStage overrides (e.g., MaxSteps=15 for planner, 20 for feasiability PRD stage).
|
||||
- A machine-readable budget field could let the skill declare its expected resource needs, overriding global defaults when appropriate.
|
||||
|
||||
**E. `plugins` (extension scripts)**
|
||||
- Not present in any current skill. The `internal/skillscript/yaegi.go` infrastructure supports running `.go` scripts from a skill's `scripts/` directory, but no skill has one.
|
||||
- Could declare scripts that the agent runtime should load and expose as tools.
|
||||
|
||||
**F. Stage/workflow structure**
|
||||
- The body text of tdd-bugfix describes a 5-stage workflow with named stages, but this is natural language only.
|
||||
- The Go workflows are hardcoded in `internal/workflow/tdd.go`, `implement.go`, `feasibility.go`, and `steps.go`.
|
||||
- A machine-readable stages list in the frontmatter or a companion YAML file would allow the workflow engine to load workflows from skills without recompilation. Currently, new workflows require Go code changes in `cmd/archied/main.go` lines 163-169.
|
||||
|
||||
**G. Gate configuration**
|
||||
- The quality gate is documented in `go-quality-gate/SKILL.md` as TOML examples but configured per-repo in the daemon's TOML config. A skill-declared gate skeleton would let the skill say "I need these commands to verify correctness."
|
||||
|
||||
---
|
||||
|
||||
### 6. Summary of the gap
|
||||
|
||||
The current system has **no Go code that reads SKILL.md files programmatically**. The files exist purely as human-readable documentation. All workflow logic is hardcoded in Go:
|
||||
|
||||
- `internal/workflow/implement.go` -- plan + build stages
|
||||
- `internal/workflow/tdd.go` -- analyse + repro-tests + capture-proof + fix + deliver
|
||||
- `internal/workflow/feasibility.go` -- assess + prd + deliver
|
||||
- `internal/workflow/steps.go` -- shared steps (prepare, commit, push, gate, diff cap, PR)
|
||||
- `internal/workflow/agent.go` -- AgentStage adapter that turns workflow config into `agentexec.Request`
|
||||
|
||||
The only code that touches skill directories is `/work/apps/archie-core/internal/skillscript/yaegi.go` (interprets `scripts/*.go` files) and the `scriptToolSet` in `/work/apps/archie-core/internal/agentexec/inprocess.go` lines 181-207 (exposes the `run_go_script` tool to agents). Both of these are unused since no skill has any `scripts/` content.
|
||||
|
||||
To make skills machine-parseable, the frontmatter would need to adopt the agentskills.io standard fields plus archie-specific extensions (tools list as per spec's `allowed-tools` or a custom `metadata.archie` block, plus possibly a declarative stage/workflow scheme in the frontmatter or a companion YAML file).
|
||||
@@ -1,350 +0,0 @@
|
||||
# daemon-task-loop
|
||||
|
||||
Here is the full report.
|
||||
|
||||
---
|
||||
|
||||
## 1. Full Lifecycle: GitHub Issue to Completed Task
|
||||
|
||||
The lifecycle spans four subsystems: **forge** (GitHub polling), **store** (SQLite persistence), **daemon** (orchestration loop), and **workflow** (pipeline engine). Here is the exact call chain.
|
||||
|
||||
### Phase 1 -- Discovery (polling)
|
||||
|
||||
**Entry:** `Daemon.Run()` at `daemon.go:72`
|
||||
- Creates a `time.NewTicker` from `d.Cfg.PollInterval` (default 60s, line 259 of config.go).
|
||||
- Calls `d.Cycle(ctx)` on every tick.
|
||||
|
||||
**`Daemon.Cycle()`** at `daemon.go:88` does three discovery activities, then drains the queue:
|
||||
|
||||
1. **`d.poll(ctx)`** -- line 89. Iterates over `d.Cfg.Repos`, calling `d.pollIssues(ctx, repo)` (line 140).
|
||||
2. **`d.reconcilePRs(ctx)`** -- line 90. Finds tasks at `StatusPROpen` and checks GitHub for merged/closed status.
|
||||
3. **`d.checkWaiting(ctx)`** -- line 91. Finds tasks at `StatusWaitingHuman` and LLM-judges human replies.
|
||||
|
||||
**`Daemon.pollIssues()`** at `daemon.go:140` routes by `d.Cfg.Dispatch.Trigger`:
|
||||
- `"assignee"`: calls `d.Forge.AssignedIssues(ctx, owner, repo, botUser)`.
|
||||
- `"label"`: calls `d.Forge.IssuesWithLabel(ctx, owner, repo, label)`.
|
||||
- `"either"`: calls `pollEither()` (line 163), which unions both lists deduped by issue number.
|
||||
|
||||
### Phase 2 -- Enqueue
|
||||
|
||||
**`Daemon.poll()`** at `daemon.go:111-133`: for each discovered `forge.Issue`, calls:
|
||||
- `d.Store.EnqueueIssue(ctx, owner, repo, number, title, body, labels)` -- this is an `INSERT ... ON CONFLICT DO NOTHING` at `store.go:115-118`. Returns `true` if a new row was inserted (idempotency via the `UNIQUE(owner, repo, issue_number)` constraint at line 99).
|
||||
- On insert: posts an emoji reaction via `d.Forge.React()` (default "eyes", config line 319), sets the "queued" state label, and emits a `KindTaskQueued` event.
|
||||
- On duplicate (inserted==false): calls `d.maybeRetryParked()` (line 134) which checks whether a human removed the `archie:parked` label and if so calls `d.Store.Requeue()`.
|
||||
|
||||
### Phase 3 -- Claim (queue drain)
|
||||
|
||||
Back in `Daemon.Cycle()` at lines 92-106, an inner `for` loop:
|
||||
1. Calls `d.Store.ClaimNext(ctx)` (line 96).
|
||||
2. If nil, returns (queue empty).
|
||||
3. Otherwise calls `d.process(ctx, task)` (line 104), then loops again.
|
||||
|
||||
**`Store.ClaimNext()`** at `store.go:129-141`: one atomic SQL statement:
|
||||
```sql
|
||||
UPDATE tasks SET status='running', attempt=attempt+1, updated_at=datetime('now')
|
||||
WHERE id = (SELECT id FROM tasks WHERE status='queued' ORDER BY id LIMIT 1)
|
||||
RETURNING id, owner, repo, issue_number, title, body, labels, status,
|
||||
workflow, stage, branch, plan, notes, pr_number, tokens_used,
|
||||
iterations, attempt, park_reason
|
||||
```
|
||||
Because SQLite locks the database file for the duration of the UPDATE, no two claimed tasks can overlap -- the daemon is effectively single-consumer. The `RETURNING` clause returns the full row.
|
||||
|
||||
### Phase 4 -- Process
|
||||
|
||||
**`Daemon.process()`** at `daemon.go:356`:
|
||||
1. Looks up the `config.Repo` for the task via `d.repoFor(task)` (line 358). Fails/tasks that belong to a now-unconfigured repo go to `StatusParked`.
|
||||
2. Routes to a workflow: `wf := workflow.Route(task, d.Workflows)` (line 362).
|
||||
3. Sets the "working" state label on the issue.
|
||||
4. Calls `workflow.Run(ctx, wf, taskContext)` (line 365).
|
||||
|
||||
### Phase 5 -- Workflow Execution
|
||||
|
||||
**`workflow.Route()`** at `workflow.go:103-141` decides the workflow:
|
||||
1. If `t.Workflow` is non-empty (set by a prior `Requeue`), use that exact workflow from the registry.
|
||||
2. Otherwise inspect labels: `"bug"` -> `"tdd"`, `"feature"` -> `"feasibility"`, `"bootstrap"` -> `"bootstrap"`.
|
||||
3. Falls back to `"implement"`, then `"default"`, then a hardcoded failing error workflow.
|
||||
|
||||
**`workflow.Run()`** at `workflow.go:145-185` executes stages sequentially:
|
||||
1. Sets `t.Workflow = wf.Name`, persists via `tc.Store.Update()`.
|
||||
2. For each stage: sets `t.Stage`, persists, emits `KindStageStart`, runs `stage.Run(ctx, tc)`, emits `KindStageFinish` with timing data.
|
||||
3. On stage error: if `ctx.Err()` is set (daemon shutting down), returns cleanly; otherwise calls `park()` at line 216, which transitions to `StatusParked`, posts a comment, and emits `KindParked`.
|
||||
4. If `tc.Outcome.Status` is non-empty after a stage, calls `finish()` at line 187, which transitions to that terminal status.
|
||||
|
||||
### Phase 6 -- Terminal States
|
||||
|
||||
A workflow terminates through `finish()` (`workflow.go:187`) or `park()` (`workflow.go:216`):
|
||||
|
||||
| Outcome | Status | What happens |
|
||||
|---|---|---|
|
||||
| `park()` | `parked` | Worktree kept, park-reason comment posted, label set. Human can remove `archie:parked` to trigger `maybeRetryParked()`. |
|
||||
| `finish()` with `StatusPROpen` | `pr_open` | PR is open on GitHub. Subsequent `Cycle()` passes call `reconcilePRs()` which checks merged/closed. |
|
||||
| `finish()` with `StatusWaitingHuman` | `waiting_human` | Daemon watches for human replies via `checkWaiting()`, LLM-judges approve/reject/unclear. |
|
||||
| `finish()` with `StatusMerged` | `merged` | Final -- PR merged, labels cleared. |
|
||||
| `finish()` with `StatusClosedWontDo` | `closed_wont_do` | Issue closed on GitHub. |
|
||||
| `finish()` with `StatusRejected` | `rejected` | PR closed without merge. |
|
||||
|
||||
### Crash Recovery
|
||||
|
||||
`Daemon.Startup()` at line 52 calls `d.Store.RecoverStale(ctx)` (`store.go:234`) which runs:
|
||||
```sql
|
||||
UPDATE tasks SET status='queued' WHERE status='running'
|
||||
```
|
||||
This re-queues any task that was `running` when the previous daemon crashed. There is no distributed locking -- the single SQLite file is the authority.
|
||||
|
||||
---
|
||||
|
||||
## 2. Event Struct and Bus
|
||||
|
||||
### `events.Event` at `events.go:31-42`
|
||||
|
||||
```go
|
||||
type Event struct {
|
||||
ID int64 `json:"id,omitempty"` // set by the store sink
|
||||
At time.Time `json:"at"`
|
||||
Kind string `json:"kind"`
|
||||
TaskID int64 `json:"task_id,omitempty"`
|
||||
Repo string `json:"repo,omitempty"`
|
||||
Issue int `json:"issue,omitempty"`
|
||||
Workflow string `json:"workflow,omitempty"`
|
||||
Stage string `json:"stage,omitempty"`
|
||||
Detail string `json:"detail,omitempty"`
|
||||
Data map[string]any `json:"data,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
**Kind constants** at `events.go:16-26`:
|
||||
- `KindTaskQueued`, `KindStageStart`, `KindStageFinish` (data: `duration_ms`, `error`), `KindAgentFinish` (data: `status`, `stop_reason`, `tokens`, `iterations`, `model`), `KindParked` (data: `reason`), `KindOutcome` (data: `status`, `detail`), `KindPRMerged`, `KindPRRejected`, `KindLog` (data: `level`, `msg`).
|
||||
|
||||
Events are intentionally the single wire type -- every consumer is just another subscriber.
|
||||
|
||||
### `events.Bus` at `events.go:59-128`
|
||||
|
||||
A simple in-process fan-out bus:
|
||||
|
||||
- **`Subscribe(buffer int) *Sub`** (line 68): creates a buffered channel (default 64), appends to `b.subs`. Returns a `*Sub` whose `.C` field is the read-only receive channel.
|
||||
- **`Publish(e Event)`** (line 86): stamps `At` if zero, then iterates all subscribers and does a non-blocking send (`select` with `default`). If a subscriber's buffer is full, the event is dropped and `s.dropped.Add(1)` is incremented. **This is the critical design property: backpressure from a slow consumer never blocks the publisher.**
|
||||
- **`unsubscribe(sub *Sub)`** (line 104): removes the sub from the slice, closes its channel.
|
||||
- **`Close()`** (line 117): sets `closed` flag, closes all subscriber channels, nils the slice.
|
||||
- **Bounded per-subscriber buffers** with drop-on-overflow, as documented at line 6: "a stalled dashboard connection can never apply backpressure to the task engine."
|
||||
|
||||
### Publish Side
|
||||
|
||||
The daemon has a convenience method `emit()` at `daemon.go:44-49`:
|
||||
```go
|
||||
func (d *Daemon) emit(e events.Event) {
|
||||
if d.Bus != nil { d.Bus.Publish(e) }
|
||||
}
|
||||
```
|
||||
Called explicitly at every lifecycle transition -- see `daemon.go:128` (queued), `daemon.go:207` (retried), `daemon.go:237` (PR merged), `daemon.go:248` (PR rejected), `daemon.go:294` (human approved), `daemon.go:303` (human rejected).
|
||||
|
||||
The workflow engine also publishes via `TaskContext.Emit()` at `workflow.go:61-75`, which stamps the task identity and delegates to `tc.Bus.Publish()`.
|
||||
|
||||
### Subscribe Side (Consumers)
|
||||
|
||||
The bus is consumed by at least two subscriber patterns visible in the codebase:
|
||||
|
||||
1. **SQLite event log** (`internal/store/events.go`): some goroutine must subscribe and call `s.InsertEvent()` to persist each event. (The subscription setup is in the composition root `cmd/archied`. I did not read that file, but `InsertEvent` at `events.go:32` clearly exists for this purpose.)
|
||||
|
||||
2. **SSE fan-out** for the dashboard: the bus is designed so each HTTP SSE connection gets its own `Sub` with a small buffer, and the drop-on-overload behavior lets the dashboard fall behind without slowing the daemon.
|
||||
|
||||
---
|
||||
|
||||
## 3. ClaimNext -- SQL, Locking, Concurrency
|
||||
|
||||
### The SQL
|
||||
|
||||
At `store.go:129-141`:
|
||||
```sql
|
||||
UPDATE tasks SET status='running', attempt=attempt+1, updated_at=datetime('now')
|
||||
WHERE id = (SELECT id FROM tasks WHERE status='queued' ORDER BY id LIMIT 1)
|
||||
RETURNING id, owner, repo, issue_number, title, body, labels, status,
|
||||
workflow, stage, branch, plan, notes, pr_number, tokens_used,
|
||||
iterations, attempt, park_reason
|
||||
```
|
||||
|
||||
Three operations in one statement:
|
||||
- **Subquery**: selects the oldest queued task (`ORDER BY id LIMIT 1`).
|
||||
- **UPDATE**: atomically marks it `running`, increments `attempt`.
|
||||
- **RETURNING**: returns the full row, scanned by `scanTask()` (line 143).
|
||||
|
||||
### Concurrency Model
|
||||
|
||||
- **Single-process, single-threaded consumers.** SQLite's locking means only one writer at a time. The daemon has a single `Store` backed by a `*sql.DB` with `_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)` (line 61). In WAL mode, readers do not block writers, but `UPDATE` still acquires a write lock -- so two concurrent `ClaimNext` calls would serialize on the database lock.
|
||||
|
||||
- **No distributed claims.** There is no advisory lock, no `SELECT ... FOR UPDATE`, no external coordination. The claim is just the SQL mutation + the WAL busy_timeout (5s). If two daemon processes somehow connected to the same SQLite file, one would get an error or timeout on concurrent UPDATE -- not a correctness issue, but not a supported deployment.
|
||||
|
||||
- **Crash recovery, not fencing.** Instead of distributed locking, `Startup()` calls `RecoverStale()` on boot (`store.go:235`): `UPDATE tasks SET status='queued' WHERE status='running'`. This re-queues any task that was in-flight when the previous process died.
|
||||
|
||||
- **Single-consumer queue semantics.** `Daemon.Cycle()` at line 92 loops: claim, process, claim, process. There is no parallelism within one daemon instance -- `process()` is synchronous. The `for` returns when the queue is empty (ClaimNext returns nil).
|
||||
|
||||
---
|
||||
|
||||
## 4. forge.Issue to store.Task Mapping
|
||||
|
||||
### `forge.Issue` at `forge.go:10-15`
|
||||
|
||||
```go
|
||||
type Issue struct {
|
||||
Number int
|
||||
Title string
|
||||
Body string
|
||||
Labels []string
|
||||
}
|
||||
```
|
||||
|
||||
A pure value type -- no methods, no identity. It represents one GitHub issue at poll time.
|
||||
|
||||
### `store.Task` at `store.go:30-52`
|
||||
|
||||
```go
|
||||
type Task struct {
|
||||
ID int64 // autoincrement PK
|
||||
Owner string // GitHub owner
|
||||
Repo string // GitHub repo
|
||||
IssueNumber int // matches Issue.Number
|
||||
Title string // matches Issue.Title
|
||||
Body string // matches Issue.Body
|
||||
Labels string // comma-separated from Issue.Labels
|
||||
Status string // lifecycle: queued, running, waiting_human, ...
|
||||
Workflow string // routed workflow name
|
||||
Stage string // current stage name
|
||||
Branch string // worktree branch
|
||||
Plan string // PRD / implementation plan
|
||||
Notes string // appended agent notes
|
||||
PRNumber int // opened PR number
|
||||
TokensUsed int
|
||||
Iterations int
|
||||
Attempt int
|
||||
ParkReason string
|
||||
WatchCommentID int64 // human-reply anchor
|
||||
}
|
||||
```
|
||||
|
||||
### Mapping at Enqueue Time
|
||||
|
||||
In `Daemon.poll()` at `daemon.go:112-113`:
|
||||
```go
|
||||
inserted, err := d.Store.EnqueueIssue(ctx,
|
||||
repo.Owner, repo.Name, is.Number, is.Title, is.Body, strings.Join(is.Labels, ","))
|
||||
```
|
||||
|
||||
The forge-side `[]string` labels are joined into a comma-separated string. The FORGE-side structure is flat (Number, Title, Body, Labels) -- it is the STORE that adds status tracking, routing, and operational fields. The `UNIQUE(owner, repo, issue_number)` constraint at `store.go:99` is the idempotency key: re-polling the same issue does not create a duplicate task.
|
||||
|
||||
### Mapping at Enqueue SQL
|
||||
|
||||
`store.go:115-118`:
|
||||
```sql
|
||||
INSERT INTO tasks (owner, repo, issue_number, title, body, labels)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(owner, repo, issue_number) DO NOTHING
|
||||
```
|
||||
|
||||
Only 6 of 18 Task fields are set at creation. The remaining fields (status defaults to `'queued'`, workflow/stage/branch etc. default to `''` or `0`) are filled in during the processing lifecycle.
|
||||
|
||||
---
|
||||
|
||||
## 5. What Changes If ClaimNext Is Replaced with NATS JetStream?
|
||||
|
||||
### Current Architecture
|
||||
```
|
||||
[Ticker] -> Cycle() -> poll() -> [SQLite EnqueueIssue]
|
||||
-> [SQLite ClaimNext] -> process(task) -> [workflow.Run]
|
||||
```
|
||||
The daemon is a single in-memory loop pulling from SQLite. It is **push-based** from SQLite's perspective (the daemon polls the DB), but effectively **single-consumer** -- no parallelism.
|
||||
|
||||
### Target Architecture with NATS JetStream
|
||||
```
|
||||
[GitHub Webhook] -> [NATS JetStream] -> [Daemon consumes from stream]
|
||||
```
|
||||
or
|
||||
```
|
||||
[Ticker] -> poll() -> [NATS JetStream Publish]
|
||||
-> [Daemon Subscribe] -> process(task)
|
||||
```
|
||||
|
||||
### What Stays the Same
|
||||
|
||||
1. **`foreach`-over-repos polling** (`Daemon.poll()`, `daemon.go:108`). The GitHub API polling logic (`IssuesWithLabel`, `AssignedIssues`) is independent of the task queue -- you still poll GitHub the same way, you just publish the discovered issue to a NATS subject instead of INSERTing into SQLite.
|
||||
|
||||
2. **`workflow.Run()` and the entire workflow engine** (`workflow.go:145-185`, `steps.go`, `agent.go`, `feasibility.go`, `implement.go`, `tdd.go`). The workflow pipeline is agnostic to how tasks arrive -- it receives a `*store.Task` and a `TaskContext`. A NATS consumer would still build the same `TaskContext` and call `workflow.Run()`.
|
||||
|
||||
3. **`workflow.Route()`** (`workflow.go:103`). Label-to-workflow routing remains identical -- it operates on `t.Labels` and `t.Workflow`.
|
||||
|
||||
4. **`forge` interface and `GitHubClient`** (`forge.go`, `github.go`). The forge is only called to interact with GitHub (comment, label, PR, react) -- not as the task queue. No forge change needed.
|
||||
|
||||
5. **`events.Bus` and all `emit()` / `Emit()` calls.** The in-process observability bus is orthogonal to task distribution. NATS JetStream would be the task queue, not the event bus; the two are conceptually separate.
|
||||
|
||||
6. **Crash recovery via `RecoverStale()`** would be replaced by NATS's exactly-once delivery (with consumer sequence tracking) or at-least-once with deduplication. But the principle is the same: the daemon should be restartable without losing work.
|
||||
|
||||
7. **`store.Update()` and `store.Transition()`** for persisting task progress. Even with NATS as the queue, you still want an audit trail of task state. SQLite is fine for that -- it just becomes the state store rather than the queue.
|
||||
|
||||
### What Changes
|
||||
|
||||
**A. `Daemon.Run()` and `Cyc1e()` (`daemon.go:72-106`)**
|
||||
|
||||
Currently `Run()` has a ticker that calls `Cycle()` which does both poll-and-drain. With NATS:
|
||||
- The poll ticker would still exist but would `Publish()` discovered issues to a NATS subject rather than `EnqueueIssue`.
|
||||
- The drain loop would be replaced by a NATS subscription callback. The daemon would register a Pull Consumer or a Push Consumer on the stream, and `process()` would be called from the message handler, not from a `ClaimNext()` call.
|
||||
|
||||
Something like:
|
||||
```go
|
||||
func (d *Daemon) Run(ctx context.Context) error {
|
||||
sub, _ := d.JS.PullSubscribe("archie.tasks", ...)
|
||||
for {
|
||||
msgs, _ := sub.Fetch(1, ...)
|
||||
for _, msg := range msgs {
|
||||
task := deserialize(msg.Data)
|
||||
d.process(ctx, task)
|
||||
msg.Ack()
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**B. `Store.ClaimNext()` is removed (`store.go:129`).**
|
||||
|
||||
The atomic SQL `UPDATE ... WHERE id = (SELECT ...)` is the heart of the single-consumer model. With NATS, the message broker provides the claim -- by delivering the message to exactly one consumer in the consumer group (JetStream's built-in `MaxDeliver` / `Ack` / `DeliveryGroup`). The entire function is replaced by `sub.Fetch()` or `sub.Consume()`.
|
||||
|
||||
**C. `Store.EnqueueIssue()` (`store.go:114`) becomes optional.**
|
||||
|
||||
You may or may not persist tasks in SQLite at enqueue time. Options:
|
||||
- Keep SQLite as the system of record: the NATS subscriber writes a row on receive, then passes `*store.Task` to `process()`. This preserves the dashboard's ability to query task history without replaying the stream.
|
||||
- Drop SQLite entirely at the queue level: the NATS message body carries the equivalent of `forge.Issue` + routing metadata, and the first workflow stage does the persistence.
|
||||
|
||||
**D. `Daemon.Startup()` crash recovery changes (`daemon.go:52`).**
|
||||
|
||||
`RecoverStale()` (`store.go:234`) re-queues tasks left `running`. With NATS:
|
||||
- If a consumer crashes mid-process, the message is either `NACK`ed or redelivered after the `AckWait` timeout. No explicit recovery step needed.
|
||||
- But: if NATS delivers a message and the daemon starts processing (updates task to `running` in SQLite), then crashes, the NATS message might already be `Ack`ed. You still need the crash-recovery query as a safety net, or you redesign to only `Ack` after the workflow completes.
|
||||
- Best practice: `Ack` *after* `workflow.Run()` returns, and keep `RecoverStale` for the SQLite side.
|
||||
|
||||
**E. Concurrency model changes fundamentally.**
|
||||
|
||||
Currently one daemon processes tasks sequentially (one `ClaimNext` -> `process()` -> next `ClaimNext`). With NATS:
|
||||
- You can horizontally scale: multiple daemon instances subscribe to the same JetStream consumer group, and NATS round-robins delivery.
|
||||
- Within one daemon, you can parallelize: call `process()` in a goroutine per message, bounded by a semaphore. Each message has an independent worktree, so parallel workflows on different repos are safe. (Current SQLite model could also parallelize, but the synchronous `Cycle()` design prevents it.)
|
||||
|
||||
**F. Idempotency moves from SQL UNIQUE constraint to JetStream dedup.**
|
||||
|
||||
Currently `EnqueueIssue` uses `ON CONFLICT DO NOTHING` at `store.go:118`. With NATS, you would use JetStream message deduplication (`Nats-Msg-Id` header set to `owner/repo/issue_number`) to prevent re-processing the same issue enqueue.
|
||||
|
||||
**G. Task ordering.**
|
||||
|
||||
Currently FIFO via `ORDER BY id LIMIT 1`. NATS JetStream supports ordered delivery within a stream, but with multiple repos and priorities this may differ. You would need a stream per repo or use a subject hierarchy like `archie.tasks.<owner>.<repo>` to preserve ordering semantics.
|
||||
|
||||
### Summary Table
|
||||
|
||||
| Component | Current (SQLite) | NATS JetStream |
|
||||
|---|---|---|
|
||||
| Task queue | `tasks` table, `status='queued'` | JetStream stream |
|
||||
| Claim | `UPDATE ... WHERE id = (SELECT ...) RETURNING` | `sub.Fetch()` / consumer group |
|
||||
| Concurrency | Single-process, sequential | Multi-process, parallel via consumer groups |
|
||||
| Idempotency | `ON CONFLICT DO NOTHING` + `UNIQUE` constraint | `Nats-Msg-Id` dedup |
|
||||
| Crash recovery | `RecoverStale()` at startup | NAck/redelivery + eventual `RecoverStale` guard |
|
||||
| Ordering | `ORDER BY id` (FIFO) | Stream ordering per subject |
|
||||
| Daemon.Run() | `time.Ticker` -> `Cycle()` -> {poll, claim-loop} | `time.Ticker` -> poll() -> publish; sub callback -> process() |
|
||||
| `Store.ClaimNext()` | **Must remove** | No equivalent needed |
|
||||
| `Store.EnqueueIssue()` | **Optional** | Still useful as system-of-record |
|
||||
| Workflow engine | Unchanged | Unchanged |
|
||||
| Forge interface | Unchanged | Unchanged |
|
||||
| events.Bus | Unchanged | Unchanged |
|
||||
@@ -1,189 +0,0 @@
|
||||
# nats-integration
|
||||
|
||||
Here is the full analysis.
|
||||
|
||||
---
|
||||
|
||||
## Report: archie-core Codebase Exploration for NATS Integration
|
||||
|
||||
### 1. Current Go Version and Key Dependencies
|
||||
|
||||
**Go version:** `go 1.26.2` (from `/work/apps/archie-core/go.mod`, line 3)
|
||||
|
||||
**Direct dependencies** (from `/work/apps/archie-core/go.mod`):
|
||||
- `github.com/BurntSushi/toml v1.6.0` -- TOML config parsing
|
||||
- `github.com/google/go-github/v78 v78.0.0` -- GitHub API client
|
||||
- `modernc.org/sqlite v1.54.0` -- pure-Go SQLite (no CGo)
|
||||
- `github.com/samcharles93/ai-sdk v0.1.17` -- local LLM SDK (replaced to `/work/projects/ai-sdk`)
|
||||
- `github.com/traefik/yaegi v0.16.1` -- Go interpreter for custom skill/workflow stages
|
||||
|
||||
**Indirect dependencies:** `go-humanize`, `go-querystring`, `uuid`, `go-isatty`, `go-strftime`, `browser`, `bigfft`, `golang.org/x/sys`, `modernc.org/libc/mathutil/memory`
|
||||
|
||||
**Key observation:** The dependency tree is very lean. There are zero messaging, queue, or streaming libraries. No `go.sum` entries match "nats" -- zero NATS transitive dependencies exist.
|
||||
|
||||
---
|
||||
|
||||
### 2. Full Config Struct (all fields, all structs)
|
||||
|
||||
File: `/work/apps/archie-core/internal/config/config.go`
|
||||
|
||||
```
|
||||
Config (205-231)
|
||||
├── WorkDir string `toml:"work_dir"`
|
||||
├── DBPath string `toml:"db_path"`
|
||||
├── PollInterval Duration `toml:"poll_interval"` (custom Duration type, TOML strings like "60s")
|
||||
├── Label string `toml:"label"`
|
||||
├── BotUser string `toml:"bot_user"`
|
||||
├── BotEmail string `toml:"bot_email"`
|
||||
├── DiffCapLines int `toml:"diff_cap_lines"`
|
||||
├── Forge Forge `toml:"forge"`
|
||||
│ ├── Type string `toml:"type"` (default: "github")
|
||||
│ ├── Host string `toml:"host"` (default: "https://github.com")
|
||||
│ └── TokenEnv string `toml:"token_env"` (default: "ARCHIE_GITHUB_TOKEN")
|
||||
├── Dispatch Dispatch `toml:"dispatch"`
|
||||
│ ├── Trigger string `toml:"trigger"` ("assignee", "label", "either"; default: "assignee")
|
||||
│ ├── AckReaction string `toml:"ack_reaction"` ("eyes"; "off" disables)
|
||||
│ └── Labels map[string]string `toml:"labels"` (keys: queued/working/waiting/pr/parked)
|
||||
├── Models map[string]string `toml:"models"` (role -> model ref, e.g. "triage", "planner", "builder")
|
||||
├── Providers map[string]Provider `toml:"providers"`
|
||||
│ ├── Class string `toml:"class"` ("openai", "ollama", etc.)
|
||||
│ ├── APIKeyEnv string `toml:"api_key_env"`
|
||||
│ └── BaseURL string `toml:"base_url"`
|
||||
├── Agent Agent `toml:"agent"`
|
||||
│ ├── Mode string `toml:"mode"` ("inprocess" or "subprocess")
|
||||
│ ├── Command string `toml:"command"` (executable path for subprocess mode)
|
||||
│ └── Env []string `toml:"env"` (extra env vars forwarded to worker)
|
||||
├── Budgets Budgets `toml:"budgets"`
|
||||
│ ├── MaxSteps int `toml:"max_steps"`
|
||||
│ ├── MaxTokens int `toml:"max_tokens"`
|
||||
│ ├── WallClock Duration `toml:"wall_clock"`
|
||||
│ └── GateMaxFailures int `toml:"gate_max_failures"`
|
||||
├── Web Web `toml:"web"`
|
||||
│ └── Listen string `toml:"listen"` (default: "127.0.0.1:8484"; "off" disables)
|
||||
├── Notify Notify `toml:"notify"`
|
||||
│ └── Webhook string `toml:"webhook"` (n8n webhook URL)
|
||||
└── Repos []Repo `toml:"repos"`
|
||||
├── Owner string `toml:"owner"`
|
||||
├── Name string `toml:"name"`
|
||||
├── Base string `toml:"base"` (branch PRs target; default "main")
|
||||
├── Gate [][]string `toml:"gate"` (quality-gate commands)
|
||||
├── Protect []string `toml:"protect"` (path suffixes agents cannot write)
|
||||
├── Ecosystem string `toml:"ecosystem"` ("go", "python", "node", "rust", "custom")
|
||||
├── Preflight [][]string `toml:"preflight"` (override ecosystem default)
|
||||
└── TestGlob string `toml:"test_glob"` (override ecosystem default)
|
||||
```
|
||||
|
||||
The `Config` struct is loaded by `config.Load(path)` which decodes TOML, then applies defaults. The `Load()` function validates: agent mode, provider URLs, dispatch trigger values, dispatch labels, bot_user requirement, repos requirement, forge type (currently only "github"), and test glob patterns.
|
||||
|
||||
---
|
||||
|
||||
### 3. How the Project is Built/Run Today
|
||||
|
||||
**No Makefile, Taskfile, justfile, Dockerfile, docker-compose, or CI config files exist anywhere in the project.**
|
||||
|
||||
The project is built and run directly with the Go toolchain:
|
||||
|
||||
- **Build:** `go build ./cmd/archied` to build the daemon binary, `go build ./cmd/archie-agent` for the subprocess worker.
|
||||
- **Run:** `archied -config ~/.config/archie/config.toml` (the `run()` func in `cmd/archied/main.go`).
|
||||
- **Entry points:**
|
||||
- `/work/apps/archie-core/cmd/archied/main.go` -- the daemon (orchestrator)
|
||||
- `/work/apps/archie-core/cmd/archie-agent/main.go` -- standalone worker (used in subprocess mode)
|
||||
- **Composition root** is `cmd/archied/main.go`. It wires: config loading, forge client creation, SQLite store, event bus, web UI server, LLM runtime, agent runner, worktree manager, workflow registry.
|
||||
- **Daemon lifecycle:** `Startup()` (recovery, invitation accept, push verify) -> `Run()` (poll loop with ticker) -> each `Cycle()` does: poll issues, reconcile PRs, check waiting tasks, drain queued tasks sequentially.
|
||||
- **Testing:** standard `go test ./...` -- no bespoke task runner.
|
||||
|
||||
---
|
||||
|
||||
### 4. Existing NATS or Messaging Patterns
|
||||
|
||||
**NATS: None whatsoever.** Zero references in Go source, zero in go.sum, zero in any config.
|
||||
|
||||
**Existing messaging system -- in-process event bus** (`/work/apps/archie-core/internal/events/events.go`):
|
||||
|
||||
This is a lightweight in-process pub/sub bus designed for observability. Key design:
|
||||
- Single `Event` type carries all observability data (timestamps, kind strings, task IDs, freeform `Data map[string]any`)
|
||||
- Mutex-based fan-out with bounded per-subscriber channels
|
||||
- **Drops on overflow** -- a stalled dashboard client cannot backpressure the task engine
|
||||
- Subscribers register via `bus.Subscribe(buffer int)` which returns a `*Sub` with a read-only `C <-chan Event`
|
||||
- Event kinds: `task_queued`, `stage_start`, `stage_finish`, `agent_finish`, `parked`, `outcome`, `pr_merged`, `pr_rejected`, `log`
|
||||
|
||||
**How it is used** (from `cmd/archied/main.go` lines 86-107):
|
||||
1. A `bus := events.NewBus()` is created in `main()`
|
||||
2. A store-sink subscriber consumes all events, writes them to SQLite, then broadcasts to the web UI
|
||||
3. The `webui.Server.Broadcast()` fans events to SSE clients
|
||||
4. The `daemon.Daemon` has a `Bus *events.Bus` field used in `emit()` calls throughout task lifecycle
|
||||
|
||||
**Current architecture is single-process:** The daemon polls GitHub issues, processes them sequentially in-process (or via subprocess `archie-agent`), and all state lives in SQLite. There is no multi-worker, no job queue, no cross-process communication beyond the subprocess protocol in `internal/agentexec/`.
|
||||
|
||||
---
|
||||
|
||||
### 5. Recommended `[nats]` Config Block
|
||||
|
||||
Following the existing TOML conventions (see `/work/apps/archie-core/config.example.toml` and the `config.go` struct patterns), a `[nats]` block should:
|
||||
|
||||
- Match the two-level key style (e.g., `[forge]`, `[dispatch]`, `[web]`)
|
||||
- Use lowercase with underscores for field names
|
||||
- Support a `"off"` / empty-string sentinel for optional features
|
||||
- Be a new struct on the `Config` type, not nested under any existing section
|
||||
|
||||
Based on the existing patterns, the convention-compliant config block would look like:
|
||||
|
||||
```toml
|
||||
# NATS JetStream configuration for distributed task queue.
|
||||
# Remove or comment out these sections to run in single-process mode.
|
||||
[nats]
|
||||
url = "nats://localhost:4222" # NATS server URL
|
||||
# token = "$NATS_TOKEN" # or user/password for auth (use env var)
|
||||
# user = "archie"
|
||||
# password_env = "NATS_PASSWORD"
|
||||
|
||||
[nats.tls] # optional TLS section as a sub-table
|
||||
# cert = "/path/to/cert.pem"
|
||||
# key = "/path/to/key.pem"
|
||||
# ca = "/path/to/ca.pem"
|
||||
|
||||
[nats.jetstream] # JetStream-specific options
|
||||
# stream = "archie-tasks" # stream name (default: auto-created)
|
||||
# bucket = "archie-state" # KV bucket for task state (optional)
|
||||
# replicas = 3 # stream replication factor
|
||||
# max_deliver = 10 # max delivery attempts for a message
|
||||
# ack_wait = "5m" # ack wait timeout
|
||||
```
|
||||
|
||||
In Go, this would map to a struct like:
|
||||
|
||||
```go
|
||||
type NATS struct {
|
||||
URL string `toml:"url"`
|
||||
Token string `toml:"token"`
|
||||
User string `toml:"user"`
|
||||
PasswordEnv string `toml:"password_env"`
|
||||
TLS *NATSTLS `toml:"tls,omitempty"`
|
||||
JetStream *JetStream `toml:"jetstream,omitempty"`
|
||||
}
|
||||
|
||||
type NATSTLS struct {
|
||||
Cert string `toml:"cert"`
|
||||
Key string `toml:"key"`
|
||||
CA string `toml:"ca"`
|
||||
}
|
||||
|
||||
type JetStream struct {
|
||||
Stream string `toml:"stream"`
|
||||
Bucket string `toml:"bucket"`
|
||||
Replicas int `toml:"replicas"`
|
||||
MaxDeliver int `toml:"max_deliver"`
|
||||
AckWait Duration `toml:"ack_wait"`
|
||||
}
|
||||
```
|
||||
|
||||
And on `Config`:
|
||||
```go
|
||||
NATS *NATS `toml:"nats,omitempty"` // nil = no NATS, single-process mode
|
||||
```
|
||||
|
||||
Key style considerations:
|
||||
- Existing config uses `password_env` pattern (see `token_env`, `api_key_env`) for secrets that come from environment variables
|
||||
- The `Duration` type is already available for timer fields like `ack_wait`
|
||||
- Using a pointer (`*NATS`) keeps the "nil means not configured" semantics, consistent with the daemon's fallback to single-process operation
|
||||
- The `[nats.tls]` sub-table pattern mirrors how `[dispatch.labels]` is a sub-table under `[dispatch]`
|
||||
@@ -1,707 +0,0 @@
|
||||
# nats-design
|
||||
|
||||
I now have a thorough understanding of the entire codebase. Let me compile my implementation plan.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan: Phase 1 -- NATS Task Distribution for archie-core
|
||||
|
||||
### Overview
|
||||
|
||||
This phase adds optional NATS JetStream-based task distribution to `archied`. When the `[nats]` config block is present, newly discovered issues are published to NATS subjects rather than enqueued directly into SQLite. A pull consumer in the same daemon process receives those messages, stores them in SQLite (idempotent dedup via `ON CONFLICT DO NOTHING`), claims them, and runs the workflow. When NATS is absent, the existing SQLite-only flow (ClaimNext) continues unchanged.
|
||||
|
||||
---
|
||||
|
||||
### Step 1: Add NATS Go Dependency
|
||||
|
||||
**File(s):** `/work/apps/archie-core/go.mod`, `/work/apps/archie-core/go.sum`
|
||||
|
||||
**Changes:**
|
||||
- Add `github.com/nats-io/nats.go v1.42.0` (or latest stable) to `require` block
|
||||
- The JetStream API is now embedded in `nats.go` (no separate `nats.go/jetstream` module since v1.33+), so only one dependency line is needed
|
||||
- Run `go mod tidy` to resolve transitive deps
|
||||
|
||||
**Connection to existing code:** This is the only new external dependency. The current module has only `BurntSushi/toml`, `google/go-github`, `modernc.org/sqlite`, `ai-sdk`, and `traefik/yaegi`.
|
||||
|
||||
**Test/verification:** `go build ./...` succeeds. `go mod tidy` is clean.
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Add `[nats]` Config Section
|
||||
|
||||
**File(s):** `/work/apps/archie-core/internal/config/config.go`, `/work/apps/archie-core/internal/config/config_test.go`, `/work/apps/archie-core/config.example.toml`
|
||||
|
||||
**New type in `config.go`:**
|
||||
```go
|
||||
// NATSConfig configures NATS JetStream for task distribution. When URL is
|
||||
// empty the existing SQLite ClaimNext flow is used.
|
||||
type NATSConfig struct {
|
||||
// URL is the NATS server address, e.g. "nats://localhost:4222".
|
||||
// Empty means NATS is not configured.
|
||||
URL string `toml:"url"`
|
||||
// TokenEnv optionally names an env var holding a NATS auth token/password.
|
||||
// When empty, no authentication is attempted.
|
||||
TokenEnv string `toml:"token_env"`
|
||||
}
|
||||
```
|
||||
|
||||
**New field in `Config` struct:**
|
||||
```go
|
||||
type Config struct {
|
||||
// ... all existing fields ...
|
||||
NATS NATSConfig `toml:"nats"`
|
||||
}
|
||||
```
|
||||
|
||||
**Validation in `Load()`:** No new validation errors needed. `cfg.NATS.URL == ""` is the disabled state. If `TokenEnv` is set but `URL` is empty, log a warning at startup (not a config error -- the token field is simply unused).
|
||||
|
||||
**Default behavior:** When the TOML has no `[nats]` block, `cfg.NATS.URL` remains `""` and NATS is not used. This is the nil-means-disabled semantics.
|
||||
|
||||
**Test in `config_test.go`:** Add `TestNATSDefaults` -- verify that (a) a config without `[nats]` has empty URL, (b) a config with `[nats]` but empty URL loads fine, (c) a config with a NATS URL parses correctly. Add to existing `TestExampleConfigLoads` so the example config stays valid.
|
||||
|
||||
**Config example in `config.example.toml`:**
|
||||
```toml
|
||||
[nats]
|
||||
# url = "nats://localhost:4222"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Create `internal/nats` Package
|
||||
|
||||
**File(s) to create:** `/work/apps/archie-core/internal/nats/client.go`, `/work/apps/archie-core/internal/nats/client_test.go`, `/work/apps/archie-core/internal/nats/subjects.go`
|
||||
|
||||
#### 3a. Subject Constants (`subjects.go`)
|
||||
|
||||
```go
|
||||
package nats
|
||||
|
||||
import "strings"
|
||||
|
||||
// Task subject constants. The subject encodes the workflow type so that
|
||||
// future horizontal scaling can route tasks to workflow-specific consumers.
|
||||
const (
|
||||
SubjectTaskBug = "archie.task.bug"
|
||||
SubjectTaskFeature = "archie.task.feature"
|
||||
SubjectTaskDefault = "archie.task.default"
|
||||
SubjectTaskBootstrap = "archie.task.bootstrap"
|
||||
)
|
||||
|
||||
// SubjectForLabels picks the NATS subject based on issue labels, following
|
||||
// the same label-to-workflow mapping as workflow.Route.
|
||||
// - "bug" -> archie.task.bug
|
||||
// - "feature" -> archie.task.feature
|
||||
// - "bootstrap" -> archie.task.bootstrap
|
||||
// - default -> archie.task.default
|
||||
func SubjectForLabels(labels []string) string {
|
||||
for _, l := range labels {
|
||||
switch strings.TrimSpace(l) {
|
||||
case "bug":
|
||||
return SubjectTaskBug
|
||||
case "feature":
|
||||
return SubjectTaskFeature
|
||||
case "bootstrap":
|
||||
return SubjectTaskBootstrap
|
||||
}
|
||||
}
|
||||
return SubjectTaskDefault
|
||||
}
|
||||
```
|
||||
|
||||
**Tests:** Verify subject routing: `SubjectForLabels([]string{"bug"})` returns `SubjectTaskBug`; `SubjectForLabels([]string{"feature"})` returns `SubjectTaskFeature`; `SubjectForLabels([]string{"enhancement"})` returns `SubjectTaskDefault`; `SubjectForLabels([]string{"bug", "feature"})` returns `SubjectTaskBug` (first match wins, matching workflow.Route behavior).
|
||||
|
||||
#### 3b. Client (`client.go`)
|
||||
|
||||
```go
|
||||
package nats
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
const (
|
||||
streamName = "ARCHIE_TASKS"
|
||||
consumerName = "archie-daemon"
|
||||
msgPrefix = "archie:" // for Msg-Id dedup
|
||||
dedupWindow = 2 * time.Minute
|
||||
pollTimeout = 2 * time.Second // Fetch wait time before returning nil
|
||||
maxFetchCount = 1
|
||||
)
|
||||
|
||||
// TaskMessage is the NATS payload for a discovered issue.
|
||||
type TaskMessage struct {
|
||||
Owner string `json:"owner"`
|
||||
Repo string `json:"repo"`
|
||||
Number int `json:"number"`
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
Labels string `json:"labels"` // comma-separated, as seen at enqueue time
|
||||
}
|
||||
|
||||
// Client manages the NATS connection and JetStream resources.
|
||||
type Client struct {
|
||||
conn *nats.Conn
|
||||
js jetstream.JetStream
|
||||
stream jetstream.Stream
|
||||
consumer jetstream.Consumer
|
||||
log *slog.Logger
|
||||
}
|
||||
|
||||
// Connect dials url, sets up the JetStream stream and pull consumer.
|
||||
// The stream is created with WorkQueue retention (message removed after ack),
|
||||
// file storage, and the subject pattern archie.task.>.
|
||||
func Connect(ctx context.Context, url string, log *slog.Logger) (*Client, error) {
|
||||
nc, err := nats.Connect(url)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("nats connect: %w", err)
|
||||
}
|
||||
js, err := jetstream.New(nc)
|
||||
if err != nil {
|
||||
nc.Close()
|
||||
return nil, fmt.Errorf("nats jetstream: %w", err)
|
||||
}
|
||||
|
||||
// Create or update stream
|
||||
stream, err := js.CreateOrUpdateStream(ctx, jetstream.StreamConfig{
|
||||
Name: streamName,
|
||||
Subjects: []string{"archie.task.>"},
|
||||
Storage: jetstream.FileStorage,
|
||||
Retention: jetstream.WorkQueuePolicy,
|
||||
MaxMsgsPerSubject: 1, // at most one message per subject per task
|
||||
Duplicates: dedupWindow,
|
||||
})
|
||||
if err != nil {
|
||||
nc.Close()
|
||||
return nil, fmt.Errorf("nats stream: %w", err)
|
||||
}
|
||||
|
||||
// Create or update pull consumer
|
||||
consumer, err := stream.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
|
||||
Name: consumerName,
|
||||
Durable: consumerName,
|
||||
FilterSubject: "archie.task.>",
|
||||
AckPolicy: jetstream.AckExplicitPolicy,
|
||||
MaxDeliver: 3, // retry 3 times before parking
|
||||
AckWait: 5 * time.Minute, // ack within 5 mins of Fetch
|
||||
InactiveThreshold: 24 * time.Hour,
|
||||
})
|
||||
if err != nil {
|
||||
nc.Close()
|
||||
return nil, fmt.Errorf("nats consumer: %w", err)
|
||||
}
|
||||
|
||||
return &Client{conn: nc, js: js, stream: stream, consumer: consumer, log: log}, nil
|
||||
}
|
||||
|
||||
// Close drains and closes the NATS connection.
|
||||
func (c *Client) Close() { c.conn.Close() }
|
||||
|
||||
// PublishTask publishes a discovered issue to the appropriate NATS subject.
|
||||
// The Msg-Id header enables JetStream dedup within the dedup window.
|
||||
// Returns nil on success.
|
||||
func (c *Client) PublishTask(ctx context.Context, owner, repo string, number int, title, body, labels string) error {
|
||||
msg := TaskMessage{
|
||||
Owner: owner,
|
||||
Repo: repo,
|
||||
Number: number,
|
||||
Title: title,
|
||||
Body: body,
|
||||
Labels: labels,
|
||||
}
|
||||
data, err := json.Marshal(msg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
subject := SubjectForLabels(splitLabels(labels))
|
||||
headers := nats.Header{}
|
||||
msgID := fmt.Sprintf("%s%s/%s/%d", msgPrefix, owner, repo, number)
|
||||
headers.Set("Nats-Msg-Id", msgID)
|
||||
|
||||
publish, err := c.js.PublishMsg(ctx, &nats.Msg{
|
||||
Subject: subject,
|
||||
Data: data,
|
||||
Header: headers,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("nats publish: %w", err)
|
||||
}
|
||||
if publish == nil || publish.Sequence == 0 {
|
||||
// Duplicate suppressed by Msg-Id dedup window
|
||||
return nil
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Fetch retrieves one task message from the pull consumer. Returns the
|
||||
// message, or nil if the context is cancelled or the fetch times out.
|
||||
// The caller MUST Ack or Nak the message after processing.
|
||||
func (c *Client) Fetch(ctx context.Context) (jetstream.Msg, error) {
|
||||
msgs, err := c.consumer.Fetch(maxFetchCount, jetstream.FetchMaxWait(pollTimeout))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("nats fetch: %w", err)
|
||||
}
|
||||
for msg := range msgs.Messages() {
|
||||
if msg == nil {
|
||||
continue
|
||||
}
|
||||
if err := msgs.Error(); err != nil {
|
||||
return nil, fmt.Errorf("nats fetch error: %w", err)
|
||||
}
|
||||
return msg, nil
|
||||
}
|
||||
return nil, nil // no message available
|
||||
}
|
||||
|
||||
// DecodeTask decodes a TaskMessage from a NATS message.
|
||||
func DecodeTask(msg jetstream.Msg) (TaskMessage, error) {
|
||||
var tm TaskMessage
|
||||
if err := json.Unmarshal(msg.Data(), &tm); err != nil {
|
||||
return tm, fmt.Errorf("nats decode: %w", err)
|
||||
}
|
||||
return tm, nil
|
||||
}
|
||||
|
||||
func splitLabels(labels string) []string {
|
||||
if labels == "" {
|
||||
return nil
|
||||
}
|
||||
return strings.Split(labels, ",")
|
||||
}
|
||||
```
|
||||
|
||||
**Key design decisions in this struct:**
|
||||
- `DedupWindow: 2 minutes` -- matching the typical poll interval. A restarted daemon within 2 minutes won't get dupes.
|
||||
- `MaxDeliver: 3` -- retries before a message is placed on the NA pile (NATS moves it to a dead-letter state internally).
|
||||
- `AckWait: 5 minutes` -- generous for long-running workflows.
|
||||
- `WorkQueuePolicy` -- message removed from stream after ack.
|
||||
- `FilterSubject: "archie.task.>"` -- consumer receives all task subjects.
|
||||
|
||||
**Tests for `client_test.go`:**
|
||||
|
||||
The test needs an embedded NATS server. Use `nats-server` binary or the `github.com/nats-io/nats-server/v2/server` package in test mode.
|
||||
|
||||
```go
|
||||
func TestConnectAndPublish(t *testing.T) {
|
||||
// Start embedded NATS server on random port
|
||||
srv := startEmbeddedNATSServer(t)
|
||||
defer srv.Shutdown()
|
||||
|
||||
url := fmt.Sprintf("nats://localhost:%d", srv.Port())
|
||||
client, err := Connect(context.Background(), url, slog.New(slog.DiscardHandler))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
// Publish a task
|
||||
if err := client.PublishTask(ctx, "sam", "todo", 1, "fix bug", "body", "bug"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Fetch it back
|
||||
msg, err := client.Fetch(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if msg == nil {
|
||||
t.Fatal("expected a message")
|
||||
}
|
||||
|
||||
tm, err := DecodeTask(msg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tm.Owner != "sam" || tm.Number != 1 {
|
||||
t.Fatalf("unexpected task: %+v", tm)
|
||||
}
|
||||
msg.Ack()
|
||||
}
|
||||
|
||||
func TestDedup(t *testing.T) {
|
||||
// Publish same task twice; second publish should be deduped
|
||||
// Verify by checking PublishTask returns no error both times
|
||||
// and only one message is consumable
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 4: Add `ClaimByIssue` to Store
|
||||
|
||||
**File(s):** `/work/apps/archie-core/internal/store/store.go`, `/work/apps/archie-core/internal/store/store_test.go`
|
||||
|
||||
**Changes to `store.go`:**
|
||||
|
||||
Add a new method that atomically claims a task by its issue identity rather than the oldest-queued:
|
||||
|
||||
```go
|
||||
// ClaimByIssue atomically claims a queued task by owner/repo/issue_number.
|
||||
// Returns nil if the task is not in queued state (already claimed, parked,
|
||||
// or completed). This is used by the NATS consumer path where the task
|
||||
// was just inserted via EnqueueIssue and needs to be claimed immediately.
|
||||
func (s *Store) ClaimByIssue(ctx context.Context, owner, repo string, number int) (*Task, error) {
|
||||
row := s.db.QueryRowContext(ctx, `
|
||||
UPDATE tasks SET status='running', attempt=attempt+1, updated_at=datetime('now')
|
||||
WHERE owner=? AND repo=? AND issue_number=? AND status='queued'
|
||||
RETURNING id, owner, repo, issue_number, title, body, labels, status,
|
||||
workflow, stage, branch, plan, notes, pr_number, tokens_used,
|
||||
iterations, attempt, park_reason`, owner, repo, number)
|
||||
t, err := scanTask(row)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, nil
|
||||
}
|
||||
return t, err
|
||||
}
|
||||
```
|
||||
|
||||
**Why a new method instead of reusing ClaimNext?** Because ClaimNext claims the *oldest* `queued` row. When multiple tasks are in the queue (e.g., after a daemon restart), claiming the one we just inserted is not guaranteed. ClaimByIssue targets the exact row.
|
||||
|
||||
**Test in `store_test.go`:**
|
||||
|
||||
Add `TestClaimByIssue`:
|
||||
```go
|
||||
func TestClaimByIssue(t *testing.T) {
|
||||
s := openTest(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Enqueue two tasks
|
||||
s.EnqueueIssue(ctx, "sam", "todo", 1, "t1", "", "bug")
|
||||
s.EnqueueIssue(ctx, "sam", "todo", 2, "t2", "", "feature")
|
||||
|
||||
// Claim task 2 by issue (not the oldest)
|
||||
task, err := s.ClaimByIssue(ctx, "sam", "todo", 2)
|
||||
if err != nil || task == nil {
|
||||
t.Fatalf("ClaimByIssue = (%v, %v)", task, err)
|
||||
}
|
||||
if task.IssueNumber != 2 || task.Status != StatusRunning {
|
||||
t.Fatalf("unexpected task: %+v", task)
|
||||
}
|
||||
|
||||
// Claiming the same issue again should return nil
|
||||
task2, err := s.ClaimByIssue(ctx, "sam", "todo", 2)
|
||||
if err != nil || task2 != nil {
|
||||
t.Fatalf("second claim must return nil, got (%v, %v)", task2, err)
|
||||
}
|
||||
|
||||
// ClaimNext should now pick up task 1 (still queued)
|
||||
task3, _ := s.ClaimNext(ctx)
|
||||
if task3 == nil || task3.IssueNumber != 1 {
|
||||
t.Fatalf("expected task 1, got %+v", task3)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 5: Modify Daemon to Support NATS
|
||||
|
||||
**File(s):** `/work/apps/archie-core/internal/daemon/daemon.go`, `/work/apps/archie-core/internal/daemon/daemon_test.go`
|
||||
|
||||
#### 5a. Add NATS Field to Daemon
|
||||
|
||||
```go
|
||||
import (
|
||||
// ... existing imports ...
|
||||
"github.com/samcharles93/archie-core/internal/nats"
|
||||
)
|
||||
|
||||
type Daemon struct {
|
||||
// ... all existing fields ...
|
||||
Nats *nats.Client // nil means NATS not configured; use SQLite path
|
||||
}
|
||||
```
|
||||
|
||||
#### 5b. Modify `poll()` for NATS Publish Path
|
||||
|
||||
The `poll()` method needs a branch for NATS. When `d.Nats != nil`, publish to NATS instead of calling `Store.EnqueueIssue`. The ack reaction, state label setting, and event emission remain the same -- they reflect discovery, not storage.
|
||||
|
||||
Refactored `poll()`:
|
||||
```go
|
||||
func (d *Daemon) poll(ctx context.Context) {
|
||||
for _, repo := range d.Cfg.Repos {
|
||||
issues := d.pollIssues(ctx, repo)
|
||||
for _, is := range issues {
|
||||
inserted := d.tryPublish(ctx, repo, is)
|
||||
if !inserted {
|
||||
d.maybeRetryParked(ctx, repo, is)
|
||||
continue
|
||||
}
|
||||
d.Log.Info("issue queued", "repo", repo.FullName(), "issue", is.Number, "title", is.Title)
|
||||
if ack := d.Cfg.Dispatch.AckReaction; ack != "" {
|
||||
if err := d.Forge.React(ctx, repo.Owner, repo.Name, is.Number, ack); err != nil {
|
||||
d.Log.Warn("ack reaction failed", "issue", is.Number, "err", err)
|
||||
}
|
||||
}
|
||||
d.Forge.SetStateLabel(ctx, repo.Owner, repo.Name, is.Number, d.Cfg.Dispatch.StateLabel("queued"), d.Cfg.Dispatch.LabelValues())
|
||||
d.emit(events.Event{
|
||||
Kind: events.KindTaskQueued, Repo: repo.FullName(),
|
||||
Issue: is.Number, Detail: is.Title,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// tryPublish returns true if this issue was accepted (either enqueued in SQLite
|
||||
// or published to NATS) for the first time. False means it was already tracked.
|
||||
func (d *Daemon) tryPublish(ctx context.Context, repo config.Repo, is forge.Issue) bool {
|
||||
labels := strings.Join(is.Labels, ",")
|
||||
|
||||
if d.Nats != nil {
|
||||
// When NATS is configured, we must check if the issue is already known
|
||||
// before publishing -- we still need maybeRetryParked to work. The
|
||||
// NATS Msg-Id dedup prevents duplicate messages but we want to detect
|
||||
// the "already tracked" case here to allow retry-parked detection.
|
||||
existing, err := d.Store.TaskByIssue(ctx, repo.Owner, repo.Name, is.Number)
|
||||
if err != nil {
|
||||
d.Log.Error("check existing failed", "repo", repo.FullName(), "issue", is.Number, "err", err)
|
||||
return false
|
||||
}
|
||||
if existing != nil {
|
||||
return false // already tracked
|
||||
}
|
||||
|
||||
if err := d.Nats.PublishTask(ctx, repo.Owner, repo.Name, is.Number, is.Title, is.Body, labels); err != nil {
|
||||
d.Log.Error("nats publish failed", "repo", repo.FullName(), "issue", is.Number, "err", err)
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// SQLite path (unchanged logic, extracted for clarity)
|
||||
inserted, err := d.Store.EnqueueIssue(ctx,
|
||||
repo.Owner, repo.Name, is.Number, is.Title, is.Body, labels)
|
||||
if err != nil {
|
||||
d.Log.Error("enqueue failed", "repo", repo.FullName(), "issue", is.Number, "err", err)
|
||||
return false
|
||||
}
|
||||
return inserted
|
||||
}
|
||||
```
|
||||
|
||||
Note: For the NATS path, we call `Store.TaskByIssue` first. This is a read-only check that only reads, never writes. The actual storage happens in the NATS consumer. This preserves the `maybeRetryParked` behavior.
|
||||
|
||||
#### 5c. Modify `Cycle()` for NATS Consumer
|
||||
|
||||
```go
|
||||
func (d *Daemon) Cycle(ctx context.Context) {
|
||||
d.poll(ctx)
|
||||
d.reconcilePRs(ctx)
|
||||
d.checkWaiting(ctx)
|
||||
if d.Nats != nil {
|
||||
d.drainNATS(ctx)
|
||||
} else {
|
||||
d.drainSQLite(ctx)
|
||||
}
|
||||
}
|
||||
|
||||
// drainSQLite processes queued tasks from SQLite (existing behavior).
|
||||
func (d *Daemon) drainSQLite(ctx context.Context) {
|
||||
for {
|
||||
task, err := d.Store.ClaimNext(ctx)
|
||||
if err != nil {
|
||||
d.Log.Error("claim failed", "err", err)
|
||||
return
|
||||
}
|
||||
if task == nil {
|
||||
return
|
||||
}
|
||||
d.process(ctx, task)
|
||||
}
|
||||
}
|
||||
|
||||
// drainNATS processes tasks from NATS, falling back to SQLite for
|
||||
// requeued tasks (waiting_human, retry-parked).
|
||||
func (d *Daemon) drainNATS(ctx context.Context) {
|
||||
for {
|
||||
// Try NATS first
|
||||
msg, err := d.Nats.Fetch(ctx)
|
||||
if err != nil {
|
||||
d.Log.Error("nats fetch failed", "err", err)
|
||||
return
|
||||
}
|
||||
if msg != nil {
|
||||
d.processNATSTask(ctx, msg)
|
||||
// After processing, try again immediately
|
||||
continue
|
||||
}
|
||||
|
||||
// Fall back to SQLite for requeued tasks that came via direct
|
||||
// calls to Store.Requeue (waiting_human approval, retry-parked).
|
||||
task, err := d.Store.ClaimNext(ctx)
|
||||
if err != nil {
|
||||
d.Log.Error("sqlite claim failed", "err", err)
|
||||
return
|
||||
}
|
||||
if task == nil {
|
||||
return
|
||||
}
|
||||
d.process(ctx, task)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Important:** The dual-drain approach means newly discovered tasks come through NATS and requeued tasks (waiting_human approval, retry-parked) come through SQLite ClaimNext. This is clean and avoids needing to publish NATS messages for every requeue.
|
||||
|
||||
#### 5d. Add `processNATSTask`
|
||||
|
||||
```go
|
||||
func (d *Daemon) processNATSTask(ctx context.Context, msg jetstream.Msg) {
|
||||
// Decode
|
||||
tm, err := nats.DecodeTask(msg)
|
||||
if err != nil {
|
||||
d.Log.Error("nats decode failed", "err", err)
|
||||
msg.Ack() // bad message, don't retry
|
||||
return
|
||||
}
|
||||
|
||||
// Store in SQLite (idempotent — ON CONFLICT DO NOTHING)
|
||||
inserted, err := d.Store.EnqueueIssue(ctx, tm.Owner, tm.Repo, tm.Number, tm.Title, tm.Body, tm.Labels)
|
||||
if err != nil {
|
||||
d.Log.Error("nats enqueue failed", "err", err)
|
||||
msg.Nak()
|
||||
return
|
||||
}
|
||||
|
||||
// Claim the task by identity
|
||||
task, err := d.Store.ClaimByIssue(ctx, tm.Owner, tm.Repo, tm.Number)
|
||||
if err != nil {
|
||||
d.Log.Error("nats claim failed", "err", err)
|
||||
msg.Nak()
|
||||
return
|
||||
}
|
||||
if task == nil {
|
||||
// Task already claimed by another consumer (multi-daemon scenario)
|
||||
// or in a terminal state. Ack and move on.
|
||||
msg.Ack()
|
||||
return
|
||||
}
|
||||
|
||||
// Process (this sets labels, runs workflow, records outcome)
|
||||
d.process(ctx, task)
|
||||
|
||||
// Ack after processing completes.
|
||||
// If the daemon crashes before ack, the NATS message is redelivered.
|
||||
// EnqueueIssue's idempotency + ClaimByIssue's nil-on-not-queued guard
|
||||
// prevents double-processing.
|
||||
msg.Ack()
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 6: Wire NATS in `cmd/archied/main.go`
|
||||
|
||||
**File(s):** `/work/apps/archie-core/cmd/archied/main.go`
|
||||
|
||||
**Changes:**
|
||||
|
||||
Add import for the nats package:
|
||||
```go
|
||||
"github.com/samcharles93/archie-core/internal/nats"
|
||||
```
|
||||
|
||||
Between store opening and daemon construction, add:
|
||||
```go
|
||||
// NATS client (optional)
|
||||
var natsClient *nats.Client
|
||||
if cfg.NATS.URL != "" {
|
||||
natsClient, err = nats.Connect(ctx, cfg.NATS.URL, log)
|
||||
if err != nil {
|
||||
log.Error("nats connect failed", "err", err)
|
||||
return 1
|
||||
}
|
||||
defer natsClient.Close()
|
||||
log.Info("nats connected", "url", cfg.NATS.URL)
|
||||
}
|
||||
```
|
||||
|
||||
Add to Daemon construction:
|
||||
```go
|
||||
d := &daemon.Daemon{
|
||||
// ... all existing fields ...
|
||||
Nats: natsClient,
|
||||
}
|
||||
```
|
||||
|
||||
The `--once` flag path should still work: `d.Cycle(ctx)` handles both NATS and SQLite paths. When NATS is configured and `--once` is used, the Cycle does poll + drainNATS once. Since Fetch has a 2-second timeout, `drainNATS` will return after the timeout if no message is available, which is fine for the one-shot case.
|
||||
|
||||
---
|
||||
|
||||
### Step 7: Integration Verification
|
||||
|
||||
**Approach:** Manual end-to-end test with a local NATS server.
|
||||
|
||||
1. Start `nats-server` locally (or via Docker: `docker run -p 4222:4222 nats`)
|
||||
2. Create a `config.toml` with:
|
||||
```toml
|
||||
bot_user = "test-bot"
|
||||
[nats]
|
||||
url = "nats://localhost:4222"
|
||||
# ... other required fields
|
||||
```
|
||||
3. Run `archied --once`
|
||||
4. Verify:
|
||||
- `nats pub` / `nats sub` shows the task subject messages
|
||||
- NATS JetStream stream `ARCHIE_TASKS` exists (`nats stream list`)
|
||||
- The task appears in SQLite after the consumer processes it
|
||||
- Workflow completes normally
|
||||
|
||||
**Automated integration test (future, not Phase 1):** A test that starts an embedded NATS server, creates a daemon with NATS configured, simulates an issue poll, and verifies the full cycle. Phase 1 keeps tests at the unit level.
|
||||
|
||||
---
|
||||
|
||||
### Summary of All Changes
|
||||
|
||||
| Step | File | Change |
|
||||
|------|------|--------|
|
||||
| 1 | `go.mod`, `go.sum` | Add `github.com/nats-io/nats.go` |
|
||||
| 2 | `internal/config/config.go` | Add `NATSConfig` struct, `NATS` field to `Config` |
|
||||
| 2 | `internal/config/config_test.go` | Test NATS config loading |
|
||||
| 2 | `config.example.toml` | Add commented-out `[nats]` block |
|
||||
| 3a | `internal/nats/subjects.go` | Subject constants + `SubjectForLabels` |
|
||||
| 3b | `internal/nats/client.go` | `Client` struct, `Connect`, `Close`, `PublishTask`, `Fetch`, `DecodeTask` |
|
||||
| 3b | `internal/nats/client_test.go` | Unit tests with embedded NATS server |
|
||||
| 4 | `internal/store/store.go` | Add `ClaimByIssue` method |
|
||||
| 4 | `internal/store/store_test.go` | Test `ClaimByIssue` |
|
||||
| 5 | `internal/daemon/daemon.go` | Add `Nats` field, refactor `poll()`, add `drainNATS`/`drainSQLite`/`processNATSTask`, modify `Cycle()` |
|
||||
| 5 | `internal/daemon/daemon_test.go` | No changes needed for Phase 1 (existing tests unchanged) |
|
||||
| 6 | `cmd/archied/main.go` | Wire `nats.Connect`, pass client to Daemon |
|
||||
|
||||
### Design Decisions and Rationale
|
||||
|
||||
1. **Dual inner loop (NATS then SQLite) for Phase 1.** Rather than modifying `Requeue` to also publish NATS messages, the NATS drain falls back to ClaimNext for requeued tasks. Simple, works, and the fallback disappears in a later phase when requeue itself publishes NATS messages.
|
||||
|
||||
2. **Msg-Id dedup instead of SQL UNIQUE for NATS path.** The existing `EnqueueIssue` with `ON CONFLICT DO NOTHING` is the idempotency mechanism in the consumer. The NATS Msg-Id dedup is a first-line defense to avoid storing duplicate messages at all. Together they form a robust dedup chain.
|
||||
|
||||
3. **Workflow subject routing duplicates `Route()` logic.** The `SubjectForLabels` function mirrors `workflow.Route`'s label-to-workflow mapping. This is intentional -- the subject encodes the workflow type so future multi-daemon setups can filter by subject. The duplication is minimal (one small function) and is kept in the `nats` package to avoid a dependency on the `workflow` package.
|
||||
|
||||
4. **`ClaimByIssue` is a targeted query, not a general claim.** It only claims a task by the exact owner/repo/number triple and only when it's in `queued` status. This is safe for the NATS consumer where we just inserted the task. It also works as a lock: two consumers racing on the same message will see one succeed and one get nil.
|
||||
|
||||
5. **Three retries, 5-minute ack wait.** NATS JetStream redelivers unacked messages. With `MaxDeliver: 3`, a failing task is retried twice after the first attempt. After 3 failures, the message goes to the NA pile (JetStream's internal dead state for work queues). This is a safety net, not a permanent error -- eventually the operator checks the NATS dead-letter state or we add a dedicated handler.
|
||||
|
||||
6. **Ack after `process()` completes, even on error.** If the workflow parks the task, that's a successful outcome from the task distribution perspective -- the message is acked and removed from the stream. Only daemon crashes leave the message unacked.
|
||||
|
||||
### Potential Challenges
|
||||
|
||||
1. **Embedded NATS server for tests.** Go tests that need a real NATS server typically use `github.com/nats-io/nats-server/v2/server` embedded mode. This adds a test-only dependency. Alternative: use a `nats-server` binary in CI. For Phase 1, the test file should use an embedded server and note it in the build tag if the binary isn't available.
|
||||
|
||||
2. **`drainNATS` blocks on Fetch.** `Fetch` has a 2-second poll timeout before returning nil. When the queue is empty, the NATS drain exits (returns nil) and the Cycle ends, which is correct. The ticker in `Run()` creates a new Cycle on the next tick, so empty-queue idle detection is fine.
|
||||
|
||||
3. **`SubjectForLabels` vs `Route()` divergence risk.** If someone adds a new label-to-workflow mapping in `Route()`, they need to update `SubjectForLabels` too. Mitigation: keep both functions small, add a comment referencing the other, and add a cross-package test that verifies they stay in sync on the known label set.
|
||||
|
||||
---
|
||||
|
||||
### Critical Files for Implementation
|
||||
- `/work/apps/archie-core/internal/daemon/daemon.go` — Core changes: poll branching, Cycle splitting, NATS consumer logic, new helper methods
|
||||
- `/work/apps/archie-core/internal/config/config.go` — Add NATSConfig struct and NATS field to Config
|
||||
- `/work/apps/archie-core/internal/nats/client.go` — New package: NATS connection management, publish, consume, JetStream setup
|
||||
- `/work/apps/archie-core/internal/store/store.go` — Add ClaimByIssue for targeted task claiming
|
||||
- `/work/apps/archie-core/cmd/archied/main.go` — Wire NATS client startup and inject into Daemon
|
||||
+156
-97
@@ -1,25 +1,27 @@
|
||||
# archie-core Architecture
|
||||
|
||||
**archied** is a resident daemon that polls GitHub for issues labelled `archie`, works each one in an isolated worktree through a routed workflow, and opens pull requests for human review.
|
||||
**archied** is a resident daemon that polls GitHub for issues labelled `archie`,
|
||||
works each one in an isolated worktree through a routed workflow, and opens pull
|
||||
requests for human review.
|
||||
|
||||
## Package Map
|
||||
|
||||
| Package | Role |
|
||||
|---|---|
|
||||
| `cmd/archied/` | Binary entry: config→forge→store→runtime→workflows→daemon |
|
||||
| `cmd/archie-agent/` | One-invocation autonomous stage worker (JSON stdin/stdout) |
|
||||
| `internal/config/` | TOML config, per-repo gates, budgets, providers, ecosystems |
|
||||
| `internal/daemon/` | Resident loop: poll, enqueue, claim, route, process, reconcile |
|
||||
| `internal/workflow/` | Engine: stage lists, routing, shared steps, workflow definitions |
|
||||
| Package | Role |
|
||||
| --------------------- | ------------------------------------------------------------------------- |
|
||||
| `cmd/archied/` | Binary entry: config→forge→store→runtime→workflows→daemon |
|
||||
| `cmd/archie-agent/` | One-invocation autonomous stage worker (JSON stdin/stdout) |
|
||||
| `internal/config/` | TOML config, per-repo gates, budgets, providers, ecosystems |
|
||||
| `internal/daemon/` | Resident loop: poll, enqueue, claim, route, process, reconcile |
|
||||
| `internal/workflow/` | Engine: stage lists, routing, shared steps, workflow definitions |
|
||||
| `internal/agentexec/` | Versioned agent-stage protocol, worker, and in-process/subprocess runners |
|
||||
| `internal/forge/` | GitHub interface: issues, labels, PRs, comments, reactions |
|
||||
| `internal/worktree/` | Git operations: clone, branch, commit, push, diff, cleanup |
|
||||
| `internal/store/` | SQLite: task rows, status transitions, crash recovery |
|
||||
| `internal/events/` | In-process event bus: publish/subscribe for observability |
|
||||
| `internal/webui/` | SSE dashboard: live event stream, task status |
|
||||
| `ai-sdk/runtime` | External: provider-agnostic LLM runtime |
|
||||
| `ai-sdk/agentloop` | External: gated agent loop with budgets and tool jails |
|
||||
| `ai-sdk/core` | External: tool definitions, generate options |
|
||||
| `internal/forge/` | GitHub interface: issues, labels, PRs, comments, reactions |
|
||||
| `internal/worktree/` | Git operations: clone, branch, commit, push, diff, cleanup |
|
||||
| `internal/store/` | SQLite: task rows, status transitions, crash recovery |
|
||||
| `internal/events/` | In-process event bus: publish/subscribe for observability |
|
||||
| `internal/webui/` | SSE dashboard: live event stream, task status |
|
||||
| `ai-sdk/runtime` | External: provider-agnostic LLM runtime |
|
||||
| `ai-sdk/agentloop` | External: gated agent loop with budgets and tool jails |
|
||||
| `ai-sdk/core` | External: tool definitions, generate options |
|
||||
|
||||
## Task Lifecycle
|
||||
|
||||
@@ -31,9 +33,11 @@ queued → running(workflow:stage) → pr_open → merged | rejected
|
||||
parked
|
||||
```
|
||||
|
||||
**Crash recovery:** tasks left in `running` are re-queued on startup. Parks are never silent -- every park posts a comment with the reason and gate output.
|
||||
**Crash recovery:** tasks left in `running` are re-queued on startup. Parks are
|
||||
never silent -- every park posts a comment with the reason and gate output.
|
||||
|
||||
**PR reconciliation:** the daemon polls open PRs, checks GitHub state (`merged`/`closed`), transitions the task, and cleans up worktrees.
|
||||
**PR reconciliation:** the daemon polls open PRs, checks GitHub state
|
||||
(`merged`/`closed`), transitions the task, and cleans up worktrees.
|
||||
|
||||
## Workflow Engine
|
||||
|
||||
@@ -41,13 +45,16 @@ queued → running(workflow:stage) → pr_open → merged | rejected
|
||||
|
||||
- `Workflow` = named list of `Stage`s
|
||||
- `Stage` = `Name` + `Run(ctx, *TaskContext) error`
|
||||
- `TaskContext` = mutable bag: Task, Repo, Config, Forge, Store, Agent runner, worktree dir/branch, scratch fields
|
||||
- `Outcome` = terminal decision (status + detail) -- ends the workflow immediately
|
||||
- `TaskContext` = mutable bag: Task, Repo, Config, Forge, Store, Agent runner,
|
||||
worktree dir/branch, scratch fields
|
||||
- `Outcome` = terminal decision (status + detail) -- ends the workflow
|
||||
immediately
|
||||
- `Registry` = `map[string]Workflow`
|
||||
|
||||
### Routing (`Route`)
|
||||
|
||||
Label-driven, no LLM involved. Pre-assigned workflow wins; then labels decide:
|
||||
|
||||
- `bug` → `tdd`
|
||||
- `feature` → `feasibility`
|
||||
- `bootstrap` → `bootstrap` (diagnostics, no LLM spend)
|
||||
@@ -56,12 +63,14 @@ Label-driven, no LLM involved. Pre-assigned workflow wins; then labels decide:
|
||||
### Execution (`Run`)
|
||||
|
||||
Stages execute sequentially. After each stage:
|
||||
|
||||
- Task is persisted to SQLite
|
||||
- Error → park the task (comment on issue, keep worktree)
|
||||
- Outcome set → finish (transition task, update labels)
|
||||
- Neither → continue to next stage
|
||||
|
||||
A workflow that ends without an outcome parks -- definition bugs must not vanish silently.
|
||||
A workflow that ends without an outcome parks -- definition bugs must not vanish
|
||||
silently.
|
||||
|
||||
## Workflows
|
||||
|
||||
@@ -71,7 +80,8 @@ A workflow that ends without an outcome parks -- definition bugs must not vani
|
||||
prepare → apply(marker file) → commit-push → diff-cap → open-pr
|
||||
```
|
||||
|
||||
Proves the plumbing: clone, push, PR. Label any test issue `bootstrap` to verify end-to-end.
|
||||
Proves the plumbing: clone, push, PR. Label any test issue `bootstrap` to verify
|
||||
end-to-end.
|
||||
|
||||
### Implement (default)
|
||||
|
||||
@@ -79,9 +89,12 @@ Proves the plumbing: clone, push, PR. Label any test issue `bootstrap` to verify
|
||||
prepare → baseline-gate → plan(planner, read-only) → build(builder, gated) → commit-push → diff-cap → open-pr
|
||||
```
|
||||
|
||||
- **baseline-gate:** runs the repo's gate at the base commit before anything is touched
|
||||
- **plan:** read-only agent explores codebase, produces implementation plan, posts it as a comment
|
||||
- **build:** full toolset agent implements against the plan behind the repo's quality gate
|
||||
- **baseline-gate:** runs the repo's gate at the base commit before anything is
|
||||
touched
|
||||
- **plan:** read-only agent explores codebase, produces implementation plan,
|
||||
posts it as a comment
|
||||
- **build:** full toolset agent implements against the plan behind the repo's
|
||||
quality gate
|
||||
- **diff-cap:** parks oversized changes (>400 lines default)
|
||||
- **open-pr:** builder's summary becomes the PR body
|
||||
|
||||
@@ -91,10 +104,13 @@ prepare → baseline-gate → plan(planner, read-only) → build(builder, gated)
|
||||
prepare → baseline-gate → analyse(planner) → repro-tests(builder, inverted gate) → capture-proof → commit-repro → fix(builder, normal gate, test files protected) → commit-push → diff-cap → open-pr + repro-evidence
|
||||
```
|
||||
|
||||
- **repro-tests:** the quality gate is **inverted** -- the test command must FAIL. Proven failure is required.
|
||||
- **capture-proof:** runs the test command deterministically, captures the failing output
|
||||
- **repro-tests:** the quality gate is **inverted** -- the test command must
|
||||
FAIL. Proven failure is required.
|
||||
- **capture-proof:** runs the test command deterministically, captures the
|
||||
failing output
|
||||
- **commit-repro:** commits the failing tests as the first commit
|
||||
- **fix:** test files are **environmentally write-protected** (not prompt-ruled). The agent fixes code, not tests.
|
||||
- **fix:** test files are **environmentally write-protected** (not
|
||||
prompt-ruled). The agent fixes code, not tests.
|
||||
- **open-pr + evidence:** posts both the PR and the captured failure proof
|
||||
|
||||
### Feasibility (feature label)
|
||||
@@ -103,121 +119,163 @@ prepare → baseline-gate → analyse(planner) → repro-tests(builder, inverted
|
||||
prepare → assess(planner, with decide tool) → [close_if_bad_fit | prd(planner)] → deliver → waiting_human
|
||||
```
|
||||
|
||||
- **assess:** reads AGENT.md, ROADMAP.md, README. Calls a `decide` tool (structured output, not parsed prose). Bad fit → close with reasons.
|
||||
- **prd:** read-only agent writes a PRD: problem, solution, files affected, acceptance criteria, non-goals, diff estimate
|
||||
- **deliver:** posts PRD as issue comment + webhook notification (n8n → email). Sets state to `waiting_human`.
|
||||
- The daemon watches for human replies, judges them with a triage LLM (APPROVE/REJECT/UNCLEAR), requeues or closes.
|
||||
- **assess:** reads AGENT.md, ROADMAP.md, README. Calls a `decide` tool
|
||||
(structured output, not parsed prose). Bad fit → close with reasons.
|
||||
- **prd:** read-only agent writes a PRD: problem, solution, files affected,
|
||||
acceptance criteria, non-goals, diff estimate
|
||||
- **deliver:** posts PRD as issue comment + webhook notification (n8n → email).
|
||||
Sets state to `waiting_human`.
|
||||
- The daemon watches for human replies, judges them with a triage LLM
|
||||
(APPROVE/REJECT/UNCLEAR), requeues or closes.
|
||||
|
||||
## Forge Interface
|
||||
|
||||
A thin abstraction over GitHub (Gitea planned):
|
||||
|
||||
| Method | Purpose |
|
||||
|---|---|
|
||||
| `AcceptInvitations` | Auto-accept repo invites |
|
||||
| `AssignedIssues` | Poll assigned issues (excluding PRs) |
|
||||
| `IssuesWithLabel` | Poll labelled issues |
|
||||
| `Comment` | Post issue/PR comment, return ID |
|
||||
| `RepliesAfter` | Find human replies after a comment ID |
|
||||
| `CreatePR` | Open pull request |
|
||||
| `PRState` | Check merged/open/closed |
|
||||
| `CloseIssue` | Close with final comment |
|
||||
| `React` | Emoji reaction (pickup acknowledgement) |
|
||||
| `SetStateLabel` | Replace old state label with new one |
|
||||
| `VerifyPush` | Confirm push permission at startup |
|
||||
| Method | Purpose |
|
||||
| ------------------- | --------------------------------------- |
|
||||
| `AcceptInvitations` | Auto-accept repo invites |
|
||||
| `AssignedIssues` | Poll assigned issues (excluding PRs) |
|
||||
| `IssuesWithLabel` | Poll labelled issues |
|
||||
| `Comment` | Post issue/PR comment, return ID |
|
||||
| `RepliesAfter` | Find human replies after a comment ID |
|
||||
| `CreatePR` | Open pull request |
|
||||
| `PRState` | Check merged/open/closed |
|
||||
| `CloseIssue` | Close with final comment |
|
||||
| `React` | Emoji reaction (pickup acknowledgement) |
|
||||
| `SetStateLabel` | Replace old state label with new one |
|
||||
| `VerifyPush` | Confirm push permission at startup |
|
||||
|
||||
State labels (`archie:queued`, `archie:working`, etc.) are created on demand, removed when transitioning.
|
||||
State labels (`archie:queued`, `archie:working`, etc.) are created on demand,
|
||||
removed when transitioning.
|
||||
|
||||
## Worktree Manager
|
||||
|
||||
Every task gets a fresh clone -- no persistent state between runs. The model never runs git.
|
||||
Every task gets a fresh clone -- no persistent state between runs. The model
|
||||
never runs git.
|
||||
|
||||
```
|
||||
~/.local/share/archie/work/<owner>-<repo>/issue-<N>/
|
||||
```
|
||||
|
||||
Token auth via a generated `GIT_ASKPASS` helper script -- the token never appears in `.git/config` or process argv.
|
||||
Token auth via a generated `GIT_ASKPASS` helper script -- the token never
|
||||
appears in `.git/config` or process argv.
|
||||
|
||||
## Config
|
||||
|
||||
Daemon-level TOML (`~/.config/archie/config.toml`):
|
||||
|
||||
- `[forge]` -- type (github), host, token secret reference
|
||||
- `[dispatch]` -- trigger (assignee/label/either), labels, ack reaction
|
||||
- `[[repos]]` -- owner, name, base branch, gate commands, protected paths, ecosystem
|
||||
- `[models]` -- planner, builder, triage (provider/model refs)
|
||||
- `[providers]` -- LLM provider configs
|
||||
- `[agent]` -- execution mode (`inprocess` or `subprocess`), worker command, environment allowlist
|
||||
- `[budgets]` -- max steps, max tokens, wall clock, gate max failures
|
||||
- `[web]` -- dashboard listen address
|
||||
- `[notify]` -- webhook URL for notifications
|
||||
- `[forge]` -- type (github), host, token secret reference
|
||||
- `[dispatch]` -- trigger (assignee/label/either), labels, ack reaction
|
||||
- `[[repos]]` -- owner, name, base branch, gate commands, protected paths,
|
||||
ecosystem
|
||||
- `[models]` -- planner, builder, triage (provider/model refs)
|
||||
- `[providers]` -- LLM provider configs
|
||||
- `[agent]` -- execution mode (`inprocess` or `subprocess`), worker command,
|
||||
environment allowlist
|
||||
- `[budgets]` -- max steps, max tokens, wall clock, gate max failures
|
||||
- `[web]` -- dashboard listen address
|
||||
- `[notify]` -- webhook URL for notifications
|
||||
|
||||
Secrets (forge token, provider API keys, channel tokens) are not embedded in config files. `internal/secret` defines a pluggable `Engine` interface (`Name`, `Version`, `Resolve(key) (string, error)`) resolved through a `Registry`; the built-in `env` engine reads an environment variable by name. Custom engines (sops, Bitwarden Secrets Manager, Vault/OpenBao) are Yaegi-interpreted `.go` plugins loaded from a directory, following the same convention as `internal/plugin`'s daemon plugins. Config fields reference a secret as `{engine, key}` (`secret.SecretRef`), e.g. `token = { engine = "env", key = "ARCHIE_GITHUB_TOKEN" }`. As of this writing only `Forge.Token` has been migrated to `SecretRef`; `Provider.APIKeyEnv`, `NATSConfig.TokenEnv`, and `TelegramConfig.TokenEnv` are still plain env-var-name strings pending a follow-up (they're forwarded by name to `ai-sdk/runtime` and subprocess workers rather than resolved centrally, which needs its own design pass before migrating).
|
||||
Secrets (forge token, provider API keys, channel tokens) are not embedded in
|
||||
config files. `internal/secret` defines a pluggable `Engine` interface (`Name`,
|
||||
`Version`, `Resolve(key) (string, error)`) resolved through a `Registry`; the
|
||||
built-in `env` engine reads an environment variable by name. Custom engines
|
||||
(sops, Bitwarden Secrets Manager, Vault/OpenBao) are Yaegi-interpreted `.go`
|
||||
plugins loaded from a directory, following the same convention as
|
||||
`internal/plugin`'s daemon plugins. Config fields reference a secret as
|
||||
`{engine, key}` (`secret.SecretRef`), e.g.
|
||||
`token = { engine = "env", key = "ARCHIE_GITHUB_TOKEN" }`. As of this writing
|
||||
only `Forge.Token` has been migrated to `SecretRef`; `Provider.APIKeyEnv`,
|
||||
`NATSConfig.TokenEnv`, and `TelegramConfig.TokenEnv` are still plain
|
||||
env-var-name strings pending a follow-up (they're forwarded by name to
|
||||
`ai-sdk/runtime` and subprocess workers rather than resolved centrally, which
|
||||
needs its own design pass before migrating).
|
||||
|
||||
Subprocess mode is a migration transport boundary, not a security sandbox: the worker still runs under the daemon UID and can access daemon-readable host resources. The "daemon never runs untrusted code" boundary requires the later container runner with a separate user, a restricted filesystem and process namespace, and explicit mounts.
|
||||
Subprocess mode is a migration transport boundary, not a security sandbox: the
|
||||
worker still runs under the daemon UID and can access daemon-readable host
|
||||
resources. The "daemon never runs untrusted code" boundary requires the later
|
||||
container runner with a separate user, a restricted filesystem and process
|
||||
namespace, and explicit mounts.
|
||||
|
||||
Per-repo gates are command lists: `[[repos.gate]] = ["go", "vet", "./..."]`. The last command is the test runner (by convention -- TDD inverts it).
|
||||
Per-repo gates are command lists: `[[repos.gate]] = ["go", "vet", "./..."]`. The
|
||||
last command is the test runner (by convention -- TDD inverts it).
|
||||
|
||||
## Observability
|
||||
|
||||
Every lifecycle event flows through an internal event bus:
|
||||
|
||||
- SQLite event log (every event stamped with a row ID)
|
||||
- Live dashboard via SSE (server-sent events)
|
||||
- Bounded subscriber buffers -- slow consumers drop events, never block the daemon
|
||||
- Bounded subscriber buffers -- slow consumers drop events, never block the
|
||||
daemon
|
||||
|
||||
## Plugin engine rule (strict)
|
||||
|
||||
“Engine” is reserved for a typed, lifecycle-managed capability family. It is
|
||||
not a synonym for a generic plugin hook. Every new plugin engine must satisfy
|
||||
all of these invariants:
|
||||
“Engine” is reserved for a typed, lifecycle-managed capability family. It is not
|
||||
a synonym for a generic plugin hook. Every new plugin engine must satisfy all of
|
||||
these invariants:
|
||||
|
||||
1. **Typed domain contract.** The engine interface exposes real
|
||||
capability-specific operations in addition to identity metadata. Examples
|
||||
are resolving a secret, delivering a message, or running an agent. A plugin
|
||||
that only exposes `Name` and `Version` is metadata, not an engine.
|
||||
capability-specific operations in addition to identity metadata. Examples are
|
||||
resolving a secret, delivering a message, or running an agent. A plugin that
|
||||
only exposes `Name` and `Version` is metadata, not an engine.
|
||||
2. **Owning family manager.** The capability package owns a `Registry` or
|
||||
`Manager` that controls provider registration and discovery,
|
||||
initialization, or access. Providers do not register arbitrary callbacks
|
||||
directly on the daemon.
|
||||
3. **Lifecycle and health.** When an implementation owns resources or
|
||||
background work, its typed family contract must expose start, health, and
|
||||
stop semantics. The owning manager invokes them and defines failure
|
||||
isolation and shutdown ordering. Stateless providers may make these
|
||||
operations explicit no-ops.
|
||||
`Manager` that controls provider registration and discovery, initialization,
|
||||
or access. Providers do not register arbitrary callbacks directly on the
|
||||
daemon.
|
||||
3. **Lifecycle and health.** When an implementation owns resources or background
|
||||
work, its typed family contract must expose start, health, and stop
|
||||
semantics. The owning manager invokes them and defines failure isolation and
|
||||
shutdown ordering. Stateless providers may make these operations explicit
|
||||
no-ops.
|
||||
4. **Narrow host access.** Plugins receive the typed family registrar and only
|
||||
the host services their declared capability requires. They never receive
|
||||
`*daemon.Daemon`, a service locator, or an untyped hook map.
|
||||
5. **Metadata-only generic plugin contract.** `plugin.Plugin` remains limited
|
||||
to `Name()` and `Version()`. Domain hooks belong on typed engine interfaces;
|
||||
5. **Metadata-only generic plugin contract.** `plugin.Plugin` remains limited to
|
||||
`Name()` and `Version()`. Domain hooks belong on typed engine interfaces;
|
||||
adding capability methods to the generic interface is forbidden.
|
||||
6. **Trust boundary stays explicit.** Explicitly trusted, operator-installed
|
||||
plugins—including third-party secret-engine plugins the operator chooses
|
||||
to trust—may run in-process and have daemon privileges. Repository-supplied
|
||||
code runs in the task container. Integrations that are not trusted as
|
||||
daemon code run out of process through MCP or a versioned protocol and, when
|
||||
a confidentiality or integrity boundary is required, inside a real
|
||||
container sandbox. A subprocess boundary alone is not a security sandbox,
|
||||
and calling Yaegi-loaded code a plugin does not isolate it.
|
||||
plugins—including third-party secret-engine plugins the operator chooses to
|
||||
trust—may run in-process and have daemon privileges. Repository-supplied code
|
||||
runs in the task container. Integrations that are not trusted as daemon code
|
||||
run out of process through MCP or a versioned protocol and, when a
|
||||
confidentiality or integrity boundary is required, inside a real container
|
||||
sandbox. A subprocess boundary alone is not a security sandbox, and calling
|
||||
Yaegi-loaded code a plugin does not isolate it.
|
||||
|
||||
`internal/plugin/architecture_test.go` enforces the mechanically testable
|
||||
parts of this rule: the generic plugin method set, the shared agent-instruction
|
||||
source, and the requirement that engine interfaces have domain behavior plus
|
||||
an owning registry or manager.
|
||||
`internal/plugin/architecture_test.go` enforces the mechanically testable parts
|
||||
of this rule: the generic plugin method set, the shared agent-instruction
|
||||
source, and the requirement that engine interfaces have domain behavior plus an
|
||||
owning registry or manager.
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
1. **Environmental constraints over prompt rules.** The gate, test-file protection, and diff cap are code enforcement -- not "please don't do that" in the system prompt.
|
||||
2. **The model never runs git.** Worktree operations are deterministic steps, not agent tools.
|
||||
3. **Tokens as cost, not features.** Token usage is tracked per task and emitted in observability events but never budgeted beyond the agent loop's own limits.
|
||||
4. **Label-driven routing.** Workflow selection is a switch statement, not an LLM call. A triage LLM is used only for the human reply judgment in waiting_human.
|
||||
5. **Shared code with tau at the module level.** `ai-sdk/agentloop` and `ai-sdk/toolkit` are extracted from tau's coordinator and promoted to shared libraries.
|
||||
6. **No daemon-internal cron.** Polling is a ticker loop, not cron. The feasibility PRD notification uses an external webhook (n8n).
|
||||
7. **Fresh worktree per task.** No persistent state between runs. Crash recovery just re-queues.
|
||||
8. **Agent execution is a data boundary.** Autonomous stages receive a versioned request and return a fenced result; SQLite, forge operations, git, workflow sequencing, and outcome handling stay daemon-owned.
|
||||
1. **Environmental constraints over prompt rules.** The gate, test-file
|
||||
protection, and diff cap are code enforcement -- not "please don't do that"
|
||||
in the system prompt.
|
||||
2. **The model never runs git.** Worktree operations are deterministic steps,
|
||||
not agent tools.
|
||||
3. **Tokens as cost, not features.** Token usage is tracked per task and emitted
|
||||
in observability events but never budgeted beyond the agent loop's own
|
||||
limits.
|
||||
4. **Label-driven routing.** Workflow selection is a switch statement, not an
|
||||
LLM call. A triage LLM is used only for the human reply judgment in
|
||||
waiting_human.
|
||||
5. **Shared code.** `ai-sdk/agentloop` and `ai-sdk/toolkit` provide the common
|
||||
agent loop and tool set.
|
||||
6. **No daemon-internal cron.** Polling is a ticker loop, not cron. The
|
||||
feasibility PRD notification uses an external webhook (n8n).
|
||||
7. **Fresh worktree per task.** No persistent state between runs. Crash recovery
|
||||
just re-queues.
|
||||
8. **Agent execution is a data boundary.** Autonomous stages receive a versioned
|
||||
request and return a fenced result; SQLite, forge operations, git, workflow
|
||||
sequencing, and outcome handling stay daemon-owned.
|
||||
|
||||
## What's Implemented vs Planned
|
||||
|
||||
**Implemented:**
|
||||
|
||||
- Daemon poll loop, enqueue, claim, process
|
||||
- Bootstrap, implement, TDD, feasibility workflows
|
||||
- GitHub forge (issues, PRs, labels, comments, reactions)
|
||||
@@ -230,7 +288,8 @@ an owning registry or manager.
|
||||
- Human reply judgment (triage LLM for waiting_human)
|
||||
|
||||
**Planned:**
|
||||
|
||||
- Gitea forge implementation
|
||||
- More ecosystems
|
||||
- Notification integrations beyond n8n webhook
|
||||
- Skills (SKILL.md support -- see `.archie/skills/`)
|
||||
- Skills (SKILL.md support -- see `.archie/skills/`)
|
||||
@@ -1,14 +1,23 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with
|
||||
code in this repository.
|
||||
|
||||
## What this is
|
||||
|
||||
**archied** is a resident daemon that polls a forge (GitHub, Gitea planned) for issues assigned/labelled to it, works each one in an isolated git worktree through a routed workflow (bootstrap / implement / TDD / feasibility), and opens pull requests for human review. See `ARCHITECTURE.md` for the full package map, workflow engine design, task lifecycle, and key design decisions — read it before making non-trivial changes, since several invariants (env-enforced gates, model never runs git, agent execution as a data boundary) aren't obvious from the code alone.
|
||||
**archied** is a resident daemon that polls a forge (GitHub, Gitea planned) for
|
||||
issues assigned/labelled to it, works each one in an isolated git worktree
|
||||
through a routed workflow (bootstrap / implement / TDD / feasibility), and opens
|
||||
pull requests for human review. See `ARCHITECTURE.md` for the full package map,
|
||||
workflow engine design, task lifecycle, and key design decisions — read it
|
||||
before making non-trivial changes, since several invariants (env-enforced gates,
|
||||
model never runs git, agent execution as a data boundary) aren't obvious from
|
||||
the code alone.
|
||||
|
||||
## Build & Test
|
||||
|
||||
Commands are defined in `Taskfile.yml` (uses [Task](https://taskfile.dev)); run `task --list` to see everything.
|
||||
Commands are defined in `Taskfile.yml` (uses [Task](https://taskfile.dev)); run
|
||||
`task --list` to see everything.
|
||||
|
||||
```bash
|
||||
task build # build archied and archie-agent binaries into bin/
|
||||
@@ -21,52 +30,121 @@ task check # fmt + go fix + vet + build + test — matches the per-repo [[r
|
||||
|
||||
Run a single test: `go test ./internal/workflow/... -run TestName -v`.
|
||||
|
||||
Docker/NATS stack (multi-process split — see below): `task docker-build`, `task docker-up`, `task docker-down`, `task docker-logs`.
|
||||
Docker/NATS stack (multi-process split — see below): `task docker-build`,
|
||||
`task docker-up`, `task docker-down`, `task docker-logs`.
|
||||
|
||||
## Architecture
|
||||
|
||||
Full details live in `ARCHITECTURE.md` — do not duplicate it here; skim it whenever touching the daemon, workflow engine, or forge/worktree/store boundaries. Summary of what's evolved since that doc was last fully accurate:
|
||||
Full details live in `ARCHITECTURE.md` — do not duplicate it here; skim it
|
||||
whenever touching the daemon, workflow engine, or forge/worktree/store
|
||||
boundaries. Summary of what's evolved since that doc was last fully accurate:
|
||||
|
||||
- The daemon has grown an RPC split (`internal/forgerpc`, `internal/storerpc`, `internal/worktreerpc`, `internal/natsrpc`, `internal/nats`) alongside the in-process model described in ARCHITECTURE.md — check `internal/nats/` and `docker-compose.yml` for the current split between `archied` and sandboxed workers.
|
||||
- `internal/gate/` — quality gate execution (the "environmental constraints over prompt rules" mechanism).
|
||||
- `internal/skill/`, `internal/skillscript/`, `internal/yaegiutil/` — SKILL.md support (listed as "Planned" in ARCHITECTURE.md; now has real implementation — check these packages for current state before assuming it's unbuilt).
|
||||
- `internal/channels/` — inbound chat/notification channels (e.g. Telegram — see `internal/channels/telegram/`, `internal/channels/email/`); channel-agnostic interface in `internal/channels/channel.go`.
|
||||
- `internal/memory/`, `internal/nell/`, `internal/gateway/`, `internal/container/`, `internal/plugin/`, `internal/tools/`, `internal/taskrun/` — not documented in ARCHITECTURE.md; read the package before modifying, its doc comment and tests are the source of truth.
|
||||
- The daemon has grown an RPC split (`internal/forgerpc`, `internal/storerpc`,
|
||||
`internal/worktreerpc`, `internal/natsrpc`, `internal/nats`) alongside the
|
||||
in-process model described in ARCHITECTURE.md — check `internal/nats/` and
|
||||
`docker-compose.yml` for the current split between `archied` and sandboxed
|
||||
workers.
|
||||
- `internal/gate/` — quality gate execution (the "environmental constraints over
|
||||
prompt rules" mechanism).
|
||||
- `internal/skill/`, `internal/skillscript/`, `internal/yaegiutil/` — SKILL.md
|
||||
support (listed as "Planned" in ARCHITECTURE.md; now has real implementation —
|
||||
check these packages for current state before assuming it's unbuilt).
|
||||
- `internal/channels/` — inbound chat/notification channels (e.g. Telegram — see
|
||||
`internal/channels/telegram/`, `internal/channels/email/`); channel-agnostic
|
||||
interface in `internal/channels/channel.go`.
|
||||
- `internal/memory/`, `internal/nell/`, `internal/gateway/`,
|
||||
`internal/container/`, `internal/plugin/`, `internal/tools/`,
|
||||
`internal/taskrun/` — not documented in ARCHITECTURE.md; read the package
|
||||
before modifying, its doc comment and tests are the source of truth.
|
||||
|
||||
**Task lifecycle** (from ARCHITECTURE.md, still current): `queued → running(workflow:stage) → pr_open → merged|rejected`, with `waiting_human` and `parked` side states. Crash recovery re-queues anything left `running`; parks always post a comment with the reason — never fail silently.
|
||||
**Task lifecycle** (from ARCHITECTURE.md, still current):
|
||||
`queued → running(workflow:stage) → pr_open → merged|rejected`, with
|
||||
`waiting_human` and `parked` side states. Crash recovery re-queues anything left
|
||||
`running`; parks always post a comment with the reason — never fail silently.
|
||||
|
||||
**Config** is daemon-level TOML (see `config.example.toml`), not per-repo files in this repo. Two live instances run on a separate host (`carina`), each with a distinct `bot_user` — see the `two-archied-instances-run-on-host-carina-ssh` beads memory (`bd memories carina`) if working on multi-identity/dispatch behavior.
|
||||
**Config** is daemon-level TOML (see `config.example.toml`), not per-repo files
|
||||
in this repo. Two live instances run on a separate host (`carina`), each with a
|
||||
distinct `bot_user` — see the `two-archied-instances-run-on-host-carina-ssh`
|
||||
beads memory (`bd memories carina`) if working on multi-identity/dispatch
|
||||
behavior.
|
||||
|
||||
## Conventions
|
||||
|
||||
- Go 1.26.3, module `github.com/samcharles93/archie-core`.
|
||||
- `ai-sdk/runtime`, `ai-sdk/agentloop`, `ai-sdk/core` are external — shared with the `tau` project at the module level, not vendored copies. Changes there affect both repos.
|
||||
- Non-interactive shell commands only: some environments alias `cp`/`mv`/`rm` to `-i`. Use `cp -f`, `mv -f`, `rm -f`/`rm -rf`, `scp -o BatchMode=yes`, `ssh -o BatchMode=yes`, `apt-get -y`.
|
||||
- Environmental enforcement over prompt rules: gates, test-file protection, and diff caps are code-level checks the agent cannot talk its way around — follow this pattern for any new safety constraint rather than adding it to a system prompt.
|
||||
- **Plugin engine rule (strict):** “engine” means a typed, lifecycle-managed capability family with an owning registry or manager; it never means another method on the generic plugin interface. Keep `plugin.Plugin` metadata-only, give plugins narrow typed registrars instead of daemon access, and apply the lifecycle and trust-boundary requirements in [ARCHITECTURE.md#plugin-engine-rule-strict](ARCHITECTURE.md#plugin-engine-rule-strict). Its mechanically testable structure is enforced by `internal/plugin/architecture_test.go`; reviewers enforce its lifecycle, access, and trust semantics.
|
||||
- The model/agent never runs git directly — worktree operations (`internal/worktree/`) are deterministic daemon-owned steps, not agent tools.
|
||||
- `ai-sdk/runtime`, `ai-sdk/agentloop`, `ai-sdk/core` are external other
|
||||
projects, not vendored copies. Changes affect other repos.
|
||||
- Non-interactive shell commands only: some environments alias `cp`/`mv`/`rm` to
|
||||
`-i`. Use `cp -f`, `mv -f`, `rm -f`/`rm -rf`, `scp -o BatchMode=yes`,
|
||||
`ssh -o BatchMode=yes`, `apt-get -y`.
|
||||
- Environmental enforcement over prompt rules: gates, test-file protection, and
|
||||
diff caps are code-level checks the agent cannot talk its way around — follow
|
||||
this pattern for any new safety constraint rather than adding it to a system
|
||||
prompt.
|
||||
- **Plugin engine rule (strict):** “engine” means a typed, lifecycle-managed
|
||||
capability family with an owning registry or manager; it never means another
|
||||
method on the generic plugin interface. Keep `plugin.Plugin` metadata-only,
|
||||
give plugins narrow typed registrars instead of daemon access, and apply the
|
||||
lifecycle and trust-boundary requirements in
|
||||
[ARCHITECTURE.md#plugin-engine-rule-strict](ARCHITECTURE.md#plugin-engine-rule-strict).
|
||||
Its mechanically testable structure is enforced by
|
||||
`internal/plugin/architecture_test.go`; reviewers enforce its lifecycle,
|
||||
access, and trust semantics.
|
||||
- The model/agent never runs git directly — worktree operations
|
||||
(`internal/worktree/`) are deterministic daemon-owned steps, not agent tools.
|
||||
|
||||
## Development Protocol (mandatory)
|
||||
|
||||
This is not optional style guidance — every code change in this repo follows this sequence. It mirrors what the `implement-and-verify` workflow (`.claude/workflows/implement-and-verify.js`) automates for `archie-core-abg` beads, and it's the same discipline archied enforces on *itself* via the TDD workflow in `ARCHITECTURE.md`.
|
||||
This is not optional style guidance — every code change in this repo follows
|
||||
this sequence. It mirrors what the `implement-and-verify` workflow
|
||||
(`.claude/workflows/implement-and-verify.js`) automates for `archie-core-abg`
|
||||
beads, and it's the same discipline archied enforces on _itself_ via the TDD
|
||||
workflow in `ARCHITECTURE.md`.
|
||||
|
||||
**`task check` is the definitive quality gate.** `task check` = `fmt` + `go fix ./...` + `vet` + `build` + `test`, and it is the one command that must pass, clean, before any change is considered done — it's the literal analogue of the `[[repos.gate]]` archied runs against every PR it opens. Don't hand-pick a subset of `go test`/`go vet`/`go build` and call it good; run `task check` (or the scoped equivalent, e.g. `go test ./internal/foo/... -count=1` + `golangci-lint run ./internal/foo/...` for a single package) and confirm it's clean.
|
||||
**`task check` is the definitive quality gate.** `task check` = `fmt` +
|
||||
`go fix ./...` + `vet` + `build` + `test`, and it is the one command that must
|
||||
pass, clean, before any change is considered done — it's the literal analogue of
|
||||
the `[[repos.gate]]` archied runs against every PR it opens. Don't hand-pick a
|
||||
subset of `go test`/`go vet`/`go build` and call it good; run `task check` (or
|
||||
the scoped equivalent, e.g. `go test ./internal/foo/... -count=1` +
|
||||
`golangci-lint run ./internal/foo/...` for a single package) and confirm it's
|
||||
clean.
|
||||
|
||||
**Red-then-green, definitively:**
|
||||
1. **Red** — write the failing test(s) first, against the *intended* behavior, before any implementation code exists. Run them and confirm they fail for the right reason (not a compile error).
|
||||
2. **Green** — write the minimum implementation to make those tests pass. No implementation code is written before its test exists and fails.
|
||||
3. Confirm via `task check` (or scoped `go test .../... -count=1`) that the suite is green, `golangci-lint run` reports zero findings, and `gofumpt -w .` reports no unformatted files.
|
||||
4. No hardcoded values in tests (use parameterized test tables), no hardcoded identity/config values in implementation code, and every error path gets a test.
|
||||
|
||||
**Adversarial verification is mandatory, and it runs as a separate pass from a fresh reviewer — never self-certified by the implementer.** After red-then-green is green:
|
||||
- Spawn a review with no memory of how the code was written (a fresh subagent, or `/code-review`/`/security-review` as appropriate) and instruct it to assume every line is wrong until proven correct.
|
||||
- The checklist: all tests pass, lint is zero-findings, formatting is clean, then a manual read of every non-test file in the changed package for — dead code paths, unchecked error returns, hardcoded values that should be parameters/constants, interface satisfaction, nil-pointer risk, goroutine leaks, and races on shared mutable state.
|
||||
- Findings are reported (use `ReportFindings` when running as a review agent), not silently fixed by the same pass that wrote the code — a finding that survives blocks closing the work, it doesn't get waved through.
|
||||
- Only close out (beads: `bd close`) once the adversarial pass reports zero surviving findings.
|
||||
1. **Red** — write the failing test(s) first, against the _intended_ behavior,
|
||||
before any implementation code exists. Run them and confirm they fail for the
|
||||
right reason (not a compile error).
|
||||
2. **Green** — write the minimum implementation to make those tests pass. No
|
||||
implementation code is written before its test exists and fails.
|
||||
3. Confirm via `task check` (or scoped `go test .../... -count=1`) that the
|
||||
suite is green, `golangci-lint run` reports zero findings, and `gofumpt -w .`
|
||||
reports no unformatted files.
|
||||
4. No hardcoded values in tests (use parameterized test tables), no hardcoded
|
||||
identity/config values in implementation code, and every error path gets a
|
||||
test.
|
||||
|
||||
**Adversarial verification is mandatory, and it runs as a separate pass from a
|
||||
fresh reviewer — never self-certified by the implementer.** After red-then-green
|
||||
is green:
|
||||
|
||||
- Spawn a review with no memory of how the code was written (a fresh subagent,
|
||||
or `/code-review`/`/security-review` as appropriate) and instruct it to assume
|
||||
every line is wrong until proven correct.
|
||||
- The checklist: all tests pass, lint is zero-findings, formatting is clean,
|
||||
then a manual read of every non-test file in the changed package for — dead
|
||||
code paths, unchecked error returns, hardcoded values that should be
|
||||
parameters/constants, interface satisfaction, nil-pointer risk, goroutine
|
||||
leaks, and races on shared mutable state.
|
||||
- Findings are reported (use `ReportFindings` when running as a review agent),
|
||||
not silently fixed by the same pass that wrote the code — a finding that
|
||||
survives blocks closing the work, it doesn't get waved through.
|
||||
- Only close out (beads: `bd close`) once the adversarial pass reports zero
|
||||
surviving findings.
|
||||
|
||||
## Beads Issue Tracker
|
||||
|
||||
This project uses **bd (beads)** for issue tracking. Run `bd prime` to see full workflow context and commands.
|
||||
This project uses **bd (beads)** for issue tracking. Run `bd prime` to see full
|
||||
workflow context and commands.
|
||||
|
||||
```bash
|
||||
bd ready # Find available work
|
||||
@@ -75,6 +153,10 @@ bd update <id> --claim # Claim work
|
||||
bd close <id> # Complete work
|
||||
```
|
||||
|
||||
- Use `bd` for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown TODO lists.
|
||||
- Use `bd remember` / `bd memories <keyword>` for persistent knowledge — do NOT use MEMORY.md files.
|
||||
- Issues live in a local Dolt DB; sync uses `refs/dolt/data` on the git remote; `.beads/issues.jsonl` is a passive export, not the source of truth. See [SYNC_CONCEPTS.md](https://github.com/gastownhall/beads/blob/main/docs/SYNC_CONCEPTS.md).
|
||||
- Use `bd` for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown
|
||||
TODO lists.
|
||||
- Use `bd remember` / `bd memories <keyword>` for persistent knowledge — do NOT
|
||||
use MEMORY.md files.
|
||||
- Issues live in a local Dolt DB; sync uses `refs/dolt/data` on the git remote;
|
||||
`.beads/issues.jsonl` is a passive export, not the source of truth. See
|
||||
[SYNC_CONCEPTS.md](https://github.com/gastownhall/beads/blob/main/docs/SYNC_CONCEPTS.md).
|
||||
@@ -215,7 +215,7 @@ func TestChatTaskProfilesUsesIdentityRepositories(t *testing.T) {
|
||||
Identities: []config.IdentityConfig{
|
||||
{Name: "builder", Repos: []config.Repo{{Owner: "sam", Name: "archie-core"}}},
|
||||
{Name: "reviewer", Repos: []config.Repo{
|
||||
{Owner: "sam", Name: "tau"},
|
||||
{Owner: "sam", Name: "example-service"},
|
||||
{Owner: "sam", Name: "ai-sdk"},
|
||||
}},
|
||||
},
|
||||
@@ -229,7 +229,7 @@ func TestChatTaskProfilesUsesIdentityRepositories(t *testing.T) {
|
||||
t.Fatalf("profiles = %d, want 2", len(profiles))
|
||||
}
|
||||
if profiles[1].Identity != "reviewer" || profiles[1].DefaultOwner != "sam" ||
|
||||
profiles[1].DefaultRepo != "tau" ||
|
||||
profiles[1].DefaultRepo != "example-service" ||
|
||||
len(profiles[1].Repos) != 2 || profiles[1].Repos[1] != "sam/ai-sdk" {
|
||||
t.Errorf("reviewer profile = %+v", profiles[1])
|
||||
}
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
import { defineConfig } from "vitepress";
|
||||
|
||||
export default defineConfig({
|
||||
title: "Archie",
|
||||
description: "Documentation for the Archie personal agent platform.",
|
||||
lastUpdated: true,
|
||||
cleanUrls: true,
|
||||
srcDir: ".",
|
||||
srcExclude: ["README.md"],
|
||||
outDir: ".vitepress/dist",
|
||||
|
||||
head: [["link", { rel: "icon", href: "/favicon.svg" }]],
|
||||
|
||||
themeConfig: {
|
||||
logo: "/favicon.svg",
|
||||
nav: [{ text: "Architecture", link: "/architecture/" }],
|
||||
search: {
|
||||
provider: "local",
|
||||
},
|
||||
footer: {
|
||||
message: "Built with VitePress.",
|
||||
copyright: "Archie",
|
||||
},
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,263 @@
|
||||
# Tau CLI Reference
|
||||
|
||||
Tau is a provider-agnostic coding agent with an interactive terminal UI. It provides an extensible, personalised
|
||||
environment for working with AI models, featuring an interactive TUI, an optional Web UI, session history, token
|
||||
tracking, and plugin integration.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
tau [global flags] [command]
|
||||
```
|
||||
|
||||
Running `tau` with no arguments starts an interactive chat session and also starts a local Web UI on `127.0.0.1`. You do
|
||||
not need a `--model` flag; if no model is configured the session starts unselected and you pick one with `/model`.
|
||||
|
||||
## Global Flags
|
||||
|
||||
| Flag | Description |
|
||||
| ----------------------------- | ------------------------------------------------------------------------- |
|
||||
| `--provider <name>` | Provider name to use (e.g. `deepseek`, `openrouter`). Env: `TAU_PROVIDER` |
|
||||
| `--model <id>` | Model ID to use. Also accepts `provider:model-id` or `provider/model-id` |
|
||||
| `--max-tokens <n>` | Maximum completion tokens per response |
|
||||
| `--temperature <f>` | Sampling temperature (0–2) |
|
||||
| `--resume`, `-r <id\|latest>` | Resume a saved session by ID, or `latest` |
|
||||
| `--execute`, `-x <text>` | Execute mode: process prompt, print response, exit (reads stdin when piped) |
|
||||
| `--prompt`, `-p <text>` | [deprecated] Use `-x`/`--execute` instead. One-shot / execute mode |
|
||||
| `--jsonl`, `--stream-json` | Output framed JSONL events on stdout instead of plain text (execute mode) |
|
||||
| `--child` | Run as a headless agent child process (internal use; hidden) |
|
||||
| `--web` | Start the Web UI and open it in the default browser |
|
||||
| `--port <n>` | HTTP port for the Web UI. Default `0` = OS-assigned ephemeral port |
|
||||
| `--no-web` | Do not start the Web UI. TUI only |
|
||||
| `--insecure` | Skip TLS certificate verification. Env: `TAU_INSECURE` |
|
||||
| `--verbose` | Show debug messages on stderr. Env: `TAU_VERBOSE` |
|
||||
|
||||
## Enabling Providers
|
||||
|
||||
Tau discovers providers automatically. To enable a provider:
|
||||
|
||||
1. **`tau setup` (recommended for first run):** Interactive walkthrough for selecting a provider and authenticating.
|
||||
Managed API keys are stored in `~/.config/tau/auth.yaml` — env vars override them when both are present.
|
||||
|
||||
2. **`tau provider` CLI:**
|
||||
```bash
|
||||
tau provider list # show all providers and auth state
|
||||
tau provider login deepseek # enter and store a managed API key
|
||||
tau provider login openai-codex # OAuth device-code flow
|
||||
tau provider login # interactive selector (TTY only)
|
||||
tau provider logout openai # disable and clear credentials
|
||||
```
|
||||
For non-OAuth providers, `tau provider login` stores a managed API key in `auth.yaml`.
|
||||
|
||||
3. **Environment variable:** Export the provider's API key env var (e.g. `DEEPSEEK_API_KEY=sk-...`). Tau
|
||||
detects it at startup.
|
||||
|
||||
4. **TUI slash commands:** `/provider [name]` to toggle a provider, `/provider login <name>` for OAuth flow.
|
||||
|
||||
5. **config.yaml:** Add a provider block (see Configuration below).
|
||||
|
||||
Run `/model` in the TUI to pick from all enabled providers' models.
|
||||
|
||||
## Web UI
|
||||
|
||||
When tau starts interactively, it binds a local HTTP/WebSocket server on `127.0.0.1`. The URL is shown in the TUI status
|
||||
bar.
|
||||
|
||||
```bash
|
||||
./tau # TUI + Web UI, URL in status bar
|
||||
./tau --web # TUI + Web UI, also opens the browser
|
||||
./tau --no-web # TUI only
|
||||
./tau --port 9343 # Fixed port
|
||||
```
|
||||
|
||||
The browser connects to `/ws` and receives the same event stream as the TUI. Commands sent from the browser are
|
||||
forwarded to the coordinator identically to TUI commands. Multiple browser tabs may connect to the same session.
|
||||
|
||||
## Slash Commands (TUI)
|
||||
|
||||
| Command | Description |
|
||||
| --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
|
||||
| `/model [id]` | Switch model; opens picker when called without an ID |
|
||||
| `/provider [name]` | Toggle a provider on/off; lists all providers when called without an argument |
|
||||
| `/provider login <provider>` | OAuth device-code sign-in for `github-copilot` or `openai-codex`; opens the browser and copies the code when possible |
|
||||
| `/provider logout <provider>` | Disable a provider / remove its saved key |
|
||||
| `/refresh` | Re-discover models from all enabled providers |
|
||||
| `/effort [level]` | Set reasoning effort (`off`, `low`, `medium`, `high`, `max`) |
|
||||
| `/reasoning [on\|off]` | Toggle display of model reasoning content |
|
||||
| `/system <prompt>` | Set the session system prompt |
|
||||
| `/session [list\|info\|export\|delete\|<id>]` | Manage saved sessions |
|
||||
| `/resume [id]` | Resume a saved session |
|
||||
| `/new` | Start a fresh conversation (aliases: `/clear`, `/reset`) |
|
||||
| `/reload` | Reload extensions/plugins |
|
||||
| `/help` | Show all available commands |
|
||||
| `/exit` | Quit tau |
|
||||
|
||||
## Subcommands
|
||||
|
||||
### `tau setup`
|
||||
|
||||
Interactive setup wizard for first-run configuration. Walks through provider selection, authentication (API key or OAuth), and model selection.
|
||||
|
||||
```bash
|
||||
tau setup
|
||||
```
|
||||
|
||||
Called automatically on first run if no providers are configured.
|
||||
|
||||
### `tau provider`
|
||||
|
||||
Manage provider authentication from the CLI.
|
||||
|
||||
```bash
|
||||
tau provider list # table of providers with status, source, and auth type
|
||||
tau provider login [name] # authenticate an OAuth or API-key provider
|
||||
tau provider login # interactive selector when stdin is a TTY
|
||||
tau provider logout <name> # disable provider and clear managed credentials
|
||||
```
|
||||
|
||||
Provider credentials are managed via the keychain or stored in `~/.config/tau/auth.yaml`. See [`docs/providers.md`](providers.md) for the full provider architecture.
|
||||
|
||||
### `tau models`
|
||||
|
||||
List available models from the configured provider.
|
||||
|
||||
```bash
|
||||
tau models [--json]
|
||||
```
|
||||
|
||||
Uses the live models.dev catalog (network). For the model list used in the interactive TUI, see the embedded snapshot
|
||||
(`internal/providers/snapshot/`).
|
||||
|
||||
### `tau refresh`
|
||||
|
||||
Force a refresh of the models.dev network catalog and list models.
|
||||
|
||||
```bash
|
||||
tau refresh
|
||||
```
|
||||
|
||||
Downloads the latest catalog for the configured provider. This affects the `tau models` output and the snapshot
|
||||
generator, but interactive sessions use the embedded offline snapshot (`internal/providers/snapshot/models.json`).
|
||||
|
||||
### `tau sessions`
|
||||
|
||||
List saved chat sessions.
|
||||
|
||||
```bash
|
||||
tau sessions
|
||||
```
|
||||
|
||||
Shows a table of sessions with ID, model, message count, tokens, cost, and date. Use `/session` inside the TUI for
|
||||
richer session management.
|
||||
|
||||
### `tau token`
|
||||
|
||||
Print the resolved API key / bearer token for the selected provider.
|
||||
|
||||
```bash
|
||||
tau token
|
||||
```
|
||||
|
||||
Useful for debugging auth configuration.
|
||||
|
||||
### `tau update`
|
||||
|
||||
Update the current tau binary from GitHub release assets, or check for available updates.
|
||||
|
||||
```bash
|
||||
tau update [--check] [--version v0.16.2] [--repo owner/repo] [--force]
|
||||
```
|
||||
|
||||
The updater downloads the platform archive and `checksums.txt`, verifies the archive checksum, extracts the tau binary,
|
||||
and applies it in place.
|
||||
|
||||
By default, update checks only happen when you run `tau update` manually. Set
|
||||
`updates.mode: warn` in [configuration](configuration.md#updates) to keep manual
|
||||
checks with update notifications, or `updates.mode: disabled` to disable all update
|
||||
checks including `tau update`. Dev builds (version `dev`) are always excluded
|
||||
from update checks — only release builds are eligible.
|
||||
|
||||
## Configuration
|
||||
|
||||
Tau loads `~/.config/tau/config.yaml` (global) and `.tau.yaml` (project-local). Example configs are in
|
||||
[`config-example.yaml`](config-example.yaml) and [`config-deepseek-example.yaml`](config-deepseek-example.yaml).
|
||||
|
||||
### Minimal configuration
|
||||
|
||||
```yaml
|
||||
# ~/.config/tau/config.yaml
|
||||
default_provider: deepseek
|
||||
default_model: deepseek-chat
|
||||
|
||||
providers:
|
||||
- name: deepseek
|
||||
base_url: https://api.deepseek.com/v1
|
||||
auth:
|
||||
type: api_key
|
||||
api_key_env: DEEPSEEK_API_KEY
|
||||
```
|
||||
|
||||
Model metadata (context window, pricing, reasoning) is loaded from the embedded snapshot - you do not need to repeat it
|
||||
in config. See [`docs/ai-sdk.md`](ai-sdk.md) for full integration details.
|
||||
|
||||
### `ProviderConfig` fields
|
||||
|
||||
| Field | Description |
|
||||
| ------------------ | ---------------------------------------------------------------------------------------- |
|
||||
| `name` | Provider identifier (matches tau's catalog ID) |
|
||||
| `base_url` | API endpoint. Must include `/v1` when the provider uses it |
|
||||
| `type` | ai-sdk class override (`anthropic` for Anthropic native API; omit for OpenAI-compatible) |
|
||||
| `auth.type` | `api_key`, `none`, or `oauth_pkce` |
|
||||
| `auth.api_key_env` | Environment variable name for the API key |
|
||||
| `auth.api_key` | Literal API key (not recommended; prefer `api_key_env`) |
|
||||
| `headers` | Optional static request headers; do not put bearer tokens here |
|
||||
| `models` | Optional list of model overrides (see below) |
|
||||
|
||||
### Model overrides
|
||||
|
||||
Tau's embedded snapshot provides model metadata. Override individual fields only when you need to customise them:
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- name: openai
|
||||
auth:
|
||||
type: api_key
|
||||
api_key_env: OPENAI_API_KEY
|
||||
models:
|
||||
- id: gpt-5.4
|
||||
context_window: 200000
|
||||
default_max_tokens: 16384
|
||||
```
|
||||
|
||||
## Documentation Index
|
||||
|
||||
### Architecture & Design
|
||||
|
||||
- [Architecture Overview](architecture.md) - system design, communication patterns, layer map
|
||||
- [Event Bus Architecture](eventbus.md) - type-based event routing, publishers, subscribers
|
||||
|
||||
### Core Systems
|
||||
|
||||
- [Agent Coordinator](agent.md) - turn loop, command handlers, tool execution, prompts
|
||||
- [Chat Types](chat-types.md) - complete command/event reference, core types
|
||||
- [Tools](tools.md) - registry, built-in tools, mutation queue, UIBridge
|
||||
- [Commands](commands.md) - slash command registry, custom commands
|
||||
|
||||
### UI Systems
|
||||
|
||||
- [Terminal UI (TUI)](tui.md) - inline rendering, widget tree, slash commands, completions, notifications
|
||||
- [Web UI](webui.md) - Vue 3 SPA, state management, components, protocol, build
|
||||
- [Server & Bridge](server.md) - HTTP/WebSocket server, event fan-out, wire format
|
||||
- [taui Framework](taui.md) - standalone TUI rendering engine, widgets, color system
|
||||
|
||||
### Infrastructure
|
||||
|
||||
- [Configuration](configuration.md) - YAML config format, env vars, CLI flags
|
||||
- [Providers](providers.md) - provider catalogue, auth state, model discovery, dynamic switching
|
||||
- [AI SDK Integration](ai-sdk.md) - embedded snapshot, dynamic streamer, URL rules, reasoning
|
||||
- [Sessions](sessions.md) - SQLite persistence, session lifecycle, JSONL export
|
||||
- [Skills](skills.md) - SKILL.md discovery, frontmatter parsing, prompt rendering
|
||||
|
||||
### Extensibility
|
||||
|
||||
- [Plugin SDK](plugins.md) - gRPC extension system, tools, slash commands, lifecycle events
|
||||
@@ -0,0 +1,16 @@
|
||||
# Architecture
|
||||
|
||||
Archie is a personal agent platform built around six domain boundaries:
|
||||
|
||||
- Identity tracks users and other actors responsible for actions.
|
||||
- Agent owns persistent assistants, specialisation, and continuity.
|
||||
- Messaging owns conversations, messages, branches, and channel correlation.
|
||||
- Workflow owns reusable, versioned behaviour definitions and their executions.
|
||||
- Work Intake admits optional durable work without constraining ordinary agent
|
||||
conversation or direct capability use.
|
||||
- Plugin owns generic extension discovery and lifecycle; each capability family
|
||||
owns its typed plugin contract.
|
||||
|
||||
The detailed architecture and migration decisions are maintained under
|
||||
`docs/prds/architecture/` in the repository while this generated site is being
|
||||
assembled.
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
layout: home
|
||||
|
||||
hero:
|
||||
name: Archie
|
||||
text: Personal agent platform
|
||||
tagline: Durable agents, conversations, workflows, tools, and integrations.
|
||||
image:
|
||||
src: /favicon.svg
|
||||
alt: Archie
|
||||
actions:
|
||||
- theme: brand
|
||||
text: Architecture
|
||||
link: /architecture/
|
||||
|
||||
features:
|
||||
- title: Personal agents
|
||||
details: Persistent assistants with their own identity, specialisation, and private memory.
|
||||
- title: Conversations
|
||||
details: Durable, branchable conversations across messaging channels.
|
||||
- title: Workflows
|
||||
details: Reusable, versioned behaviour definitions for structured and automated work.
|
||||
- title: Extensible capabilities
|
||||
details: Typed plugins, tools, channels, and infrastructure integrations.
|
||||
---
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+autocomplete-plugin-algolia-insights@1.17.7_@algolia+client-search@5.55.1_algo_0b7c744a1ea982691c999438340aa842/node_modules/@algolia/autocomplete-plugin-algolia-insights
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+autocomplete-shared@1.17.7_@algolia+client-search@5.55.1_algoliasearch@5.55.1/node_modules/@algolia/autocomplete-shared
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+autocomplete-shared@1.17.7_@algolia+client-search@5.55.1_algoliasearch@5.55.1/node_modules/@algolia/autocomplete-shared
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../search-insights@2.17.3/node_modules/search-insights
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+autocomplete-shared@1.17.7_@algolia+client-search@5.55.1_algoliasearch@5.55.1/node_modules/@algolia/autocomplete-shared
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-search@5.55.1/node_modules/@algolia/client-search
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../algoliasearch@5.55.1/node_modules/algoliasearch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-search@5.55.1/node_modules/@algolia/client-search
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../algoliasearch@5.55.1/node_modules/algoliasearch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
docs-2/node_modules/.pnpm/@algolia+client-abtesting@5.55.1/node_modules/@algolia/requester-node-http
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
docs-2/node_modules/.pnpm/@algolia+client-analytics@5.55.1/node_modules/@algolia/requester-node-http
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
docs-2/node_modules/.pnpm/@algolia+client-personalization@5.55.1/node_modules/@algolia/client-common
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-browser-xhr@5.55.1/node_modules/@algolia/requester-browser-xhr
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-fetch@5.55.1/node_modules/@algolia/requester-fetch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+requester-node-http@5.55.1/node_modules/@algolia/requester-node-http
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+client-common@5.55.1/node_modules/@algolia/client-common
|
||||
+1
@@ -0,0 +1 @@
|
||||
../../../@babel+types@7.29.7/node_modules/@babel/types
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@babel+helper-string-parser@7.29.7/node_modules/@babel/helper-string-parser
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@babel+helper-validator-identifier@7.29.7/node_modules/@babel/helper-validator-identifier
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@docsearch+react@3.8.2_@algolia+client-search@5.55.1_search-insights@2.17.3/node_modules/@docsearch/react
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../preact@10.29.3/node_modules/preact
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+autocomplete-core@1.17.7_@algolia+client-search@5.55.1_algoliasearch@5.55.1_search-insights@2.17.3/node_modules/@algolia/autocomplete-core
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@algolia+autocomplete-preset-algolia@1.17.7_@algolia+client-search@5.55.1_algoliasearch@5.55.1/node_modules/@algolia/autocomplete-preset-algolia
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@docsearch+css@3.8.2/node_modules/@docsearch/css
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../algoliasearch@5.55.1/node_modules/algoliasearch
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../search-insights@2.17.3/node_modules/search-insights
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@iconify+types@2.0.0/node_modules/@iconify/types
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+engine-javascript@2.5.0/node_modules/@shikijs/engine-javascript
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+engine-oniguruma@2.5.0/node_modules/@shikijs/engine-oniguruma
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+types@2.5.0/node_modules/@shikijs/types
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+vscode-textmate@10.0.2/node_modules/@shikijs/vscode-textmate
|
||||
+1
@@ -0,0 +1 @@
|
||||
../../../@types+hast@3.0.4/node_modules/@types/hast
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../hast-util-to-html@9.0.5/node_modules/hast-util-to-html
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+types@2.5.0/node_modules/@shikijs/types
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+vscode-textmate@10.0.2/node_modules/@shikijs/vscode-textmate
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../oniguruma-to-es@3.1.1/node_modules/oniguruma-to-es
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+types@2.5.0/node_modules/@shikijs/types
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+vscode-textmate@10.0.2/node_modules/@shikijs/vscode-textmate
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+types@2.5.0/node_modules/@shikijs/types
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+types@2.5.0/node_modules/@shikijs/types
|
||||
Generated
Vendored
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../../@shikijs+core@2.5.0/node_modules/@shikijs/core
|
||||
Loaded 100 of 311 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user