From 6e5117f01a5ef5d7d9bbe23545dfddfe6f2ff2ae Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Sat, 26 Sep 2026 02:06:50 +1000 Subject: [PATCH] docs(prds): store --- docs/data/generated/dev-docs.json | 12 ++- docs/prds/extensions-center.md | 2 + docs/prds/store.md | 132 ++++++++++++++++++++++++++++++ 3 files changed, 145 insertions(+), 1 deletion(-) create mode 100644 docs/prds/store.md diff --git a/docs/data/generated/dev-docs.json b/docs/data/generated/dev-docs.json index ef2d62e60..bae096f72 100644 --- a/docs/data/generated/dev-docs.json +++ b/docs/data/generated/dev-docs.json @@ -322,7 +322,7 @@ "url": "/dev/prds/extensions-center/", "title": "Extensions centre", "section": "PRDs", - "body": "# Extensions centre\n\n**Status:** Draft\n\nEpic: `archie-core-1786637490708-35-38424ead` / GitHub `#56` (\"Replace Skills\nwith a bundled Extensions centre\").\n\nSkills, Workflows, and Tools/MCP already exist as separate, working\ninventories with their own APIs (`api_skills.go`, `api_workflows.go`,\nconfig-driven MCP servers) and their own family-owned controller pattern\n(`curator.Registry`, `health.Registry`, `image.Registry`,\n`workflow.Registry` all already prove this shape). `plugin.Plugin`\n(`internal/plugin/plugin.go`) is already metadata-only, per\n`docs/architecture/plugins-and-extensions.md`. None of that is rebuilt here.\nWhat's missing, confirmed by grep, is: any config schema for enabling or\ndisabling a capability, any \"pending restart\" state tracking, and any single\ninventory that reconciles configured/bundled/discovered/active state across\nfamilies. This document decides those three things and the UI surface that\npresents them. It extends `plugins-and-extensions.md` rather than replacing\nit — that doc's decisions (metadata-only `Plugin`, typed capability\nfamilies, no generic dispatch) are unchanged.\n\nOut of scope: any remote or arbitrary-URL plugin installation path (the\nepic's own acceptance criteria forbid this, and `plugin.LoadDir`\n(`plugin.go`) already only reads `~/.config/archie/plugins/*.go` — this\ndocument does not add a fetch-by-URL variant to it).\n\n## Problem\n\n`internal/config/config.go`'s `MCPServer` and `ToolsConfig` structs have no\n`enabled` field (confirmed by the investigation grep). `api_config.go`'s\n`UpdateConfig`/`handleConfigUpdate` seam persists changes but has no state\ndistinguishing \"saved, takes effect now\" from \"saved, needs restart\" — the\nonly existing reload path (`ReloadChannel`, `server.go`) is\nchannel-specific and explicitly does not cover plugins or MCP servers.\n`server.go` already wires five separate optional inventories\n(Workflows, Skills, Curators, Channels, Captures/Mappings/Bindings) as\nindependent fields with no unifying type — an operator has no single\nplace to see \"what's available, what's enabled, what's actually running,\nand what's degraded\" across all of them. `ui/src/` has `settings/`,\n`skills/`, and `workflows/` directories already; there is no\n`ui/src/extensions/`.\n\n## Design\n\n### Unified inventory type\n\n```go\npackage extensions // internal/domain/extensions — new, app-level aggregator\n\ntype State string\n\nconst (\n StateAvailable State = \"available\" // bundled, not configured\n StateEnabled State = \"enabled\" // configured on, not yet live\n StateActive State = \"active\" // configured on and running\n StateDegraded State = \"degraded\" // active but unhealthy\n StateSkipped State = \"skipped\" // optional, disabled by config\n StatePendingRestart State = \"pending_restart\" // config changed, not yet live\n)\n\ntype Entry struct {\n Family string // \"skill\" | \"workflow\" | \"tool\" | \"plugin\" | \"integration\"\n Name string\n Core bool // non-disableable\n State State\n Health string // reuses whatever the family's own registry already reports\n}\n\ntype Inventory interface {\n List(ctx context.Context) ([]Entry, error)\n}\n```\n\n**Decision.** `extensions.Inventory` is a read-only aggregator, not a new\ndomain that owns state. It queries each family's existing registry\n(`curator.Registry`, `health.Registry`, `workflow.Registry`, the MCP client\nset, `plugin.LoadDir`'s result) for live state, and compares that against\nthe config-desired state (below) to compute `StatePendingRestart` — the diff\nis computed on read, not persisted, so there is no new durable state to keep\nconsistent with the registries themselves. This follows\n`migration-decisions.md`'s rule that cross-cutting aggregation composes\nexisting domain-owned contracts rather than duplicating their state.\n\n**Call site.** New package `internal/domain/extensions/inventory.go`.\nComposed in `internal/app/archied` (wherever `server.go`'s other optional\nfields — Workflows, Skills, Curators — are wired) since it needs\nreferences to every family's registry, which only application-layer\ncomposition has.\n\n### Config-backed enablement\n\n**Decision.** Add `enabled bool` (default `true` for pre-existing configured\nentries, so no upgrade silently disables anything already configured) to\neach family's config element: `MCPServer.Enabled`, and equivalent fields on\nwhatever config types back skills/workflows/plugins if they don't already\nhave an on/off switch. Core capabilities (identity, workflow engine, State\nStore connectivity — named in `organisation.md`'s cross-cutting packages)\nhave no `enabled` field at all; a config value can't be added to something\nthat doesn't expose the toggle, which is the \"non-disableable\" property\nenforced structurally rather than by a runtime check that could be\nbypassed.\n\n**Call site.** `internal/config/config.go` — add the field per family\nstruct, not one generic `map[string]bool` (typed fields keep\n`docsgen`/`contracts.json` generation working, per `tools/docsgen`'s\n\"treat flags as executable truth\" rule — a generic map has no type to\ngenerate from). `config.example.toml` documents each new field with its\ndefault.\n\n**Reuse check.** The config `Holder` overlay system\n(`internal/infrastructure/configuration`) already handles safe saves; this\nadds fields to existing structs, it does not add a second config-write path.\n\n### Pending-restart and reconciliation\n\n**Decision.** No new \"pending restart\" flag is persisted. `Inventory.List`\ncomputes `StatePendingRestart` by comparing the config's `Enabled` value\nagainst whether the family's registry reports that entry as\nlive — if they disagree, the entry is pending restart. This is simpler than\ntracking a flag through a save-then-restart cycle and can never go stale,\nbecause it's recomputed on every read.\n\n**Call site.** `internal/domain/extensions/inventory.go`'s comparison logic.\n\n### API and UI surface\n\n**Decision.** New `api_extensions.go` alongside the existing `api_skills.go`\n/`api_workflows.go` pattern, exposing `Inventory.List` and a save endpoint\nthat writes through the existing `UpdateConfig` seam (`api_config.go`) —\nthis document does not add a second config-write endpoint. New\n`ui/src/extensions/` (colocated `.js`/`.css` per CLAUDE.md's frontend rule)\nas an index/dashboard that **links to**, not replaces, the existing\n`ui/src/skills/` and `ui/src/workflows/` detail views — Extensions centre is\nwhere an operator sees everything and toggles enablement; the family-specific\npages remain where they inspect a family's own detail.\n\n**Call site.** `internal/webui/api_extensions.go` (new, follows\n`api_skills.go`'s shape), `ui/src/extensions/` (new).\n\n## Call site inventory\n\n| concern | file | change |\n| ------------------------------------------------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------- |\n| Plugin metadata type | `internal/plugin/plugin.go` | none — already handles this |\n| Family controllers (curator/health/image/workflow registries) | `internal/domain/curator/registry.go`, `internal/domain/health/health.go`, etc. | none — pattern already proven, reused not rewritten |\n| Unified inventory | `internal/domain/extensions/inventory.go` | new |\n| Composition wiring | `internal/app/archied/server.go` (near existing optional fields) | new |\n| Config enablement fields | `internal/config/config.go` | new fields per family |\n| Config docs | `config.example.toml` | new field documentation |\n| API surface | `internal/webui/api_extensions.go` | new |\n| Existing skills/workflows APIs | `internal/webui/api_skills.go`, `api_workflows.go` | none — linked from, not replaced |\n| Config save path | `internal/webui/api_config.go` (`UpdateConfig`) | reused, not duplicated |\n| UI | `ui/src/extensions/` | new |\n| Existing UI detail views | `ui/src/skills/`, `ui/src/workflows/` | none — linked from Extensions centre |\n\n## Execution: multi-agent team breakdown\n\n| sub-feature | issue | implementer scope | suggested council lenses | why |\n| ------------------------------------ | ---------------------- | ---------------------------------------------------------------- | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |\n| Unified inventory type + composition | file as child of `#56` | `internal/domain/extensions/inventory.go`, wiring in `server.go` | `lens-boundary`, `lens-deletionist` | boundary: aggregator must not duplicate family-owned state; deletionist: must not become a new domain that shadows existing registries |\n| Config enablement fields | file as child of `#56` | `config.go` per-family `Enabled` fields, `config.example.toml` | `lens-contract` | config field is a wire contract other tooling (`docsgen`) generates from |\n| API + UI surface | file as child of `#56` | `api_extensions.go`, `ui/src/extensions/` | `lens-maintainer` | new user-facing state machine (available/enabled/active/degraded/skipped/pending-restart) must be legible to an operator with no prior context |\n\n## File and link the beads\n\n- `#56` itself has no filed children yet — file the three rows above as\n child issues linking to this doc's matching `##` heading.\n- Each issue's acceptance criteria should restate this doc's per-family\n `Enabled` default (`true` for existing config, no field at all for core\n capabilities) so \"non-disableable core\" isn't re-litigated per bead.\n- Add the suggested lenses to each issue.\n" + "body": "# Extensions centre\n\n**Status:** Draft\n\n\u003e Superseded by `docs/prds/store.md`.\n\nEpic: `archie-core-1786637490708-35-38424ead` / GitHub `#56` (\"Replace Skills\nwith a bundled Extensions centre\").\n\nSkills, Workflows, and Tools/MCP already exist as separate, working\ninventories with their own APIs (`api_skills.go`, `api_workflows.go`,\nconfig-driven MCP servers) and their own family-owned controller pattern\n(`curator.Registry`, `health.Registry`, `image.Registry`,\n`workflow.Registry` all already prove this shape). `plugin.Plugin`\n(`internal/plugin/plugin.go`) is already metadata-only, per\n`docs/architecture/plugins-and-extensions.md`. None of that is rebuilt here.\nWhat's missing, confirmed by grep, is: any config schema for enabling or\ndisabling a capability, any \"pending restart\" state tracking, and any single\ninventory that reconciles configured/bundled/discovered/active state across\nfamilies. This document decides those three things and the UI surface that\npresents them. It extends `plugins-and-extensions.md` rather than replacing\nit — that doc's decisions (metadata-only `Plugin`, typed capability\nfamilies, no generic dispatch) are unchanged.\n\nOut of scope: any remote or arbitrary-URL plugin installation path (the\nepic's own acceptance criteria forbid this, and `plugin.LoadDir`\n(`plugin.go`) already only reads `~/.config/archie/plugins/*.go` — this\ndocument does not add a fetch-by-URL variant to it).\n\n## Problem\n\n`internal/config/config.go`'s `MCPServer` and `ToolsConfig` structs have no\n`enabled` field (confirmed by the investigation grep). `api_config.go`'s\n`UpdateConfig`/`handleConfigUpdate` seam persists changes but has no state\ndistinguishing \"saved, takes effect now\" from \"saved, needs restart\" — the\nonly existing reload path (`ReloadChannel`, `server.go`) is\nchannel-specific and explicitly does not cover plugins or MCP servers.\n`server.go` already wires five separate optional inventories\n(Workflows, Skills, Curators, Channels, Captures/Mappings/Bindings) as\nindependent fields with no unifying type — an operator has no single\nplace to see \"what's available, what's enabled, what's actually running,\nand what's degraded\" across all of them. `ui/src/` has `settings/`,\n`skills/`, and `workflows/` directories already; there is no\n`ui/src/extensions/`.\n\n## Design\n\n### Unified inventory type\n\n```go\npackage extensions // internal/domain/extensions — new, app-level aggregator\n\ntype State string\n\nconst (\n StateAvailable State = \"available\" // bundled, not configured\n StateEnabled State = \"enabled\" // configured on, not yet live\n StateActive State = \"active\" // configured on and running\n StateDegraded State = \"degraded\" // active but unhealthy\n StateSkipped State = \"skipped\" // optional, disabled by config\n StatePendingRestart State = \"pending_restart\" // config changed, not yet live\n)\n\ntype Entry struct {\n Family string // \"skill\" | \"workflow\" | \"tool\" | \"plugin\" | \"integration\"\n Name string\n Core bool // non-disableable\n State State\n Health string // reuses whatever the family's own registry already reports\n}\n\ntype Inventory interface {\n List(ctx context.Context) ([]Entry, error)\n}\n```\n\n**Decision.** `extensions.Inventory` is a read-only aggregator, not a new\ndomain that owns state. It queries each family's existing registry\n(`curator.Registry`, `health.Registry`, `workflow.Registry`, the MCP client\nset, `plugin.LoadDir`'s result) for live state, and compares that against\nthe config-desired state (below) to compute `StatePendingRestart` — the diff\nis computed on read, not persisted, so there is no new durable state to keep\nconsistent with the registries themselves. This follows\n`migration-decisions.md`'s rule that cross-cutting aggregation composes\nexisting domain-owned contracts rather than duplicating their state.\n\n**Call site.** New package `internal/domain/extensions/inventory.go`.\nComposed in `internal/app/archied` (wherever `server.go`'s other optional\nfields — Workflows, Skills, Curators — are wired) since it needs\nreferences to every family's registry, which only application-layer\ncomposition has.\n\n### Config-backed enablement\n\n**Decision.** Add `enabled bool` (default `true` for pre-existing configured\nentries, so no upgrade silently disables anything already configured) to\neach family's config element: `MCPServer.Enabled`, and equivalent fields on\nwhatever config types back skills/workflows/plugins if they don't already\nhave an on/off switch. Core capabilities (identity, workflow engine, State\nStore connectivity — named in `organisation.md`'s cross-cutting packages)\nhave no `enabled` field at all; a config value can't be added to something\nthat doesn't expose the toggle, which is the \"non-disableable\" property\nenforced structurally rather than by a runtime check that could be\nbypassed.\n\n**Call site.** `internal/config/config.go` — add the field per family\nstruct, not one generic `map[string]bool` (typed fields keep\n`docsgen`/`contracts.json` generation working, per `tools/docsgen`'s\n\"treat flags as executable truth\" rule — a generic map has no type to\ngenerate from). `config.example.toml` documents each new field with its\ndefault.\n\n**Reuse check.** The config `Holder` overlay system\n(`internal/infrastructure/configuration`) already handles safe saves; this\nadds fields to existing structs, it does not add a second config-write path.\n\n### Pending-restart and reconciliation\n\n**Decision.** No new \"pending restart\" flag is persisted. `Inventory.List`\ncomputes `StatePendingRestart` by comparing the config's `Enabled` value\nagainst whether the family's registry reports that entry as\nlive — if they disagree, the entry is pending restart. This is simpler than\ntracking a flag through a save-then-restart cycle and can never go stale,\nbecause it's recomputed on every read.\n\n**Call site.** `internal/domain/extensions/inventory.go`'s comparison logic.\n\n### API and UI surface\n\n**Decision.** New `api_extensions.go` alongside the existing `api_skills.go`\n/`api_workflows.go` pattern, exposing `Inventory.List` and a save endpoint\nthat writes through the existing `UpdateConfig` seam (`api_config.go`) —\nthis document does not add a second config-write endpoint. New\n`ui/src/extensions/` (colocated `.js`/`.css` per CLAUDE.md's frontend rule)\nas an index/dashboard that **links to**, not replaces, the existing\n`ui/src/skills/` and `ui/src/workflows/` detail views — Extensions centre is\nwhere an operator sees everything and toggles enablement; the family-specific\npages remain where they inspect a family's own detail.\n\n**Call site.** `internal/webui/api_extensions.go` (new, follows\n`api_skills.go`'s shape), `ui/src/extensions/` (new).\n\n## Call site inventory\n\n| concern | file | change |\n| ------------------------------------------------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------- |\n| Plugin metadata type | `internal/plugin/plugin.go` | none — already handles this |\n| Family controllers (curator/health/image/workflow registries) | `internal/domain/curator/registry.go`, `internal/domain/health/health.go`, etc. | none — pattern already proven, reused not rewritten |\n| Unified inventory | `internal/domain/extensions/inventory.go` | new |\n| Composition wiring | `internal/app/archied/server.go` (near existing optional fields) | new |\n| Config enablement fields | `internal/config/config.go` | new fields per family |\n| Config docs | `config.example.toml` | new field documentation |\n| API surface | `internal/webui/api_extensions.go` | new |\n| Existing skills/workflows APIs | `internal/webui/api_skills.go`, `api_workflows.go` | none — linked from, not replaced |\n| Config save path | `internal/webui/api_config.go` (`UpdateConfig`) | reused, not duplicated |\n| UI | `ui/src/extensions/` | new |\n| Existing UI detail views | `ui/src/skills/`, `ui/src/workflows/` | none — linked from Extensions centre |\n\n## Execution: multi-agent team breakdown\n\n| sub-feature | issue | implementer scope | suggested council lenses | why |\n| ------------------------------------ | ---------------------- | ---------------------------------------------------------------- | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |\n| Unified inventory type + composition | file as child of `#56` | `internal/domain/extensions/inventory.go`, wiring in `server.go` | `lens-boundary`, `lens-deletionist` | boundary: aggregator must not duplicate family-owned state; deletionist: must not become a new domain that shadows existing registries |\n| Config enablement fields | file as child of `#56` | `config.go` per-family `Enabled` fields, `config.example.toml` | `lens-contract` | config field is a wire contract other tooling (`docsgen`) generates from |\n| API + UI surface | file as child of `#56` | `api_extensions.go`, `ui/src/extensions/` | `lens-maintainer` | new user-facing state machine (available/enabled/active/degraded/skipped/pending-restart) must be legible to an operator with no prior context |\n\n## File and link the beads\n\n- `#56` itself has no filed children yet — file the three rows above as\n child issues linking to this doc's matching `##` heading.\n- Each issue's acceptance criteria should restate this doc's per-family\n `Enabled` default (`true` for existing config, no field at all for core\n capabilities) so \"non-disableable core\" isn't re-litigated per bead.\n- Add the suggested lenses to each issue.\n" }, { "url": "/dev/prds/external-agent-harness/", @@ -408,6 +408,12 @@ "section": "PRDs", "body": "# Playbook data-flow and condition syntax\n\n**Status:** Approved\n**Date:** 2026-09-03\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-t2db.14` (expression environment).\nUnbuilt decisions: `archie-core-t2db.25` (J1 action ids),\n`archie-core-t2db.26` (J2 args as CEL),\n`archie-core-t2db.27` (Result schemas declared to CEL).\n\n**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression\nmechanism covers both an action's `when` condition and its `args` values.\n\n## The requirement, stated directly\n\nA playbook action is gated and parametrised by runtime data: (a) a condition\ndeciding whether the action runs; (b) a later action's `args` values referencing\nan earlier action's `Result` and the triggering event's fields. Both are **CEL\nexpressions**, evaluated against one evaluation context. There is no separate\ninterpolation syntax: CEL reads nested maps and fields natively, so `args`\nvalues that need data are written as CEL, and only plain literal args remain\nplain YAML scalars.\n\nBoth halves are read-only: expressions may read context values and compute a\nresult from them, never mutate, never call host functions, never perform I/O.\nCEL enforces this structurally (no side effects, linear evaluation when macros\nare bounded, cost-limited).\n\n```yaml\nactions:\n - id: build\n position: module\n kind: log\n args:\n message: '\"build finished\"' # literal\n when: 'event.label == \"bugfix\"' # gate on event field\n - position: module\n kind: log\n args:\n message: '\"done: \" + string(actions.build.result.written)'\n when: \"actions.build.result.written == true\"\n```\n\n## Trust boundary of the expression\n\nA playbook YAML is **operator-installed, in-process, daemon-privileged** -- the\nsame tier as `ModuleDir`/`PluginDir`/`SecretEngineDir` (module-position.md's\ntrust-boundary section): loaded from a configured directory at startup, never\nfrom a webhook body or task worktree. The schema-by-example flow uses live\nevents to _design against_; the saved playbook is operator-authored file content\nin an operator-configured location.\n\nTwo consequences:\n\n1. The expression string is **trusted**. Operator-authored config, not attacker\n input; no sandboxing against a hostile expression string is required. If a\n future playbook source is less trusted (auto-generated bindings from\n unauthenticated captures), that is the playbook-file trust tier moving -- the\n loader's acceptance check is the right place, not the evaluator.\n2. The **data the expression reads is NOT trusted** (decoded webhook body, prior\n results through the same pipeline). Robustness against hostile data is\n required: no panic on wrong-typed/missing fields, no unbounded cost on\n deep/cyclic maps, bounded evaluation time. CEL provides this structurally\n (cost limits, linear non-Turing-complete evaluation, error values instead of\n panics -- verified in the vetting record below).\n\n## The gate precedent is deliberately NOT copied\n\n`internal/gate/gateeval` evaluates arbitrary interpreted Go (`.archie/gate.go`)\nas the repo's conditional-logic precedent. That is the wrong precedent for\nplaybook conditions because this document's premise is: **YAML is pure\norchestration data; Yaegi is where imperative logic lives** (Problem section;\nmodule-position.md's \"no generic hook\" section). Yaegi snippets in YAML would\nmove imperative logic into the data layer -- the inversion this design exists to\nprevent -- and make playbooks un-lintable.\n\nThe line: conditions are **declarative predicates over runtime data, not\nprograms**. CEL is non-Turing-complete and side-effect-free, so it cannot become\na program; a condition that needs real logic gets a Module kind (Yaegi),\nreferenced by the action, not a stronger expression body.\n\n## Plugin-engine-rule interaction\n\nThe plugin engine rule forbids a generic `Module` interface with an `any`\npayload; it exists to keep static implementation contracts typed. Evaluating CEL\nagainst runtime `map[string]any` is **not the same problem**: the rule concerns\n_implementation contracts_ (static operations a capability family exposes), not\n_runtime data values_ (legitimately dynamic -- an event payload's shape is\nunknown until it arrives). The Module contract stays fully typed (generated\n`Args`/`Result` schemas); CEL is a read-only data reader under the playbook\nengine family, not a new capability contract. Stated once.\n\n## Minimum viable evaluation context (what expressions see)\n\nOne flat context, namespaced from the playbook run, built at dispatch time:\n\n| Name | Type | Source |\n| --------- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `event` | `map(string, dyn)` | The triggering event's decoded payload (webhook body / forge issue / schedule tick). Field access via `event.\u003cfield\u003e`, map access via `event[\"field\"]`, presence via `has(event.\u003cfield\u003e)`. |\n| `actions` | per-playbook object type | Previous actions' results, one field per declared action `id` (`actions.\u003cid\u003e`), each field typed `{ result: \u003cKindResult\u003e }` so a later action reads an earlier one's `Result` struct (`actions.\u003cid\u003e.result.\u003cfield\u003e`) with field-level checking. The current action and later actions are not present. |\n\nExpressions may also read literal-only state (numbers, strings, booleans)\ndirectly. CEL's `has()` macro covers the missing-key case (`has(event.labels)`\n-\u003e bool), which replaces any bespoke `has`/ `contains` operator.\n\n`actions.\u003cid\u003e.result` is the kind's hand-written `Result` struct:\n`ModuleRegistry.DecodeResult` converts `Module.Invoke`'s flat `map[string]any`\n(`archie-core-t2db.13`) back into that struct before a later expression reads\nit, so the checker and the run read the same typed value.\n\nJudgement call J1: `action` id is a required field on every action that later\nactions reference; ids are validated at load (unique, stable-identifier-shaped),\nand a `when`/`args` expression referencing an unknown `actions.\u003cid\u003e` is a **load\nfailure**, not a runtime miss -- the same reject-at-load rule as collisions (see\nbelow).\n\n## Load-time validation (consistent with the collision rule)\n\nThe collision-handling rule is: schema defines the accepted message; anything\noutside it is rejected at load, logged plainly, reported to the caller -- never\ndeferred to runtime. Expressions are validated the same way, at playbook load:\n\n- **Syntax**: every `when` and every CEL-typed `args` value is parsed and\n checked by CEL's checker when the playbook is loaded. A syntax or type error\n is a reported load failure (the playbook is dropped and the error goes to the\n caller), not a runtime surprise.\n- **Context conformance**: with the context declared to CEL (`event` as\n `map(string, dyn)`, `actions` as a per-playbook object type with one typed\n field per declared id), unknown top-level identifiers are rejected at compile\n time. Field-level typing inside `dyn` values is deferred to runtime by CEL's\n design (verified: `data.items.lenght` with `data` dyn compiles clean and\n errors at eval) -- so the loader declares as much type as the schema provides\n (the action kinds' generated `Result` structs become CEL type declarations at\n load, giving field-level rejection for result reads), and `event` stays `dyn`\n because its shape is unknown by design (schema-by-example).\n- **Names**: unknown `actions.\u003cid\u003e` references and unknown roots are\n compile-time/lint-visible errors (J1).\n\nThis means the lint tool and the daemon share one checker -- same single-source\nargument as the playbook loaders.\n\n## Candidate comparison\n\n| Candidate | Maintenance | Safety surface | Cost (measured) | Verdict |\n| ------------------------ | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| **CEL (cel.dev/cel-go)** | Actively maintained (google/cel, monthly releases) | Non-Turing-complete, linear eval, `CostLimit`/`CostTracking`/`ParserRecursionLimit`, no side effects, error values not panics | **+13.0MB** (2.34MB -\u003e 15.25MB) | **Chosen** |\n| `expr-lang/expr` v1.17.8 | Actively maintained | Side-effect-free, `DisableAllBuiltins`, depth limits (CVE fixed w/ tests) | +3.9MB (2.34MB -\u003e 6.28MB) | Not chosen: two safety CVEs this cycle; smaller ecosystem; no protobuf/type-model fit with future schema-gen |\n| `gval` v1.2.3 | Maintained, slower | No builtin-restriction surface; extras (ternary/`??`) not wanted | +3.0MB (2 transitive deps) | Not chosen |\n| `govaluate` | **ARCHIVED** (author archived 2024) | n/a | n/a | Not chosen: dead |\n| `starlark-go` | Maintained (Bazel) | Turing-complete with step limits | large (full language) | Not chosen: whole-language surface |\n\n## Vetting record\n\nSource reviewed at HEAD:\n\n- **Repo hygiene**: clean. No binaries/media in tree (the only non-source\n artifacts are expected generated protobuf files); 16-line `.gitignore`.\n- **Hard-boundary correctness**: the place correctness is hard for an expression\n evaluator is bounding evaluation and surviving hostile input. Dedicated\n regression tests exist and pass at HEAD: `TestCostLimit`, `TestCostTracking*`,\n `TestParserRecursionLimit`, and a large conformance suite. Verified by\n execution: evaluation against a 2000-deep nested map under `CostLimit`\n completes in ~10 microseconds and returns an error value for a missing key --\n no panic, no unbounded walk.\n- **Dependency footprint**: antlr (parser), protobuf + genproto (CEL's type\n model), `cel.dev/expr` (CEL spec types), yaml.v3. All mainstream, canonical,\n vendored widely by Google-adjacent projects. No runtime-impure third-party\n code; the REPL-only deps (readline, tview, tcell) are not in the library\n import graph.\n- **Security record**: no advisories on record for `cel.dev/cel-go` (GitHub\n Advisory Database).\n- **Module path**: migrated to `cel.dev/cel-go` (v0.32.0 is current); the old\n `cel.dev/cel-go/cel` path is the pre-migration identity. Pin `cel.dev/cel-go`.\n- **Binary size cost**: +13.0MB is the largest of the candidates (measured:\n baseline 2.34MB -\u003e 15.25MB with CEL). Accepted because: the daemon already\n embeds a large Yaegi runtime; CEL's type model is the load-bearing win\n (compile-time rejection of result-path typos, matching the reject-at-load\n rule); and the alternative bespoke grammar would re-implement CEL's checker to\n get the same guarantee. Flagged for sign-off: if 13MB is unacceptable for a\n specific deployment target, the fallback is a bespoke grammar with a\n re-implemented subset of this checker -- a strictly worse trade.\n\n## Why bespoke was not sufficient\n\nA bespoke grammar for the shipped cases (path read + comparison + bool\ncomposition) is small, but the load-time-validation requirement makes the\ncomparison: the reject-at-load rule requires _field-level_ static checking of\nresult paths against generated `Result` schemas, which is CEL's type-checker's\njob. Re-implementing that on a hand-rolled grammar means re-implementing a type\nsystem -- more code than the grammar itself, with worse diagnostics, no\nconformance suite, and no community surface for future expression needs (even\ndeclarative ones). The bespoke option only wins on binary size, and the size\ncost of CEL is measured and bounded.\n\n## Judgement calls (flagged for sign-off)\n\n- **J1 -- `id` required on referenced actions, unknown id = load failure.**\n Matches the reject-at-load rule. Alternative considered: an alternative is\n positional `actions[0]` access, which breaks on reorder; named is the\n operator-friendlier shape.\n- **J2 -- every `args` value is evaluated as a CEL expression.** No split\n between \"literal args\" and \"expression args\": a plain string value is a CEL\n string literal, a number is a CEL number literal, and any value that reads\n context data is written in CEL. One mechanism, no interpolation marker, no\n second syntax to learn. Alternative considered: cEL string literals must be\n quoted inside YAML (e.g. `message: '\"build finished\"'`), which is slightly\n noisy for mostly-literal args; accepted for uniformity -- the linter catches\n quoting mistakes at load.\n- **J3 -- condition failure semantics.** A condition that errors at runtime\n (missing/wrong-typed field) evaluates to **false**: the action is skipped,\n logged, run continues. This is the simplest candidate consistent with gap 1's\n bias against clever recovery; gap 1 (`archie-core-t2db.18`) commits\n stop-on-first-failure, so per-action control is not coming and J3 stands.\n Alternative considered: cEL separates false from error; we treat evaluation\n error as false + log.\n- **J4 -- `event` stays `dyn` (schema-by-example).** Load-time we cannot know\n event field types; declaring them typed would reject valid plays. Result reads\n get field-level checking via generated schema declarations. Alternative\n considered: when schema-by-example ships, tighten event field declarations\n from captured samples.\n- **J5 -- cost limit default.** A `CostLimit` at load time (e.g. 100,000) is\n applied to every expression; the linter and daemon use the same value so they\n agree. Alternative considered: exact number is a constant to tune at first\n use.\n\n## Interaction with the execution-time gaps\n\nThis resolution does not resolve the execution-time gaps (both are resolved\nseparately: gap 1 by `archie-core-t2db.18`, gap 2 by the gap-2 decision).\nConditions make gap 1 concrete (condition-failure vs. action-failure semantics,\nJ3) and CEL's read-only result access makes gap 2's keying derivation readable\n(an idempotency key expression can read event fields) -- both flagged as the\ngaps' first real exercise. Gap 2's exercise is subsequently solved by that\ndecision: the answer is a fixed structural derivation, and idempotency keys are\ndeliberately NOT CEL expressions (see the \"No CEL in the key\" passage).\n\n## Implementation placement (for the follow-up slice)\n\n- New `internal/domain/eda/expr` package: builds the CEL environment (context\n declarations, generated `Result` schema types as CEL type declarations, cost\n limit), compiles/validates playbook expressions at load, evaluates them at\n dispatch. Domain layer; no infrastructure imports; owned by the playbook\n engine family. One new dependency: `cel.dev/cel-go` (pinned), `go.mod`\n otherwise unchanged.\n- Consumption: the playbook event coordinator (open question 4's loader)\n evaluates `when` before dispatch and evaluates `args` values before\n `Module.Invoke`; the lint tool (`archie-core-t2db.12`) shares the same\n compile/validate path for author-time diagnostics.\n- Schema integration: action kinds' generated `Result`/`Args` structs\n (`archie-core-t2db.13`'s `logextract` pattern) are declared to CEL at load so\n `actions.\u003cid\u003e.result.\u003cfield\u003e` type-checks; `args` expressions type-check\n against the target kind's `Args` schema before the playbook is accepted.\n" }, + { + "url": "/dev/prds/pr-review-agent/", + "title": "PR review agent", + "section": "PRDs", + "body": "# PR review agent\n\n**Status:** Draft\n**Authority:** `docs/architecture/agent-system.md`; supersedes the reviewer\nexecutor and blocking rule in `docs/architecture/adversarial-review.md`, and\nthe designs in `docs/prds/operator-pr-review.md` and\n`docs/prds/review-calibration.md`.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (every agent\ncall is a StepExecution), `docs/prds/external-agent-harness.md` (any agent\nstep may run on a harness), `docs/prds/inline-review.md` (posting),\n`docs/prds/pr-review-remediation.md` (consumes the findings).\n\n## Decision\n\nArchie ships a pull request reviewer as its first end-to-end product. It\nreviews any pull request on a watched repository, a PR an operator names, and\narchie's own PRs before they open.\n\nThe reviewer is a fixed pipeline, not a single agent. The review strategy is\ngenerated per PR: three lens agents propose review dimensions from the PR's\ncontent, one reviewer definition runs once per dimension, and later agents\nverify, challenge and combine the findings. Everything that can be computed\nis code: diff parsing, clustering, blast radius, evidence extraction,\nscoring, deduplication, line mapping and the review event.\n\nThe pipeline is **recall-first**. It posts every evidenced finding above the\nconfidence threshold, up to the comment cap. A precision gate exists as an\noperator dial and is off by default.\n\nAcceptance is measured, not asserted: see Verification.\n\n## Pipeline\n\nEach numbered phase is a stage of the `pr-review` workflow. Each agent call is\nan `agent` StepExecution; fan-out phases record one child per call.\n\n1. **Intake.** Code reads the PR metadata and diff statistics: type,\n complexity, languages, areas touched, risk signals. Depth comes from lines\n changed: under 100 is `quick` (at most 3 dimensions), under 500 is\n `standard` (6), otherwise `deep` (12). An operator may set depth\n explicitly.\n2. **Anatomy.** Code parses the diff into files and hunks, clusters related\n files, and computes the blast radius: unchanged files that import or are\n imported by changed files. One agent writes the PR narrative, risk\n surfaces, changes that do not fit the narrative, and gaps between the\n description and the diff.\n3. **Dimension selection.** Three lens agents run in parallel over the intake,\n anatomy and diff, each able to read the repository:\n - **behaviour:** where the old and new code diverge for any input; API\n contracts, concurrency and state, security, error handling, data flow;\n - **mechanics:** whether the code runs at the language and framework\n level; types, signatures and every caller, decorators and middleware,\n framework contracts, imports;\n - **fit:** patterns, complexity, abstraction, test adequacy, documentation\n that no longer matches, dependencies, migration completeness.\n\n Each lens returns review dimensions: a name, a reviewer prompt written for\n this PR, target files, context files and a priority. Code merges them,\n drops duplicates and keeps the highest-priority dimensions up to the\n depth's cap.\n4. **Review.** One reviewer definition runs once per dimension, in parallel,\n at most 8 at a time. Each reads its target and context files and may\n follow references. Each finding carries file, line range, severity\n (`critical`, `important`, `suggestion`, `nitpick`), title, body, an\n optional suggested fix, quoted evidence, confidence and tags.\n5. **Verification layer**, over the review findings:\n - code extracts an evidence package per finding: the code at the cited\n lines and snippets from its callers;\n - an evidence verifier checks high-priority findings against their\n packages;\n - an adversary confirms or challenges each finding, looks for issues every\n reviewer missed, and is more sceptical when intake scores the PR as\n likely machine-written;\n - a compound agent looks, within each cluster of confirmed findings, for\n defects that only exist in combination.\n6. **Coverage and consistency**, in parallel:\n - a coverage gate checks every cluster and high-exposure blast-radius file\n was reviewed, and runs gap reviewers for what was not, at most 2 rounds;\n - consistency verification lists every cross-location obligation the\n changed code creates (what must be true elsewhere for this line to be\n correct), then verifies each by reading both ends. A broken obligation\n becomes a finding.\n7. **Synthesis**, code only:\n - score = severity weight (1.0, 0.7, 0.3, 0.1) × confidence × multipliers:\n compound 1.5, adversary-confirmed 1.3, adversary-challenged 0.5, likely\n machine-written PR 1.2, blast radius over 10 files 1.2;\n - drop findings under the severity's confidence floor: critical 0.2,\n important 0.3, suggestion 0.4, nitpick 0.4;\n - merge exact duplicates (file, overlapping lines, category) in code, and\n near-duplicates through a small classification call that keeps both\n when unsure;\n - map lines to the diff's coordinates, sort by score, cap at 25 inline\n comments.\n8. **Merge gate.** One classification call per finding decides blocking or\n advisory: blocking only for broken builds, security, data loss, contract\n breaks and regressions. A failed call means advisory.\n9. **Output.** One call per comment tightens its wording, keeping the\n original on failure. The review is posted as one forge review with inline\n comments. Any blocking finding requests changes; otherwise the review is a\n comment. Archie never posts an approval.\n\n## Precision dial\n\nWith `review.precision_gate = true`, a post-worthiness pass runs after phase 4\nand before the verification layer. It keeps every concrete, evidenced defect\nand drops nitpicks, style and unverifiable claims, keeping a finding when\nunsure. It trades recall for precision and cuts verification cost by\nfiltering early.\n\n## Operator approval\n\nWith `review.approve_before_post = true`, synthesis ends in\n`waiting_human`. The operator chooses which findings to post, rejects the\nreview, or asks for a re-review with instructions. A re-review reruns phases\n3 to 8 with the instructions added to the lens and reviewer prompts, at most\n2 times. A review with no findings posts nothing and does not wait.\n\n## Isolation\n\nEvery agent in the pipeline reads a `.git`-free snapshot of the PR head, with\nread-only tools rooted at it, and never the implementer's worktree or\ntranscript. No pipeline agent holds a forge credential; posting is code.\n\n## Budget\n\nA run has a cost cap and a wall-clock cap, split into per-phase shares. A\nphase whose share is spent is skipped and recorded on its StepExecution. The\nposted review names every skipped phase, so an exhausted run never reads as a\nclean one. A run whose review stage fails posts nothing and\nfails.\n\n## Triggers\n\n- **Watched repositories:** a PR opened or updated on a repository with\n `review.enabled` starts a run. An update cancels the in-flight run for the\n same PR.\n- **Operator:** a PR URL from the dashboard or chat starts a run on any\n repository the identity can read.\n- **Archie's own PRs:** the implement workflow runs the pipeline before\n opening its PR. A blocking finding the adversary did not challenge parks\n the task with the findings. A pipeline that fails parks the task. Advisory\n findings are posted after the PR opens.\n\n## Models\n\nEach agent phase names a model role. Lens, reviewer, verification,\nadversary, compound and consistency agents use the review role. Intake,\ncoverage, dedup, merge gate and polish calls use the classification role.\nEither role may run on the built-in runner or a harness.\n\n## Out of scope\n\n- SARIF and standalone report formats. The findings are StepExecution outputs\n and the forge review.\n- Reviewing a diff without a forge PR.\n- Self-consistency reruns.\n\n## Plan\n\n1. Deterministic core: diff parsing, clustering, blast radius, evidence\n extraction, scoring, dedup, line mapping, review event. Table-driven tests.\n2. The benchmark harness (Verification), run against the deterministic core\n with fixture findings.\n3. `pr-review` workflow: intake, anatomy, lenses, reviewers, synthesis,\n output, with the execution tree recording each call.\n4. Verification layer, coverage loop and consistency verification.\n5. Merge gate, polish, budget shares and skipped-phase reporting.\n6. Benchmark acceptance run.\n7. Triggers: operator, watched repositories, archie's own PRs replacing the\n single-reviewer stage.\n8. Precision dial and operator approval.\n\n## Verification\n\n- **Benchmark acceptance.** On the 38 runnable PRs of the Martian\n Code-Review-Bench offline set, one blind run per PR at `deep` with the\n review and classification roles on GLM-5.2, and an independent judge\n (`anthropic/claude-sonnet-4.6`) matching posted comments to golden\n comments: golden micro-recall is at least 0.70. Golden-only precision is\n reported alongside it. The run is reproducible from a script in the\n repository.\n- Scoring, thresholds, dedup and line mapping give identical output for\n identical findings.\n- A run whose phase share is exhausted posts a review naming the skipped\n phase.\n- An update to a PR under review cancels the running review and its child\n steps.\n- No pipeline agent can write to the snapshot, and none can reach a forge\n credential.\n- Archie's own PR with an unchallenged blocking finding is parked, not\n opened.\n- Archie never submits an approving review.\n" + }, { "url": "/dev/prds/pr-review-remediation/", "title": "PR review comment remediation -- decision", @@ -803,6 +809,10 @@ "title": "Playbook data-flow and condition syntax", "url": "/dev/prds/playbook-expression-syntax/" }, + { + "title": "PR review agent", + "url": "/dev/prds/pr-review-agent/" + }, { "title": "PR review comment remediation -- decision", "url": "/dev/prds/pr-review-remediation/" diff --git a/docs/prds/extensions-center.md b/docs/prds/extensions-center.md index bba5a187f..7c4d39fe2 100644 --- a/docs/prds/extensions-center.md +++ b/docs/prds/extensions-center.md @@ -2,6 +2,8 @@ **Status:** Draft +> Superseded by `docs/prds/store.md`. + Epic: `archie-core-1786637490708-35-38424ead` / GitHub `#56` ("Replace Skills with a bundled Extensions centre"). diff --git a/docs/prds/store.md b/docs/prds/store.md new file mode 100644 index 000000000..5413518d5 --- /dev/null +++ b/docs/prds/store.md @@ -0,0 +1,132 @@ +# Store + +**Status:** Draft +**Authority:** `docs/architecture/plugins-and-extensions.md`; supersedes +`docs/prds/extensions-center.md`, whose installed-inventory states are +adopted below. +**Compounds with:** `docs/prds/external-agent-harness.md` (harness Kits), +`docs/prds/pr-review-agent.md` (the first package). + +## Decision + +Operators get capabilities from a store: browse, install, update and remove +packages from the dashboard, with no config file editing. A package is an +OCI image in a registry. A store catalogue is a list of package references. +Installing pins a digest. Every package declares the authority it needs, and +an update that asks for more stops for the operator's approval. + +## Packages + +Two kinds, both ordinary OCI images: + +- **Harness Kits.** Kit v3 images, resolved as + `docs/prds/external-agent-harness.md` decides. Their authority is their + Kit permission surface. +- **Archie packages.** An image whose manifest annotation + `dev.archie.package.v1` carries the descriptor and whose single layer + carries files. A package contributes any of: workflow definitions, + prompts, skills, MCP server definitions, and default settings for them. + +Archie packages are declarative. They carry no Go, Yaegi or script code that +archie executes on the host. Anything that runs code ships as a Kit and runs +in a sandbox. An archie package that references a Kit names it by digest, +and installing the package installs the Kit. + +The descriptor declares: display name, description, version, the families +it contributes, the packages it requires, and its authority. + +## Authority + +A package's authority is the set of grants it needs to work: + +- credential services (from its Kits and MCP servers); +- egress hosts (from its Kits); +- forge permissions its workflows use: read, comment, review, push, open + pull request; +- triggers it registers: watched-repository events, schedules; +- tools it enables for agents. + +Install shows this list and records the operator's acceptance against the +digest. An update whose authority is equal or narrower applies under the +package's update policy. An update whose authority is wider waits for +approval, showing only what widened. Nothing a package does at runtime can +exceed its accepted authority: archie checks forge actions, triggers and +tool grants against it, and the egress proxy enforces the Kit's grants. + +## Catalogues + +A catalogue is an OCI artifact listing package references with digests, +signed by the catalogue's key. Archie ships with the default catalogue and +its public key configured. An operator can add catalogues, for example a +private registry, by reference and public key. A catalogue without a key is +refused unless the operator marks it unsigned, which the dashboard shows on +every package it supplies. + +Install and update verify the catalogue signature and the package digest. +Archie never installs a tag it has not resolved to a digest through a +verified catalogue. + +## Install, update, remove + +- Installed packages, their pinned digests, accepted authority and update + policy are State Store resources. They apply without a restart. +- **Update policy** per package: `manual` (default) or `auto`. `auto` applies + non-widening updates when the catalogue lists them; widening updates always + wait. +- **Setup.** A package whose authority includes a credential with no binding + installs as `needs_setup`, and links to the binding form or the harness + setup terminal. Its triggers do not fire until setup completes. +- **Remove** stops its triggers, removes its contributions, and leaves any + credential bindings for the operator to delete. A package another + installed package requires cannot be removed first. +- Rolling back reinstalls the previously accepted digest. + +## Inventory + +The installed view lists every package and every bundled capability with +one state, computed on read and never persisted: + +| State | Meaning | +| ------------- | -------------------------------------------------------- | +| `available` | in a catalogue or bundled, not installed | +| `needs_setup` | installed, waiting for a credential binding | +| `active` | installed and running | +| `degraded` | active, and its family's health check reports a fault | +| `pending` | an update waiting for approval of widened authority | + +Core capabilities (identity, workflow engine, State Store) are listed and +cannot be removed. + +## Out of scope + +- Payment, licensing and paid packages. +- Publishing from the dashboard. Packages are built and pushed with standard + image tooling. +- Ratings and reviews. + +## Plan + +1. Archie package descriptor and its validation, including declarative-only + content. +2. Installed-package resources, digest pinning, install and remove for + archie packages from a local registry. +3. Authority model: extraction from packages and Kits, acceptance records, + runtime checks for forge actions, triggers and tools. +4. Signed catalogues and verification; the default catalogue. +5. Updates: widening detection, approval, `auto` policy, rollback. +6. Kit packages through the same flow; `needs_setup`. +7. Inventory states across packages and bundled capabilities. +8. The PR review agent published as the first package. + +## Verification + +- Installing the PR review package from the default catalogue on a fresh + instance, then binding a credential, reviews a PR without editing + configuration. +- An update adding a forge permission or egress host waits as `pending`, + showing only the added grant, and does not apply under `auto`. +- A workflow from a package that tries a forge action outside its accepted + authority fails the step. +- A tampered package digest or an unsigned catalogue refuses to install. +- A package containing executable host code is rejected at validation. +- Rolling back restores the prior digest and its accepted authority.