diff --git a/docs/data/generated/dev-docs.json b/docs/data/generated/dev-docs.json index 02f5d3a1b..91b5b11a1 100644 --- a/docs/data/generated/dev-docs.json +++ b/docs/data/generated/dev-docs.json @@ -226,7 +226,7 @@ "url": "/dev/prds/eda-playbook-engine/", "title": "EDA playbook engine -- design", "section": "PRDs", - "body": "# EDA playbook engine -- design\n\n**Status:** Draft\n**Date:** 2026-09-03\n**Beads epic:** `archie-core-t2db`\n\n## Problem\n\n\"Which workflow runs for which trigger\" is hardcoded Go: a literal\nlabel vocabulary (`workintake.labelKinds`) and a literal label-\u003eworkflow map\n(`workflow/routing.go`), extended only by adding a Go case or shipping a\nskill whose `metadata.archie.workflow` overrides one *named* built-in\nworkflow. Extending the system to a new trigger, a new action, or a new\nevent source means a feature branch, a code change in a hardcoded\npath, and new Go tests -- for every addition. That is the barrier this\ndocument exists to remove.\n\nThe goal: triggers, the actions they run, and the\nconditions that connect them should be **drop-in and load**, not **drop-in\nand touch application code**. Yaegi-evaluated Go is the unit of imperative\n\"do a thing\" logic; YAML playbooks are pure orchestration data -- no logic,\njust \"on this event, with this data, call these actions, in this order,\nunder these conditions.\" The daemon's event coordinator doesn't know what a\n`bugfix` or `image-gen` action does, it resolves the name against a\nregistry and pipes typed data through.\n\nThis is explicitly **not** scoped to code-generation workflows. A playbook\naction can be any registered capability -- run a codegen workflow, send a\nmessage on a channel, open something on a forge, generate an image -- so\nlong as it's implemented behind one of the typed engine families below.\n\n## Relationship to existing decisions\n\n- `docs/prds/event-sources-and-reactions.md` ruled\n that reactions in the two epics it covers are **producer-only** (no veto,\n no mutation of in-flight work) and explicitly rejected a single generic\n `Source` interface as \"the untyped-hook shape the plugin engine rule\n exists to prevent.\" **This document extends that scope**: playbooks can\n dispatch to multiple typed action kinds, not just \"produce a\n `TaskEnvelope`.\" It does not relitigate producer-only-ness (still no\n veto/mutation of in-flight work here) and it does not introduce one\n generic interface -- see \"Typed positions, not one generic hook\" below,\n which is how it stays compliant with the plugin engine rule instead of\n reopening that document's #4.\n- `docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`\n governs every extension point\n introduced here. Every position below is a typed domain contract with an\n owning registry, not a generic callback map. This is a hard constraint,\n confirmed against the existing rule text before writing this doc.\n- Yaegi is already a proven in-process extension mechanism\n (`internal/plugin`, `internal/domain/workflow/wfeval/yaegi.go`,\n `internal/gate/gateeval/yaegi.go`, `internal/skill/plugin.go`) -- this\n design generalizes its existing role, it does not introduce a new\n execution mechanism.\n- `internal/domain/workflow/skillbuild.BuildRegistry` already composes a\n `workflow.Registry` from a skills catalog, plugin-defined overriding\n built-in. The playbook loader described here supersedes label routing\n specifically; it does not replace `skillbuild`'s workflow-composition role,\n it becomes an additional input the daemon composes at startup.\n- Webhook event intake is a separate design. This\n document defines the router each webhook-sourced event should be able to\n reach; it does not redesign webhook receipt itself.\n\n## Shape: four logic positions, each a typed engine family\n\nEvery entrypoint kind (mechanism, action, loop, schedule, route, trigger) is itself a schema emitter: defining a new kind produces the\ngenerated contract downstream tooling consumes. The kinds slot into a fixed\nhierarchy of logic positions:\n\n```\nWorkflow -\u003e Module -\u003e Channel (telegram/whatsapp/email/...)\n -\u003e Forge (GitHub/Gitea/GitLab/...)\n```\n\nEach position is its own typed engine family per the plugin engine rule:\nidentity + capability-specific operations, an owning `Registry`/`Manager`,\nexplicit lifecycle where the implementation owns resources, narrow host\naccess. Concretely:\n\n- **Workflow** -- already exists (`internal/domain/workflow`). A playbook\n action of kind `workflow` invokes a registered `workflow.Registry` entry\n by name, unchanged from the existing execution path.\n- **Module** -- the new general-purpose action position. A `Module` is a\n Yaegi-evaluated implementation of a generated interface (args in, typed\n result out) for one declared action kind (e.g. `image-gen`, `notify`,\n `open-pr-comment`). This is the position that makes actions \"not\n restricted to code-generation.\" Full design (schema-per-kind, the\n registry shape, the trust boundary, the recommended first slice):\n `docs/prds/module-position.md`.\n- **Channel** -- already exists as `channels.Channel` (telegram, email,\n webhook). A playbook action of kind `channel` sends through a named,\n already-registered channel. No new mechanism; this document just makes it\n addressable from a playbook.\n- **Forge** -- forge clients (GitHub, Gitea, GitLab) become an addressable\n action position the same way Channel is, so a playbook can say \"comment\n on this forge issue\" without that being a special case wired only into\n the poller.\n\n**Typed positions, not one generic hook.** There is no single `Action`\ninterface with an `any` payload. Each position keeps its own typed\ncontract; the playbook YAML names a position and an implementation, and the\nevent coordinator resolves that pair against the matching typed registry.\nThis is what keeps the design compliant with the plugin engine rule and\nwith `event-sources-and-reactions.md`'s rejection of a generic `Source`/\nhook interface -- the generality lives in the YAML data model (which\npositions exist, which named implementations are registered), not in a\nruntime `any`-typed dispatch path.\n\n## `go:generate`-driven schemas, from day one\n\nSchemas are not a future nicety bolted onto an ad hoc\nYAML shape. They are the struct+interface contracts for each logic\nposition, and they drive code generation from the start.\n\n- Each logic position (Workflow/Module/Channel/Forge) has a schema source\n (Go-annotated types, following the existing pattern elsewhere in the\n repo) from which `go:generate` emits the interface and struct scaffolding\n an implementation must satisfy.\n- A Yaegi module implements the generated interface for its position. The\n generated contract is what makes an implementation valid, not hand-review.\n- The schema is also what the linter and LSP (below) validate a playbook\n YAML against -- one source of truth for codegen, startup loading, and\n authoring-time tooling, so the three can't drift from each other.\n- Schema versioning is explicit per position, independent of a playbook's\n own workflow version (see collision handling below).\n\nThe generator itself is not designed here. The constraint is\nschema-first, `go:generate`-driven, per logic position.\n\n## Playbook YAML: pure orchestration data\n\nA playbook declares no embedded logic -- only:\n\n- **trigger**: the event predicate (e.g. forge issue assigned + labelled\n `bugfix`, a schedule tick, a webhook payload shape).\n- **actions**: an ordered list, each naming a logic position and a\n registered implementation, with typed argument data and optional\n conditions.\n- **data flow between actions**: results/facts from one action available to\n later actions in the same playbook run -- exact interpolation syntax is\n an open point, not decided here (see Open questions).\n\nPlaybooks live in a configured directory (repo/directory-scoped, per Sam's\n\"define repos, directories, playbooks\" framing) and are loaded at archied\nstartup, alongside the existing skills-catalog composition.\n\n## Dedup: detect and log, no arbitration mechanism\n\nNo runtime auto-resolution rule. An automatic \"pick a winner\" algorithm\n(e.g. by version) is itself a mechanism that can fail and need debugging\nwhen a collision happens -- the wrong kind of complexity to introduce for\na case that should just be visible and fixed by a human. Instead:\n\n- **At daemon startup:** if independently-sourced playbook directories\n declare a collision on the same trigger, that is a definition failure.\n The colliding definitions are dropped (not loaded) and the failure is\n logged plainly (which sources, which trigger) and reported back to the\n caller that requested the load -- not swallowed as a background log line\n only. The schema is the definition; a schema conflict is a reported\n failure, not something the daemon silently arbitrates.\n- **Linter**: a CI/dev-time tool, reading the same generated schemas, that\n flags duplicate or conflicting trigger definitions *within* a single\n maintained tree before merge.\n\n **Resolved** (`archie-core-t2db.12`). A standalone\n gopls-shaped binary (`cmd/archie-playbooks`, lint subcommand) validates\n playbook directories against the exact loaders the daemon uses\n (`LoadPlaybookDirs`/`LoadKindWorkflowsYAML`/`LoadLabelWorkflowsYAML`),\n exiting non-zero on any collision / malformed file / invalid binding.\n A finding about one binding key leads with the key's `file:line`, read\n from the `yaml.Node` the loader decodes; the linter agrees with runtime\n validation by construction, which is the load-bearing property. That\n agreement covers the routing loaders only: EDA playbooks under\n `eda_playbook_dir` (`playbook.Load`, with its CEL and action-id checks)\n have no lint path yet and are validated only at daemon startup.\n Discoverable via `task lint:playbooks` or direct `go run\n ./cmd/archie-playbooks lint -dir ...`. The LSP/serve mode is a later\n entrypoint of the SAME binary, per the shared-package decision above.\n- **LSP**: same schema source again, feeding author-time hover/completion/\n inline errors while someone edits a playbook YAML in an editor. Kept\n explicitly in scope per Sam (not deferred), because it's the same schema\n artifact as the linter's, just a different consumer.\n\nThe startup log-and-refuse path exists for what the linter/LSP structurally\ncannot see -- sources composed together only at daemon startup, never\nchecked against each other by either tool. If the linter and LSP do the job\nthey're built for, a real collision reaching that startup path should be\nrare in practice, not a routine occurrence the design leans on.\n\n**Shared package, thin consumers -- not a bundled-vs-standalone choice.**\nThe validation logic (`LoadKindWorkflowsYAML`, `LoadLabelWorkflowsYAML`, the\ndirectory-merge from t2db.11) lives in `internal/domain/workflow`, a plain\nGo package with no knowledge of which binary calls it. This makes \"should\nthe linter/LSP be bundled into archied or a separate tool\" a non-question:\n- `archied` already imports the package directly and validates in-process\n at startup -- no subprocess, no external tool call.\n- An agent that needs to validate a playbook in-process can import the same\n package directly, for the same reason.\n- The standalone `cmd/` binary (t2db.12, gopls-shaped: one binary, a lint\n mode now, an LSP/serve mode later) exists only because CI and editors are\n out-of-process consumers by nature -- a CI step needs an exit code, an\n editor speaks LSP over a process boundary, neither can `import` a Go\n package. It is a thin wrapper per `organisation.md`'s `cmd/` rule, not a\n second implementation.\n\nFolding CLI/LSP protocol handling into `archied` itself would grow the\ndaemon into unrelated concerns, against the plugin-engine rule's spirit of\nnarrow capability families.\n\n## Execution-time gaps (both resolved)\n\nEverything above covers *loading* a playbook. Nothing above covers what\nhappens while one *runs*. Two gaps, both load-bearing enough to resolve\nbefore implementation, not defer:\n\n### 1. Mid-playbook action failure\n\n**Resolved** (`archie-core-t2db.18`). The default below\nis a committed decision, not a candidate list; changing it later would\nrequire its own decision. Gap 2 below is resolved by `archie-core-t2db.17`.\n\nA playbook is an ordered action list with data flowing between steps. If\naction 2 of 4 fails at runtime (an image-gen API times out, a forge call\n403s), the run needs a defined outcome. This\ncodebase already answers a structurally identical question for its\nexisting workflow engine, precisely enough to reuse rather than\nre-derive: `workflow.Run` (`internal/domain/workflow/workflow.go`)\ndraws a three-way distinction on a stage failure, and a playbook run\ndraws the same one on an action failure.\n\n**Decision: reuse `Run`'s three-way distinction directly.** The stated\ndefault is **stop-on-first-failure**: a real action error halts the\nplaybook run immediately and is reported, with no continue-to-next-action\nand no automatic retry or compensating follow-up. The three points below\nare the complete decision.\n\n1. **A real action error** (the action's own logic failed -- an API\n error, a validation failure inside the action) stops the run\n immediately. No later actions execute. The failure is reported the\n same way a load collision is reported -- dropped, logged plainly\n (which action and why), and returned to the caller, never swallowed\n as a background log line only (the Dedup section's drop-and-report\n rule). `Run` is the mechanical\n precedent for the stop itself: a stage error sets the park reason and\n returns without running further stages (`workflow.go`). This\n is stop-on-first-failure, matching the document's existing bias\n against clever automatic recovery, and it is now a stated decision\n rather than a nobody-chose default.\n2. **Interruption** (the daemon is shutting down mid-dispatch, `ctx.Err()\n != nil`) is explicitly **not** a failure. `Run` already treats this\n case specially -- \"Daemon shutdown is not a workflow failure. Leave\n the task running ...; parking here would publish a false failure and\n require manual intervention\" (`workflow.go`) -- and the same\n reasoning applies to a playbook run: an in-flight dispatch interrupted\n by restart must not be recorded as a broken playbook. What \"leave it\n alone for restart\" means for a playbook run (there is no `store.Task`\n row a playbook run inherently owns the way a workflow stage does) is\n an open implementation detail for whoever builds the coordinator, not\n decided here -- but the *semantic* (interruption ≠ failure) is decided.\n3. **A run that produces no outcome at all** (every action ran without\n error, but nothing was assigned to happen) is itself an error\n condition, matching `Run`'s own \"a workflow must end with an explicit\n outcome; not doing so is a definition bug, which still must not vanish\n silently\" (`workflow.go`). A playbook with actions that all ran\n but never reached a coherent terminal state is a definition bug in the\n playbook, reported as such.\n\n**Rollback:** explicitly **not attempted**, for exactly the reason the\ngap's own description named -- a posted forge comment is not revocable.\nThis is not a limitation to work around later; it is the correct\nsemantic. An operator who needs a corrective action after a failure\nwrites a *new* playbook/action for that, they do not get automatic\nundo.\n\n**This stays inside the producer-only rule.** Stop-on-first-failure is the\nrun deciding its own terminal state from its own action's error. Nothing\nalready in flight is blocked or altered; the run only declines to start\nactions that had not started yet, and it records the failure on its own\nunit the way `Run` sets a park reason and transitions the task to\n`StatusParked`. A future compensating action (\"if the notify action\nfailed, also log an incident\") would be producing new work, which the rule\nalready permits. No syntax for it is designed here.\n\n**J3 relationship, stated explicitly:** J3 (a CEL `when` condition\nerroring at eval time → treated as false → skip that action → run\ncontinues) is a **different case from an action failure** and is not\nrevisited by this resolution. A condition error means \"we couldn't\ndetermine whether to run this action,\" which this document already\nresolved as \"then don't, and move on.\" An action failure means \"we tried\nto run this action and it broke,\" which stops the run per point 1 above.\nThe two must not be conflated: a coordinator implementation should be\nable to point at one `when`-evaluation code path and one\naction-invocation code path and show they use different outcomes.\n\n### 2. Idempotency at execution time\n\n**Resolved** (`archie-core-t2db.17`, a\ndesign-investigation ticket: resolve, do not implement). The decision\nbelow is the answer; the keying scheme itself is separate follow-up work\nand is intentionally not built here. It **extends**\n`event-sources-and-reactions.md`'s existing contract -- it does not\ncontradict it (relationship argued at the end).\n\n**Problem (unchanged):** `event-sources-and-reactions.md` decided\nreactions ride `internal/eventbus`'s at-least-once delivery specifically\nso `PublishUnique`/idempotency keys absorb redelivery without duplicate\nside effects. This document inherits that for free only for\n`workflow`-kind actions, because `TaskEnvelope` already carries an\nidempotency key. It said nothing for `channel`, `forge`, or `module`\nactions with real side effects -- a posted comment, a sent message, a\nbilled image generation.\n\nOne correction to the problem statement, checked against the code rather\nthan assumed, because it changes what the answer must be:\n**`PublishUnique`'s dedup is not a consumer-redelivery guard at all.**\n`PublishUnique` only sets the JetStream `Nats-Msg-Id` header on the\noutgoing message (`internal/infrastructure/eventbus/nats/publisher.go`;\nkey constant `message.go`); JetStream suppresses *republished*\nmessages carrying a repeated key inside `Config.DedupWindow`\n(`client.go`, `Duplicates: cfg.DedupWindow`), default 2 minutes\n(`config.go`, `DefaultDedupWindow = 2 * time.Minute`). A consumer\nredelivery is the same *stored* message re-delivered on Nak or\nacknowledge-timeout (`internal/eventbus/eventbus.go`: \"a handler\nreturning an error causes redelivery\") -- not a republish; its\n`Nats-Msg-Id` is never re-firewalled, and even if it were, two minutes is\nnot an at-most-once guarantee for a side effect that must never repeat.\n**Conclusion: side-effecting action kinds need a consumer-side, durable\ndedup record; the bus cannot supply it.** That is precisely the situation\n`binding_dispatches` was built for -- so the decision below reuses its\nmechanism rather than designing a new one.\n\n#### Decision\n\n**Granularity: per-action-per-event -- one dedup record per\n(action, event) pair, not per playbook run.** Whole-run keying, the\nobvious alternative, has two crash windows: the run row is written when the run starts, so a crash mid-run\nleaves the run \"started\" and a redelivery skips *every remaining\naction* (work silently lost); a crash before the row write re-runs\n*every already-fired action* (duplicate side effects -- the exact harm\nthis gap exists to prevent). Per-action keys collapse both into one\nwindow and one behavior: an action's record is written immediately\nbefore that action's side effect is invoked, so an action fires iff its\nown record is absent, no fired action ever re-fires, and a redelivered\nevent resumes at the first unrecorded action. For the shipped\nsingle-action playbooks the two granularities are equivalent in effect;\nthe difference starts with multi-action runs -- where whole-run keying\nwould already be wrong.\n\n**Storage/lookup: a new durable ledger table `playbook_dispatches` in\n`internal/store`, copying `binding_dispatches`'s conventions exactly**\n(durable table, `INSERT OR IGNORE`, sentinel error, rows freed with\ntheir parent, survives daemon restarts -- `internal/store/bindings.go`):\n\n```sql\nCREATE TABLE playbook_dispatches (\n playbook_id TEXT NOT NULL,\n playbook_version TEXT NOT NULL,\n event_id TEXT NOT NULL,\n action_id TEXT NOT NULL,\n dispatched_at TEXT NOT NULL,\n PRIMARY KEY (playbook_id, playbook_version, event_id, action_id)\n);\n```\n\nWritten with the same `INSERT OR IGNORE` + sentinel-error convention\n(`ErrAlreadyDispatched`, `bindings.go`), so a duplicate is a no-op\nwrite rather than a constraint error, and the caller matches it with\n`errors.Is` the way the binding dispatch loop already does\n(`internal/daemon/daemon.go` -- \"another cycle already won this\nrace,\" not an error to surface). The record is consumed by the playbook\ncoordinator at dispatch time through a domain-side interface implemented\nby `internal/store` -- the same shape as `BindingDispatcher`\n(`internal/store/interface.go`) -- and the table lands in the\nstore's schema application (`internal/store/store.go`). No change\nto `internal/eventbus`; no per-message dedup state on the client.\n\n**Lifetime: no time-based expiry.** Rows live as long as their playbook\nexists; when a playbook is removed from the configured directories, the\ncoordinator deletes its rows -- mirroring binding-dispatches-rows-deleted-\nwith-binding (`bindings.go`, same transaction as the delete). The\nat-most-once guarantee must not silently decay, which is exactly the\nproperty that disqualifies `DedupWindow` above; table growth is bounded\nby distinct (playbook, event, action) pairs over the playbook's lifetime\nand is accepted, the same tradeoff `binding_dispatches` already makes.\n\n**Redelivery behavior: skip the recorded action and continue the run.**\nA detected duplicate is not re-invoked, is not reported to the caller as\nan error, and is logged at debug level rather than as a warning -- this\nis the expected, correct behavior of at-least-once delivery, not an\nanomaly (same treatment as `ErrAlreadyDispatched` at\n`daemon.go`).\nThe run continues with the next action (or ends, for a single-action\nplaybook whose one action is recorded), which is what makes a redelivery\na resume rather than a dead restart.\n\n**Key derivation: a fixed composition of four structural identities,\ncomputed by the coordinator -- not a CEL expression.** The key is the\n`(playbook_id, playbook_version, event_id, action_id)` tuple above, where:\n\n- `playbook_id` is `Playbook.ID` -- the playbook file's path relative to\n its configured directory root, slash-normalized, unique within the load\n composition by construction (`internal/domain/eda/playbook`).\n- `playbook_version` is `Playbook.Version` -- a SHA-256 of the loaded\n file, recomputed on every load (`internal/domain/eda/playbook`),\n mirroring `Binding.Version`'s provenance-pinning purpose: the dispatch\n is pinned to the exact definition that was active when it fired. A\n changed playbook definition is a different key and may fire again for\n the same event -- deliberate, not a bug.\n- `event_id` is `DispatchInput.TaskID` (`playbook.go`), which\n carries the `TaskEnvelope.IdempotencyKey()` value\n (`\"archie:\" + owner/repo/number`,\n `internal/domain/workintake/envelope.go`). It is chosen over\n `store.Task.ID` because the trigger vocabulary is issue-level\n workintake label/kind, dispatched at discovery time (pollNATS, forge\n webhook receiver), where no task row exists yet (`playbook.go`\n comment; `playbook_test.go`: \"the value available at the\n discovery/dispatch point... NOT a store.Task.ID int64\"). Keying on the\n issue identity rather than the delivery source is the load-bearing\n property the existing contract already established for `TaskEnvelope`\n (`event-sources-and-reactions.md` question 4: a webhook-sourced\n envelope for the same issue produces the *same* idempotency key as a\n poll-discovered one; `internal/forge/webhook/receiver.go` --\n \"Idempotency is not this package's job\"); this resolution inherits that\n property rather than re-deciding it. Consequence, stated explicitly:\n same-issue re-discoveries (a re-label that re-triggers the same kind)\n are the *same event* by design -- the workintake trigger vocabulary is\n issue-granular, exactly as `TaskEnvelope`'s own key is. A future\n capture/binding-originated trigger type keys on `captured_events.id`\n instead; derivation is per-trigger-type, not one formula for all\n triggers (rule carried over unchanged from the draft).\n- `action_id` is the action's declared `id` -- J1's named ids, \"a\n required field on every action that later actions reference\", unique\n and stable-identifier-shaped, validated at load (the CEL resolution's\n J1 and its context table above in this document). For an action declaring no `id`\n (permitted by J1 while unreferenced), the key uses the action's 1-based\n position in the playbook: deterministic and stable for the\n single-action playbooks, and a later reorder is a file edit whose\n content hash changes `playbook_version` with it, so a positional\n fallback cannot silently collide with a prior definition.\n\n**No CEL in the key -- stated explicitly because the CEL resolution\nflagged this as gap 2's \"first real exercise\"** (\"CEL's read-only result\naccess makes gap 2's keying derivation readable\" -- the CEL resolution's\nclosing note). Readability\nis not the binding constraint; **determinism under redelivery is**, and\nCEL key expressions lose three ways. First, they are not load-checkable:\n`event` stays `dyn` by the resolved J4 (schema-by-example), so\na key expression over event fields cannot be type-checked at load,\nviolating the document's reject-at-load philosophy. Second, they can\nfail at dispatch (a missing field is a CEL error), and a key that cannot\nbe computed is an unfireable state -- neither \"skip\" nor \"fire\" is safe\nwithout losing or duplicating work. Third, the contract needs one\nderivation, and two authors writing different key expressions would\nfragment the ledger. The key is therefore structural by construction:\nevery component above is an immutable identity of the run's inputs, so\ntwo deliveries of the same event derive the same key with no evaluation.\nThe exercise's actual outcome is the derivation rule above; CEL remains\na *condition* surface (`when`), never a key surface. An action kind that\ngenuinely needs finer granularity than (action, event), such as \"per\ncomment, not per issue\", arrives with its own trigger type and its own\nevent identity, per the per-trigger-type rule.\n\n**Record-before-invoke -- one deliberate divergence from\n`binding_dispatches`, justified by the harm profile.** The binding\ndispatch loop enqueues the task first and records the ledger row after\n(`daemon.go`), accepting that racing cycles may both enqueue\n(\"the duplicate task remains queued... delete-on-races is its own can of\nworms\"); the selection query excludes already-dispatched captures\n(`bindings.go`), but the race window remains. That ordering is\ncorrect there because the duplicate artifact is an internal task row --\nrecoverable, low-harm. The playbook ledger's ordering is reversed on\npurpose: **the row is written and committed before the action's side\neffect is invoked** -- the `INSERT OR IGNORE` is the atomic gate, and\nthe lost-write loser skips without invoking. The cost is stated, not\nhidden: a crash between the row commit and the invoke loses that one\naction's side effect, repairable by hand -- the reverse direction of the\nsame window that would otherwise duplicate a posted comment or\nre-bill an image generation. For side effects whose duplication is\nexternally visible and not revocable, at-most-once is the required\ndirection; the binding system's at-least-once-tolerant ordering is not\nthe precedent for this ledger.\n\n**Gap-1 interplay, stated to keep the two mechanisms separate:** the\nledger answers one question per action -- has this action already fired\nfor this event? It does not answer whether the run continues, stops, or\nis re-attempted; that is gap 1's coordinator concern (a run-outcome\nrecord; gap 1 explicitly leaves \"what leave it alone for restart means\nfor a playbook run\" to whoever builds the coordinator). One consequence\nof record-before: an action that FAILED (gap 1's real-action-error case)\nhas a ledger row, so a later redelivery of the same event skips it --\nconsistent with gap 1's \"no automatic retry\" resolve (a corrective\naction is a *new* playbook/action, per gap 1's rollback paragraph). A\nfailed action is never re-attempted by the bus; if a run must be\nrecovered, an operator edits the playbook -- new version, new key --\nrather than fighting ledger state.\n\n**Relationship to `event-sources-and-reactions.md` -- extension, not\ncontradiction.** Keyed to that document's sections:\n\n1. Its decision 2 commits reactions to at-least-once delivery via\n `PublishUnique`'s \"without duplicate enqueue\" guarantee -- which, with\n the code facts above, is accurate for the *publish* path only:\n `PublishUnique` absorbs duplicate publishes of the same event (poll\n vs. webhook), which is what that decision was actually about. This\n document's own problem statement, taken literally, overstates it\n (\"keys absorb redelivery without duplicate side effects\") -- that\n overstatement is exactly what this gap exists to correct. The\n existing contract never claimed consumer redeliveries of\n side-effecting reactions were absorbed, so nothing it says contradicts\n a consumer-side ledger; this resolution pins down the boundary the\n sentence left implicit (publish-path dedup vs. consumer-path dedup)\n rather than changing either.\n2. Its decision 3 is untouched: the ledger is the run's own bookkeeping\n about its own actions, suppressing a re-fire and nothing else.\n3. Its decision 4 (no generic `Source` interface; `TaskEnvelope` is the\n typed contract, keyed on issue identity, not delivery source): the\n event half of the playbook key IS that same identity\n (`TaskEnvelope.IdempotencyKey()`), and the per-trigger-type rule\n means any future trigger type brings its own identity rather than a\n new generic one -- the playbook ledger introduces no second identity\n vocabulary and no untyped payload. The only new type in this\n resolution is the ledger's own value object, consumed through a\n narrow interface like `BindingDispatcher` -- the same shape the\n document's \"typed families\" rule (decision 1, plugin engine rule)\n already blesses.\n\n**Scope: this is the `channel`/`forge`/`module` decision, not a\n`workflow`-position change.** Workflow-kind actions keep their existing\nkeying claim unchanged -- the `TaskEnvelope` path that this document\nalready inherits for free (`daemon.go`, `PublishTask` publishes\nwith `task.IdempotencyKey()`), with its cross-poll/webhook property\nuntouched. The ledger gate applies to the side-effecting positions\n(channel, forge, module), which is why the closing statement below\nstill holds: the first slice (Module + loader + workflow-kind dispatch)\nwas buildable without this resolution, and channel/forge/multi-action\nplaybooks were not. A coordinator MAY implement one position-uniform\ngate as a simplification; what this resolution requires is only that\nthe side-effecting positions are gated.\n\n## Resolved questions\n\n1. **Data-flow/condition syntax.** Does this project want a small existing\n Go expression evaluator (there may be one already in the codebase worth\n checking, e.g. anything backing gate conditions), or a bespoke minimal\n grammar?\n\n **Resolved** (`archie-core-t2db.14`). CEL (`cel.dev/cel-go`, pinned) is\n the one mechanism for both an action's `when` condition and its `args`\n values. The trust boundary, evaluation context, load-time validation and\n the five judgment calls are in\n `docs/prds/playbook-expression-syntax.md`.\n\n2. **Playbook directory config field.** Where this lives in\n `internal/config` / `configuration.Document`, and whether it's one\n directory or a list (mirroring `SkillsDir`'s shape) -- follow existing\n config precedent, not invented fresh.\n\n **Resolved** (`archie-core-t2db.11`). The directory\n shape landed as `playbook_dirs` (a LIST of directories of\n `*.yaml`/`*.yml` binding files), loaded at startup as an additional\n input to the two single-file fields, which remain supported unchanged.\n A single `playbook_dir` would contradict\n supporting multiple independently-maintained playbook sources, which\n the Dedup section already assumes ('independently-sourced playbook\n directories collide'), so the field is a list. Cross-source collision\n (same key in two directories, or in a single-file field and a\n directory) is reported like an in-directory collision -- nothing is\n arbitrated by source precedence. This is what finally exercises the\n design doc's drop-and-report collision rule, since a single file\n cannot collide with itself. Repo-scoping (tying directories to\n `Config.Repos` owner/name) is explicitly deferred to a separate,\n later decision; org/tenant keying is out of scope (no auth/identity\n model).\n\n Decisions recorded from the label-vocabulary slice:\n - Arbitrary labels bind to registered workflow names via\n `LabelWorkflows` + `LoadLabelWorkflowsYAML`/`SetLabelWorkflows`\n (`internal/domain/workflow/routing.go`).\n - The closed `Kind`/NATS-subject set is untouched; the label map only\n extends binding authority for labels the kind layer does not own.\n - Collision rule: a label already owned by the\n kind set (bug/feature/bootstrap), an empty label, an empty workflow\n name, or a duplicate binding is a reported load failure -- dropped\n and logged, the error returned to the caller. Nothing is silently\n arbitrated by schema/version. This is the \"schema defines the\n accepted message; anything else is the caller's fault\" rule applied\n to the label layer.\n - Precedence in `Route()`: explicit `t.Workflow` → arbitrary-label\n binding → kind binding → triage → implement → default.\n3. **Trust boundary for Module Yaegi code.** The plugin engine rule's\n invariant 6 draws a line between operator-trusted in-process code and\n repository-supplied code that must run in a container.\n\n **Resolved** in `docs/prds/module-position.md`, \"Trust boundary\": a\n Module is operator-installed, in-process and daemon-privileged, the same\n tier as `PluginDir` and `SecretEngineDir`, never repository-supplied\n task code.\n4. **First implementation slice.** Recommend: Module position + the\n playbook loader + trigger-to-workflow dispatch only (subsuming the existing\n label routing) first, proving the schema-gen -\u003e Yaegi -\u003e playbook path\n end to end on the smallest useful case, before adding Channel/Forge\n action kinds or the linter/LSP.\n\n **Resolved** (`archie-core-t2db.13`, `.14`, `.15`). The Module position\n (registry + log kind, t2db.13), the CEL expression environment\n (t2db.14), and the playbook document + event coordinator with\n single-action workflow-kind dispatch (t2db.15) are shipped.\n\n **Resolved** (`archie-core-t2db.23`): the coordinator selects a real\n task's workflow. The wiring point is the daemon's definition pin\n (`Daemon.resolveWorkflowID`, consumed by `pinWorkflowFromCollection` in\n `internal/daemon`), not `pollNATS`/`publishTask`. Routing belongs to the\n pin because a workflow definition is a\n database row pinned before dispatch, which makes the pin the one place a\n task's workflow is chosen. Precedence there is explicit `Task.Workflow`\n (the waiting_human -\u003e approved requeue) → matching playbook →\n `workflow.ResolveWorkflowID`'s label binding → kind binding → triage →\n implement. A playbook naming an undefined workflow is a reported failure\n that parks the task rather than a silent fall-through to the binding's\n choice.\n\n `Store.Dispatch` returns the selected workflow *name* and the matching\n playbook's id and version, not a compiled `workflow.Workflow`: the\n production caller decides against the active definition collection, and\n the id and version are the first two components of the gap-2 idempotency\n key. The playbook package does not import `internal/domain/workflow`.\n\n A `when` condition at this point reads what the intake knows about the\n originating issue: `kind`, `labels`, `owner`, `repo`, `number`, `title`,\n `body`, `source`, built by `playbookInput` in `internal/daemon`.\n\n The trigger shape reuses the existing workintake kind/label vocabulary.\n The single-action hard boundary is superseded by\n `docs/prds/multi-action-playbooks.md` (two-shape rule); Module, Channel,\n and Forge positions still have no production dispatch path.\n## Standing constraint\n\nThis may be commercialized, and is to be discernible from other open-source\nevent-driven-automation tooling. No design decision follows from that yet;\nit bears on licensing or feature-gating questions about playbook authoring\nand the LSP when they arise.\n\n## Packages this touches (first slice)\n\n- New: a schema-generation package/convention (`go:generate` source +\n emitted contracts) for the Module position.\n- New: playbook loader + event coordinator (name/location TBD -- likely\n `internal/domain/eda` or similar, following the domain-layer rule: no\n infrastructure imports, no untyped hooks).\n- `internal/domain/workflow`: dispatch target for `workflow`-kind actions,\n unchanged in its own contracts.\n- `internal/domain/workintake`: `labelKinds`/routing becomes one playbook\n among others once the loader subsumes it; existing Go fallback stays for\n backward compatibility until playbooks fully replace it (mirrors the\n `dynamic-workflow-triage.md` precedent of a two-line fallback change, not\n a rip-and-replace).\n" + "body": "# EDA playbook engine -- design\n\n**Status:** Draft\n**Date:** 2026-09-03\n**Beads epic:** `archie-core-t2db`\n\n## Problem\n\n\"Which workflow runs for which trigger\" is hardcoded Go: a literal\nlabel vocabulary (`workintake.labelKinds`) and a literal label-\u003eworkflow map\n(`workflow/routing.go`), extended only by adding a Go case or shipping a\nskill whose `metadata.archie.workflow` overrides one *named* built-in\nworkflow. Extending the system to a new trigger, a new action, or a new\nevent source means a feature branch, a code change in a hardcoded\npath, and new Go tests -- for every addition. That is the barrier this\ndocument exists to remove.\n\nThe goal: triggers, the actions they run, and the\nconditions that connect them should be **drop-in and load**, not **drop-in\nand touch application code**. Yaegi-evaluated Go is the unit of imperative\n\"do a thing\" logic; YAML playbooks are pure orchestration data -- no logic,\njust \"on this event, with this data, call these actions, in this order,\nunder these conditions.\" The daemon's event coordinator doesn't know what a\n`bugfix` or `image-gen` action does, it resolves the name against a\nregistry and pipes typed data through.\n\nThis is explicitly **not** scoped to code-generation workflows. A playbook\naction can be any registered capability -- run a codegen workflow, send a\nmessage on a channel, open something on a forge, generate an image -- so\nlong as it's implemented behind one of the typed engine families below.\n\n## Relationship to existing decisions\n\n- `docs/prds/event-sources-and-reactions.md` ruled\n that reactions in the two epics it covers are **producer-only** (no veto,\n no mutation of in-flight work) and explicitly rejected a single generic\n `Source` interface as \"the untyped-hook shape the plugin engine rule\n exists to prevent.\" **This document extends that scope**: playbooks can\n dispatch to multiple typed action kinds, not just \"produce a\n `TaskEnvelope`.\" It does not relitigate producer-only-ness (still no\n veto/mutation of in-flight work here) and it does not introduce one\n generic interface -- see \"Typed positions, not one generic hook\" below,\n which is how it stays compliant with the plugin engine rule instead of\n reopening that document's #4.\n- `docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`\n governs every extension point\n introduced here. Every position below is a typed domain contract with an\n owning registry, not a generic callback map. This is a hard constraint,\n confirmed against the existing rule text before writing this doc.\n- Yaegi is already a proven in-process extension mechanism\n (`internal/plugin`, `internal/domain/workflow/wfeval/yaegi.go`,\n `internal/gate/gateeval/yaegi.go`, `internal/skill/plugin.go`) -- this\n design generalizes its existing role, it does not introduce a new\n execution mechanism.\n- `internal/domain/workflow/skillbuild.BuildRegistry` already composes a\n `workflow.Registry` from a skills catalog, plugin-defined overriding\n built-in. The playbook loader described here supersedes label routing\n specifically; it does not replace `skillbuild`'s workflow-composition role,\n it becomes an additional input the daemon composes at startup.\n- Webhook event intake is a separate design. This\n document defines the router each webhook-sourced event should be able to\n reach; it does not redesign webhook receipt itself.\n\n## Shape: four logic positions, each a typed engine family\n\nEvery entrypoint kind (mechanism, action, loop, schedule, route, trigger) is itself a schema emitter: defining a new kind produces the\ngenerated contract downstream tooling consumes. The kinds slot into a fixed\nhierarchy of logic positions:\n\n```\nWorkflow -\u003e Module -\u003e Channel (telegram/whatsapp/email/...)\n -\u003e Forge (GitHub/Gitea/GitLab/...)\n```\n\nEach position is its own typed engine family per the plugin engine rule:\nidentity + capability-specific operations, an owning `Registry`/`Manager`,\nexplicit lifecycle where the implementation owns resources, narrow host\naccess. Concretely:\n\n- **Workflow** -- already exists (`internal/domain/workflow`). A playbook\n action of kind `workflow` invokes a registered `workflow.Registry` entry\n by name, unchanged from the existing execution path.\n- **Module** -- the new general-purpose action position. A `Module` is a\n Yaegi-evaluated implementation of a generated interface (args in, typed\n result out) for one declared action kind (e.g. `image-gen`, `notify`,\n `open-pr-comment`). This is the position that makes actions \"not\n restricted to code-generation.\" Full design (schema-per-kind, the\n registry shape, the trust boundary, the recommended first slice):\n `docs/prds/module-position.md`.\n- **Channel** -- already exists as `channels.Channel` (telegram, email,\n webhook). A playbook action of kind `channel` sends through a named,\n already-registered channel. No new mechanism; this document just makes it\n addressable from a playbook.\n- **Forge** -- forge clients (GitHub, Gitea, GitLab) become an addressable\n action position the same way Channel is, so a playbook can say \"comment\n on this forge issue\" without that being a special case wired only into\n the poller.\n\n**Typed positions, not one generic hook.** There is no single `Action`\ninterface with an `any` payload. Each position keeps its own typed\ncontract; the playbook YAML names a position and an implementation, and the\nevent coordinator resolves that pair against the matching typed registry.\nThis is what keeps the design compliant with the plugin engine rule and\nwith `event-sources-and-reactions.md`'s rejection of a generic `Source`/\nhook interface -- the generality lives in the YAML data model (which\npositions exist, which named implementations are registered), not in a\nruntime `any`-typed dispatch path.\n\n## `go:generate`-driven schemas, from day one\n\nSchemas are not a future nicety bolted onto an ad hoc\nYAML shape. They are the struct+interface contracts for each logic\nposition, and they drive code generation from the start.\n\n- Each logic position (Workflow/Module/Channel/Forge) has a schema source\n (Go-annotated types, following the existing pattern elsewhere in the\n repo) from which `go:generate` emits the interface and struct scaffolding\n an implementation must satisfy.\n- A Yaegi module implements the generated interface for its position. The\n generated contract is what makes an implementation valid, not hand-review.\n- The schema is also what the linter and LSP (below) validate a playbook\n YAML against -- one source of truth for codegen, startup loading, and\n authoring-time tooling, so the three can't drift from each other.\n- Schema versioning is explicit per position, independent of a playbook's\n own workflow version (see collision handling below).\n\nThe generator itself is not designed here. The constraint is\nschema-first, `go:generate`-driven, per logic position.\n\n## Playbook YAML: pure orchestration data\n\nA playbook declares no embedded logic -- only:\n\n- **trigger**: the event predicate (e.g. forge issue assigned + labelled\n `bugfix`, a schedule tick, a webhook payload shape).\n- **actions**: an ordered list, each naming a logic position and a\n registered implementation, with typed argument data and optional\n conditions.\n- **data flow between actions**: results/facts from one action available to\n later actions in the same playbook run -- exact interpolation syntax is\n an open point, not decided here (see Open questions).\n\nPlaybooks live in a configured directory (repo/directory-scoped, per Sam's\n\"define repos, directories, playbooks\" framing) and are loaded at archied\nstartup, alongside the existing skills-catalog composition.\n\n## Dedup: detect and log, no arbitration mechanism\n\nNo runtime auto-resolution rule. An automatic \"pick a winner\" algorithm\n(e.g. by version) is itself a mechanism that can fail and need debugging\nwhen a collision happens -- the wrong kind of complexity to introduce for\na case that should just be visible and fixed by a human. Instead:\n\n- **At daemon startup:** if independently-sourced playbook directories\n declare a collision on the same trigger, that is a definition failure.\n The colliding definitions are dropped (not loaded) and the failure is\n logged plainly (which sources, which trigger) and reported back to the\n caller that requested the load -- not swallowed as a background log line\n only. The schema is the definition; a schema conflict is a reported\n failure, not something the daemon silently arbitrates.\n- **Linter**: a CI/dev-time tool, reading the same generated schemas, that\n flags duplicate or conflicting trigger definitions *within* a single\n maintained tree before merge.\n\n **Resolved** (`archie-core-t2db.12`). A standalone\n gopls-shaped binary (`cmd/archie-playbooks`, lint subcommand) validates\n playbook directories against the exact loaders the daemon uses\n (`LoadPlaybookDirs`/`LoadKindWorkflowsYAML`/`LoadLabelWorkflowsYAML`),\n exiting non-zero on any collision / malformed file / invalid binding.\n A finding about one binding key leads with the key's `file:line`, read\n from the `yaml.Node` the loader decodes; the linter agrees with runtime\n validation by construction, which is the load-bearing property. That\n agreement covers the routing loaders only: EDA playbooks under\n `eda_playbook_dir` (`playbook.Load`, with its CEL and action-id checks)\n have no lint path yet and are validated only at daemon startup.\n Discoverable via `task lint:playbooks` or direct `go run\n ./cmd/archie-playbooks lint -dir ...`. The LSP/serve mode is a later\n entrypoint of the SAME binary, per the shared-package decision above.\n- **LSP**: same schema source again, feeding author-time hover/completion/\n inline errors while someone edits a playbook YAML in an editor. Kept\n explicitly in scope per Sam (not deferred), because it's the same schema\n artifact as the linter's, just a different consumer.\n\nThe startup log-and-refuse path exists for what the linter/LSP structurally\ncannot see -- sources composed together only at daemon startup, never\nchecked against each other by either tool. If the linter and LSP do the job\nthey're built for, a real collision reaching that startup path should be\nrare in practice, not a routine occurrence the design leans on.\n\n**Shared package, thin consumers -- not a bundled-vs-standalone choice.**\nThe validation logic (`LoadKindWorkflowsYAML`, `LoadLabelWorkflowsYAML`, the\ndirectory-merge from t2db.11) lives in `internal/domain/workflow`, a plain\nGo package with no knowledge of which binary calls it. This makes \"should\nthe linter/LSP be bundled into archied or a separate tool\" a non-question:\n- `archied` already imports the package directly and validates in-process\n at startup -- no subprocess, no external tool call.\n- An agent that needs to validate a playbook in-process can import the same\n package directly, for the same reason.\n- The standalone `cmd/` binary (t2db.12, gopls-shaped: one binary, a lint\n mode now, an LSP/serve mode later) exists only because CI and editors are\n out-of-process consumers by nature -- a CI step needs an exit code, an\n editor speaks LSP over a process boundary, neither can `import` a Go\n package. It is a thin wrapper per `organisation.md`'s `cmd/` rule, not a\n second implementation.\n\nFolding CLI/LSP protocol handling into `archied` itself would grow the\ndaemon into unrelated concerns, against the plugin-engine rule's spirit of\nnarrow capability families.\n\n## Execution-time gaps (both resolved)\n\nEverything above covers *loading* a playbook. Nothing above covers what\nhappens while one *runs*. Two gaps, both load-bearing enough to resolve\nbefore implementation, not defer:\n\n### 1. Mid-playbook action failure\n\n**Resolved** (`archie-core-t2db.18`). The default below\nis a committed decision, not a candidate list; changing it later would\nrequire its own decision. Gap 2 below is resolved by `archie-core-t2db.17`.\n\nA playbook is an ordered action list with data flowing between steps. If\naction 2 of 4 fails at runtime (an image-gen API times out, a forge call\n403s), the run needs a defined outcome. This\ncodebase already answers a structurally identical question for its\nexisting workflow engine, precisely enough to reuse rather than\nre-derive: `workflow.Run` (`internal/domain/workflow/workflow.go`)\ndraws a three-way distinction on a stage failure, and a playbook run\ndraws the same one on an action failure.\n\n**Decision: reuse `Run`'s three-way distinction directly.** The stated\ndefault is **stop-on-first-failure**: a real action error halts the\nplaybook run immediately and is reported, with no continue-to-next-action\nand no automatic retry or compensating follow-up. The three points below\nare the complete decision.\n\n1. **A real action error** (the action's own logic failed -- an API\n error, a validation failure inside the action) stops the run\n immediately. No later actions execute. The failure is reported the\n same way a load collision is reported -- dropped, logged plainly\n (which action and why), and returned to the caller, never swallowed\n as a background log line only (the Dedup section's drop-and-report\n rule). `Run` is the mechanical\n precedent for the stop itself: a stage error sets the park reason and\n returns without running further stages (`workflow.go`). This\n is stop-on-first-failure, matching the document's existing bias\n against clever automatic recovery, and it is now a stated decision\n rather than a nobody-chose default.\n2. **Interruption** (the daemon is shutting down mid-dispatch, `ctx.Err()\n != nil`) is explicitly **not** a failure. `Run` already treats this\n case specially -- \"Daemon shutdown is not a workflow failure. Leave\n the task running ...; parking here would publish a false failure and\n require manual intervention\" (`workflow.go`) -- and the same\n reasoning applies to a playbook run: an in-flight dispatch interrupted\n by restart must not be recorded as a broken playbook. What \"leave it\n alone for restart\" means for a playbook run (there is no `store.Task`\n row a playbook run inherently owns the way a workflow stage does) is\n an open implementation detail for whoever builds the coordinator, not\n decided here -- but the *semantic* (interruption ≠ failure) is decided.\n3. **A run that produces no outcome at all** (every action ran without\n error, but nothing was assigned to happen) is itself an error\n condition, matching `Run`'s own \"a workflow must end with an explicit\n outcome; not doing so is a definition bug, which still must not vanish\n silently\" (`workflow.go`). A playbook with actions that all ran\n but never reached a coherent terminal state is a definition bug in the\n playbook, reported as such.\n\n**Rollback:** explicitly **not attempted**, for exactly the reason the\ngap's own description named -- a posted forge comment is not revocable.\nThis is not a limitation to work around later; it is the correct\nsemantic. An operator who needs a corrective action after a failure\nwrites a *new* playbook/action for that, they do not get automatic\nundo.\n\n**This stays inside the producer-only rule.** Stop-on-first-failure is the\nrun deciding its own terminal state from its own action's error. Nothing\nalready in flight is blocked or altered; the run only declines to start\nactions that had not started yet, and it records the failure on its own\nunit the way `Run` sets a park reason and transitions the task to\n`StatusParked`. A future compensating action (\"if the notify action\nfailed, also log an incident\") would be producing new work, which the rule\nalready permits. No syntax for it is designed here.\n\n**J3 relationship, stated explicitly:** J3 (a CEL `when` condition\nerroring at eval time → treated as false → skip that action → run\ncontinues) is a **different case from an action failure** and is not\nrevisited by this resolution. A condition error means \"we couldn't\ndetermine whether to run this action,\" which this document already\nresolved as \"then don't, and move on.\" An action failure means \"we tried\nto run this action and it broke,\" which stops the run per point 1 above.\nThe two must not be conflated: a coordinator implementation should be\nable to point at one `when`-evaluation code path and one\naction-invocation code path and show they use different outcomes.\n\n### 2. Idempotency at execution time\n\n**Resolved** (`archie-core-t2db.17`, a\ndesign-investigation ticket: resolve, do not implement). The decision\nbelow is the answer; the keying scheme itself is separate follow-up work\nand is intentionally not built here. It **extends**\n`event-sources-and-reactions.md`'s existing contract -- it does not\ncontradict it (relationship argued at the end).\n\n**Problem (unchanged):** `event-sources-and-reactions.md` decided\nreactions ride `internal/eventbus`'s at-least-once delivery specifically\nso `PublishUnique`/idempotency keys absorb redelivery without duplicate\nside effects. This document inherits that for free only for\n`workflow`-kind actions, because `TaskEnvelope` already carries an\nidempotency key. It said nothing for `channel`, `forge`, or `module`\nactions with real side effects -- a posted comment, a sent message, a\nbilled image generation.\n\nOne correction to the problem statement, checked against the code rather\nthan assumed, because it changes what the answer must be:\n**`PublishUnique`'s dedup is not a consumer-redelivery guard at all.**\n`PublishUnique` only sets the JetStream `Nats-Msg-Id` header on the\noutgoing message (`internal/infrastructure/eventbus/nats/publisher.go`;\nkey constant `message.go`); JetStream suppresses *republished*\nmessages carrying a repeated key inside `Config.DedupWindow`\n(`client.go`, `Duplicates: cfg.DedupWindow`), default 2 minutes\n(`config.go`, `DefaultDedupWindow = 2 * time.Minute`). A consumer\nredelivery is the same *stored* message re-delivered on Nak or\nacknowledge-timeout (`internal/eventbus/eventbus.go`: \"a handler\nreturning an error causes redelivery\") -- not a republish; its\n`Nats-Msg-Id` is never re-firewalled, and even if it were, two minutes is\nnot an at-most-once guarantee for a side effect that must never repeat.\n**Conclusion: side-effecting action kinds need a consumer-side, durable\ndedup record; the bus cannot supply it.** That is precisely the situation\n`binding_dispatches` was built for -- so the decision below reuses its\nmechanism rather than designing a new one.\n\n#### Decision\n\n**Granularity: per-action-per-event -- one dedup record per\n(action, event) pair, not per playbook run.** Whole-run keying, the\nobvious alternative, has two crash windows: the run row is written when the run starts, so a crash mid-run\nleaves the run \"started\" and a redelivery skips *every remaining\naction* (work silently lost); a crash before the row write re-runs\n*every already-fired action* (duplicate side effects -- the exact harm\nthis gap exists to prevent). Per-action keys collapse both into one\nwindow and one behavior: an action's record is written immediately\nbefore that action's side effect is invoked, so an action fires iff its\nown record is absent, no fired action ever re-fires, and a redelivered\nevent resumes at the first unrecorded action. For the shipped\nsingle-action playbooks the two granularities are equivalent in effect;\nthe difference starts with multi-action runs -- where whole-run keying\nwould already be wrong.\n\n**Storage/lookup: a new durable ledger table `playbook_dispatches` in\n`internal/store`, copying `binding_dispatches`'s conventions exactly**\n(durable table, `INSERT OR IGNORE`, sentinel error, rows freed with\ntheir parent, survives daemon restarts -- `internal/store/bindings.go`):\n\n```sql\nCREATE TABLE playbook_dispatches (\n playbook_id TEXT NOT NULL,\n playbook_version TEXT NOT NULL,\n event_id TEXT NOT NULL,\n action_id TEXT NOT NULL,\n dispatched_at TEXT NOT NULL,\n PRIMARY KEY (playbook_id, playbook_version, event_id, action_id)\n);\n```\n\nWritten with the same `INSERT OR IGNORE` + sentinel-error convention\n(`ErrAlreadyDispatched`, `bindings.go`), so a duplicate is a no-op\nwrite rather than a constraint error, and the caller matches it with\n`errors.Is` the way the binding dispatch loop already does\n(`internal/daemon/daemon.go` -- \"another cycle already won this\nrace,\" not an error to surface). The record is consumed by the playbook\ncoordinator at dispatch time through a domain-side interface implemented\nby `internal/store` -- the same shape as `BindingDispatcher`\n(`internal/store/interface.go`) -- and the table lands in the\nstore's schema application (`internal/store/store.go`). No change\nto `internal/eventbus`; no per-message dedup state on the client.\n\n**Lifetime: no time-based expiry, and no automatic reclamation.** Rows are\nretained until an explicit `DeletePlaybookDispatches` for their playbook.\nRemoving a playbook file does not delete its rows: playbooks are read only\nat boot, so the only place to notice a removal is a boot-time reconcile,\nand a boot that finds the directory missing or unmounted loads as \"no\nplaybooks\" and would reclaim the whole ledger, letting redelivered events\nfire their side effects again. A binding's rows go with an explicit\noperator delete (`bindings.go`, same transaction); a file disappearing is\nnot an equally deliberate act. The at-most-once guarantee must not\nsilently decay, which is exactly the property that disqualifies\n`DedupWindow` above; table growth is bounded by distinct (playbook,\nevent, action) pairs and is accepted, the same tradeoff\n`binding_dispatches` already makes.\n\n**Redelivery behavior: skip the recorded action and continue the run.**\nA detected duplicate is not re-invoked, is not reported to the caller as\nan error, and is logged at debug level rather than as a warning -- this\nis the expected, correct behavior of at-least-once delivery, not an\nanomaly (same treatment as `ErrAlreadyDispatched` at\n`daemon.go`).\nThe run continues with the next action (or ends, for a single-action\nplaybook whose one action is recorded), which is what makes a redelivery\na resume rather than a dead restart.\n\n**Key derivation: a fixed composition of four structural identities,\ncomputed by the coordinator -- not a CEL expression.** The key is the\n`(playbook_id, playbook_version, event_id, action_id)` tuple above, where:\n\n- `playbook_id` is `Playbook.ID` -- the playbook file's path relative to\n its configured directory root, slash-normalized, unique within the load\n composition by construction (`internal/domain/eda/playbook`).\n- `playbook_version` is `Playbook.Version` -- a SHA-256 of the loaded\n file, recomputed on every load (`internal/domain/eda/playbook`),\n mirroring `Binding.Version`'s provenance-pinning purpose: the dispatch\n is pinned to the exact definition that was active when it fired. A\n changed playbook definition is a different key and may fire again for\n the same event -- deliberate, not a bug.\n- `event_id` is `DispatchInput.TaskID` (`playbook.go`), which\n carries the `TaskEnvelope.IdempotencyKey()` value\n (`\"archie:\" + owner/repo/number`,\n `internal/domain/workintake/envelope.go`). It is chosen over\n `store.Task.ID` because the trigger vocabulary is issue-level\n workintake label/kind, dispatched at discovery time (pollNATS, forge\n webhook receiver), where no task row exists yet (`playbook.go`\n comment; `playbook_test.go`: \"the value available at the\n discovery/dispatch point... NOT a store.Task.ID int64\"). Keying on the\n issue identity rather than the delivery source is the load-bearing\n property the existing contract already established for `TaskEnvelope`\n (`event-sources-and-reactions.md` question 4: a webhook-sourced\n envelope for the same issue produces the *same* idempotency key as a\n poll-discovered one; `internal/forge/webhook/receiver.go` --\n \"Idempotency is not this package's job\"); this resolution inherits that\n property rather than re-deciding it. Consequence, stated explicitly:\n same-issue re-discoveries (a re-label that re-triggers the same kind)\n are the *same event* by design -- the workintake trigger vocabulary is\n issue-granular, exactly as `TaskEnvelope`'s own key is. A future\n capture/binding-originated trigger type keys on `captured_events.id`\n instead; derivation is per-trigger-type, not one formula for all\n triggers (rule carried over unchanged from the draft).\n- `action_id` is the action's declared `id` -- J1's named ids, \"a\n required field on every action that later actions reference\", unique\n and stable-identifier-shaped, validated at load (the CEL resolution's\n J1 and its context table above in this document). For an action declaring no `id`\n (permitted by J1 while unreferenced), the key uses the action's 1-based\n position in the playbook: deterministic and stable for the\n single-action playbooks, and a later reorder is a file edit whose\n content hash changes `playbook_version` with it, so a positional\n fallback cannot silently collide with a prior definition.\n\n**No CEL in the key -- stated explicitly because the CEL resolution\nflagged this as gap 2's \"first real exercise\"** (\"CEL's read-only result\naccess makes gap 2's keying derivation readable\" -- the CEL resolution's\nclosing note). Readability\nis not the binding constraint; **determinism under redelivery is**, and\nCEL key expressions lose three ways. First, they are not load-checkable:\n`event` stays `dyn` by the resolved J4 (schema-by-example), so\na key expression over event fields cannot be type-checked at load,\nviolating the document's reject-at-load philosophy. Second, they can\nfail at dispatch (a missing field is a CEL error), and a key that cannot\nbe computed is an unfireable state -- neither \"skip\" nor \"fire\" is safe\nwithout losing or duplicating work. Third, the contract needs one\nderivation, and two authors writing different key expressions would\nfragment the ledger. The key is therefore structural by construction:\nevery component above is an immutable identity of the run's inputs, so\ntwo deliveries of the same event derive the same key with no evaluation.\nThe exercise's actual outcome is the derivation rule above; CEL remains\na *condition* surface (`when`), never a key surface. An action kind that\ngenuinely needs finer granularity than (action, event), such as \"per\ncomment, not per issue\", arrives with its own trigger type and its own\nevent identity, per the per-trigger-type rule.\n\n**Record-before-invoke -- one deliberate divergence from\n`binding_dispatches`, justified by the harm profile.** The binding\ndispatch loop enqueues the task first and records the ledger row after\n(`daemon.go`), accepting that racing cycles may both enqueue\n(\"the duplicate task remains queued... delete-on-races is its own can of\nworms\"); the selection query excludes already-dispatched captures\n(`bindings.go`), but the race window remains. That ordering is\ncorrect there because the duplicate artifact is an internal task row --\nrecoverable, low-harm. The playbook ledger's ordering is reversed on\npurpose: **the row is written and committed before the action's side\neffect is invoked** -- the `INSERT OR IGNORE` is the atomic gate, and\nthe lost-write loser skips without invoking. The cost is stated, not\nhidden: a crash between the row commit and the invoke loses that one\naction's side effect, repairable by hand -- the reverse direction of the\nsame window that would otherwise duplicate a posted comment or\nre-bill an image generation. For side effects whose duplication is\nexternally visible and not revocable, at-most-once is the required\ndirection; the binding system's at-least-once-tolerant ordering is not\nthe precedent for this ledger.\n\n**Gap-1 interplay, stated to keep the two mechanisms separate:** the\nledger answers one question per action -- has this action already fired\nfor this event? It does not answer whether the run continues, stops, or\nis re-attempted; that is gap 1's coordinator concern (a run-outcome\nrecord; gap 1 explicitly leaves \"what leave it alone for restart means\nfor a playbook run\" to whoever builds the coordinator). One consequence\nof record-before: an action that FAILED (gap 1's real-action-error case)\nhas a ledger row, so a later redelivery of the same event skips it --\nconsistent with gap 1's \"no automatic retry\" resolve (a corrective\naction is a *new* playbook/action, per gap 1's rollback paragraph). A\nfailed action is never re-attempted by the bus; if a run must be\nrecovered, an operator edits the playbook -- new version, new key --\nrather than fighting ledger state.\n\n**Relationship to `event-sources-and-reactions.md` -- extension, not\ncontradiction.** Keyed to that document's sections:\n\n1. Its decision 2 commits reactions to at-least-once delivery via\n `PublishUnique`'s \"without duplicate enqueue\" guarantee -- which, with\n the code facts above, is accurate for the *publish* path only:\n `PublishUnique` absorbs duplicate publishes of the same event (poll\n vs. webhook), which is what that decision was actually about. This\n document's own problem statement, taken literally, overstates it\n (\"keys absorb redelivery without duplicate side effects\") -- that\n overstatement is exactly what this gap exists to correct. The\n existing contract never claimed consumer redeliveries of\n side-effecting reactions were absorbed, so nothing it says contradicts\n a consumer-side ledger; this resolution pins down the boundary the\n sentence left implicit (publish-path dedup vs. consumer-path dedup)\n rather than changing either.\n2. Its decision 3 is untouched: the ledger is the run's own bookkeeping\n about its own actions, suppressing a re-fire and nothing else.\n3. Its decision 4 (no generic `Source` interface; `TaskEnvelope` is the\n typed contract, keyed on issue identity, not delivery source): the\n event half of the playbook key IS that same identity\n (`TaskEnvelope.IdempotencyKey()`), and the per-trigger-type rule\n means any future trigger type brings its own identity rather than a\n new generic one -- the playbook ledger introduces no second identity\n vocabulary and no untyped payload. The only new type in this\n resolution is the ledger's own value object, consumed through a\n narrow interface like `BindingDispatcher` -- the same shape the\n document's \"typed families\" rule (decision 1, plugin engine rule)\n already blesses.\n\n**Scope: this is the `channel`/`forge`/`module` decision, not a\n`workflow`-position change.** Workflow-kind actions keep their existing\nkeying claim unchanged -- the `TaskEnvelope` path that this document\nalready inherits for free (`daemon.go`, `PublishTask` publishes\nwith `task.IdempotencyKey()`), with its cross-poll/webhook property\nuntouched. The ledger gate applies to the side-effecting positions\n(channel, forge, module), which is why the closing statement below\nstill holds: the first slice (Module + loader + workflow-kind dispatch)\nwas buildable without this resolution, and channel/forge/multi-action\nplaybooks were not. A coordinator MAY implement one position-uniform\ngate as a simplification; what this resolution requires is only that\nthe side-effecting positions are gated.\n\n## Resolved questions\n\n1. **Data-flow/condition syntax.** Does this project want a small existing\n Go expression evaluator (there may be one already in the codebase worth\n checking, e.g. anything backing gate conditions), or a bespoke minimal\n grammar?\n\n **Resolved** (`archie-core-t2db.14`). CEL (`cel.dev/cel-go`, pinned) is\n the one mechanism for both an action's `when` condition and its `args`\n values. The trust boundary, evaluation context, load-time validation and\n the five judgment calls are in\n `docs/prds/playbook-expression-syntax.md`.\n\n2. **Playbook directory config field.** Where this lives in\n `internal/config` / `configuration.Document`, and whether it's one\n directory or a list (mirroring `SkillsDir`'s shape) -- follow existing\n config precedent, not invented fresh.\n\n **Resolved** (`archie-core-t2db.11`). The directory\n shape landed as `playbook_dirs` (a LIST of directories of\n `*.yaml`/`*.yml` binding files), loaded at startup as an additional\n input to the two single-file fields, which remain supported unchanged.\n A single `playbook_dir` would contradict\n supporting multiple independently-maintained playbook sources, which\n the Dedup section already assumes ('independently-sourced playbook\n directories collide'), so the field is a list. Cross-source collision\n (same key in two directories, or in a single-file field and a\n directory) is reported like an in-directory collision -- nothing is\n arbitrated by source precedence. This is what finally exercises the\n design doc's drop-and-report collision rule, since a single file\n cannot collide with itself. Repo-scoping (tying directories to\n `Config.Repos` owner/name) is explicitly deferred to a separate,\n later decision; org/tenant keying is out of scope (no auth/identity\n model).\n\n Decisions recorded from the label-vocabulary slice:\n - Arbitrary labels bind to registered workflow names via\n `LabelWorkflows` + `LoadLabelWorkflowsYAML`/`SetLabelWorkflows`\n (`internal/domain/workflow/routing.go`).\n - The closed `Kind`/NATS-subject set is untouched; the label map only\n extends binding authority for labels the kind layer does not own.\n - Collision rule: a label already owned by the\n kind set (bug/feature/bootstrap), an empty label, an empty workflow\n name, or a duplicate binding is a reported load failure -- dropped\n and logged, the error returned to the caller. Nothing is silently\n arbitrated by schema/version. This is the \"schema defines the\n accepted message; anything else is the caller's fault\" rule applied\n to the label layer.\n - Precedence in `Route()`: explicit `t.Workflow` → arbitrary-label\n binding → kind binding → triage → implement → default.\n3. **Trust boundary for Module Yaegi code.** The plugin engine rule's\n invariant 6 draws a line between operator-trusted in-process code and\n repository-supplied code that must run in a container.\n\n **Resolved** in `docs/prds/module-position.md`, \"Trust boundary\": a\n Module is operator-installed, in-process and daemon-privileged, the same\n tier as `PluginDir` and `SecretEngineDir`, never repository-supplied\n task code.\n4. **First implementation slice.** Recommend: Module position + the\n playbook loader + trigger-to-workflow dispatch only (subsuming the existing\n label routing) first, proving the schema-gen -\u003e Yaegi -\u003e playbook path\n end to end on the smallest useful case, before adding Channel/Forge\n action kinds or the linter/LSP.\n\n **Resolved** (`archie-core-t2db.13`, `.14`, `.15`). The Module position\n (registry + log kind, t2db.13), the CEL expression environment\n (t2db.14), and the playbook document + event coordinator with\n single-action workflow-kind dispatch (t2db.15) are shipped.\n\n **Resolved** (`archie-core-t2db.23`): the coordinator selects a real\n task's workflow. The wiring point is the daemon's definition pin\n (`Daemon.resolveWorkflowID`, consumed by `pinWorkflowFromCollection` in\n `internal/daemon`), not `pollNATS`/`publishTask`. Routing belongs to the\n pin because a workflow definition is a\n database row pinned before dispatch, which makes the pin the one place a\n task's workflow is chosen. Precedence there is explicit `Task.Workflow`\n (the waiting_human -\u003e approved requeue) → matching playbook →\n `workflow.ResolveWorkflowID`'s label binding → kind binding → triage →\n implement. A playbook naming an undefined workflow is a reported failure\n that parks the task rather than a silent fall-through to the binding's\n choice.\n\n `Store.Dispatch` returns the selected workflow *name* and the matching\n playbook's id and version, not a compiled `workflow.Workflow`: the\n production caller decides against the active definition collection, and\n the id and version are the first two components of the gap-2 idempotency\n key. The playbook package does not import `internal/domain/workflow`.\n\n A `when` condition at this point reads what the intake knows about the\n originating issue: `kind`, `labels`, `owner`, `repo`, `number`, `title`,\n `body`, `source`, built by `playbookInput` in `internal/daemon`.\n\n The trigger shape reuses the existing workintake kind/label vocabulary.\n The single-action hard boundary is superseded by\n `docs/prds/multi-action-playbooks.md` (two-shape rule); Module, Channel,\n and Forge positions still have no production dispatch path.\n## Standing constraint\n\nThis may be commercialized, and is to be discernible from other open-source\nevent-driven-automation tooling. No design decision follows from that yet;\nit bears on licensing or feature-gating questions about playbook authoring\nand the LSP when they arise.\n\n## Packages this touches (first slice)\n\n- New: a schema-generation package/convention (`go:generate` source +\n emitted contracts) for the Module position.\n- New: playbook loader + event coordinator (name/location TBD -- likely\n `internal/domain/eda` or similar, following the domain-layer rule: no\n infrastructure imports, no untyped hooks).\n- `internal/domain/workflow`: dispatch target for `workflow`-kind actions,\n unchanged in its own contracts.\n- `internal/domain/workintake`: `labelKinds`/routing becomes one playbook\n among others once the loader subsumes it; existing Go fallback stays for\n backward compatibility until playbooks fully replace it (mirrors the\n `dynamic-workflow-triage.md` precedent of a two-line fallback change, not\n a rip-and-replace).\n" }, { "url": "/dev/prds/embedded-nats/", @@ -394,7 +394,7 @@ "url": "/dev/prds/state-store-contract/", "title": "State Store — contract boundary \u0026 transport (ratification)", "section": "PRDs", - "body": "# State Store — contract boundary \u0026 transport (ratification)\n\n**Status:** Approved\n**Revision:** 2e\n**Date:** 2026-09-06\n\nThis is the pre-implementation design document that Phase 2 subtasks `.4.2` (generalize\n`storerpc` as the State Store gRPC contract) and `.4.3` (stand up `archie-state-store`)\nimplement against.\n**Beads milestone:** archie-core-8cda.4.1\n**Parent:** `docs/prds/service-decomposition.md` (open-question Q4 **RESOLVED**)\n\nThis document fixes the **contract boundary** and **transport** for the State Store service.\nIt does not redesign Archie's domain model, does not invent a second store, and does **not**\nsplit `archie.db`. It records method ownership, DTO/wire representation, error semantics,\nlimits, mode/config shape, local-adapter compatibility, and cutover/deletion rules so that\n`.4.2` and `.4.3` have exactly one authoritative surface to build against.\n\n\u003e **Current state — read before `.4.2` starts.** The §12 step 2 relocation (`archie-core-8cda.\n\u003e 4.8`) has landed: `Task`/`Status*`/`Source*` and the 3-method `workflow.Store` now live in\n\u003e `internal/domain/workflow`, the six production workflow files import no `internal/store`\n\u003e symbol, and `storerpc`/the NATS transport assert against `workflow.Store` instead of\n\u003e `store.WorkflowStore`. The broader Phase 2 acceptance criterion (State Store as its own\n\u003e service behind gRPC) is still open — that is what `.4.2`/`.4.3` build next.\n\n\u003e **Relationship to migration-decisions §4 — not replaced.** `docs/architecture/\n\u003e migration-decisions.md` §4 (`store.Task → WorkflowExecution`, `mutable Stage →\n\u003e StepExecution history`, `RetryCount/Attempt → Attempt records`, versioned Workflow defs)\n\u003e **remains the authoritative, open document for the long-term Workflow migration.** This PRD\n\u003e does **not** replace it. It only pulls the minimal `Task`/`Status`/`Source` type + interface\n\u003e relocation forward as a Phase 2 prerequisite (because the acceptance criterion requires it);\n\u003e the rest of §4 stays its own migration and stays open.\n\n---\n\n## 1. Decision (the invariant everything follows)\n\nArchie is composed such that a single **`archie-state-store`** service owns the one existing\nSQLite file (`archie.db`) and exposes it over **gRPC behind narrow typed contracts**. There is\nexactly one store, one file, one service. No database-per-service, no distributed\ntransactions, no second store abstraction.\n\nTwo adapter families satisfy the contract interfaces:\n\n- **Local adapter** — `*store.Store` (the in-process SQLite implementation).\n- **Remote adapter** — `*staterpc.Client` (wraps the generated gRPC client).\n\nEach adapter carries an explicit compile-time assertion against the contract interfaces it\nserves. Composition picks local vs. remote by the presence of `[services.state].target`\n(empty = local; set = dial the gRPC service). Default is **local** (the State Store is not yet\nextracted), which is the PRD §3 posture; it differs from the gateway's remote-default only\nbecause the gateway is already a separate deployment and the State Store is not.\n\n\u003e **Contract-ownership fork, decided (rev. 2c — resolves the domain contradiction).**\n\u003e Ownership splits by the consumer's *layer*, per the dependency contract's rule #2\n\u003e (\"a domain defines the smallest interfaces required to perform its work\") and rule #7\n\u003e (\"database rows … MUST NOT leak into domain models\"):\n\u003e\n\u003e - The **workflow domain** (`internal/domain/workflow`) is the only domain-layer consumer of\n\u003e store state. It defines its own **consumer-owned** narrow `Store` interface\n\u003e (Update/Transition/InsertEvent) and its own **domain-owned** `Task` type plus\n\u003e `Status`/`Source` vocabulary. It imports **zero** `internal/store` symbols — satisfying the\n\u003e Phase 2 acceptance criterion. This is the minimal bound of migration-decisions §4\n\u003e (`store.Task → WorkflowExecution`), pulled forward **because** the acceptance criterion\n\u003e requires it.\n\u003e - The **daemon / webui / intake** store surfaces (`TaskStore`, `CaptureStore`,\n\u003e `MappingStore`, `BindingStore`, `BindingDispatcher`, `BindingTaskCreator`) stay\n\u003e **producer-owned** in `internal/store`, because those consumers are application and\n\u003e infrastructure layers, not domains — rule #2 does not apply to them.\n\nSo the State Store contract is **not** uniformly producer-owned: the domain-facing workflow\ncontract is consumer-owned (in the domain), while the daemon/webui store surfaces remain in\n`internal/store`. `internal/store` imports `internal/domain/workflow` for the `Task`/`Status`/\n`Source` types it persists (persistence → domain, the correct direction), and its `Store`\nimplementation satisfies `workflow.Store`. This is the single decision that makes the whole\nPhase 2 acceptance criterion and the architecture's rules simultaneously satisfiable.\n\n\u003e **`internal/store` is a TRANSITIONAL compatibility location (rev. 2c).** The repository's target\n\u003e architecture (`docs/architecture/dependencies-and-contracts.md`, `organisation.md`) places\n\u003e persistence implementations under `internal/infrastructure/\u003ccapability\u003e`. `internal/store`\n\u003e holding the task/capture/mapping/binding implementation and ALSO being the home of the\n\u003e producer-owned daemon/webui interfaces is acceptable **only as a transitional location**. The\n\u003e long-term home for the store implementation is `internal/infrastructure/store`; the\n\u003e producer-owned interfaces may move to the same infrastructure package (or an approved\n\u003e contract location) when the store implementation relocates. Agents must not treat\n\u003e `internal/store` as the final infrastructure boundary.\n\u003e\n\u003e **REV. 2d (archie-core-8cda.5.6): the producer-owned interfaces HAVE moved to an approved\n\u003e contract location** — `internal/domain/storecontract` (package `storecontract`), ahead of the\n\u003e store implementation's own relocation. The UI Service deletion gate required the archie-ui\n\u003e process to hold the contracts without linking `internal/store`'s SQL implementation, and\n\u003e `internal/contracts/` itself is ruled out: that tree is the `buf generate` output and\n\u003e `tools/proto.sh check` deletes hand-written files there. `internal/store` keeps type aliases\n\u003e (`type TaskStore = storecontract.TaskStore`, the sentinel `var`s, the value types) so\n\u003e daemon-side callers are unaffected; the sentinel message strings are unchanged and remain\n\u003e the wire contract (§4).\n\n---\n\n## 2. Contract boundary\n\n**Ownership split** (see the fork in §1): the **workflow-domain consumer contract** is\n`workflow.Store` (consumer-owned, in `internal/domain/workflow`); the **daemon/webui/intake\nstore surfaces** are the producer-owned interfaces in `internal/store/interface.go`.\n`store.WorkflowStore` is **superseded** — it is replaced by the domain-owned `workflow.Store`,\nand `internal/store`'s `Store` implementation satisfies `workflow.Store` directly. `TaskStore`\nremains the only composite; every other interface is flat. Narrow slices are split along\nconsumer-capability lines so no consumer acquires a surface it does not need\n(`interfacebloat` cap = 8).\n\n| Interface (owner) | Embeds | Methods | Local impl | Remote impl |\n|---|---|---|---|---|\n| `TaskStore` (store) | `TaskLifecycle`, `TaskEvents`, `TaskQueries`, `TaskArchiver`, `TaskRetryer` | 24 (union) | `*Store` | `*staterpc.Client` |\n| `TaskLifecycle` (store) | — | 8 | `*Store` | `*staterpc.Client` |\n| `TaskEvents` (store) | — | 7 | `*Store` | `*staterpc.Client` |\n| `TaskQueries` (store) | — | 7 | `*Store` | `*staterpc.Client` |\n| `TaskArchiver` (store) | — | 1 | `*Store` | `*staterpc.Client` |\n| `TaskRetryer` (store) | — | 1 | `*Store` | `*staterpc.Client` |\n| `workflow.Store` (workflow domain) | — | 3 | `*Store` | `*staterpc.Client` |\n| `CaptureStore` (store) | — | 2 | `*Store` | `*staterpc.Client` |\n| `MappingStore` (store) | — | 5 | `*Store` | `*staterpc.Client` |\n| `BindingStore` (store) | — | 6 | `*Store` | `*staterpc.Client` |\n| `BindingDispatcher` (store) | — | 3 | `*Store` | `*staterpc.Client` |\n| `BindingTaskCreator` (store) | — | 1 | `*Store` | `*staterpc.Client` |\n| `PlaybookDispatcher` (store) | — | 2 | `*Store` | `*staterpc.Client` |\n\n**Key property of the remote client:** `*staterpc.Client` is a *single* adapter that wraps one\n`StateStoreClient` and is asserted against **whichever narrow Go interfaces its caller needs**\n(exactly as `gatewayrpc.Client` asserts both `gateway.ChatContract` and\n`gateway.SessionStore`). The Go contracts stay narrow (≤8, the `interfacebloat` cap); the wire\nis one service. `workflow.Store` introduces **no new RPCs** — it is a narrow consumer face over\nthe same `Update`/`Transition`/`InsertEvent` RPCs.\n\n---\n\n## 3. Method ownership\n\n`*Store` remains the single implementation of the daemon/webui store surfaces and of\n`workflow.Store`. Ownership below is about which **process** is the primary consumer of each\nmethod and therefore which methods are genuinely cross-process on the wire vs.\ndaemon-internal.\n\nProcess legend: **DAEMON** = `archied` (incl. in-process webui + gateway runtime +\n`storerpc.Server`); **AGENT** = `archie-agent` worker (no DB connection); **GATEWAY** = store\naccess through daemon-side narrow adapter interfaces.\n\n| Contract | Primary consumers (process) | Remote on the wire? |\n|---|---|---|\n| `TaskLifecycle` (8) | DAEMON (dispatch, claim, action) + **AGENT** (`Transition`, `Update` via storerpc) | `Transition`/`Update` = **YES** (agent); rest DAEMON |\n| `TaskArchiver` (1) | DAEMON (`taskactions` archive) | not yet |\n| `TaskRetryer` (1) | DAEMON (`taskactions` retry) | not yet |\n| `TaskQueries` (7) | DAEMON + GATEWAY (status/list adapters) | not yet (`OpenPRs`/`Tasks`/`StatusCounts` cross to gateway adapters in-process) |\n| `TaskEvents` (7) | DAEMON + **AGENT** (`InsertEvent` via storerpc) | `InsertEvent` = **YES** (agent); rest DAEMON |\n| `workflow.Store` (3) | **AGENT** only | **YES** (the sole genuinely remote consumer) |\n| `CaptureStore` (2) | DAEMON (webhook intake + mapping editor) | not yet |\n| `MappingStore` (5) | DAEMON (dashboard editor + dispatch loop) | not yet |\n| `BindingStore` (6) | DAEMON (dashboard editor + dispatch loop) | not yet |\n| `BindingDispatcher` (3) | DAEMON (dispatch loop) | not yet |\n| `BindingTaskCreator` (1) | DAEMON (dispatch loop) | not yet |\n| `PlaybookDispatcher` (2) | DAEMON (EDA playbook dispatch loop) | **YES** (side-effecting-action idempotency ledger over gRPC) |\n\n**Consequence for migration order:** the **agent's `workflow.Store` is already cross-process\nalready** (via the NATS `storerpc` surrogate), so it is the first contract to move to gRPC. The\ndaemon-internal surfaces (`Capture`/`Mapping`/`Binding`) stay in-process until the daemon\nitself points at `archie-state-store`. `TaskStore` (the large composite) moves last.\n\n**Two orphan methods** — `TaskQueries.ClearTerminalTasks` and\n`TaskQueries.IncrementRetryCount` have **no production consumer** (only `*Store` tests). They\nremain part of the Go `TaskQueries`/`TaskStore` surface, and therefore part of the wire\ncontract, but are flagged for deletion: they must **not** gain dedicated consumers or wire\nmethods in this work. Do not add the rest of the codebase to them.\n\n---\n\n## 4. Domain-facing types \u0026 consumer-owned contract — resolves the contradiction\n\nThe Phase 2 acceptance criterion (\"`internal/domain/workflow` … no longer import\n`internal/store` directly — they call the State Store contract\") **cannot be met while**\n`store.Task`, `store.Status*`, `store.Source*`, and `store.WorkflowStore` live in\n`internal/store`. This section fixes where they move.\n\n**The relocation (a prerequisite that must land before or with `.4.2`):**\n\n| Current symbol (`internal/store`) | Moves to | Notes |\n|---|---|---|\n| `Task` type | `internal/domain/workflow.Task` | The task-execution record the workflow domain operates on. `internal/store` persists this type (persistence → domain). |\n| `Status` + `Status*` constants | `internal/domain/workflow` | e.g. `StatusClosedWontDo`, `StatusMerged`, `StatusPROpen`, `StatusWaitingHuman`, `StatusRunning` — task-lifecycle vocabulary. |\n| `Source` + `Source*` constants | `internal/domain/workflow` | e.g. `SourceChat` — provenance vocabulary, follows `Task`. |\n| `WorkflowStore` interface (3 methods) | `internal/domain/workflow.Store` | **Consumer-owned** (dependency rule #2). Supersedes `store.WorkflowStore`. |\n\n**Resulting dependency direction (correct per the architecture):**\n\n- `internal/domain/workflow` imports **no** `internal/store` symbols → the acceptance\n criterion is satisfied. It consumes `store.Task`'s replacement as `workflow.Task`, uses\n `workflow.Status*`/`workflow.Source*`, and depends on its own `workflow.Store` interface.\n- `internal/store` imports `internal/domain/workflow` for the `Task`/`Status`/`Source` types it\n reads and writes (infrastructure → domain). **NOTE: `internal/store` is a transitional\n compatibility location only** — its long-term home is `internal/infrastructure/store` (see\n the callout in §1). Do not treat `internal/store` as the final infrastructure boundary.\n- Every store test that constructs `store.Task{…}` is updated to the relocated type as part of\n this prerequisite. This is a mechanical rename across ~6 workflow production files\n (`workflow.go`, `steps.go`, `triage.go`, `review.go`, `feasibility.go`, `implement.go`) and\n their tests plus the store tests. `TestRecordDispatchViaExplicitTx` is removed rather than\n updated — its non-nil `*sql.Tx` path has no production consumer (see §5's `RecordDispatch`\n blocker).\n\n**What this does NOT do:** it does not implement the full §4 Workflow migration\n(`mutable Stage → StepExecution history`, `RetryCount/Attempt → Attempt records`,\n`workflow.Registry → versioned Workflow definitions`, etc.). It lands only the minimal\ntask-type + `Status`/`Source` + consumer-owned-interface relocation that the Phase 2\ndomain-decoupling criterion requires. The residue of §4 remains its own migration.\n\n**Forward-compatibility:** because the wire DTOs are derived from the domain `Task`/`Status`\n(§5), when the full §4 migration later renames `workflow.Task` (or refines it into\n`WorkflowExecution`), only `values.go` and the proto field mapping change — the RPC surface and\nconsumer interfaces are unaffected. The wire contract is deliberately type-agnostic at this\nlevel.\n\n---\n\n## 5. DTO / wire representation\n\n### Proto layout\n\n- **Service:** `proto/state/v1/state.proto`, `package state.v1`, **one** `StateStore` service\n (mirrors the gateway's single `ChatService`; proto bypasses the Go `interfacebloat` cap, as\n `ChatContract` does via `//nolint:interfacebloat`).\n- **Codegen:** the existing buf v2 pipeline. `proto/buf.yaml` (module, lint STANDARD, breaking\n FILE against `main`), `proto/buf.gen.yaml` **managed mode** with\n `go_package_prefix → github.com/samcharles93/archie-core/internal/contracts`, plugins\n `protoc-gen-go` + `protoc-gen-go-grpc`, `out: ../internal/contracts`,\n `opt: paths=source_relative`. **No `go_package` in source** — managed mode owns it. Generated\n Go lands in `internal/contracts/state/v1/` (package `statev1`), committed (`proto:check`\n fails on uncommitted/unmodified generated files).\n- **Adapter package:** `internal/infrastructure/staterpc/` with `server.go` (registers the\n generated server over the store), `client.go` (wraps the generated client into `store.*` and\n `workflow.Store` facades), `values.go` (proto ⇄ domain mappers), and `conformance_test.go`\n (bufconn local-vs-grpc dual-adapter suite), mirroring `internal/infrastructure/gatewayrpc/`.\n- **Binary:** `cmd/archie-state-store/main.go`, mirroring `cmd/archie-gateway/main.go` and the\n serve pattern in `internal/app/archied/gateway.go` (`serveGateway`).\n\n### The `StateStore` RPC set (union of the contracts, grouped)\n\nMethods are named after the store methods so the mapping is unambiguous. `workflow.Store` is a\n*view*, not new RPCs — `Update`/`Transition`/`InsertEvent` are shared with\n`TaskLifecycle`/`TaskEvents` and appear once.\n\n| Group | RPCs |\n|---|---|\n| Lifecycle | `EnqueueIssue`, `EnqueueChatTask`, `ClaimNext`, `ClaimByIssue`, `Transition`, `Update`, `Requeue`, `RecoverStale` |\n| Archive / Retry | `ArchiveTask`, `RetryTask` |\n| Queries | `TaskByIssue`, `OpenTaskByPR`, `TaskByID`, `OpenPRs`, `ClearTerminalTasks`, `Tasks`, `StatusCounts`, `IncrementRetryCount` |\n| Events | `InsertEvent`, `EventsSince`, `TaskEvents`, `WorkflowStats`, `StageStats`, `TokensByDay` |\n| Capture | `InsertCapture`, `ListCaptures` |\n| Mapping | `InsertMapping`, `GetMapping`, `ListMappings`, `UpdateMapping`, `DeleteMapping` |\n| Binding | `InsertBinding`, `GetBinding`, `ListBindings`, `UpdateBinding`, `DeleteBinding`, `ApproveBinding` |\n| Dispatch | `ArmedBindingsForSource`, `RecordDispatch`, `ListUndispatchedCaptures` |\n| Playbook | `RecordPlaybookDispatch`, `DeletePlaybookDispatches` |\n| BindingTaskCreator | `EnqueueBindingTask` |\n| Config snapshot | `PutConfigSnapshot`, `GetConfigSnapshot` |\n| Apply status | `PutApplyStatus`, `ListApplyStatus` |\n| Channel status | `PutChannelStatus`, `ListChannelStatus` |\n| Task log | `ReadTaskLog`, `StreamTaskLogContent` |\n\nThat is **51 unique contract RPCs** across one service (the `TaskEvents.Close` method is dropped).\n`Close()` is **excluded** from the wire (it is server lifecycle, not a client call) — see §11.\n\nThe channel-status pair follows the config snapshot's split for the same reason: the writer is the process that HOSTS the channels -- the Messaging Service, which alone observes whether a channel is starting, running or failed -- and the reader is the UI process. Both RPCs are administrative, so a task-scoped grant reaches neither. A channel's state is runtime state rather than a settings document, which is why it is not a `ControlPlaneService` resource kind: a failed channel is not a document anybody edits. Asking for a reload travels on its own surface rather than through this one; see `docs/architecture/migration-decisions.md`, \"Channel state to the dashboard\".\n\nThe config-snapshot pair was added by `archie-core-ymut` (see\n`docs/architecture/migration-decisions.md`, \"Dashboard configuration page\"). It is the one\nsurface whose writer is the daemon and whose reader is the UI process, and both RPCs are\nadministrative: `authorizesTaskScopedCall` is deny-by-default, so a task-scoped grant reaches\nneither. `ConfigSnapshot.document` is opaque to this contract — the store neither parses nor\nvalidates it, and `schema` names the projection so a reader can refuse a shape it does not\nunderstand.\n\nThe apply-status pair was added by `archie-core-pskb` (see\n`docs/prds/control-plane-apply-status.md`). It is administrative for the same reason as the\nconfig-snapshot pair, and deny-by-default in `authorizesTaskScopedCall` covers it without a\nnew rule. Unlike that pair, the read is a list: `ListApplyStatus` returns every process's\nrecord and an empty slice means nothing has reported yet, so the `found=false` convention for\nabsent single rows does not apply. The rows are keyed on `(process, kind)` and replaced in\nplace, because each process re-stamps its record on an interval and no reader wants a history\nof re-stamps.\n\nThe task-log pair was added by `archie-core-iaqx` (see `docs/prds/ui-service-boundary.md`'s\nroute inventory). It is the one surface whose payload is not a row: the store service reads\nthem out of the state directory it owns with `internal/logging`'s own reader, so the wire\ncarries `logging.Entry`/`logging.Query` values and neither side restates the JSONL format.\n`ReadTaskLog` returns a decoded page and `StreamTaskLogContent` returns the raw bytes\nserver-streamed, because a log file rotates at 20 MB — past gRPC's 4 MiB unary cap — and the\nsame reasoning that made `StreamCaptures` supersede `ListCaptures` applies. `found=false` with\nno error means the attempt has no log; a service with no reader answers `Unavailable` instead,\nand the two answers are load-bearing for the dashboard: only the second is a deployment\ncondition, and the panel that collapsed them told operators task logging was switched off when\nit was not. The Go facade is `storecontract.TaskLogStore` (2 methods), carried by\n`*staterpc.Client` alongside the rest.\n\nThe playbook pair was added by `archie-core-t2db.17`'s implementation (see\n`docs/prds/eda-playbook-engine.md`, execution-time gap 2). It is administrative\nfor the same reason as the config-snapshot and apply-status pairs, and\ndeny-by-default in `authorizesTaskScopedCall` covers it without a new rule. The\nrow is the structural `(playbook_id, playbook_version, event_id, action_id)`\ntuple, written **before** the action's side effect fires (the reverse of\n`binding_dispatches`), so a duplicate returns `store.ErrAlreadyDispatched` and\nthe coordinator skips the invoke rather than repeating a non-revocable side\neffect. `DeletePlaybookDispatches` is the reclamation primitive for a playbook\nremoved from the configured directory. The Go facade is\n`storecontract.PlaybookDispatcher` (2 methods), carried by `*staterpc.Client`\nalongside the rest.\n\n### Domain types and `values.go` mapping\n\n`values.go` maps proto ⇄ these types. The task model for the **agent** path is the domain-owned\n`workflow.Task` (§4); the daemon/webui store surfaces return the same `workflow.Task` (the store\npersists the domain type).\n\n| Domain type | Package | Notes for `values.go` |\n|---|---|---|\n| `workflow.Task` | `internal/domain/workflow` | time fields stored as layout strings; convert to/from `google.protobuf.Timestamp` |\n| `events.Event` | `internal/events` | `Timestamp` → `google.protobuf.Timestamp` |\n| `store.CapturedEvent` | `internal/store` | headers/body/remote_addr; body capped 256 KiB |\n| `mapping.Mapping` | `internal/domain/mapping` | JSON-path field bindings |\n| `binding.Binding` | `internal/domain/binding` | secret is **already decrypted** before returning; keep decrypted form out of any durable/logged representation |\n| `workflow.Status` / `workflow.Source` | `internal/domain/workflow` | enum-like; map to proto enum or string, keeping the names stable for compatibility |\n| `store.WorkflowStat` | `internal/store` | aggregate row |\n| `store.StageStat` | `internal/store` | aggregate row |\n| `store.DayTokens` | `internal/store` | aggregate row |\n\n### Wire-contract blocker fixed here: `RecordDispatch`\n\n`store.BindingDispatcher.RecordDispatch(ctx, tx *sql.Tx, bindingID, bindingVersion int64,\ncaptureID int64, taskID int64) error` takes a **`*sql.Tx`** that cannot cross a gRPC boundary.\n\n**Evidence that the `*sql.Tx` is removable:** the dispatch loop calls `RecordDispatch(ctx, nil,\n…)` (`internal/daemon/daemon.go`). The implementation falls back to the store's own\n`*sql.DB` when `tx == nil` (\"best-effort, not atomic\"; the comment and the\n`TestRecordDispatchViaExplicitTx` test are the only consumers of the non-nil `tx` path).\nProduction never opens the explicit transaction.\n\n**Decision:** the contract drops `*sql.Tx` from `RecordDispatch` (the method becomes\n`RecordDispatch(ctx, bindingID, bindingVersion, captureID, taskID) error`). The store service\nowns any transaction boundary internally when atomicity is wanted; the at-most-once\nguarantee is the `INSERT OR IGNORE` dedup row, which is already transactionless in production.\n`TestRecordDispatchViaExplicitTx` is superseded by the store service's own internal-transaction\ntest. **This is a required interface migration that must land before or with `.4.2`** (the\ncurrent `interface.go` signature and test still use the transaction-bearing form).\n\n---\n\n## 6. Agent State Store client handoff\n\nMoving the agent from NATS `storerpc` to gRPC `workflow.Store` requires the agent process to\nreceive a State Store endpoint. The agent already receives its NATS connection settings via\nboot-time env/flags (see `cmd/archie-agent/main.go`: `-nats-url` / `NATS_URL`, `NATS_TOKEN`\nenv), and the daemon injects them into the task container. The State Store handoff follows the\nsame seam.\n\n**Boot handoff contract (additive, not replacing the NATS settings):**\n\n| Field | Where | Source | Notes |\n|---|---|---|---|\n| `StateStoreTarget` | `agentworker.Settings` | env `STATE_STORE_URL` (or `-state-store-url` flag), set by the daemon in the container env | `host:port` of the State Store gRPC server. Loopback/Docker-bridge address — the agent reaches it the same way it reaches the daemon's NATS. |\n| `StateStoreToken` | `agentworker.Settings` | env `STATE_STORE_TOKEN` (or `-state-store-token` flag) | Per-task bearer token carried in gRPC metadata by a client interceptor (see §9). |\n| `StoreTimeout` | `agentworker.Settings` | a bounded per-call timeout | Mirrors `natsrpc.Client.Timeout` / the existing `rpcTimeout`; bounds each State Store call when the caller `ctx` has no deadline. |\n\n**Wiring change in the transport façade:**\n\n- `internal/infrastructure/agenttransport/nats.Transport` — `Store(timeout) store.WorkflowStore`\n returns `\u0026storerpc.Client{Conn: t.conn, Timeout: timeout}`. It becomes\n `Store(timeout) workflow.Store` and returns a *single long-lived* `staterpc.Client` dialed to\n `settings.StateStoreTarget` (with the token interceptor and per-call timeout).\n- `internal/app/agentworker/worker.go` `Settings` gains the three fields above; `taskServiceTransport`\n and `taskDependencies` change their `store` type from `store.WorkflowStore` to\n `workflow.Store` (via `domain/workflow.TaskContext{Store: …}`).\n- Reconnect/deadline: `grpc.NewClient` (or the existing dial path) provides transparent reconnect;\n each RPC is bounded by the per-call context deadline (`StoreTimeout`), matching the\n request/reply timeout the NATS path used. No streaming is involved, so no server-sent\n reconnect semantics apply.\n\n**Deletion rule for the NATS path:** the NATS `storerpc` (Server, Client, subjects\n`archie.store.update|transition|insert_event`) is deleted **only after** the agent has fully\nmoved to gRPC and the container env carries `STATE_STORE_URL`. Until then both paths may coexist\n(agent flips when the daemon starts injecting `STATE_STORE_URL`); there is no dual-live window\nacross a single agent process, which is either NATS-backed or gRPC-backed, never both.\n\n### Operational specifics (in-process listener → address → per-task injection)\n\n- **Who binds the in-process State Store listener:** the daemon (`archied`), at bootstrap, when\n the State Store is served in-process/multiplexed (mirroring `serveGateway` in\n `internal/app/archied/gateway.go`): a `grpc.NewServer()` + `RegisterServer` + graceful stop on\n `ctx.Done()`.\n- **Address selection (Docker-bridge reachable):** reuse the host-gateway resolution already used\n for embedded NATS (`internal/container/network.go` `RequireHostGateway` /\n `resolveHostGateway`). Per §9's single listener-topology rule, the listener binds the\n **host-gateway bridge address** when agent containers consume it (the Phase 2 default), or\n loopback-only when the daemon is the sole consumer. The chosen address is stored on the daemon\n as `d.ConnectedStateStore.URL` (analogous to `d.ConnectedNATS.URL`).\n- **Per-task injection:** `containerEnv(task)` (`internal/daemon/daemon.go`) appends\n `STATE_STORE_URL=\u003cd.ConnectedStateStore.URL\u003e` and `STATE_STORE_TOKEN=\u003ctoken\u003e` alongside the\n existing `NATS_URL`/`NATS_TOKEN`. The agent reads them via env (`-state-store-url` /\n `STATE_STORE_URL`, `-state-store-token` / `STATE_STORE_TOKEN`), the same seam as `NATS_URL`. A\n reloaded `[services.state]` must not point new containers at a store the daemon is not serving\n — the same immutability rule as NATS.\n\n---\n\n## 7. Error semantics\n\n### Current state (both boundaries lose error identity)\n\n- **NATS `natsrpc.Envelope`** (`internal/natsrpc/natsrpc.go`) serialises any error to a single\n `err.Error()` string and rehydrates a fresh `errors.New(string)`. The original type, `%w`\n chain, and sentinel identity are lost → `errors.Is(err, store.ErrStaleTransition)` is\n **false** across the boundary.\n- **gRPC `gatewayrpc`** returns raw `err` from handlers; gRPC-Go converts to\n `codes.Unknown` with only the message. The **only** explicit code in the repo is\n `codes.Unavailable` for a missing session store (`server.go`). No status-code mapping for\n store errors exists.\n\n### Decision — structured mapping so `errors.Is` survives\n\nThe `StateStore` service must map the store's sentinel errors to canonical gRPC codes and\nrehydrate them to the *same* sentinels on the client, so consumer `errors.Is` checks keep\nworking — an improvement over the flattening it replaces, not a regression.\n\n| Store sentinel / outcome | gRPC code | Client rehydrates to |\n|---|---|---|\n| `store.ErrStaleTransition` | `FailedPrecondition` | `store.ErrStaleTransition` |\n| `store.ErrBindingNotFound` | `NotFound` | `store.ErrBindingNotFound` |\n| `store.ErrMappingNotFound` | `NotFound` | `store.ErrMappingNotFound` |\n| `store.ErrBindingOverlap` | `FailedPrecondition` | `store.ErrBindingOverlap` |\n| `store.ErrBindingTransition` | `FailedPrecondition` | `store.ErrBindingTransition` |\n| `store.ErrAlreadyDispatched` | `AlreadyExists` | `store.ErrAlreadyDispatched` |\n| not-found-as-`(nil,nil)` (`TaskByID`, `GetBinding`, `GetMapping`, `ClaimNext`, `ClaimByIssue`, …) | `NotFound` with a `found=false` field (**not** an error) | `(nil, nil)` |\n| infra / wrapped error | `Internal` (**sanitised** message, see below) | wrapped `fmt.Errorf(...%w)` |\n| capability absent | `Unavailable` | context/typed error |\n\nThe `found=false`-not-error convention matters: the repo deliberately returns `(nil, nil)` for\nnot-found in reads and a sentinel for not-found in writes. The wire must preserve **both**\nconventions (a `GoogleRPCStatus` with `NotFound` is an *error*, which would wrongly surface to\ncallers that expect `(nil, nil)`). Implement `found` as an explicit boolean field on read\nresponses, and map `(nil,nil)` ⇄ `found=false` + `OK`.\n\n### Error-detail sanitisation (rev. 2c)\n\nFull internal error messages must **not** be placed in gRPC `details` for remote callers. SQL\npaths, provider/forge details, secrets, and stack traces are sensitive internals. The rule:\n\n- **Sentinel errors** — the client rehydrates the sentinel from the code; the message string in\n `status.Error` is a short, public, stable phrase (e.g. `\"stale transition\"`), not the raw\n wrapped chain.\n- **Infra errors** — the wire carries a **sanitised** public message + the `Internal` code.\n The **full** error (with `%w` chain, SQL, provider detail) is logged **server-side only** by\n the store service. The client rehydrates a generic wrapped error and must not surface the raw\n text to users.\n\n---\n\n## 8. Limits\n\n**Interface bloat (`interfacebloat` cap = 8)** — `.golangci.yml` `interfacebloat: max: 8`,\nwhich overrides the upstream default of 10. The Go consumer facades must stay ≤8 methods.\n`TaskLifecycle` is exactly 8 (at the cap); `TaskStore` is the 24-method composite that the cap\nis why it is decomposed; proto services bypass the cap (like `ChatContract`).\n\n**Payload caps:**\n- Capture body: `defaultCaptureMaxBodyBytes = 256 KiB` (may reduce further; the wire message\n must not exceed gRPC's default 4 MiB receive limit for a single capture). A single capture\n fits, but a *batch* of them did not: the caller-side cap of up to 100 captures at 256 KiB each\n could exceed 4 MiB in one unary `ListCaptures`/`ListUndispatchedCaptures` response. `.4.7` adds\n `StreamCaptures`/`StreamUndispatchedCaptures` (server-streaming, one capture per message) as\n the path both `staterpc.Client` methods actually call; the original unary RPCs stay defined,\n unmodified and deprecated, only so `buf breaking` protects an in-flight rolling deploy.\n- Task `detail`/`park_reason`/event `Detail`: 4000-char hard cap (`clip(s, 4000)`).\n\n**Retention (stays server-side in the store; the wire does not carry pruning decisions):**\n- `events` table: **no prune, unbounded growth** — a known risk to revisit, not changed here.\n- `captured_events`: prune-on-write, default 7 days / 5000 events.\n\n**Pagination/arity:** store read methods take a raw `int limit` (no store-side cap); the\ncaller-side caps (task list 100, log 2000, captures 100, capture scan window 5000, binding\ndispatch batch 100, session list 20) remain caller-side. The wire passes `limit int` and returns\n`[]*` slices, mirroring the existing Go signatures — **no cursor/paging protocol is introduced**\nin this work (the current `EventsSince(sinceID, limit)` offset cursor is the only one, and it\nstays as-is).\n\n---\n\n## 9. Transport security boundary\n\nThe gateway precedent is loopback-only, unauthenticated gRPC. The State Store holds the\ncrown-jewel `archie.db` (task/event/capture/mapping/binding data), so its boundary must be\nstated explicitly rather than left implicit.\n\n**The State Store's equivalent boundary (rev. 2c):**\n\n- **Listener topology — single rule (rev. 2c).** One in-process listener; its bind address is\n chosen by the consumer set, never both at once:\n - *Agent containers consume it* (the Phase 2 driver during multiplexed serving) → bind to the\n **host-gateway bridge address** so containers can reach it, and **token auth is mandatory**\n (a bridge host is reachable by any process on that bridge). A container **cannot reach the\n host's `127.0.0.1`**, so loopback-only is NOT usable for the agent-consumed path.\n - *Only the daemon consumes it* (no agent) → bind **loopback-only** (`--listen 127.0.0.1`),\n `insecure` is fine because it is not network-reachable; no token required.\n - There is **no dual-listener** design. The practical default during `.4.2` (the reason\n in-process serving exists is to give the agent a gRPC target) is the **bridge address +\n token**. `127.0.0.1` is reserved for the daemon-local-only case.\n- **Agent access → the container bridge.** The agent runs in a task-scoped container on the\n host and reaches the State Store over the Docker bridge gateway (the bind address from the\n topology rule above), exactly as it reaches the daemon's NATS. To close the \"any host\n process could reach the store\" gap, the agent **authenticates with a per-task bearer token**\n carried in gRPC metadata by a client interceptor (`StateStoreToken`, §6). Network\n reachability + the token is the trust boundary for the default bridge topology.\n- **The daemon** uses the local adapter (`*store.Store`) in-process by default, so no network\n boundary exists until `[services.state].target` is set; when it is set, the daemon client\n authenticates with a token supplied alongside the target.\n- **Non-loopback / k8s:** TLS/mTLS is an **operator decision**, not a default. Any non-loopback\n exposure (e.g. `--listen 0.0.0.0`, a CDN, a cross-host service) requires an explicit decision\n and mTLS or token auth — mirroring the gateway's `--listen`/`--enable-cors-header`\n confirmation caveat. The NATS-KV discovery fallback supplies endpoints but does not add\n transport security; that stays the operator's responsibility.\n\n### Token lifecycle (rev. 2c; scoping added in `.4.7`)\n\n- **Generation:** the daemon generates a per-task bearer token when it acquires the container\n (or per incumbence) and injects it via `containerEnv` as `STATE_STORE_TOKEN`. A single token\n may cover a task's container lifetime.\n- **Registration (`.4.7`, post-standalone-cutover):** the State Store is now its own process\n (`.4.3`/`.4.6`), so the daemon can no longer just hold an in-process issued-token set for the\n server to check — it registers the token at the remote authority via the\n `RegisterTaskGrant`/`RevokeTaskGrant` RPCs (`staterpc.GrantIssuer`, `daemon.StateStoreGrantIssuer`),\n admin-token-authenticated calls a task-scoped token itself can never make.\n- **Lifetime:** tied to the task + container grace period; rotated on a new container acquisition\n for the same task; a token for a released container is invalidated (`RevokeTaskGrant`, deferred\n alongside `ContainerPool.Release`). An unrevoked grant still expires server-side (bounded by\n `RegisterTaskGrantRequest.lifetime_seconds`, capped at 7 days) so a daemon crash before revoke\n cannot leave a grant valid indefinitely.\n- **Validation and scope (`.4.7`):** a gRPC interceptor (`staterpc.TaskGrants.UnaryInterceptor`)\n validates the token against the State Store's own issued-grant set; missing/unknown/expired →\n `codes.Unauthenticated`. Unlike the daemon's own administrative token (full access to all ~40\n RPCs), a task-scoped grant additionally authorizes only `Update`/`Transition`/`InsertEvent` on\n its own task ID — every other RPC, including the streaming capture surface and\n `RegisterTaskGrant`/`RevokeTaskGrant` themselves, is `codes.PermissionDenied` for a task-scoped\n caller. The token is carried in gRPC metadata, never in a URL.\n\n### Fail-closed on non-loopback (rev. 2c)\n\nIf `[services.state].target` points to a non-loopback address and **neither** TLS **nor** a token\nis configured, the State Store client (daemon or agent) **fails closed** — it refuses to\nstart/dial rather than fall back to a plaintext, unauthenticated remote connection. This mirrors\nthe gateway's `--listen`/`--enable-cors-header` confinement caveat and prevents a silent\nunauthenticated remote store. The loopback + token default is the only path that starts without\nan explicit operator decision.\n\n---\n\n## 10. Mode / config shape\n\n**Follow the gateway's presence-based `target` seam, not the NATS `mode` enum.**\n\n- **Services are keyed by name, not named as struct fields.** A\n `State ServiceConnection` field per service is the duplication\n `docs/prds/service-registry.md` removes, and this document's summary describes\n the seam as `[services.\u003cname\u003e]`:\n ```go\n type Services map[string]ServiceConnection\n ```\n registered once via `RegisterService(context, name, target, listen, tokenEnv)`. The TOML\n shape below is unchanged, so nothing else in this section is affected; read\n `cfg.Services.Get(\"state\")` wherever it says `cfg.Services.State`.\n- **Empty `Target` → startup error.** The local `*store.Store` adapter default was\n withdrawn when the State Store was extracted (§12 step 7): the standalone `archie-state-store` process\n exclusively owns `archie.db`, so both `archied` and `archie-gateway` now reject an empty\n target (`internal/app/archied/bootstrap.go` and `state_store_client.go`: `services.state.target\n is required`). **Set `Target` → dial gRPC** with `*staterpc.Client`; no client opens the\n database itself. This differs from the gateway, where `Target` is defaulted to\n `127.0.0.1:8585` because `archie-gateway` is already a separate process whose client and\n server share one default.\n- `[services.state]` is present and **uncommented** in `config.example.toml`, with `target`\n marked REQUIRED, because the value is load-bearing for every client. A commented-out\n `target` cannot boot, so the section ships uncommented.\n- Composition root: `internal/app/archied/bootstrap.go` (the daemon) and\n `internal/app/archieui/config.go` (the UI process) both read `cfg.Services.Get(\"state\")` and\n dial the remote State Store. `openProductionTaskStore` is **not** called from the daemon: it\n lives in the standalone `archie-state-store` process (`internal/app/archied/state_store.go`),\n which is the only path that opens `archie.db`.\n\n\u003e **Mode fork, decided.** The gateway uses a single `target` address (no enum). The State Store\n\u003e boundary uses the same seam for symmetry, and **both now require a non-empty `target`**: the\n\u003e State Store's local-store default was withdrawn when the process was extracted (§12 step 7).\n\u003e No `mode` field is added. A companion `STATE_STORE_TOKEN` secret (or a\n\u003e `[services.state] target_token` key) carries authentication.\n\n---\n\n## 11. Local-adapter compatibility + conformance\n\n- `*store.Store` (local) and `*staterpc.Client` (remote) satisfy the **same** contract\n interfaces. Use **explicit** `var _` assertions on **both** adapters (the gateway's local\n adapters use only structural/return-type assertions; the store precedent's explicit `var _`\n style is stronger and should be adopted on both sides).\n ```go\n var _ workflow.Store = (*staterpc.Client)(nil) // first contract to move (agent)\n var _ store.TaskStore = (*staterpc.Client)(nil) // daemon path, added progressively\n var _ store.CaptureStore = (*staterpc.Client)(nil)\n var _ store.MappingStore = (*staterpc.Client)(nil)\n var _ store.BindingStore = (*staterpc.Client)(nil)\n var _ store.BindingDispatcher = (*staterpc.Client)(nil)\n var _ store.BindingTaskCreator = (*staterpc.Client)(nil)\n var _ store.PlaybookDispatcher = (*staterpc.Client)(nil)\n // local side (in store), already present:\n var _ workflow.Store = (*Store)(nil)\n ```\n- **Conformance suite:** a `staterpc/conformance_test.go` bufconn suite runs the **same**\n contract through both `\"local\"` and `\"grpc\"` adapters (mirroring\n `gatewayrpc/conformance_test.go`), asserting error-sentinel fidelity and the\n not-found-as-`(nil,nil)` convention both ways.\n- **`Close()`:** stays on the Go store interfaces (so `TaskStore`/`TaskEvents` remain fully\n satisfiable) but is **not** a `StateStore` RPC. The remote `staterpc.Client` implements\n `Close()` as a **no-op**; the store service owns its own DB lifecycle. When the daemon hands\n the store to `archie-state-store`, `b.cleanup()` no longer calls `store.Close()` on a remote\n client (it is the daemon's own lifecycle only when it owns the store in-process).\n\n---\n\n## 12. Cutover / deletion rules\n\n1. **Ratify the contract (this document).** No code.\n2. **Domain type + interface relocation (prerequisite, coupled with §4).** Move `Task`/`Status`/\n `Source` and the `WorkflowStore` interface into `internal/domain/workflow` (as\n `workflow.Task`/`workflow.Status`/`workflow.Source`/`workflow.Store`), so\n `internal/domain/workflow` imports no `internal/store`. Update the store tests and the 6\n workflow production files. **This must land before or with `.4.2`, because `.4.2` cannot move\n the agent's store contract off `store.WorkflowStore` while the domain still imports it.**\n3. **`.4.2` — generalize `storerpc` to the gRPC contract.** Move the agent's `workflow.Store`\n (`Update`/`Transition`/`InsertEvent`) from NATS JSON `storerpc` to `staterpc.Client` over\n gRPC, using the §6 handoff. The daemon serves the gRPC `StateStore` server **in-process\n (multiplexed)** first so the agent has a gRPC target before the standalone binary exists\n (PRD §3 \"multiplexed in-process is the migration default\"). Bind the in-process listener and\n inject `STATE_STORE_URL`/`STATE_STORE_TOKEN` per task (see §6 \"Operational specifics\"). Also\n land the `RecordDispatch` `*sql.Tx` drop (see §5) in the same span. Flip the agent's\n `Transport.Store` to a gRPC client once `STATE_STORE_URL` is injected.\n4. **Delete NATS `storerpc` only after the agent is fully on gRPC** — delete `internal/storerpc`\n (Server, Client, subjects `archie.store.update|transition|insert_event`), the\n `natsrpc.RegisterAll` registration, and the now-unused `store.WorkflowStore` NATS path in\n `internal/infrastructure/agenttransport/nats`, in the same commit.\n5. **`[services.state].target` stays empty (inproc)** through `.4.2`; the `archie-state-store`\n `cmd/` binary is **`.4.3`** and hosts the same `StateStore` gRPC service.\n6. **Swap daemon-internal contracts one at a time** (only once the daemon points at\n `archie-state-store`): `Capture → Mapping → Binding → TaskStore`. Each swap is independent —\n add the remote-client composition path, run the per-contract conformance suite, flip\n `[services.state].target`, delete the local use. A consumer mixes in-process and remote\n contracts for a contract-code (no dual-live *window*; each contract is either local or remote\n at a time).\n7. **Final flip / deletion of the in-process serving path:** when the daemon no longer owns the\n store (i.e. `TaskStore` has moved), delete the in-process `openProductionTaskStore` /\n `wireWebStoreSurfaces` path in the daemon in a **same-commit** deletion — no dual-store\n ownership. The single SQLite file remains owned by `archie-state-store`.\n8. **Never** split `archie.db` or add a second store. The gateway keeps its own already-separate\n session SQLite; that is unchanged and out of state-store scope.\n\n---\n\n## 13. Decisions recorded (for the review trail)\n\n- **`internal/store` is a transitional compatibility location** (target\n `internal/infrastructure/store`), not the final infrastructure boundary. The producer-owned\n daemon/webui store surfaces live there now but may relocate with the implementation.\n- **In-process listener + token lifecycle:** the daemon binds the in-process State Store\n listener to the **host-gateway bridge address** when agent containers consume it (the Phase 2\n default) or loopback-only when the daemon is the sole consumer — one listener, one address\n rule, never both (rev. 2c). It selects the bridge address by reusing the embedded-NATS\n host-gateway resolution and injects `STATE_STORE_URL`/`STATE_STORE_TOKEN` per task via\n `containerEnv`. Token is per-incumbence (task + container grace period) and validated by a\n gRPC interceptor → `codes.Unauthenticated` on missing/unknown/expired. Non-loopback targets\n fail closed without TLS or a token.\n- **§12 step 2 (domain type + interface relocation) is done** — `internal/domain/workflow`\n imports no `internal/store` symbol and `store.WorkflowStore` is gone from `TaskContext`. The\n broader Phase 2 acceptance criterion (State Store as its own gRPC service) is still open;\n `.4.2` picks up from here.\n- **Ownership split:** domain-facing workflow contract is **consumer-owned** in the domain\n (`workflow.Store` + `workflow.Task`/`Status`/`Source`, per dependency rules #2 and #7);\n daemon/webui store surfaces stay **producer-owned** in `internal/store`. `store.WorkflowStore`\n is superseded by `workflow.Store`.\n- **One `StateStore` gRPC service** (49 contract RPCs, grouped by contract) + narrow Go consumer facades\n (≤8) on `staterpc.Client` — mirrors the single-`ChatService` precedent.\n- **Domain type relocation** (`Task`/`Status`/`Source` → `internal/domain/workflow`) is a\n Phase 2 prerequisite, pulled forward from migration-decisions §4 (minimal bound,\n `Task → WorkflowExecution`); the rest of §4 stays a separate migration.\n- **`RecordDispatch` drops `*sql.Tx`** (wire-blocker; production passes `nil`) — a\n required interface migration landing before/with `.4.2`.\n- **`Close()` not on the wire**; remote client `Close()` is a no-op.\n- **Structured gRPC error codes** with client rehydration to store sentinels (preserves\n `errors.Is`), a `found=false` + `OK` convention for read not-found, and **sanitised public\n error details** (full detail logged server-side only).\n- **Agent handoff:** `agentworker.Settings` gains `StateStoreTarget`/`StateStoreToken`/\n `StoreTimeout`, injected as `STATE_STORE_URL`/`STATE_STORE_TOKEN` env (mirrors the NATS\n handoff); `Transport.Store` returns a long-lived gRPC `workflow.Store` client.\n- **Transport security + listener topology (rev. 2c):** one in-process listener, bound to the\n host-gateway bridge address when agent containers consume it (the Phase 2 default) or\n loopback-only for a daemon-local-only consumer. `insecure` only on loopback; mandatory\n per-task bearer-token auth on the bridge; TLS/mTLS is an operator choice for any\n non-loopback/k8s topology.\n- **Mode = presence-based `[services.state].target`**, default **local** (not yet extracted);\n add `[services.state]` to `config.example.toml`.\n- **Explicit `var _` assertions on both adapters**; per-contract bufconn conformance.\n- **Migration order:** domain type/interface relocation → `workflow.Store` (agent) → stand up\n `archie-state-store` → `Capture` → `Mapping` → `Binding` → `TaskStore`; NATS `storerpc`\n deleted once the agent is on gRPC; in-process store path deleted in a same-commit flip.\n- **Orphans `ClearTerminalTasks`/`IncrementRetryCount`** retained but not built upon.\n" + "body": "# State Store — contract boundary \u0026 transport (ratification)\n\n**Status:** Approved\n**Revision:** 2e\n**Date:** 2026-09-06\n\nThis is the pre-implementation design document that Phase 2 subtasks `.4.2` (generalize\n`storerpc` as the State Store gRPC contract) and `.4.3` (stand up `archie-state-store`)\nimplement against.\n**Beads milestone:** archie-core-8cda.4.1\n**Parent:** `docs/prds/service-decomposition.md` (open-question Q4 **RESOLVED**)\n\nThis document fixes the **contract boundary** and **transport** for the State Store service.\nIt does not redesign Archie's domain model, does not invent a second store, and does **not**\nsplit `archie.db`. It records method ownership, DTO/wire representation, error semantics,\nlimits, mode/config shape, local-adapter compatibility, and cutover/deletion rules so that\n`.4.2` and `.4.3` have exactly one authoritative surface to build against.\n\n\u003e **Current state — read before `.4.2` starts.** The §12 step 2 relocation (`archie-core-8cda.\n\u003e 4.8`) has landed: `Task`/`Status*`/`Source*` and the 3-method `workflow.Store` now live in\n\u003e `internal/domain/workflow`, the six production workflow files import no `internal/store`\n\u003e symbol, and `storerpc`/the NATS transport assert against `workflow.Store` instead of\n\u003e `store.WorkflowStore`. The broader Phase 2 acceptance criterion (State Store as its own\n\u003e service behind gRPC) is still open — that is what `.4.2`/`.4.3` build next.\n\n\u003e **Relationship to migration-decisions §4 — not replaced.** `docs/architecture/\n\u003e migration-decisions.md` §4 (`store.Task → WorkflowExecution`, `mutable Stage →\n\u003e StepExecution history`, `RetryCount/Attempt → Attempt records`, versioned Workflow defs)\n\u003e **remains the authoritative, open document for the long-term Workflow migration.** This PRD\n\u003e does **not** replace it. It only pulls the minimal `Task`/`Status`/`Source` type + interface\n\u003e relocation forward as a Phase 2 prerequisite (because the acceptance criterion requires it);\n\u003e the rest of §4 stays its own migration and stays open.\n\n---\n\n## 1. Decision (the invariant everything follows)\n\nArchie is composed such that a single **`archie-state-store`** service owns the one existing\nSQLite file (`archie.db`) and exposes it over **gRPC behind narrow typed contracts**. There is\nexactly one store, one file, one service. No database-per-service, no distributed\ntransactions, no second store abstraction.\n\nTwo adapter families satisfy the contract interfaces:\n\n- **Local adapter** — `*store.Store` (the in-process SQLite implementation).\n- **Remote adapter** — `*staterpc.Client` (wraps the generated gRPC client).\n\nEach adapter carries an explicit compile-time assertion against the contract interfaces it\nserves. Composition picks local vs. remote by the presence of `[services.state].target`\n(empty = local; set = dial the gRPC service). Default is **local** (the State Store is not yet\nextracted), which is the PRD §3 posture; it differs from the gateway's remote-default only\nbecause the gateway is already a separate deployment and the State Store is not.\n\n\u003e **Contract-ownership fork, decided (rev. 2c — resolves the domain contradiction).**\n\u003e Ownership splits by the consumer's *layer*, per the dependency contract's rule #2\n\u003e (\"a domain defines the smallest interfaces required to perform its work\") and rule #7\n\u003e (\"database rows … MUST NOT leak into domain models\"):\n\u003e\n\u003e - The **workflow domain** (`internal/domain/workflow`) is the only domain-layer consumer of\n\u003e store state. It defines its own **consumer-owned** narrow `Store` interface\n\u003e (Update/Transition/InsertEvent) and its own **domain-owned** `Task` type plus\n\u003e `Status`/`Source` vocabulary. It imports **zero** `internal/store` symbols — satisfying the\n\u003e Phase 2 acceptance criterion. This is the minimal bound of migration-decisions §4\n\u003e (`store.Task → WorkflowExecution`), pulled forward **because** the acceptance criterion\n\u003e requires it.\n\u003e - The **daemon / webui / intake** store surfaces (`TaskStore`, `CaptureStore`,\n\u003e `MappingStore`, `BindingStore`, `BindingDispatcher`, `BindingTaskCreator`) stay\n\u003e **producer-owned** in `internal/store`, because those consumers are application and\n\u003e infrastructure layers, not domains — rule #2 does not apply to them.\n\nSo the State Store contract is **not** uniformly producer-owned: the domain-facing workflow\ncontract is consumer-owned (in the domain), while the daemon/webui store surfaces remain in\n`internal/store`. `internal/store` imports `internal/domain/workflow` for the `Task`/`Status`/\n`Source` types it persists (persistence → domain, the correct direction), and its `Store`\nimplementation satisfies `workflow.Store`. This is the single decision that makes the whole\nPhase 2 acceptance criterion and the architecture's rules simultaneously satisfiable.\n\n\u003e **`internal/store` is a TRANSITIONAL compatibility location (rev. 2c).** The repository's target\n\u003e architecture (`docs/architecture/dependencies-and-contracts.md`, `organisation.md`) places\n\u003e persistence implementations under `internal/infrastructure/\u003ccapability\u003e`. `internal/store`\n\u003e holding the task/capture/mapping/binding implementation and ALSO being the home of the\n\u003e producer-owned daemon/webui interfaces is acceptable **only as a transitional location**. The\n\u003e long-term home for the store implementation is `internal/infrastructure/store`; the\n\u003e producer-owned interfaces may move to the same infrastructure package (or an approved\n\u003e contract location) when the store implementation relocates. Agents must not treat\n\u003e `internal/store` as the final infrastructure boundary.\n\u003e\n\u003e **REV. 2d (archie-core-8cda.5.6): the producer-owned interfaces HAVE moved to an approved\n\u003e contract location** — `internal/domain/storecontract` (package `storecontract`), ahead of the\n\u003e store implementation's own relocation. The UI Service deletion gate required the archie-ui\n\u003e process to hold the contracts without linking `internal/store`'s SQL implementation, and\n\u003e `internal/contracts/` itself is ruled out: that tree is the `buf generate` output and\n\u003e `tools/proto.sh check` deletes hand-written files there. `internal/store` keeps type aliases\n\u003e (`type TaskStore = storecontract.TaskStore`, the sentinel `var`s, the value types) so\n\u003e daemon-side callers are unaffected; the sentinel message strings are unchanged and remain\n\u003e the wire contract (§4).\n\n---\n\n## 2. Contract boundary\n\n**Ownership split** (see the fork in §1): the **workflow-domain consumer contract** is\n`workflow.Store` (consumer-owned, in `internal/domain/workflow`); the **daemon/webui/intake\nstore surfaces** are the producer-owned interfaces in `internal/store/interface.go`.\n`store.WorkflowStore` is **superseded** — it is replaced by the domain-owned `workflow.Store`,\nand `internal/store`'s `Store` implementation satisfies `workflow.Store` directly. `TaskStore`\nremains the only composite; every other interface is flat. Narrow slices are split along\nconsumer-capability lines so no consumer acquires a surface it does not need\n(`interfacebloat` cap = 8).\n\n| Interface (owner) | Embeds | Methods | Local impl | Remote impl |\n|---|---|---|---|---|\n| `TaskStore` (store) | `TaskLifecycle`, `TaskEvents`, `TaskQueries`, `TaskArchiver`, `TaskRetryer` | 24 (union) | `*Store` | `*staterpc.Client` |\n| `TaskLifecycle` (store) | — | 8 | `*Store` | `*staterpc.Client` |\n| `TaskEvents` (store) | — | 7 | `*Store` | `*staterpc.Client` |\n| `TaskQueries` (store) | — | 7 | `*Store` | `*staterpc.Client` |\n| `TaskArchiver` (store) | — | 1 | `*Store` | `*staterpc.Client` |\n| `TaskRetryer` (store) | — | 1 | `*Store` | `*staterpc.Client` |\n| `workflow.Store` (workflow domain) | — | 3 | `*Store` | `*staterpc.Client` |\n| `CaptureStore` (store) | — | 2 | `*Store` | `*staterpc.Client` |\n| `MappingStore` (store) | — | 5 | `*Store` | `*staterpc.Client` |\n| `BindingStore` (store) | — | 6 | `*Store` | `*staterpc.Client` |\n| `BindingDispatcher` (store) | — | 3 | `*Store` | `*staterpc.Client` |\n| `BindingTaskCreator` (store) | — | 1 | `*Store` | `*staterpc.Client` |\n| `PlaybookDispatcher` (store) | — | 2 | `*Store` | `*staterpc.Client` |\n\n**Key property of the remote client:** `*staterpc.Client` is a *single* adapter that wraps one\n`StateStoreClient` and is asserted against **whichever narrow Go interfaces its caller needs**\n(exactly as `gatewayrpc.Client` asserts both `gateway.ChatContract` and\n`gateway.SessionStore`). The Go contracts stay narrow (≤8, the `interfacebloat` cap); the wire\nis one service. `workflow.Store` introduces **no new RPCs** — it is a narrow consumer face over\nthe same `Update`/`Transition`/`InsertEvent` RPCs.\n\n---\n\n## 3. Method ownership\n\n`*Store` remains the single implementation of the daemon/webui store surfaces and of\n`workflow.Store`. Ownership below is about which **process** is the primary consumer of each\nmethod and therefore which methods are genuinely cross-process on the wire vs.\ndaemon-internal.\n\nProcess legend: **DAEMON** = `archied` (incl. in-process webui + gateway runtime +\n`storerpc.Server`); **AGENT** = `archie-agent` worker (no DB connection); **GATEWAY** = store\naccess through daemon-side narrow adapter interfaces.\n\n| Contract | Primary consumers (process) | Remote on the wire? |\n|---|---|---|\n| `TaskLifecycle` (8) | DAEMON (dispatch, claim, action) + **AGENT** (`Transition`, `Update` via storerpc) | `Transition`/`Update` = **YES** (agent); rest DAEMON |\n| `TaskArchiver` (1) | DAEMON (`taskactions` archive) | not yet |\n| `TaskRetryer` (1) | DAEMON (`taskactions` retry) | not yet |\n| `TaskQueries` (7) | DAEMON + GATEWAY (status/list adapters) | not yet (`OpenPRs`/`Tasks`/`StatusCounts` cross to gateway adapters in-process) |\n| `TaskEvents` (7) | DAEMON + **AGENT** (`InsertEvent` via storerpc) | `InsertEvent` = **YES** (agent); rest DAEMON |\n| `workflow.Store` (3) | **AGENT** only | **YES** (the sole genuinely remote consumer) |\n| `CaptureStore` (2) | DAEMON (webhook intake + mapping editor) | not yet |\n| `MappingStore` (5) | DAEMON (dashboard editor + dispatch loop) | not yet |\n| `BindingStore` (6) | DAEMON (dashboard editor + dispatch loop) | not yet |\n| `BindingDispatcher` (3) | DAEMON (dispatch loop) | not yet |\n| `BindingTaskCreator` (1) | DAEMON (dispatch loop) | not yet |\n| `PlaybookDispatcher` (2) | DAEMON (EDA playbook dispatch loop) | **YES** (side-effecting-action idempotency ledger over gRPC) |\n\n**Consequence for migration order:** the **agent's `workflow.Store` is already cross-process\nalready** (via the NATS `storerpc` surrogate), so it is the first contract to move to gRPC. The\ndaemon-internal surfaces (`Capture`/`Mapping`/`Binding`) stay in-process until the daemon\nitself points at `archie-state-store`. `TaskStore` (the large composite) moves last.\n\n**Two orphan methods** — `TaskQueries.ClearTerminalTasks` and\n`TaskQueries.IncrementRetryCount` have **no production consumer** (only `*Store` tests). They\nremain part of the Go `TaskQueries`/`TaskStore` surface, and therefore part of the wire\ncontract, but are flagged for deletion: they must **not** gain dedicated consumers or wire\nmethods in this work. Do not add the rest of the codebase to them.\n\n---\n\n## 4. Domain-facing types \u0026 consumer-owned contract — resolves the contradiction\n\nThe Phase 2 acceptance criterion (\"`internal/domain/workflow` … no longer import\n`internal/store` directly — they call the State Store contract\") **cannot be met while**\n`store.Task`, `store.Status*`, `store.Source*`, and `store.WorkflowStore` live in\n`internal/store`. This section fixes where they move.\n\n**The relocation (a prerequisite that must land before or with `.4.2`):**\n\n| Current symbol (`internal/store`) | Moves to | Notes |\n|---|---|---|\n| `Task` type | `internal/domain/workflow.Task` | The task-execution record the workflow domain operates on. `internal/store` persists this type (persistence → domain). |\n| `Status` + `Status*` constants | `internal/domain/workflow` | e.g. `StatusClosedWontDo`, `StatusMerged`, `StatusPROpen`, `StatusWaitingHuman`, `StatusRunning` — task-lifecycle vocabulary. |\n| `Source` + `Source*` constants | `internal/domain/workflow` | e.g. `SourceChat` — provenance vocabulary, follows `Task`. |\n| `WorkflowStore` interface (3 methods) | `internal/domain/workflow.Store` | **Consumer-owned** (dependency rule #2). Supersedes `store.WorkflowStore`. |\n\n**Resulting dependency direction (correct per the architecture):**\n\n- `internal/domain/workflow` imports **no** `internal/store` symbols → the acceptance\n criterion is satisfied. It consumes `store.Task`'s replacement as `workflow.Task`, uses\n `workflow.Status*`/`workflow.Source*`, and depends on its own `workflow.Store` interface.\n- `internal/store` imports `internal/domain/workflow` for the `Task`/`Status`/`Source` types it\n reads and writes (infrastructure → domain). **NOTE: `internal/store` is a transitional\n compatibility location only** — its long-term home is `internal/infrastructure/store` (see\n the callout in §1). Do not treat `internal/store` as the final infrastructure boundary.\n- Every store test that constructs `store.Task{…}` is updated to the relocated type as part of\n this prerequisite. This is a mechanical rename across ~6 workflow production files\n (`workflow.go`, `steps.go`, `triage.go`, `review.go`, `feasibility.go`, `implement.go`) and\n their tests plus the store tests. `TestRecordDispatchViaExplicitTx` is removed rather than\n updated — its non-nil `*sql.Tx` path has no production consumer (see §5's `RecordDispatch`\n blocker).\n\n**What this does NOT do:** it does not implement the full §4 Workflow migration\n(`mutable Stage → StepExecution history`, `RetryCount/Attempt → Attempt records`,\n`workflow.Registry → versioned Workflow definitions`, etc.). It lands only the minimal\ntask-type + `Status`/`Source` + consumer-owned-interface relocation that the Phase 2\ndomain-decoupling criterion requires. The residue of §4 remains its own migration.\n\n**Forward-compatibility:** because the wire DTOs are derived from the domain `Task`/`Status`\n(§5), when the full §4 migration later renames `workflow.Task` (or refines it into\n`WorkflowExecution`), only `values.go` and the proto field mapping change — the RPC surface and\nconsumer interfaces are unaffected. The wire contract is deliberately type-agnostic at this\nlevel.\n\n---\n\n## 5. DTO / wire representation\n\n### Proto layout\n\n- **Service:** `proto/state/v1/state.proto`, `package state.v1`, **one** `StateStore` service\n (mirrors the gateway's single `ChatService`; proto bypasses the Go `interfacebloat` cap, as\n `ChatContract` does via `//nolint:interfacebloat`).\n- **Codegen:** the existing buf v2 pipeline. `proto/buf.yaml` (module, lint STANDARD, breaking\n FILE against `main`), `proto/buf.gen.yaml` **managed mode** with\n `go_package_prefix → github.com/samcharles93/archie-core/internal/contracts`, plugins\n `protoc-gen-go` + `protoc-gen-go-grpc`, `out: ../internal/contracts`,\n `opt: paths=source_relative`. **No `go_package` in source** — managed mode owns it. Generated\n Go lands in `internal/contracts/state/v1/` (package `statev1`), committed (`proto:check`\n fails on uncommitted/unmodified generated files).\n- **Adapter package:** `internal/infrastructure/staterpc/` with `server.go` (registers the\n generated server over the store), `client.go` (wraps the generated client into `store.*` and\n `workflow.Store` facades), `values.go` (proto ⇄ domain mappers), and `conformance_test.go`\n (bufconn local-vs-grpc dual-adapter suite), mirroring `internal/infrastructure/gatewayrpc/`.\n- **Binary:** `cmd/archie-state-store/main.go`, mirroring `cmd/archie-gateway/main.go` and the\n serve pattern in `internal/app/archied/gateway.go` (`serveGateway`).\n\n### The `StateStore` RPC set (union of the contracts, grouped)\n\nMethods are named after the store methods so the mapping is unambiguous. `workflow.Store` is a\n*view*, not new RPCs — `Update`/`Transition`/`InsertEvent` are shared with\n`TaskLifecycle`/`TaskEvents` and appear once.\n\n| Group | RPCs |\n|---|---|\n| Lifecycle | `EnqueueIssue`, `EnqueueChatTask`, `ClaimNext`, `ClaimByIssue`, `Transition`, `Update`, `Requeue`, `RecoverStale` |\n| Archive / Retry | `ArchiveTask`, `RetryTask` |\n| Queries | `TaskByIssue`, `OpenTaskByPR`, `TaskByID`, `OpenPRs`, `ClearTerminalTasks`, `Tasks`, `StatusCounts`, `IncrementRetryCount` |\n| Events | `InsertEvent`, `EventsSince`, `TaskEvents`, `WorkflowStats`, `StageStats`, `TokensByDay` |\n| Capture | `InsertCapture`, `ListCaptures` |\n| Mapping | `InsertMapping`, `GetMapping`, `ListMappings`, `UpdateMapping`, `DeleteMapping` |\n| Binding | `InsertBinding`, `GetBinding`, `ListBindings`, `UpdateBinding`, `DeleteBinding`, `ApproveBinding` |\n| Dispatch | `ArmedBindingsForSource`, `RecordDispatch`, `ListUndispatchedCaptures` |\n| Playbook | `RecordPlaybookDispatch`, `DeletePlaybookDispatches` |\n| BindingTaskCreator | `EnqueueBindingTask` |\n| Config snapshot | `PutConfigSnapshot`, `GetConfigSnapshot` |\n| Apply status | `PutApplyStatus`, `ListApplyStatus` |\n| Channel status | `PutChannelStatus`, `ListChannelStatus` |\n| Task log | `ReadTaskLog`, `StreamTaskLogContent` |\n\nThat is **51 unique contract RPCs** across one service (the `TaskEvents.Close` method is dropped).\n`Close()` is **excluded** from the wire (it is server lifecycle, not a client call) — see §11.\n\nThe channel-status pair follows the config snapshot's split for the same reason: the writer is the process that HOSTS the channels -- the Messaging Service, which alone observes whether a channel is starting, running or failed -- and the reader is the UI process. Both RPCs are administrative, so a task-scoped grant reaches neither. A channel's state is runtime state rather than a settings document, which is why it is not a `ControlPlaneService` resource kind: a failed channel is not a document anybody edits. Asking for a reload travels on its own surface rather than through this one; see `docs/architecture/migration-decisions.md`, \"Channel state to the dashboard\".\n\nThe config-snapshot pair was added by `archie-core-ymut` (see\n`docs/architecture/migration-decisions.md`, \"Dashboard configuration page\"). It is the one\nsurface whose writer is the daemon and whose reader is the UI process, and both RPCs are\nadministrative: `authorizesTaskScopedCall` is deny-by-default, so a task-scoped grant reaches\nneither. `ConfigSnapshot.document` is opaque to this contract — the store neither parses nor\nvalidates it, and `schema` names the projection so a reader can refuse a shape it does not\nunderstand.\n\nThe apply-status pair was added by `archie-core-pskb` (see\n`docs/prds/control-plane-apply-status.md`). It is administrative for the same reason as the\nconfig-snapshot pair, and deny-by-default in `authorizesTaskScopedCall` covers it without a\nnew rule. Unlike that pair, the read is a list: `ListApplyStatus` returns every process's\nrecord and an empty slice means nothing has reported yet, so the `found=false` convention for\nabsent single rows does not apply. The rows are keyed on `(process, kind)` and replaced in\nplace, because each process re-stamps its record on an interval and no reader wants a history\nof re-stamps.\n\nThe task-log pair was added by `archie-core-iaqx` (see `docs/prds/ui-service-boundary.md`'s\nroute inventory). It is the one surface whose payload is not a row: the store service reads\nthem out of the state directory it owns with `internal/logging`'s own reader, so the wire\ncarries `logging.Entry`/`logging.Query` values and neither side restates the JSONL format.\n`ReadTaskLog` returns a decoded page and `StreamTaskLogContent` returns the raw bytes\nserver-streamed, because a log file rotates at 20 MB — past gRPC's 4 MiB unary cap — and the\nsame reasoning that made `StreamCaptures` supersede `ListCaptures` applies. `found=false` with\nno error means the attempt has no log; a service with no reader answers `Unavailable` instead,\nand the two answers are load-bearing for the dashboard: only the second is a deployment\ncondition, and the panel that collapsed them told operators task logging was switched off when\nit was not. The Go facade is `storecontract.TaskLogStore` (2 methods), carried by\n`*staterpc.Client` alongside the rest.\n\nThe playbook pair was added by `archie-core-t2db.17`'s implementation (see\n`docs/prds/eda-playbook-engine.md`, execution-time gap 2). It is administrative\nfor the same reason as the config-snapshot and apply-status pairs, and\ndeny-by-default in `authorizesTaskScopedCall` covers it without a new rule. The\nrow is the structural `(playbook_id, playbook_version, event_id, action_id)`\ntuple, written **before** the action's side effect fires (the reverse of\n`binding_dispatches`), so a duplicate returns `store.ErrAlreadyDispatched` and\nthe coordinator skips the invoke rather than repeating a non-revocable side\neffect. `DeletePlaybookDispatches` is the explicit reclamation primitive; nothing\ncalls it automatically, because a playbook file's absence at boot is not a\ndeliberate removal (`eda-playbook-engine.md`, \"Lifetime\"). The Go facade is\n`storecontract.PlaybookDispatcher` (2 methods), carried by `*staterpc.Client`\nalongside the rest.\n\n### Domain types and `values.go` mapping\n\n`values.go` maps proto ⇄ these types. The task model for the **agent** path is the domain-owned\n`workflow.Task` (§4); the daemon/webui store surfaces return the same `workflow.Task` (the store\npersists the domain type).\n\n| Domain type | Package | Notes for `values.go` |\n|---|---|---|\n| `workflow.Task` | `internal/domain/workflow` | time fields stored as layout strings; convert to/from `google.protobuf.Timestamp` |\n| `events.Event` | `internal/events` | `Timestamp` → `google.protobuf.Timestamp` |\n| `store.CapturedEvent` | `internal/store` | headers/body/remote_addr; body capped 256 KiB |\n| `mapping.Mapping` | `internal/domain/mapping` | JSON-path field bindings |\n| `binding.Binding` | `internal/domain/binding` | secret is **already decrypted** before returning; keep decrypted form out of any durable/logged representation |\n| `workflow.Status` / `workflow.Source` | `internal/domain/workflow` | enum-like; map to proto enum or string, keeping the names stable for compatibility |\n| `store.WorkflowStat` | `internal/store` | aggregate row |\n| `store.StageStat` | `internal/store` | aggregate row |\n| `store.DayTokens` | `internal/store` | aggregate row |\n\n### Wire-contract blocker fixed here: `RecordDispatch`\n\n`store.BindingDispatcher.RecordDispatch(ctx, tx *sql.Tx, bindingID, bindingVersion int64,\ncaptureID int64, taskID int64) error` takes a **`*sql.Tx`** that cannot cross a gRPC boundary.\n\n**Evidence that the `*sql.Tx` is removable:** the dispatch loop calls `RecordDispatch(ctx, nil,\n…)` (`internal/daemon/daemon.go`). The implementation falls back to the store's own\n`*sql.DB` when `tx == nil` (\"best-effort, not atomic\"; the comment and the\n`TestRecordDispatchViaExplicitTx` test are the only consumers of the non-nil `tx` path).\nProduction never opens the explicit transaction.\n\n**Decision:** the contract drops `*sql.Tx` from `RecordDispatch` (the method becomes\n`RecordDispatch(ctx, bindingID, bindingVersion, captureID, taskID) error`). The store service\nowns any transaction boundary internally when atomicity is wanted; the at-most-once\nguarantee is the `INSERT OR IGNORE` dedup row, which is already transactionless in production.\n`TestRecordDispatchViaExplicitTx` is superseded by the store service's own internal-transaction\ntest. **This is a required interface migration that must land before or with `.4.2`** (the\ncurrent `interface.go` signature and test still use the transaction-bearing form).\n\n---\n\n## 6. Agent State Store client handoff\n\nMoving the agent from NATS `storerpc` to gRPC `workflow.Store` requires the agent process to\nreceive a State Store endpoint. The agent already receives its NATS connection settings via\nboot-time env/flags (see `cmd/archie-agent/main.go`: `-nats-url` / `NATS_URL`, `NATS_TOKEN`\nenv), and the daemon injects them into the task container. The State Store handoff follows the\nsame seam.\n\n**Boot handoff contract (additive, not replacing the NATS settings):**\n\n| Field | Where | Source | Notes |\n|---|---|---|---|\n| `StateStoreTarget` | `agentworker.Settings` | env `STATE_STORE_URL` (or `-state-store-url` flag), set by the daemon in the container env | `host:port` of the State Store gRPC server. Loopback/Docker-bridge address — the agent reaches it the same way it reaches the daemon's NATS. |\n| `StateStoreToken` | `agentworker.Settings` | env `STATE_STORE_TOKEN` (or `-state-store-token` flag) | Per-task bearer token carried in gRPC metadata by a client interceptor (see §9). |\n| `StoreTimeout` | `agentworker.Settings` | a bounded per-call timeout | Mirrors `natsrpc.Client.Timeout` / the existing `rpcTimeout`; bounds each State Store call when the caller `ctx` has no deadline. |\n\n**Wiring change in the transport façade:**\n\n- `internal/infrastructure/agenttransport/nats.Transport` — `Store(timeout) store.WorkflowStore`\n returns `\u0026storerpc.Client{Conn: t.conn, Timeout: timeout}`. It becomes\n `Store(timeout) workflow.Store` and returns a *single long-lived* `staterpc.Client` dialed to\n `settings.StateStoreTarget` (with the token interceptor and per-call timeout).\n- `internal/app/agentworker/worker.go` `Settings` gains the three fields above; `taskServiceTransport`\n and `taskDependencies` change their `store` type from `store.WorkflowStore` to\n `workflow.Store` (via `domain/workflow.TaskContext{Store: …}`).\n- Reconnect/deadline: `grpc.NewClient` (or the existing dial path) provides transparent reconnect;\n each RPC is bounded by the per-call context deadline (`StoreTimeout`), matching the\n request/reply timeout the NATS path used. No streaming is involved, so no server-sent\n reconnect semantics apply.\n\n**Deletion rule for the NATS path:** the NATS `storerpc` (Server, Client, subjects\n`archie.store.update|transition|insert_event`) is deleted **only after** the agent has fully\nmoved to gRPC and the container env carries `STATE_STORE_URL`. Until then both paths may coexist\n(agent flips when the daemon starts injecting `STATE_STORE_URL`); there is no dual-live window\nacross a single agent process, which is either NATS-backed or gRPC-backed, never both.\n\n### Operational specifics (in-process listener → address → per-task injection)\n\n- **Who binds the in-process State Store listener:** the daemon (`archied`), at bootstrap, when\n the State Store is served in-process/multiplexed (mirroring `serveGateway` in\n `internal/app/archied/gateway.go`): a `grpc.NewServer()` + `RegisterServer` + graceful stop on\n `ctx.Done()`.\n- **Address selection (Docker-bridge reachable):** reuse the host-gateway resolution already used\n for embedded NATS (`internal/container/network.go` `RequireHostGateway` /\n `resolveHostGateway`). Per §9's single listener-topology rule, the listener binds the\n **host-gateway bridge address** when agent containers consume it (the Phase 2 default), or\n loopback-only when the daemon is the sole consumer. The chosen address is stored on the daemon\n as `d.ConnectedStateStore.URL` (analogous to `d.ConnectedNATS.URL`).\n- **Per-task injection:** `containerEnv(task)` (`internal/daemon/daemon.go`) appends\n `STATE_STORE_URL=\u003cd.ConnectedStateStore.URL\u003e` and `STATE_STORE_TOKEN=\u003ctoken\u003e` alongside the\n existing `NATS_URL`/`NATS_TOKEN`. The agent reads them via env (`-state-store-url` /\n `STATE_STORE_URL`, `-state-store-token` / `STATE_STORE_TOKEN`), the same seam as `NATS_URL`. A\n reloaded `[services.state]` must not point new containers at a store the daemon is not serving\n — the same immutability rule as NATS.\n\n---\n\n## 7. Error semantics\n\n### Current state (both boundaries lose error identity)\n\n- **NATS `natsrpc.Envelope`** (`internal/natsrpc/natsrpc.go`) serialises any error to a single\n `err.Error()` string and rehydrates a fresh `errors.New(string)`. The original type, `%w`\n chain, and sentinel identity are lost → `errors.Is(err, store.ErrStaleTransition)` is\n **false** across the boundary.\n- **gRPC `gatewayrpc`** returns raw `err` from handlers; gRPC-Go converts to\n `codes.Unknown` with only the message. The **only** explicit code in the repo is\n `codes.Unavailable` for a missing session store (`server.go`). No status-code mapping for\n store errors exists.\n\n### Decision — structured mapping so `errors.Is` survives\n\nThe `StateStore` service must map the store's sentinel errors to canonical gRPC codes and\nrehydrate them to the *same* sentinels on the client, so consumer `errors.Is` checks keep\nworking — an improvement over the flattening it replaces, not a regression.\n\n| Store sentinel / outcome | gRPC code | Client rehydrates to |\n|---|---|---|\n| `store.ErrStaleTransition` | `FailedPrecondition` | `store.ErrStaleTransition` |\n| `store.ErrBindingNotFound` | `NotFound` | `store.ErrBindingNotFound` |\n| `store.ErrMappingNotFound` | `NotFound` | `store.ErrMappingNotFound` |\n| `store.ErrBindingOverlap` | `FailedPrecondition` | `store.ErrBindingOverlap` |\n| `store.ErrBindingTransition` | `FailedPrecondition` | `store.ErrBindingTransition` |\n| `store.ErrAlreadyDispatched` | `AlreadyExists` | `store.ErrAlreadyDispatched` |\n| not-found-as-`(nil,nil)` (`TaskByID`, `GetBinding`, `GetMapping`, `ClaimNext`, `ClaimByIssue`, …) | `NotFound` with a `found=false` field (**not** an error) | `(nil, nil)` |\n| infra / wrapped error | `Internal` (**sanitised** message, see below) | wrapped `fmt.Errorf(...%w)` |\n| capability absent | `Unavailable` | context/typed error |\n\nThe `found=false`-not-error convention matters: the repo deliberately returns `(nil, nil)` for\nnot-found in reads and a sentinel for not-found in writes. The wire must preserve **both**\nconventions (a `GoogleRPCStatus` with `NotFound` is an *error*, which would wrongly surface to\ncallers that expect `(nil, nil)`). Implement `found` as an explicit boolean field on read\nresponses, and map `(nil,nil)` ⇄ `found=false` + `OK`.\n\n### Error-detail sanitisation (rev. 2c)\n\nFull internal error messages must **not** be placed in gRPC `details` for remote callers. SQL\npaths, provider/forge details, secrets, and stack traces are sensitive internals. The rule:\n\n- **Sentinel errors** — the client rehydrates the sentinel from the code; the message string in\n `status.Error` is a short, public, stable phrase (e.g. `\"stale transition\"`), not the raw\n wrapped chain.\n- **Infra errors** — the wire carries a **sanitised** public message + the `Internal` code.\n The **full** error (with `%w` chain, SQL, provider detail) is logged **server-side only** by\n the store service. The client rehydrates a generic wrapped error and must not surface the raw\n text to users.\n\n---\n\n## 8. Limits\n\n**Interface bloat (`interfacebloat` cap = 8)** — `.golangci.yml` `interfacebloat: max: 8`,\nwhich overrides the upstream default of 10. The Go consumer facades must stay ≤8 methods.\n`TaskLifecycle` is exactly 8 (at the cap); `TaskStore` is the 24-method composite that the cap\nis why it is decomposed; proto services bypass the cap (like `ChatContract`).\n\n**Payload caps:**\n- Capture body: `defaultCaptureMaxBodyBytes = 256 KiB` (may reduce further; the wire message\n must not exceed gRPC's default 4 MiB receive limit for a single capture). A single capture\n fits, but a *batch* of them did not: the caller-side cap of up to 100 captures at 256 KiB each\n could exceed 4 MiB in one unary `ListCaptures`/`ListUndispatchedCaptures` response. `.4.7` adds\n `StreamCaptures`/`StreamUndispatchedCaptures` (server-streaming, one capture per message) as\n the path both `staterpc.Client` methods actually call; the original unary RPCs stay defined,\n unmodified and deprecated, only so `buf breaking` protects an in-flight rolling deploy.\n- Task `detail`/`park_reason`/event `Detail`: 4000-char hard cap (`clip(s, 4000)`).\n\n**Retention (stays server-side in the store; the wire does not carry pruning decisions):**\n- `events` table: **no prune, unbounded growth** — a known risk to revisit, not changed here.\n- `captured_events`: prune-on-write, default 7 days / 5000 events.\n\n**Pagination/arity:** store read methods take a raw `int limit` (no store-side cap); the\ncaller-side caps (task list 100, log 2000, captures 100, capture scan window 5000, binding\ndispatch batch 100, session list 20) remain caller-side. The wire passes `limit int` and returns\n`[]*` slices, mirroring the existing Go signatures — **no cursor/paging protocol is introduced**\nin this work (the current `EventsSince(sinceID, limit)` offset cursor is the only one, and it\nstays as-is).\n\n---\n\n## 9. Transport security boundary\n\nThe gateway precedent is loopback-only, unauthenticated gRPC. The State Store holds the\ncrown-jewel `archie.db` (task/event/capture/mapping/binding data), so its boundary must be\nstated explicitly rather than left implicit.\n\n**The State Store's equivalent boundary (rev. 2c):**\n\n- **Listener topology — single rule (rev. 2c).** One in-process listener; its bind address is\n chosen by the consumer set, never both at once:\n - *Agent containers consume it* (the Phase 2 driver during multiplexed serving) → bind to the\n **host-gateway bridge address** so containers can reach it, and **token auth is mandatory**\n (a bridge host is reachable by any process on that bridge). A container **cannot reach the\n host's `127.0.0.1`**, so loopback-only is NOT usable for the agent-consumed path.\n - *Only the daemon consumes it* (no agent) → bind **loopback-only** (`--listen 127.0.0.1`),\n `insecure` is fine because it is not network-reachable; no token required.\n - There is **no dual-listener** design. The practical default during `.4.2` (the reason\n in-process serving exists is to give the agent a gRPC target) is the **bridge address +\n token**. `127.0.0.1` is reserved for the daemon-local-only case.\n- **Agent access → the container bridge.** The agent runs in a task-scoped container on the\n host and reaches the State Store over the Docker bridge gateway (the bind address from the\n topology rule above), exactly as it reaches the daemon's NATS. To close the \"any host\n process could reach the store\" gap, the agent **authenticates with a per-task bearer token**\n carried in gRPC metadata by a client interceptor (`StateStoreToken`, §6). Network\n reachability + the token is the trust boundary for the default bridge topology.\n- **The daemon** uses the local adapter (`*store.Store`) in-process by default, so no network\n boundary exists until `[services.state].target` is set; when it is set, the daemon client\n authenticates with a token supplied alongside the target.\n- **Non-loopback / k8s:** TLS/mTLS is an **operator decision**, not a default. Any non-loopback\n exposure (e.g. `--listen 0.0.0.0`, a CDN, a cross-host service) requires an explicit decision\n and mTLS or token auth — mirroring the gateway's `--listen`/`--enable-cors-header`\n confirmation caveat. The NATS-KV discovery fallback supplies endpoints but does not add\n transport security; that stays the operator's responsibility.\n\n### Token lifecycle (rev. 2c; scoping added in `.4.7`)\n\n- **Generation:** the daemon generates a per-task bearer token when it acquires the container\n (or per incumbence) and injects it via `containerEnv` as `STATE_STORE_TOKEN`. A single token\n may cover a task's container lifetime.\n- **Registration (`.4.7`, post-standalone-cutover):** the State Store is now its own process\n (`.4.3`/`.4.6`), so the daemon can no longer just hold an in-process issued-token set for the\n server to check — it registers the token at the remote authority via the\n `RegisterTaskGrant`/`RevokeTaskGrant` RPCs (`staterpc.GrantIssuer`, `daemon.StateStoreGrantIssuer`),\n admin-token-authenticated calls a task-scoped token itself can never make.\n- **Lifetime:** tied to the task + container grace period; rotated on a new container acquisition\n for the same task; a token for a released container is invalidated (`RevokeTaskGrant`, deferred\n alongside `ContainerPool.Release`). An unrevoked grant still expires server-side (bounded by\n `RegisterTaskGrantRequest.lifetime_seconds`, capped at 7 days) so a daemon crash before revoke\n cannot leave a grant valid indefinitely.\n- **Validation and scope (`.4.7`):** a gRPC interceptor (`staterpc.TaskGrants.UnaryInterceptor`)\n validates the token against the State Store's own issued-grant set; missing/unknown/expired →\n `codes.Unauthenticated`. Unlike the daemon's own administrative token (full access to all ~40\n RPCs), a task-scoped grant additionally authorizes only `Update`/`Transition`/`InsertEvent` on\n its own task ID — every other RPC, including the streaming capture surface and\n `RegisterTaskGrant`/`RevokeTaskGrant` themselves, is `codes.PermissionDenied` for a task-scoped\n caller. The token is carried in gRPC metadata, never in a URL.\n\n### Fail-closed on non-loopback (rev. 2c)\n\nIf `[services.state].target` points to a non-loopback address and **neither** TLS **nor** a token\nis configured, the State Store client (daemon or agent) **fails closed** — it refuses to\nstart/dial rather than fall back to a plaintext, unauthenticated remote connection. This mirrors\nthe gateway's `--listen`/`--enable-cors-header` confinement caveat and prevents a silent\nunauthenticated remote store. The loopback + token default is the only path that starts without\nan explicit operator decision.\n\n---\n\n## 10. Mode / config shape\n\n**Follow the gateway's presence-based `target` seam, not the NATS `mode` enum.**\n\n- **Services are keyed by name, not named as struct fields.** A\n `State ServiceConnection` field per service is the duplication\n `docs/prds/service-registry.md` removes, and this document's summary describes\n the seam as `[services.\u003cname\u003e]`:\n ```go\n type Services map[string]ServiceConnection\n ```\n registered once via `RegisterService(context, name, target, listen, tokenEnv)`. The TOML\n shape below is unchanged, so nothing else in this section is affected; read\n `cfg.Services.Get(\"state\")` wherever it says `cfg.Services.State`.\n- **Empty `Target` → startup error.** The local `*store.Store` adapter default was\n withdrawn when the State Store was extracted (§12 step 7): the standalone `archie-state-store` process\n exclusively owns `archie.db`, so both `archied` and `archie-gateway` now reject an empty\n target (`internal/app/archied/bootstrap.go` and `state_store_client.go`: `services.state.target\n is required`). **Set `Target` → dial gRPC** with `*staterpc.Client`; no client opens the\n database itself. This differs from the gateway, where `Target` is defaulted to\n `127.0.0.1:8585` because `archie-gateway` is already a separate process whose client and\n server share one default.\n- `[services.state]` is present and **uncommented** in `config.example.toml`, with `target`\n marked REQUIRED, because the value is load-bearing for every client. A commented-out\n `target` cannot boot, so the section ships uncommented.\n- Composition root: `internal/app/archied/bootstrap.go` (the daemon) and\n `internal/app/archieui/config.go` (the UI process) both read `cfg.Services.Get(\"state\")` and\n dial the remote State Store. `openProductionTaskStore` is **not** called from the daemon: it\n lives in the standalone `archie-state-store` process (`internal/app/archied/state_store.go`),\n which is the only path that opens `archie.db`.\n\n\u003e **Mode fork, decided.** The gateway uses a single `target` address (no enum). The State Store\n\u003e boundary uses the same seam for symmetry, and **both now require a non-empty `target`**: the\n\u003e State Store's local-store default was withdrawn when the process was extracted (§12 step 7).\n\u003e No `mode` field is added. A companion `STATE_STORE_TOKEN` secret (or a\n\u003e `[services.state] target_token` key) carries authentication.\n\n---\n\n## 11. Local-adapter compatibility + conformance\n\n- `*store.Store` (local) and `*staterpc.Client` (remote) satisfy the **same** contract\n interfaces. Use **explicit** `var _` assertions on **both** adapters (the gateway's local\n adapters use only structural/return-type assertions; the store precedent's explicit `var _`\n style is stronger and should be adopted on both sides).\n ```go\n var _ workflow.Store = (*staterpc.Client)(nil) // first contract to move (agent)\n var _ store.TaskStore = (*staterpc.Client)(nil) // daemon path, added progressively\n var _ store.CaptureStore = (*staterpc.Client)(nil)\n var _ store.MappingStore = (*staterpc.Client)(nil)\n var _ store.BindingStore = (*staterpc.Client)(nil)\n var _ store.BindingDispatcher = (*staterpc.Client)(nil)\n var _ store.BindingTaskCreator = (*staterpc.Client)(nil)\n var _ store.PlaybookDispatcher = (*staterpc.Client)(nil)\n // local side (in store), already present:\n var _ workflow.Store = (*Store)(nil)\n ```\n- **Conformance suite:** a `staterpc/conformance_test.go` bufconn suite runs the **same**\n contract through both `\"local\"` and `\"grpc\"` adapters (mirroring\n `gatewayrpc/conformance_test.go`), asserting error-sentinel fidelity and the\n not-found-as-`(nil,nil)` convention both ways.\n- **`Close()`:** stays on the Go store interfaces (so `TaskStore`/`TaskEvents` remain fully\n satisfiable) but is **not** a `StateStore` RPC. The remote `staterpc.Client` implements\n `Close()` as a **no-op**; the store service owns its own DB lifecycle. When the daemon hands\n the store to `archie-state-store`, `b.cleanup()` no longer calls `store.Close()` on a remote\n client (it is the daemon's own lifecycle only when it owns the store in-process).\n\n---\n\n## 12. Cutover / deletion rules\n\n1. **Ratify the contract (this document).** No code.\n2. **Domain type + interface relocation (prerequisite, coupled with §4).** Move `Task`/`Status`/\n `Source` and the `WorkflowStore` interface into `internal/domain/workflow` (as\n `workflow.Task`/`workflow.Status`/`workflow.Source`/`workflow.Store`), so\n `internal/domain/workflow` imports no `internal/store`. Update the store tests and the 6\n workflow production files. **This must land before or with `.4.2`, because `.4.2` cannot move\n the agent's store contract off `store.WorkflowStore` while the domain still imports it.**\n3. **`.4.2` — generalize `storerpc` to the gRPC contract.** Move the agent's `workflow.Store`\n (`Update`/`Transition`/`InsertEvent`) from NATS JSON `storerpc` to `staterpc.Client` over\n gRPC, using the §6 handoff. The daemon serves the gRPC `StateStore` server **in-process\n (multiplexed)** first so the agent has a gRPC target before the standalone binary exists\n (PRD §3 \"multiplexed in-process is the migration default\"). Bind the in-process listener and\n inject `STATE_STORE_URL`/`STATE_STORE_TOKEN` per task (see §6 \"Operational specifics\"). Also\n land the `RecordDispatch` `*sql.Tx` drop (see §5) in the same span. Flip the agent's\n `Transport.Store` to a gRPC client once `STATE_STORE_URL` is injected.\n4. **Delete NATS `storerpc` only after the agent is fully on gRPC** — delete `internal/storerpc`\n (Server, Client, subjects `archie.store.update|transition|insert_event`), the\n `natsrpc.RegisterAll` registration, and the now-unused `store.WorkflowStore` NATS path in\n `internal/infrastructure/agenttransport/nats`, in the same commit.\n5. **`[services.state].target` stays empty (inproc)** through `.4.2`; the `archie-state-store`\n `cmd/` binary is **`.4.3`** and hosts the same `StateStore` gRPC service.\n6. **Swap daemon-internal contracts one at a time** (only once the daemon points at\n `archie-state-store`): `Capture → Mapping → Binding → TaskStore`. Each swap is independent —\n add the remote-client composition path, run the per-contract conformance suite, flip\n `[services.state].target`, delete the local use. A consumer mixes in-process and remote\n contracts for a contract-code (no dual-live *window*; each contract is either local or remote\n at a time).\n7. **Final flip / deletion of the in-process serving path:** when the daemon no longer owns the\n store (i.e. `TaskStore` has moved), delete the in-process `openProductionTaskStore` /\n `wireWebStoreSurfaces` path in the daemon in a **same-commit** deletion — no dual-store\n ownership. The single SQLite file remains owned by `archie-state-store`.\n8. **Never** split `archie.db` or add a second store. The gateway keeps its own already-separate\n session SQLite; that is unchanged and out of state-store scope.\n\n---\n\n## 13. Decisions recorded (for the review trail)\n\n- **`internal/store` is a transitional compatibility location** (target\n `internal/infrastructure/store`), not the final infrastructure boundary. The producer-owned\n daemon/webui store surfaces live there now but may relocate with the implementation.\n- **In-process listener + token lifecycle:** the daemon binds the in-process State Store\n listener to the **host-gateway bridge address** when agent containers consume it (the Phase 2\n default) or loopback-only when the daemon is the sole consumer — one listener, one address\n rule, never both (rev. 2c). It selects the bridge address by reusing the embedded-NATS\n host-gateway resolution and injects `STATE_STORE_URL`/`STATE_STORE_TOKEN` per task via\n `containerEnv`. Token is per-incumbence (task + container grace period) and validated by a\n gRPC interceptor → `codes.Unauthenticated` on missing/unknown/expired. Non-loopback targets\n fail closed without TLS or a token.\n- **§12 step 2 (domain type + interface relocation) is done** — `internal/domain/workflow`\n imports no `internal/store` symbol and `store.WorkflowStore` is gone from `TaskContext`. The\n broader Phase 2 acceptance criterion (State Store as its own gRPC service) is still open;\n `.4.2` picks up from here.\n- **Ownership split:** domain-facing workflow contract is **consumer-owned** in the domain\n (`workflow.Store` + `workflow.Task`/`Status`/`Source`, per dependency rules #2 and #7);\n daemon/webui store surfaces stay **producer-owned** in `internal/store`. `store.WorkflowStore`\n is superseded by `workflow.Store`.\n- **One `StateStore` gRPC service** (49 contract RPCs, grouped by contract) + narrow Go consumer facades\n (≤8) on `staterpc.Client` — mirrors the single-`ChatService` precedent.\n- **Domain type relocation** (`Task`/`Status`/`Source` → `internal/domain/workflow`) is a\n Phase 2 prerequisite, pulled forward from migration-decisions §4 (minimal bound,\n `Task → WorkflowExecution`); the rest of §4 stays a separate migration.\n- **`RecordDispatch` drops `*sql.Tx`** (wire-blocker; production passes `nil`) — a\n required interface migration landing before/with `.4.2`.\n- **`Close()` not on the wire**; remote client `Close()` is a no-op.\n- **Structured gRPC error codes** with client rehydration to store sentinels (preserves\n `errors.Is`), a `found=false` + `OK` convention for read not-found, and **sanitised public\n error details** (full detail logged server-side only).\n- **Agent handoff:** `agentworker.Settings` gains `StateStoreTarget`/`StateStoreToken`/\n `StoreTimeout`, injected as `STATE_STORE_URL`/`STATE_STORE_TOKEN` env (mirrors the NATS\n handoff); `Transport.Store` returns a long-lived gRPC `workflow.Store` client.\n- **Transport security + listener topology (rev. 2c):** one in-process listener, bound to the\n host-gateway bridge address when agent containers consume it (the Phase 2 default) or\n loopback-only for a daemon-local-only consumer. `insecure` only on loopback; mandatory\n per-task bearer-token auth on the bridge; TLS/mTLS is an operator choice for any\n non-loopback/k8s topology.\n- **Mode = presence-based `[services.state].target`**, default **local** (not yet extracted);\n add `[services.state]` to `config.example.toml`.\n- **Explicit `var _` assertions on both adapters**; per-contract bufconn conformance.\n- **Migration order:** domain type/interface relocation → `workflow.Store` (agent) → stand up\n `archie-state-store` → `Capture` → `Mapping` → `Binding` → `TaskStore`; NATS `storerpc`\n deleted once the agent is on gRPC; in-process store path deleted in a same-commit flip.\n- **Orphans `ClearTerminalTasks`/`IncrementRetryCount`** retained but not built upon.\n" }, { "url": "/dev/prds/status-health-surface/", diff --git a/docs/prds/eda-playbook-engine.md b/docs/prds/eda-playbook-engine.md index 3ebb8974b..47f6fb129 100644 --- a/docs/prds/eda-playbook-engine.md +++ b/docs/prds/eda-playbook-engine.md @@ -374,14 +374,19 @@ by `internal/store` -- the same shape as `BindingDispatcher` store's schema application (`internal/store/store.go`). No change to `internal/eventbus`; no per-message dedup state on the client. -**Lifetime: no time-based expiry.** Rows live as long as their playbook -exists; when a playbook is removed from the configured directories, the -coordinator deletes its rows -- mirroring binding-dispatches-rows-deleted- -with-binding (`bindings.go`, same transaction as the delete). The -at-most-once guarantee must not silently decay, which is exactly the -property that disqualifies `DedupWindow` above; table growth is bounded -by distinct (playbook, event, action) pairs over the playbook's lifetime -and is accepted, the same tradeoff `binding_dispatches` already makes. +**Lifetime: no time-based expiry, and no automatic reclamation.** Rows are +retained until an explicit `DeletePlaybookDispatches` for their playbook. +Removing a playbook file does not delete its rows: playbooks are read only +at boot, so the only place to notice a removal is a boot-time reconcile, +and a boot that finds the directory missing or unmounted loads as "no +playbooks" and would reclaim the whole ledger, letting redelivered events +fire their side effects again. A binding's rows go with an explicit +operator delete (`bindings.go`, same transaction); a file disappearing is +not an equally deliberate act. The at-most-once guarantee must not +silently decay, which is exactly the property that disqualifies +`DedupWindow` above; table growth is bounded by distinct (playbook, +event, action) pairs and is accepted, the same tradeoff +`binding_dispatches` already makes. **Redelivery behavior: skip the recorded action and continue the run.** A detected duplicate is not re-invoked, is not reported to the caller as diff --git a/docs/prds/state-store-contract.md b/docs/prds/state-store-contract.md index 959a72ee9..ff78e1abc 100644 --- a/docs/prds/state-store-contract.md +++ b/docs/prds/state-store-contract.md @@ -305,8 +305,9 @@ row is the structural `(playbook_id, playbook_version, event_id, action_id)` tuple, written **before** the action's side effect fires (the reverse of `binding_dispatches`), so a duplicate returns `store.ErrAlreadyDispatched` and the coordinator skips the invoke rather than repeating a non-revocable side -effect. `DeletePlaybookDispatches` is the reclamation primitive for a playbook -removed from the configured directory. The Go facade is +effect. `DeletePlaybookDispatches` is the explicit reclamation primitive; nothing +calls it automatically, because a playbook file's absence at boot is not a +deliberate removal (`eda-playbook-engine.md`, "Lifetime"). The Go facade is `storecontract.PlaybookDispatcher` (2 methods), carried by `*staterpc.Client` alongside the rest. diff --git a/internal/domain/storecontract/storecontract.go b/internal/domain/storecontract/storecontract.go index 5f53ab7ab..ad2d24224 100644 --- a/internal/domain/storecontract/storecontract.go +++ b/internal/domain/storecontract/storecontract.go @@ -227,10 +227,10 @@ type BindingTaskCreator interface { // BindingDispatcher.RecordDispatch -- so a duplicate returns // ErrAlreadyDispatched and the caller skips the invoke rather than repeating // a non-revocable side effect. DeletePlaybookDispatches deletes one -// playbook's ledger rows and is the reclamation primitive for when a playbook -// is removed from the configured directory -- a removal path that does not -// exist yet and is tracked by archie-core-t2db.30. There is deliberately no -// *sql.Tx: any transaction stays server-side inside internal/store. +// playbook's ledger rows on explicit request only: a playbook file missing at +// boot may be an unmounted directory, and reclaiming on that would let +// redelivered events fire again. There is deliberately no *sql.Tx: any +// transaction stays server-side inside internal/store. type PlaybookDispatcher interface { RecordPlaybookDispatch(ctx context.Context, playbookID, playbookVersion, eventID, actionID string) error DeletePlaybookDispatches(ctx context.Context, playbookID string) error