Plan item 3 of docs/prds/execution-tree-state-machine.md (archie-core-1220), on top of the transition tables and ErrIllegalTransition enforcement (1214/1215, archie-core-qdbo.5): - Migration 0017 adds step_executions -- id, org_id, workspace_id, execution_id, attempt, parent_id, depth, kind, name, called_execution_id, status, detail, tokens_used, started_at, finished_at, indexed on (execution_id, attempt, id). Ownership fields are stamped from the execution's row; depth is derived from the parent, never supplied by a caller. Existing rows gain no backfilled steps. - task.Store (the workflow engine's mid-run facade) gains StartStep/FinishStep (5 methods, within the 8 cap), with storecontract.StepRecorder fronting them on the server; deadlineStore wraps both with its per-call timeout. - The store enforces each write as one transaction: the guarded row write under the shared step transition table plus the transition's own stage_start/stage_finish event row. Staleness is decided before legality, the way guardTransition decides it, and the store returns the persisted event for the caller's post-commit publish -- the EmitDurable convention (the daemon's event sink skips already-persisted rows by assigned ID), which is why workflow.Run stops emitting stage events beside the store and no longer discards step-write errors: a failed start or finish parks the execution, and the stage does not run unrecorded. An interrupted stage records interrupted rather than failed; a recording failure on that shutdown path is logged and left to RecoverStale, which owns the interrupted-execution sweep in plan item 6. - The run credential of docs/prds/orgs-and-access.md authorises StartStep and FinishStep on its own execution only: both requests carry the execution ID the grant check verifies (FinishStep carries it because the interceptor never reads the database), and FinishStep verifies handler-side that the step really is that execution's. authorizesTaskScopedCall's sanctioned widenings are now held as a lookup table so the next one stops growing a switch. - The PRD's wire contract is StartStep(execution, attempt, parent, kind, name) -> step id; the responses also carry the persisted event (needed for the post-commit publish) and FinishStep's request carries execution_id (needed for the grant check) -- both noted here as deliberate additions to the PRD's shorthand. called_execution_id exists in the table but not yet on the wire: its named consumer is the workflow-call recording of plan item 4 (#1216). Conformance drives both adapters; grant tests pin the own-execution scoping; the wire sentinel set gains ErrInvalidStep (InvalidArgument, "step execution invalid") for a step kind no consumer names.
749 lines
57 KiB
Markdown
749 lines
57 KiB
Markdown
# State Store — contract boundary & transport (ratification)
|
|
|
|
**Status:** Approved
|
|
**Revision:** 2e
|
|
**Date:** 2026-09-06
|
|
|
|
This is the pre-implementation design document that Phase 2 subtasks `.4.2` (generalise
|
|
`storerpc` as the State Store gRPC contract) and `.4.3` (stand up `archie-state-store`)
|
|
implement against.
|
|
**Beads milestone:** archie-core-8cda.4.1
|
|
**Parent:** `docs/prds/service-decomposition.md` (open-question Q4 **RESOLVED**)
|
|
|
|
This document fixes the **contract boundary** and **transport** for the State Store service.
|
|
It does not redesign Archie's domain model, does not invent a second store, and does **not**
|
|
split `archie.db`. It records method ownership, DTO/wire representation, error semantics,
|
|
limits, mode/config shape, local-adapter compatibility, and cutover/deletion rules so that
|
|
`.4.2` and `.4.3` have exactly one authoritative surface to build against.
|
|
|
|
> **Storage superseded.** The store behind this contract is PostgreSQL:
|
|
> `internal/infrastructure/postgres` implements the producer-owned interfaces in
|
|
> `internal/domain/storecontract` and `workflow.Store`, over the database
|
|
> `database_url` names. `internal/store`, the SQLite file and the local adapter
|
|
> are gone; where this document says `archie.db`, `*store.Store` or
|
|
> `internal/store`, read the State Store's PostgreSQL database and
|
|
> `internal/infrastructure/postgres`. The contract, ownership split, error
|
|
> semantics and cutover rules stand.
|
|
|
|
> **Current state — read before `.4.2` starts.** The §12 step 2 relocation (`archie-core-8cda.
|
|
4.8`) has landed: `Task`/`Status*`/`Source*` and the 3-method `workflow.Store` now live in
|
|
> `internal/domain/workflow`, the six production workflow files import no `internal/store`
|
|
> symbol, and `storerpc`/the NATS transport assert against `workflow.Store` instead of
|
|
> `store.WorkflowStore`. The broader Phase 2 acceptance criterion (State Store as its own
|
|
> service behind gRPC) is still open — that is what `.4.2`/`.4.3` build next.
|
|
|
|
> **Relationship to migration-decisions §4 — not replaced.** `docs/architecture/
|
|
migration-decisions.md` §4 (`store.Task → WorkflowExecution`, `mutable Stage →
|
|
StepExecution history`, `RetryCount/Attempt → Attempt records`, versioned Workflow defs)
|
|
> **remains the authoritative, open document for the long-term Workflow migration.** This PRD
|
|
> does **not** replace it. It only pulls the minimal `Task`/`Status`/`Source` type + interface
|
|
> relocation forward as a Phase 2 prerequisite (because the acceptance criterion requires it);
|
|
> the rest of §4 stays its own migration and stays open.
|
|
|
|
---
|
|
|
|
## 1. Decision (the invariant everything follows)
|
|
|
|
Archie is composed such that a single **`archie-state-store`** service owns the one existing
|
|
SQLite file (`archie.db`) and exposes it over **gRPC behind narrow typed contracts**. There is
|
|
exactly one store, one file, one service. No database-per-service, no distributed
|
|
transactions, no second store abstraction.
|
|
|
|
Two adapter families satisfy the contract interfaces:
|
|
|
|
- **Local adapter** — `*store.Store` (the in-process SQLite implementation).
|
|
- **Remote adapter** — `*staterpc.Client` (wraps the generated gRPC client).
|
|
|
|
Each adapter carries an explicit compile-time assertion against the contract interfaces it
|
|
serves. Composition picks local vs. remote by the presence of `[services.state].target`
|
|
(empty = local; set = dial the gRPC service). Default is **local** (the State Store is not yet
|
|
extracted), which is the PRD §3 posture; it differs from the gateway's remote-default only
|
|
because the gateway is already a separate deployment and the State Store is not.
|
|
|
|
> **Contract-ownership fork, decided (rev. 2c — resolves the domain contradiction).**
|
|
> Ownership splits by the consumer's _layer_, per the dependency contract's rule #2
|
|
> ("a domain defines the smallest interfaces required to perform its work") and rule #7
|
|
> ("database rows … MUST NOT leak into domain models"):
|
|
>
|
|
> - The **workflow domain** (`internal/domain/workflow`) is the only domain-layer consumer of
|
|
> store state. It defines its own **consumer-owned** narrow `Store` interface
|
|
> (Update/Transition/InsertEvent) and its own **domain-owned** `Task` type plus
|
|
> `Status`/`Source` vocabulary. It imports **zero** `internal/store` symbols — satisfying the
|
|
> Phase 2 acceptance criterion. This is the minimal bound of migration-decisions §4
|
|
> (`store.Task → WorkflowExecution`), pulled forward **because** the acceptance criterion
|
|
> requires it.
|
|
> - The **daemon / webui / intake** store surfaces (`TaskStore`, `CaptureStore`,
|
|
> `MappingStore`, `BindingStore`, `BindingDispatcher`, `BindingTaskCreator`) stay
|
|
> **producer-owned** in `internal/store`, because those consumers are application and
|
|
> infrastructure layers, not domains — rule #2 does not apply to them.
|
|
|
|
So the State Store contract is **not** uniformly producer-owned: the domain-facing workflow
|
|
contract is consumer-owned (in the domain), while the daemon/webui store surfaces remain in
|
|
`internal/store`. `internal/store` imports `internal/domain/workflow` for the `Task`/`Status`/
|
|
`Source` types it persists (persistence → domain, the correct direction), and its `Store`
|
|
implementation satisfies `workflow.Store`. This is the single decision that makes the whole
|
|
Phase 2 acceptance criterion and the architecture's rules simultaneously satisfiable.
|
|
|
|
> **`internal/store` is a TRANSITIONAL compatibility location (rev. 2c).** The repository's target
|
|
> architecture (`docs/architecture/dependencies-and-contracts.md`, `organisation.md`) places
|
|
> persistence implementations under `internal/infrastructure/<capability>`. `internal/store`
|
|
> holding the task/capture/mapping/binding implementation and ALSO being the home of the
|
|
> producer-owned daemon/webui interfaces is acceptable **only as a transitional location**. The
|
|
> long-term home for the store implementation is `internal/infrastructure/store`; the
|
|
> producer-owned interfaces may move to the same infrastructure package (or an approved
|
|
> contract location) when the store implementation relocates. Agents must not treat
|
|
> `internal/store` as the final infrastructure boundary.
|
|
>
|
|
> **REV. 2d (archie-core-8cda.5.6): the producer-owned interfaces HAVE moved to an approved
|
|
> contract location** — `internal/domain/storecontract` (package `storecontract`), ahead of the
|
|
> store implementation's own relocation. The UI Service deletion gate required the archie-ui
|
|
> process to hold the contracts without linking `internal/store`'s SQL implementation, and
|
|
> `internal/contracts/` itself is ruled out: that tree is the `buf generate` output and
|
|
> `tools/proto.sh check` deletes hand-written files there. `internal/store` keeps type aliases
|
|
> (`type TaskStore = storecontract.TaskStore`, the sentinel `var`s, the value types) so
|
|
> daemon-side callers are unaffected; the sentinel message strings are unchanged and remain
|
|
> the wire contract (§4).
|
|
|
|
---
|
|
|
|
## 2. Contract boundary
|
|
|
|
**Ownership split** (see the fork in §1): the **workflow-domain consumer contract** is
|
|
`workflow.Store` (consumer-owned, in `internal/domain/workflow`); the **daemon/webui/intake
|
|
store surfaces** are the producer-owned interfaces in `internal/store/interface.go`.
|
|
`store.WorkflowStore` is **superseded** — it is replaced by the domain-owned `workflow.Store`,
|
|
and `internal/store`'s `Store` implementation satisfies `workflow.Store` directly. `TaskStore`
|
|
remains the only composite; every other interface is flat. Narrow slices are split along
|
|
consumer-capability lines so no consumer acquires a surface it does not need
|
|
(`interfacebloat` cap = 8).
|
|
|
|
| Interface (owner) | Embeds | Methods | Local impl | Remote impl |
|
|
| ---------------------------------- | --------------------------------------------------------------------------- | ---------- | ---------- | ------------------ |
|
|
| `TaskStore` (store) | `TaskLifecycle`, `TaskEvents`, `TaskQueries`, `TaskArchiver`, `TaskRetryer` | 24 (union) | `*Store` | `*staterpc.Client` |
|
|
| `TaskLifecycle` (store) | — | 8 | `*Store` | `*staterpc.Client` |
|
|
| `TaskEvents` (store) | — | 7 | `*Store` | `*staterpc.Client` |
|
|
| `TaskQueries` (store) | — | 7 | `*Store` | `*staterpc.Client` |
|
|
| `TaskArchiver` (store) | — | 1 | `*Store` | `*staterpc.Client` |
|
|
| `TaskRetryer` (store) | — | 1 | `*Store` | `*staterpc.Client` |
|
|
| `workflow.Store` (workflow domain) | — | 3 | `*Store` | `*staterpc.Client` |
|
|
| `CaptureStore` (store) | — | 2 | `*Store` | `*staterpc.Client` |
|
|
| `MappingStore` (store) | — | 5 | `*Store` | `*staterpc.Client` |
|
|
| `BindingStore` (store) | — | 6 | `*Store` | `*staterpc.Client` |
|
|
| `BindingDispatcher` (store) | — | 3 | `*Store` | `*staterpc.Client` |
|
|
| `BindingTaskCreator` (store) | — | 1 | `*Store` | `*staterpc.Client` |
|
|
| `PlaybookDispatcher` (store) | — | 2 | `*Store` | `*staterpc.Client` |
|
|
|
|
**Key property of the remote client:** `*staterpc.Client` is a _single_ adapter that wraps one
|
|
`StateStoreClient` and is asserted against **whichever narrow Go interfaces its caller needs**
|
|
(exactly as `gatewayrpc.Client` asserts both `gateway.ChatContract` and
|
|
`gateway.SessionStore`). The Go contracts stay narrow (≤8, the `interfacebloat` cap); the wire
|
|
is one service. `workflow.Store` introduces **no new RPCs** — it is a narrow consumer face over
|
|
the same `Update`/`Transition`/`InsertEvent` RPCs.
|
|
|
|
---
|
|
|
|
## 3. Method ownership
|
|
|
|
`*Store` remains the single implementation of the daemon/webui store surfaces and of
|
|
`workflow.Store`. Ownership below is about which **process** is the primary consumer of each
|
|
method and therefore which methods are genuinely cross-process on the wire vs.
|
|
daemon-internal.
|
|
|
|
Process legend: **DAEMON** = `archied` (incl. in-process webui + gateway runtime +
|
|
`storerpc.Server`); **AGENT** = `archie-agent` worker (no DB connection); **GATEWAY** = store
|
|
access through daemon-side narrow adapter interfaces.
|
|
|
|
| Contract | Primary consumers (process) | Remote on the wire? |
|
|
| ------------------------ | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
|
|
| `TaskLifecycle` (8) | DAEMON (dispatch, claim, action) + **AGENT** (`Transition`, `Update` via storerpc) | `Transition`/`Update` = **YES** (agent); rest DAEMON |
|
|
| `TaskArchiver` (1) | DAEMON (`taskactions` archive) | not yet |
|
|
| `TaskRetryer` (1) | DAEMON (`taskactions` retry) | not yet |
|
|
| `TaskQueries` (7) | DAEMON + GATEWAY (status/list adapters) | not yet (`OpenPRs`/`Tasks`/`StatusCounts` cross to gateway adapters in-process) |
|
|
| `TaskEvents` (7) | DAEMON + **AGENT** (`InsertEvent` via storerpc) | `InsertEvent` = **YES** (agent); rest DAEMON |
|
|
| `workflow.Store` (3) | **AGENT** only | **YES** (the sole genuinely remote consumer) |
|
|
| `CaptureStore` (2) | DAEMON (webhook intake + mapping editor) | not yet |
|
|
| `MappingStore` (5) | DAEMON (dashboard editor + dispatch loop) | not yet |
|
|
| `BindingStore` (6) | DAEMON (dashboard editor + dispatch loop) | not yet |
|
|
| `BindingDispatcher` (3) | DAEMON (dispatch loop) | not yet |
|
|
| `BindingTaskCreator` (1) | DAEMON (dispatch loop) | not yet |
|
|
| `PlaybookDispatcher` (2) | DAEMON (EDA playbook dispatch loop) | **YES** (side-effecting-action idempotency ledger over gRPC) |
|
|
|
|
**Consequence for migration order:** the **agent's `workflow.Store` is already cross-process
|
|
already** (via the NATS `storerpc` surrogate), so it is the first contract to move to gRPC. The
|
|
daemon-internal surfaces (`Capture`/`Mapping`/`Binding`) stay in-process until the daemon
|
|
itself points at `archie-state-store`. `TaskStore` (the large composite) moves last.
|
|
|
|
**Two orphan methods** — `TaskQueries.ClearTerminalTasks` and
|
|
`TaskQueries.IncrementRetryCount` have **no production consumer** (only `*Store` tests). They
|
|
remain part of the Go `TaskQueries`/`TaskStore` surface, and therefore part of the wire
|
|
contract, but are flagged for deletion: they must **not** gain dedicated consumers or wire
|
|
methods in this work. Do not add the rest of the codebase to them.
|
|
|
|
---
|
|
|
|
## 4. Domain-facing types & consumer-owned contract — resolves the contradiction
|
|
|
|
The Phase 2 acceptance criterion ("`internal/domain/workflow` … no longer import
|
|
`internal/store` directly — they call the State Store contract") **cannot be met while**
|
|
`store.Task`, `store.Status*`, `store.Source*`, and `store.WorkflowStore` live in
|
|
`internal/store`. This section fixes where they move.
|
|
|
|
**The relocation (a prerequisite that must land before or with `.4.2`):**
|
|
|
|
| Current symbol (`internal/store`) | Moves to | Notes |
|
|
| ------------------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
|
|
| `Task` type | `internal/domain/workflow.Task` | The task-execution record the workflow domain operates on. `internal/store` persists this type (persistence → domain). |
|
|
| `Status` + `Status*` constants | `internal/domain/workflow` | e.g. `StatusClosedWontDo`, `StatusMerged`, `StatusPROpen`, `StatusWaitingHuman`, `StatusRunning` — task-lifecycle vocabulary. |
|
|
| `Source` + `Source*` constants | `internal/domain/workflow` | e.g. `SourceChat` — provenance vocabulary, follows `Task`. |
|
|
| `WorkflowStore` interface (3 methods) | `internal/domain/workflow.Store` | **Consumer-owned** (dependency rule #2). Supersedes `store.WorkflowStore`. |
|
|
|
|
**Resulting dependency direction (correct per the architecture):**
|
|
|
|
- `internal/domain/workflow` imports **no** `internal/store` symbols → the acceptance
|
|
criterion is satisfied. It consumes `store.Task`'s replacement as `workflow.Task`, uses
|
|
`workflow.Status*`/`workflow.Source*`, and depends on its own `workflow.Store` interface.
|
|
- `internal/store` imports `internal/domain/workflow` for the `Task`/`Status`/`Source` types it
|
|
reads and writes (infrastructure → domain). **NOTE: `internal/store` is a transitional
|
|
compatibility location only** — its long-term home is `internal/infrastructure/store` (see
|
|
the callout in §1). Do not treat `internal/store` as the final infrastructure boundary.
|
|
- Every store test that constructs `store.Task{…}` is updated to the relocated type as part of
|
|
this prerequisite. This is a mechanical rename across ~6 workflow production files
|
|
(`workflow.go`, `steps.go`, `triage.go`, `review.go`, `feasibility.go`, `implement.go`) and
|
|
their tests plus the store tests. `TestRecordDispatchViaExplicitTx` is removed rather than
|
|
updated — its non-nil `*sql.Tx` path has no production consumer (see §5's `RecordDispatch`
|
|
blocker).
|
|
|
|
**What this does NOT do:** it does not implement the full §4 Workflow migration
|
|
(`mutable Stage → StepExecution history`, `RetryCount/Attempt → Attempt records`,
|
|
`workflow.Registry → versioned Workflow definitions`, etc.). It lands only the minimal
|
|
task-type + `Status`/`Source` + consumer-owned-interface relocation that the Phase 2
|
|
domain-decoupling criterion requires. The residue of §4 remains its own migration.
|
|
|
|
**Forward-compatibility:** because the wire DTOs are derived from the domain `Task`/`Status`
|
|
(§5), when the full §4 migration later renames `workflow.Task` (or refines it into
|
|
`WorkflowExecution`), only `values.go` and the proto field mapping change — the RPC surface and
|
|
consumer interfaces are unaffected. The wire contract is deliberately type-agnostic at this
|
|
level.
|
|
|
|
---
|
|
|
|
## 5. DTO / wire representation
|
|
|
|
### Proto layout
|
|
|
|
- **Service:** `proto/state/v1/state.proto`, `package state.v1`, **one** `StateStore` service
|
|
(mirrors the gateway's single `ChatService`; proto bypasses the Go `interfacebloat` cap, as
|
|
`ChatContract` does via `//nolint:interfacebloat`).
|
|
- **Codegen:** the existing buf v2 pipeline. `proto/buf.yaml` (module, lint STANDARD, breaking
|
|
FILE against `main`), `proto/buf.gen.yaml` **managed mode** with
|
|
`go_package_prefix → github.com/samcharles93/archie-core/internal/contracts`, plugins
|
|
`protoc-gen-go` + `protoc-gen-go-grpc`, `out: ../internal/contracts`,
|
|
`opt: paths=source_relative`. **No `go_package` in source** — managed mode owns it. Generated
|
|
Go lands in `internal/contracts/state/v1/` (package `statev1`), committed (`proto:check`
|
|
fails on uncommitted/unmodified generated files).
|
|
- **Adapter package:** `internal/infrastructure/staterpc/` with `server.go` (registers the
|
|
generated server over the store), `client.go` (wraps the generated client into `store.*` and
|
|
`workflow.Store` facades), `values.go` (proto ⇄ domain mappers), and `conformance_test.go`
|
|
(bufconn local-vs-grpc dual-adapter suite), mirroring `internal/infrastructure/gatewayrpc/`.
|
|
- **Binary:** `cmd/archie-state-store/main.go`, mirroring `cmd/archie-gateway/main.go` and the
|
|
serve pattern in `internal/app/archied/gateway.go` (`serveGateway`).
|
|
|
|
### The `StateStore` RPC set (union of the contracts, grouped)
|
|
|
|
Methods are named after the store methods so the mapping is unambiguous. `workflow.Store` is a
|
|
_view_, not new RPCs — `Update`/`Transition`/`InsertEvent` are shared with
|
|
`TaskLifecycle`/`TaskEvents` and appear once.
|
|
|
|
| Group | RPCs |
|
|
| ------------------ | -------------------------------------------------------------------------------------------------------------------------- |
|
|
| Lifecycle | `EnqueueIssue`, `EnqueueChatTask`, `ClaimNext`, `ClaimByIssue`, `Transition`, `Update`, `Requeue`, `RecoverStale` |
|
|
| Archive / Retry | `ArchiveTask`, `RetryTask` |
|
|
| Queries | `TaskByIssue`, `OpenTaskByPR`, `TaskByID`, `OpenPRs`, `ClearTerminalTasks`, `Tasks`, `StatusCounts`, `IncrementRetryCount` |
|
|
| Events | `InsertEvent`, `EventsSince`, `TaskEvents`, `WorkflowStats`, `StageStats`, `TokensByDay` |
|
|
| Capture | `InsertCapture`, `ListCaptures` |
|
|
| Mapping | `InsertMapping`, `GetMapping`, `ListMappings`, `UpdateMapping`, `DeleteMapping` |
|
|
| Binding | `InsertBinding`, `GetBinding`, `ListBindings`, `UpdateBinding`, `DeleteBinding`, `ApproveBinding` |
|
|
| Dispatch | `ArmedBindingsForSource`, `RecordDispatch`, `ListUndispatchedCaptures` |
|
|
| Playbook | `RecordPlaybookDispatch`, `DeletePlaybookDispatches` |
|
|
| WorkflowCaller | `EnqueueCallTask`, `WorkflowCallStatus` |
|
|
| Step executions | `StartStep`, `FinishStep` |
|
|
| BindingTaskCreator | `EnqueueBindingTask` |
|
|
| Config snapshot | `PutConfigSnapshot`, `GetConfigSnapshot` |
|
|
| Apply status | `PutApplyStatus`, `ListApplyStatus` |
|
|
| Channel status | `PutChannelStatus`, `ListChannelStatus` |
|
|
| Task log | `ReadTaskLog`, `StreamTaskLogContent` |
|
|
|
|
That is **55 unique contract RPCs** across one service (the `TaskEvents.Close` method is dropped).
|
|
`Close()` is **excluded** from the wire (it is server lifecycle, not a client call) — see §11.
|
|
|
|
The channel-status pair follows the config snapshot's split for the same reason: the writer is the process that HOSTS the channels -- the Messaging Service, which alone observes whether a channel is starting, running or failed -- and the reader is the UI process. Both RPCs are administrative, so a task-scoped grant reaches neither. A channel's state is runtime state rather than a settings document, which is why it is not a `ControlPlaneService` resource kind: a failed channel is not a document anybody edits. Asking for a reload travels on its own surface rather than through this one; see `docs/architecture/migration-decisions.md`, "Channel state to the dashboard".
|
|
|
|
The config-snapshot pair was added by `archie-core-ymut` (see
|
|
`docs/architecture/migration-decisions.md`, "Dashboard configuration page"). It is the one
|
|
surface whose writer is the daemon and whose reader is the UI process, and both RPCs are
|
|
administrative: `authorizesTaskScopedCall` is deny-by-default, so a task-scoped grant reaches
|
|
neither. `ConfigSnapshot.document` is opaque to this contract — the store neither parses nor
|
|
validates it, and `schema` names the projection so a reader can refuse a shape it does not
|
|
understand.
|
|
|
|
The apply-status pair was added by `archie-core-pskb` (see
|
|
`docs/prds/control-plane-apply-status.md`). It is administrative for the same reason as the
|
|
config-snapshot pair, and deny-by-default in `authorizesTaskScopedCall` covers it without a
|
|
new rule. Unlike that pair, the read is a list: `ListApplyStatus` returns every process's
|
|
record and an empty slice means nothing has reported yet, so the `found=false` convention for
|
|
absent single rows does not apply. The rows are keyed on `(process, kind)` and replaced in
|
|
place, because each process re-stamps its record on an interval and no reader wants a history
|
|
of re-stamps.
|
|
|
|
The task-log pair was added by `archie-core-iaqx` (see `docs/prds/ui-service-boundary.md`'s
|
|
route inventory). It is the one surface whose payload is not a row: the store service reads
|
|
them out of the state directory it owns with `internal/logging`'s own reader, so the wire
|
|
carries `logging.Entry`/`logging.Query` values and neither side restates the JSONL format.
|
|
`ReadTaskLog` returns a decoded page and `StreamTaskLogContent` returns the raw bytes
|
|
server-streamed, because a log file rotates at 20 MB — past gRPC's 4 MiB unary cap — and the
|
|
same reasoning that made `StreamCaptures` supersede `ListCaptures` applies. `found=false` with
|
|
no error means the attempt has no log; a service with no reader answers `Unavailable` instead,
|
|
and the two answers are load-bearing for the dashboard: only the second is a deployment
|
|
condition, and the panel that collapsed them told operators task logging was switched off when
|
|
it was not. The Go facade is `storecontract.TaskLogStore` (2 methods), carried by
|
|
`*staterpc.Client` alongside the rest.
|
|
|
|
The step-execution pair was added by the execution tree's plan item 3 (see
|
|
`docs/prds/execution-tree-state-machine.md`), the surface a task-scoped grant reaches for
|
|
its own execution beyond the three `task.Store` RPCs and the workflow-caller pair: the
|
|
container records its stages and its own agent calls with the run credential. Both requests
|
|
carry the caller's execution ID, which the grant check verifies; `FinishStep` additionally
|
|
verifies, handler-side, that the step really is that execution's. The store writes each
|
|
transition's `stage_start`/`stage_finish` event row in the same transaction and returns it
|
|
for the caller's post-commit bus publish. `called_execution_id` is stored for the `call`
|
|
kind but not yet carried on the wire: the named consumer is the workflow-call recording
|
|
that will set it.
|
|
|
|
The workflow-caller pair was added by `archie-core-t2db.41` (see everything else from the caller's
|
|
row and refuses a caller that is not running or a call past the depth limit
|
|
(`ErrCallCallerNotRunning` / `ErrCallDepthExceeded`), and `WorkflowCallStatus`
|
|
re-checks the parent-child relation against the row before answering
|
|
(`ErrCallNotYours`), so the interceptor itself never reads the database. The
|
|
facade is `workflow/task.Caller` (2 methods), carried by `*staterpc.Client`
|
|
and the PostgreSQL store alike.
|
|
|
|
The playbook pair was added by `archie-core-t2db.17`'s implementation (see
|
|
`docs/prds/eda-playbook-engine.md`, execution-time gap 2). It is administrative
|
|
for the same reason as the config-snapshot and apply-status pairs, and
|
|
deny-by-default in `authorizesTaskScopedCall` covers it without a new rule. The
|
|
row is the structural `(playbook_id, playbook_version, event_id, action_id)`
|
|
tuple, written **before** the action's side effect fires (the reverse of
|
|
`binding_dispatches`), so a duplicate returns `store.ErrAlreadyDispatched` and
|
|
the coordinator skips the invoke rather than repeating a non-revocable side
|
|
effect. `DeletePlaybookDispatches` is the explicit reclamation primitive; nothing
|
|
calls it automatically, because a playbook file's absence at boot is not a
|
|
deliberate removal (`eda-playbook-engine.md`, "Lifetime"). The Go facade is
|
|
`storecontract.PlaybookDispatcher` (2 methods), carried by `*staterpc.Client`
|
|
alongside the rest.
|
|
|
|
### Domain types and `values.go` mapping
|
|
|
|
`values.go` maps proto ⇄ these types. The task model for the **agent** path is the domain-owned
|
|
`workflow.Task` (§4); the daemon/webui store surfaces return the same `workflow.Task` (the store
|
|
persists the domain type).
|
|
|
|
| Domain type | Package | Notes for `values.go` |
|
|
| ------------------------------------- | -------------------------- | -------------------------------------------------------------------------------------------------------------- |
|
|
| `workflow.Task` | `internal/domain/workflow` | time fields stored as layout strings; convert to/from `google.protobuf.Timestamp` |
|
|
| `events.Event` | `internal/events` | `Timestamp` → `google.protobuf.Timestamp` |
|
|
| `store.CapturedEvent` | `internal/store` | headers/body/remote_addr; body capped 256 KiB |
|
|
| `mapping.Mapping` | `internal/domain/mapping` | JSON-path field bindings |
|
|
| `binding.Binding` | `internal/domain/binding` | secret is **already decrypted** before returning; keep decrypted form out of any durable/logged representation |
|
|
| `workflow.Status` / `workflow.Source` | `internal/domain/workflow` | enum-like; map to proto enum or string, keeping the names stable for compatibility |
|
|
| `store.WorkflowStat` | `internal/store` | aggregate row |
|
|
| `store.StageStat` | `internal/store` | aggregate row |
|
|
| `store.DayTokens` | `internal/store` | aggregate row |
|
|
|
|
### Wire-contract blocker fixed here: `RecordDispatch`
|
|
|
|
`store.BindingDispatcher.RecordDispatch(ctx, tx *sql.Tx, bindingID, bindingVersion int64,
|
|
captureID int64, taskID int64) error` takes a **`*sql.Tx`** that cannot cross a gRPC boundary.
|
|
|
|
**Evidence that the `*sql.Tx` is removable:** the dispatch loop calls `RecordDispatch(ctx, nil,
|
|
…)` (`internal/daemon/daemon.go`). The implementation falls back to the store's own
|
|
`*sql.DB` when `tx == nil` ("best-effort, not atomic"; the comment and the
|
|
`TestRecordDispatchViaExplicitTx` test are the only consumers of the non-nil `tx` path).
|
|
Production never opens the explicit transaction.
|
|
|
|
**Decision:** the contract drops `*sql.Tx` from `RecordDispatch` (the method becomes
|
|
`RecordDispatch(ctx, bindingID, bindingVersion, captureID, taskID) error`). The store service
|
|
owns any transaction boundary internally when atomicity is wanted; the at-most-once
|
|
guarantee is the `INSERT OR IGNORE` dedup row, which is already transactionless in production.
|
|
`TestRecordDispatchViaExplicitTx` is superseded by the store service's own internal-transaction
|
|
test. **This is a required interface migration that must land before or with `.4.2`** (the
|
|
current `interface.go` signature and test still use the transaction-bearing form).
|
|
|
|
---
|
|
|
|
## 6. Agent State Store client handoff
|
|
|
|
Moving the agent from NATS `storerpc` to gRPC `workflow.Store` requires the agent process to
|
|
receive a State Store endpoint. The agent already receives its NATS connection settings via
|
|
boot-time env/flags (see `cmd/archie-agent/main.go`: `-nats-url` / `NATS_URL`, `NATS_TOKEN`
|
|
env), and the daemon injects them into the task container. The State Store handoff follows the
|
|
same seam.
|
|
|
|
**Boot handoff contract (additive, not replacing the NATS settings):**
|
|
|
|
| Field | Where | Source | Notes |
|
|
| ------------------ | ---------------------- | ------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `StateStoreTarget` | `agentworker.Settings` | env `STATE_STORE_URL` (or `-state-store-url` flag), set by the daemon in the container env | `host:port` of the State Store gRPC server. Loopback/Docker-bridge address — the agent reaches it the same way it reaches the daemon's NATS. |
|
|
| `StateStoreToken` | `agentworker.Settings` | env `STATE_STORE_TOKEN` (or `-state-store-token` flag) | Per-task bearer token carried in gRPC metadata by a client interceptor (see §9). |
|
|
| `StoreTimeout` | `agentworker.Settings` | a bounded per-call timeout | Mirrors `natsrpc.Client.Timeout` / the existing `rpcTimeout`; bounds each State Store call when the caller `ctx` has no deadline. |
|
|
|
|
**Wiring change in the transport façade:**
|
|
|
|
- `internal/infrastructure/agenttransport/nats.Transport` — `Store(timeout) store.WorkflowStore`
|
|
returns `&storerpc.Client{Conn: t.conn, Timeout: timeout}`. It becomes
|
|
`Store(timeout) workflow.Store` and returns a _single long-lived_ `staterpc.Client` dialled to
|
|
`settings.StateStoreTarget` (with the token interceptor and per-call timeout).
|
|
- `internal/app/agentworker/worker.go` `Settings` gains the three fields above; `taskServiceTransport`
|
|
and `taskDependencies` change their `store` type from `store.WorkflowStore` to
|
|
`workflow.Store` (via `domain/workflow.TaskContext{Store: …}`).
|
|
- Reconnect/deadline: `grpc.NewClient` (or the existing dial path) provides transparent reconnect;
|
|
each RPC is bounded by the per-call context deadline (`StoreTimeout`), matching the
|
|
request/reply timeout the NATS path used. No streaming is involved, so no server-sent
|
|
reconnect semantics apply.
|
|
|
|
**Deletion rule for the NATS path:** the NATS `storerpc` (Server, Client, subjects
|
|
`archie.store.update|transition|insert_event`) is deleted **only after** the agent has fully
|
|
moved to gRPC and the container env carries `STATE_STORE_URL`. Until then both paths may coexist
|
|
(agent flips when the daemon starts injecting `STATE_STORE_URL`); there is no dual-live window
|
|
across a single agent process, which is either NATS-backed or gRPC-backed, never both.
|
|
|
|
### Operational specifics (in-process listener → address → per-task injection)
|
|
|
|
- **Who binds the in-process State Store listener:** the daemon (`archied`), at bootstrap, when
|
|
the State Store is served in-process/multiplexed (mirroring `serveGateway` in
|
|
`internal/app/archied/gateway.go`): a `grpc.NewServer()` + `RegisterServer` + graceful stop on
|
|
`ctx.Done()`.
|
|
- **Address selection (Docker-bridge reachable):** reuse the host-gateway resolution already used
|
|
for embedded NATS (`internal/container/network.go` `RequireHostGateway` /
|
|
`resolveHostGateway`). Per §9's single listener-topology rule, the listener binds the
|
|
**host-gateway bridge address** when agent containers consume it (the Phase 2 default), or
|
|
loopback-only when the daemon is the sole consumer. The chosen address is stored on the daemon
|
|
as `d.ConnectedStateStore.URL` (analogous to `d.ConnectedNATS.URL`).
|
|
- **Per-task injection:** `containerEnv(task)` (`internal/daemon/daemon.go`) appends
|
|
`STATE_STORE_URL=<d.ConnectedStateStore.URL>` and `STATE_STORE_TOKEN=<token>` alongside the
|
|
existing `NATS_URL`/`NATS_TOKEN`. The agent reads them via env (`-state-store-url` /
|
|
`STATE_STORE_URL`, `-state-store-token` / `STATE_STORE_TOKEN`), the same seam as `NATS_URL`. A
|
|
reloaded `[services.state]` must not point new containers at a store the daemon is not serving
|
|
— the same immutability rule as NATS.
|
|
|
|
---
|
|
|
|
## 7. Error semantics
|
|
|
|
### Current state (both boundaries lose error identity)
|
|
|
|
- **NATS `natsrpc.Envelope`** (`internal/natsrpc/natsrpc.go`) serialises any error to a single
|
|
`err.Error()` string and rehydrates a fresh `errors.New(string)`. The original type, `%w`
|
|
chain, and sentinel identity are lost → `errors.Is(err, store.ErrStaleTransition)` is
|
|
**false** across the boundary.
|
|
- **gRPC `gatewayrpc`** returns raw `err` from handlers; gRPC-Go converts to
|
|
`codes.Unknown` with only the message. The **only** explicit code in the repo is
|
|
`codes.Unavailable` for a missing session store (`server.go`). No status-code mapping for
|
|
store errors exists.
|
|
|
|
### Decision — structured mapping so `errors.Is` survives
|
|
|
|
The `StateStore` service must map the store's sentinel errors to canonical gRPC codes and
|
|
rehydrate them to the _same_ sentinels on the client, so consumer `errors.Is` checks keep
|
|
working — an improvement over the flattening it replaces, not a regression.
|
|
|
|
| Store sentinel / outcome | gRPC code | Client rehydrates to |
|
|
| ------------------------------------------------------------------------------------------------- | -------------------------------------------------------- | ---------------------------- |
|
|
| `store.ErrStaleTransition` | `FailedPrecondition` | `store.ErrStaleTransition` |
|
|
| `store.ErrBindingNotFound` | `NotFound` | `store.ErrBindingNotFound` |
|
|
| `store.ErrMappingNotFound` | `NotFound` | `store.ErrMappingNotFound` |
|
|
| `store.ErrBindingOverlap` | `FailedPrecondition` | `store.ErrBindingOverlap` |
|
|
| `store.ErrBindingTransition` | `FailedPrecondition` | `store.ErrBindingTransition` |
|
|
| `store.ErrAlreadyDispatched` | `AlreadyExists` | `store.ErrAlreadyDispatched` |
|
|
| not-found-as-`(nil,nil)` (`TaskByID`, `GetBinding`, `GetMapping`, `ClaimNext`, `ClaimByIssue`, …) | `NotFound` with a `found=false` field (**not** an error) | `(nil, nil)` |
|
|
| infra / wrapped error | `Internal` (**sanitised** message, see below) | wrapped `fmt.Errorf(...%w)` |
|
|
| capability absent | `Unavailable` | context/typed error |
|
|
|
|
The `found=false`-not-error convention matters: the repo deliberately returns `(nil, nil)` for
|
|
not-found in reads and a sentinel for not-found in writes. The wire must preserve **both**
|
|
conventions (a `GoogleRPCStatus` with `NotFound` is an _error_, which would wrongly surface to
|
|
callers that expect `(nil, nil)`). Implement `found` as an explicit boolean field on read
|
|
responses, and map `(nil,nil)` ⇄ `found=false` + `OK`.
|
|
|
|
### Error-detail sanitisation (rev. 2c)
|
|
|
|
Full internal error messages must **not** be placed in gRPC `details` for remote callers. SQL
|
|
paths, provider/forge details, secrets, and stack traces are sensitive internals. The rule:
|
|
|
|
- **Sentinel errors** — the client rehydrates the sentinel from the code; the message string in
|
|
`status.Error` is a short, public, stable phrase (e.g. `"stale transition"`), not the raw
|
|
wrapped chain.
|
|
- **Infra errors** — the wire carries a **sanitised** public message + the `Internal` code.
|
|
The **full** error (with `%w` chain, SQL, provider detail) is logged **server-side only** by
|
|
the store service. The client rehydrates a generic wrapped error and must not surface the raw
|
|
text to users.
|
|
|
|
---
|
|
|
|
## 8. Limits
|
|
|
|
**Interface bloat (`interfacebloat` cap = 8)** — `.golangci.yml` `interfacebloat: max: 8`,
|
|
which overrides the upstream default of 10. The Go consumer facades must stay ≤8 methods.
|
|
`TaskLifecycle` is exactly 8 (at the cap); `TaskStore` is the 24-method composite that the cap
|
|
is why it is decomposed; proto services bypass the cap (like `ChatContract`).
|
|
|
|
**Payload caps:**
|
|
|
|
- Capture body: `defaultCaptureMaxBodyBytes = 256 KiB` (may reduce further; the wire message
|
|
must not exceed gRPC's default 4 MiB receive limit for a single capture). A single capture
|
|
fits, but a _batch_ of them did not: the caller-side cap of up to 100 captures at 256 KiB each
|
|
could exceed 4 MiB in one unary `ListCaptures`/`ListUndispatchedCaptures` response. `.4.7` adds
|
|
`StreamCaptures`/`StreamUndispatchedCaptures` (server-streaming, one capture per message) as
|
|
the path both `staterpc.Client` methods actually call; the original unary RPCs stay defined,
|
|
unmodified and deprecated, only so `buf breaking` protects an in-flight rolling deploy.
|
|
- Task `detail`/`park_reason`/event `Detail`: 4000-char hard cap (`clip(s, 4000)`).
|
|
|
|
**Retention (stays server-side in the store; the wire does not carry pruning decisions):**
|
|
|
|
- `events` table: **no prune, unbounded growth** — a known risk to revisit, not changed here.
|
|
- `captured_events`: prune-on-write, default 7 days / 5000 events.
|
|
|
|
**Pagination/arity:** store read methods take a raw `int limit` (no store-side cap); the
|
|
caller-side caps (task list 100, log 2000, captures 100, capture scan window 5000, binding
|
|
dispatch batch 100, session list 20) remain caller-side. The wire passes `limit int` and returns
|
|
`[]*` slices, mirroring the existing Go signatures — **no cursor/paging protocol is introduced**
|
|
in this work (the current `EventsSince(sinceID, limit)` offset cursor is the only one, and it
|
|
stays as-is).
|
|
|
|
---
|
|
|
|
## 9. Transport security boundary
|
|
|
|
The gateway precedent is loopback-only, unauthenticated gRPC. The State Store holds the
|
|
crown-jewel `archie.db` (task/event/capture/mapping/binding data), so its boundary must be
|
|
stated explicitly rather than left implicit.
|
|
|
|
**The State Store's equivalent boundary (rev. 2c):**
|
|
|
|
- **Listener topology — single rule (rev. 2c).** One in-process listener; its bind address is
|
|
chosen by the consumer set, never both at once:
|
|
- _Agent containers consume it_ (the Phase 2 driver during multiplexed serving) → bind to the
|
|
**host-gateway bridge address** so containers can reach it, and **token auth is mandatory**
|
|
(a bridge host is reachable by any process on that bridge). A container **cannot reach the
|
|
host's `127.0.0.1`**, so loopback-only is NOT usable for the agent-consumed path.
|
|
- _Only the daemon consumes it_ (no agent) → bind **loopback-only** (`--listen 127.0.0.1`),
|
|
`insecure` is fine because it is not network-reachable; no token required.
|
|
- There is **no dual-listener** design. The practical default during `.4.2` (the reason
|
|
in-process serving exists is to give the agent a gRPC target) is the **bridge address +
|
|
token**. `127.0.0.1` is reserved for the daemon-local-only case.
|
|
- **Agent access → the container bridge.** The agent runs in a task-scoped container on the
|
|
host and reaches the State Store over the Docker bridge gateway (the bind address from the
|
|
topology rule above), exactly as it reaches the daemon's NATS. To close the "any host
|
|
process could reach the store" gap, the agent **authenticates with a per-task bearer token**
|
|
carried in gRPC metadata by a client interceptor (`StateStoreToken`, §6). Network
|
|
reachability + the token is the trust boundary for the default bridge topology.
|
|
- **The daemon** uses the local adapter (`*store.Store`) in-process by default, so no network
|
|
boundary exists until `[services.state].target` is set; when it is set, the daemon client
|
|
authenticates with a token supplied alongside the target.
|
|
- **Non-loopback / k8s:** TLS/mTLS is an **operator decision**, not a default. Any non-loopback
|
|
exposure (e.g. `--listen 0.0.0.0`, a CDN, a cross-host service) requires an explicit decision
|
|
and mTLS or token auth — mirroring the gateway's `--listen`/`--enable-cors-header`
|
|
confirmation caveat. The NATS-KV discovery fallback supplies endpoints but does not add
|
|
transport security; that stays the operator's responsibility.
|
|
|
|
### Token lifecycle (rev. 2c; scoping added in `.4.7`)
|
|
|
|
- **Generation:** the daemon generates a per-task bearer token when it acquires the container
|
|
(or per incumbence) and injects it via `containerEnv` as `STATE_STORE_TOKEN`. A single token
|
|
may cover a task's container lifetime.
|
|
- **Registration (`.4.7`, post-standalone-cutover):** the State Store is now its own process
|
|
(`.4.3`/`.4.6`), so the daemon can no longer just hold an in-process issued-token set for the
|
|
server to check — it registers the token at the remote authority via the
|
|
`RegisterTaskGrant`/`RevokeTaskGrant` RPCs (`staterpc.GrantIssuer`, `daemon.StateStoreGrantIssuer`),
|
|
admin-token-authenticated calls a task-scoped token itself can never make.
|
|
- **Lifetime:** tied to the task + container grace period; rotated on a new container acquisition
|
|
for the same task; a token for a released container is invalidated (`RevokeTaskGrant`, deferred
|
|
alongside `ContainerPool.Release`). An unrevoked grant still expires server-side (bounded by
|
|
`RegisterTaskGrantRequest.lifetime_seconds`, capped at 7 days) so a daemon crash before revoke
|
|
cannot leave a grant valid indefinitely.
|
|
- **Validation and scope (`.4.7`):** a gRPC interceptor (`staterpc.TaskGrants.UnaryInterceptor`)
|
|
validates the token against the State Store's own issued-grant set; missing/unknown/expired →
|
|
`codes.Unauthenticated`. Unlike the daemon's own administrative token (full access to all RPCs),
|
|
a task-scoped grant additionally authorises only `Update`/`Transition`/`InsertEvent` on
|
|
its own task ID, plus the two workflow-caller RPCs naming that same task ID
|
|
(`EnqueueCallTask`/`WorkflowCallStatus`, `docs/prds/workflow-calls.md` — the one
|
|
sanctioned widening: the callee is the caller's child). Every other RPC, including the
|
|
streaming capture surface and
|
|
`RegisterTaskGrant`/`RevokeTaskGrant` themselves, is `codes.PermissionDenied` for a task-scoped
|
|
caller. The token is carried in gRPC metadata, never in a URL.
|
|
|
|
### Fail-closed on non-loopback (rev. 2c)
|
|
|
|
If `[services.state].target` points to a non-loopback address and **neither** TLS **nor** a token
|
|
is configured, the State Store client (daemon or agent) **fails closed** — it refuses to
|
|
start/dial rather than fall back to a plaintext, unauthenticated remote connection. This mirrors
|
|
the gateway's `--listen`/`--enable-cors-header` confinement caveat and prevents a silent
|
|
unauthenticated remote store. The loopback + token default is the only path that starts without
|
|
an explicit operator decision.
|
|
|
|
---
|
|
|
|
## 10. Mode / config shape
|
|
|
|
**Follow the gateway's presence-based `target` seam, not the NATS `mode` enum.**
|
|
|
|
- **Services are keyed by name, not named as struct fields.** A
|
|
`State ServiceConnection` field per service is the duplication
|
|
`docs/prds/service-registry.md` removes, and this document's summary describes
|
|
the seam as `[services.<name>]`:
|
|
```go
|
|
type Services map[string]ServiceConnection
|
|
```
|
|
registered once via `RegisterService(context, name, target, listen, tokenEnv)`. The TOML
|
|
shape below is unchanged, so nothing else in this section is affected; read
|
|
`cfg.Services.Get("state")` wherever it says `cfg.Services.State`.
|
|
- **Empty `Target` → startup error.** The local `*store.Store` adapter default was
|
|
withdrawn when the State Store was extracted (§12 step 7): the standalone `archie-state-store` process
|
|
exclusively owns `archie.db`, so both `archied` and `archie-gateway` now reject an empty
|
|
target (`internal/app/archied/bootstrap.go` and `state_store_client.go`: `services.state.target
|
|
is required`). **Set `Target` → dial gRPC** with `*staterpc.Client`; no client opens the
|
|
database itself. This differs from the gateway, where `Target` is defaulted to
|
|
`127.0.0.1:8585` because `archie-gateway` is already a separate process whose client and
|
|
server share one default.
|
|
- `[services.state]` is present and **uncommented** in `config.example.toml`, with `target`
|
|
marked REQUIRED, because the value is load-bearing for every client. A commented-out
|
|
`target` cannot boot, so the section ships uncommented.
|
|
- Composition root: `internal/app/archied/bootstrap.go` (the daemon) and
|
|
`internal/app/archieui/config.go` (the UI process) both read `cfg.Services.Get("state")` and
|
|
dial the remote State Store. The task store is built only in the standalone
|
|
`archie-state-store` process (`internal/app/archied/state_store.go`), which is the only path
|
|
that opens the task tables.
|
|
|
|
> **Mode fork, decided.** The gateway uses a single `target` address (no enum). The State Store
|
|
> boundary uses the same seam for symmetry, and **both now require a non-empty `target`**: the
|
|
> State Store's local-store default was withdrawn when the process was extracted (§12 step 7).
|
|
> No `mode` field is added. A companion `STATE_STORE_TOKEN` secret (or a
|
|
> `[services.state] target_token` key) carries authentication.
|
|
|
|
---
|
|
|
|
## 11. Local-adapter compatibility + conformance
|
|
|
|
- `*store.Store` (local) and `*staterpc.Client` (remote) satisfy the **same** contract
|
|
interfaces. Use **explicit** `var _` assertions on **both** adapters (the gateway's local
|
|
adapters use only structural/return-type assertions; the store precedent's explicit `var _`
|
|
style is stronger and should be adopted on both sides).
|
|
```go
|
|
var _ workflow.Store = (*staterpc.Client)(nil) // first contract to move (agent)
|
|
var _ store.TaskStore = (*staterpc.Client)(nil) // daemon path, added progressively
|
|
var _ store.CaptureStore = (*staterpc.Client)(nil)
|
|
var _ store.MappingStore = (*staterpc.Client)(nil)
|
|
var _ store.BindingStore = (*staterpc.Client)(nil)
|
|
var _ store.BindingDispatcher = (*staterpc.Client)(nil)
|
|
var _ store.BindingTaskCreator = (*staterpc.Client)(nil)
|
|
var _ store.PlaybookDispatcher = (*staterpc.Client)(nil)
|
|
// local side (in store), already present:
|
|
var _ workflow.Store = (*Store)(nil)
|
|
```
|
|
- **Conformance suite:** a `staterpc/conformance_test.go` bufconn suite runs the **same**
|
|
contract through both `"local"` and `"grpc"` adapters (mirroring
|
|
`gatewayrpc/conformance_test.go`), asserting error-sentinel fidelity and the
|
|
not-found-as-`(nil,nil)` convention both ways.
|
|
- **`Close()`:** stays on the Go store interfaces (so `TaskStore`/`TaskEvents` remain fully
|
|
satisfiable) but is **not** a `StateStore` RPC. The remote `staterpc.Client` implements
|
|
`Close()` as a **no-op**; the store service owns its own DB lifecycle. When the daemon hands
|
|
the store to `archie-state-store`, `b.cleanup()` no longer calls `store.Close()` on a remote
|
|
client (it is the daemon's own lifecycle only when it owns the store in-process).
|
|
|
|
---
|
|
|
|
## 12. Cutover / deletion rules
|
|
|
|
1. **Ratify the contract (this document).** No code.
|
|
2. **Domain type + interface relocation (prerequisite, coupled with §4).** Move `Task`/`Status`/
|
|
`Source` and the `WorkflowStore` interface into `internal/domain/workflow` (as
|
|
`workflow.Task`/`workflow.Status`/`workflow.Source`/`workflow.Store`), so
|
|
`internal/domain/workflow` imports no `internal/store`. Update the store tests and the 6
|
|
workflow production files. **This must land before or with `.4.2`, because `.4.2` cannot move
|
|
the agent's store contract off `store.WorkflowStore` while the domain still imports it.**
|
|
3. **`.4.2` — generalise `storerpc` to the gRPC contract.** Move the agent's `workflow.Store`
|
|
(`Update`/`Transition`/`InsertEvent`) from NATS JSON `storerpc` to `staterpc.Client` over
|
|
gRPC, using the §6 handoff. The daemon serves the gRPC `StateStore` server **in-process
|
|
(multiplexed)** first so the agent has a gRPC target before the standalone binary exists
|
|
(PRD §3 "multiplexed in-process is the migration default"). Bind the in-process listener and
|
|
inject `STATE_STORE_URL`/`STATE_STORE_TOKEN` per task (see §6 "Operational specifics"). Also
|
|
land the `RecordDispatch` `*sql.Tx` drop (see §5) in the same span. Flip the agent's
|
|
`Transport.Store` to a gRPC client once `STATE_STORE_URL` is injected.
|
|
4. **Delete NATS `storerpc` only after the agent is fully on gRPC** — delete `internal/storerpc`
|
|
(Server, Client, subjects `archie.store.update|transition|insert_event`), the
|
|
`natsrpc.RegisterAll` registration, and the now-unused `store.WorkflowStore` NATS path in
|
|
`internal/infrastructure/agenttransport/nats`, in the same commit.
|
|
5. **`[services.state].target` stays empty (inproc)** through `.4.2`; the `archie-state-store`
|
|
`cmd/` binary is **`.4.3`** and hosts the same `StateStore` gRPC service.
|
|
6. **Swap daemon-internal contracts one at a time** (only once the daemon points at
|
|
`archie-state-store`): `Capture → Mapping → Binding → TaskStore`. Each swap is independent —
|
|
add the remote-client composition path, run the per-contract conformance suite, flip
|
|
`[services.state].target`, delete the local use. A consumer mixes in-process and remote
|
|
contracts for a contract-code (no dual-live _window_; each contract is either local or remote
|
|
at a time).
|
|
7. **Final flip / deletion of the in-process serving path:** when the daemon no longer owns the
|
|
store (i.e. `TaskStore` has moved), delete the in-process task-store /
|
|
`wireWebStoreSurfaces` path in the daemon in a **same-commit** deletion — no dual-store
|
|
ownership. The single SQLite file remains owned by `archie-state-store`.
|
|
8. **Never** split `archie.db` or add a second store. The gateway keeps its own already-separate
|
|
session SQLite; that is unchanged and out of state-store scope.
|
|
|
|
---
|
|
|
|
## 13. Decisions recorded (for the review trail)
|
|
|
|
- **`internal/store` is a transitional compatibility location** (target
|
|
`internal/infrastructure/store`), not the final infrastructure boundary. The producer-owned
|
|
daemon/webui store surfaces live there now but may relocate with the implementation.
|
|
- **In-process listener + token lifecycle:** the daemon binds the in-process State Store
|
|
listener to the **host-gateway bridge address** when agent containers consume it (the Phase 2
|
|
default) or loopback-only when the daemon is the sole consumer — one listener, one address
|
|
rule, never both (rev. 2c). It selects the bridge address by reusing the embedded-NATS
|
|
host-gateway resolution and injects `STATE_STORE_URL`/`STATE_STORE_TOKEN` per task via
|
|
`containerEnv`. Token is per-incumbence (task + container grace period) and validated by a
|
|
gRPC interceptor → `codes.Unauthenticated` on missing/unknown/expired. Non-loopback targets
|
|
fail closed without TLS or a token.
|
|
- **§12 step 2 (domain type + interface relocation) is done** — `internal/domain/workflow`
|
|
imports no `internal/store` symbol and `store.WorkflowStore` is gone from `TaskContext`. The
|
|
broader Phase 2 acceptance criterion (State Store as its own gRPC service) is still open;
|
|
`.4.2` picks up from here.
|
|
- **Ownership split:** domain-facing workflow contract is **consumer-owned** in the domain
|
|
(`workflow.Store` + `workflow.Task`/`Status`/`Source`, per dependency rules #2 and #7);
|
|
daemon/webui store surfaces stay **producer-owned** in `internal/store`. `store.WorkflowStore`
|
|
is superseded by `workflow.Store`.
|
|
- **One `StateStore` gRPC service** (49 contract RPCs, grouped by contract) + narrow Go consumer facades
|
|
(≤8) on `staterpc.Client` — mirrors the single-`ChatService` precedent.
|
|
- **Domain type relocation** (`Task`/`Status`/`Source` → `internal/domain/workflow`) is a
|
|
Phase 2 prerequisite, pulled forward from migration-decisions §4 (minimal bound,
|
|
`Task → WorkflowExecution`); the rest of §4 stays a separate migration.
|
|
- **`RecordDispatch` drops `*sql.Tx`** (wire-blocker; production passes `nil`) — a
|
|
required interface migration landing before/with `.4.2`.
|
|
- **`Close()` not on the wire**; remote client `Close()` is a no-op.
|
|
- **Structured gRPC error codes** with client rehydration to store sentinels (preserves
|
|
`errors.Is`), a `found=false` + `OK` convention for read not-found, and **sanitised public
|
|
error details** (full detail logged server-side only).
|
|
- **Agent handoff:** `agentworker.Settings` gains `StateStoreTarget`/`StateStoreToken`/
|
|
`StoreTimeout`, injected as `STATE_STORE_URL`/`STATE_STORE_TOKEN` env (mirrors the NATS
|
|
handoff); `Transport.Store` returns a long-lived gRPC `workflow.Store` client.
|
|
- **Transport security + listener topology (rev. 2c):** one in-process listener, bound to the
|
|
host-gateway bridge address when agent containers consume it (the Phase 2 default) or
|
|
loopback-only for a daemon-local-only consumer. `insecure` only on loopback; mandatory
|
|
per-task bearer-token auth on the bridge; TLS/mTLS is an operator choice for any
|
|
non-loopback/k8s topology.
|
|
- **Mode = presence-based `[services.state].target`**, default **local** (not yet extracted);
|
|
add `[services.state]` to `config.example.toml`.
|
|
- **Explicit `var _` assertions on both adapters**; per-contract bufconn conformance.
|
|
- **Migration order:** domain type/interface relocation → `workflow.Store` (agent) → stand up
|
|
`archie-state-store` → `Capture` → `Mapping` → `Binding` → `TaskStore`; NATS `storerpc`
|
|
deleted once the agent is on gRPC; in-process store path deleted in a same-commit flip.
|
|
- **Orphans `ClearTerminalTasks`/`IncrementRetryCount`** retained but not built upon.
|