Landing three PRD refinements found while studying prior reviewer art before starting archie-core-afbk.3: dimension selection now sees intake's machine-written score directly and adds a hallucination-check dimension for it; a reviewer that exhausts its turn cap or deadline records a distinct StepExecution status from one that completed and found nothing; phase 9 polish runs in a bounded pool of 8, matching phase 4.
916 lines
997 KiB
JSON
916 lines
997 KiB
JSON
{
|
||
"pages": [
|
||
{
|
||
"url": "/dev/architecture/",
|
||
"title": "Architecture",
|
||
"section": "Architecture",
|
||
"body": "# Architecture\n\nArchie is a personal agent platform built around six domain boundaries:\n\n- Identity tracks users and other actors responsible for actions.\n- Agent owns persistent assistants, specialisation, and continuity.\n- Messaging owns conversations, messages, branches, and channel correlation.\n- Workflow owns reusable, versioned behaviour definitions and their executions.\n- Work Intake admits optional durable work without constraining ordinary agent\n conversation or direct capability use.\n- Plugin owns generic extension discovery and lifecycle; each capability family\n owns its typed plugin contract.\n\nThe detailed architecture and migration decisions are maintained under\n`docs/architecture/` in the repository while this generated site is being\nassembled.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/adversarial-review/",
|
||
"title": "Adversarial self-review",
|
||
"section": "Architecture",
|
||
"body": "# Adversarial self-review\n\nArchie runs a fresh adversarial review of the diff before it opens a pull\nrequest, so a PR arrives with a findings record rather than an unreviewed\nclaim. The reviewer provably cannot see the implementer's context: it reviews a\n`.git`-free snapshot of the committed tree, not the worktree, the commit\nhistory, or the implementer's transcript.\n\nPromoted from `docs/prds/adversarial-self-review.md` (the design decision;\n`archie-core-h019`). Describes the stage as shipped.\n\n## Contract (`internal/domain/workflow`)\n\nThe workflow domain owns the contract and the blocking decision; it never names\nthe executor.\n\n- `Reviewer.Review(ctx, ReviewRequest) ReviewReport` — the interface the\n workflow stage consumes.\n- `ReviewRequest` is the reviewer's entire input: `SnapshotDir` (a `.git`-free\n export of the reviewed commit), `Diff` (base...head), `IssueText` (the\n originating issue title/body), and `MaxSteps`.\n- `ReviewFinding` / `ReviewReport` are the structured result. The load-bearing\n rule is `Blocking() == (Verdict == confirmed \u0026\u0026 Level == error)`: only a\n confirmed error-level finding blocks; a plausible finding never blocks\n however severe it claims to be.\n\n`ReviewReport` distinguishes `not_run`, `skipped`, and `completed` (zero\nfindings) structurally, so \"ran and found nothing\" is not the same as \"never\nran\" — collapsing them would make an outage indistinguishable from a clean\nreview.\n\n## Execution (`internal/app/agentworker`)\n\nThe executor is an ai-sdk `agent.Subagent`: a nested synchronous generation in\nthe same worker process, with its own model, system prompt, toolset and step\nbudget.\n\n- **Conversation isolation is structural.** `Subagent.Run` sends only a prompt\n string, never the implementer's message history.\n- **Workspace isolation is the `.git`-free snapshot.** The reviewer's toolset is\n read-only (`read`, `grep`, `find`) and rooted at `SnapshotDir` — never the\n worker's own toolset, which carries the `worktreerpc` publication grant.\n- **Findings return through a typed capture tool** (`record_finding`), validated\n on the way in, so the reviewer reports through the contract rather than free\n text the caller regexes.\n- **Fail-closed.** A reviewer that errors, truncates, or reaches no conclusion\n yields `ReviewStatusNotRun`, which the stage treats as a park — never a silent\n \"PR opened unreviewed\".\n\n## Lifecycle effect\n\n`StageReview` runs in the implement workflow after the deterministic gates and\nbefore `StageOpenPR`, gated by `repo.review_enabled` (opt-in, default off). A\nsurviving blocking finding — or a review that failed to run — parks the task\n(`StatusParked`, not `waiting_human`) with the findings rendered into the park\nreason; `StageOpenPR` never runs. Warn/plausible findings and the zero-finding\ncase leave the outcome unset, so the PR opens with a findings section rendered\nonto its body. `StagePostReviewComments` then runs after `StageOpenPR` and posts\nthe **line-anchored** findings as inline review comments, carrying a one-click\nsuggestion on a confirmed finding the reviewer could state a fix for. That stage\nis best-effort — a comment that fails to post is logged, never a park — because\nthe body section already lists every finding, and it is the record a whole-file\nfinding (no line to anchor to) only ever appears in.\n\n## Model and budget\n\nThe reviewer model is `models.reviewer`, falling back to `models.builder`.\nEffort is the per-stage `MaxSteps` budget, mapped onto `Subagent.MaxSteps`\n(default 10).\n\n## Operator-triggered review\n\nThe same reviewer runs against an _existing_ PR via the channel-neutral\n`review_pr` chat command: fetch the PR's head, snapshot it, run the isolated\nreviewer, and return the structured findings. Authorisation is identity-scoped\nto the operator's configured repositories; duplicate or concurrent reviews of\nthe same PR are deduplicated.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/agent-system/",
|
||
"title": "Agent System",
|
||
"section": "Architecture",
|
||
"body": "# Agent System\n\n**Status:** Current-state investigation in progress \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Domain framing\n\nThe domain is the Agent System.\n\n`Task` is not a core Agent System concept. Forge tasks, forge issues, Jira\nissues, email, Telegram messages, webhooks, and future external inputs are\nchannel-specific messages. Channels translate those messages into requests to\nthe Agent System; external issue or task models MUST NOT shape execution state.\n\nThe current `store.Task` is a persisted workflow execution. Treating that row as\na task-domain aggregate would incorrectly turn Archie into an agentic issue\nboard.\n\n## Canonical agent interaction model\n\nAn **Agent** is a persistent assistant selected for an interaction. It is not a\nmodel invocation and is not a WorkflowExecution.\n\nEach Agent has its own:\n\n- immutable acting `IdentityID` for ownership and audit;\n- durable memory and learned information about the user;\n- perspective, specialisation, instructions, and communication behaviour;\n- conversation history and channel bindings;\n- available tools, capabilities, and policy;\n- continuity across model or provider changes.\n\nThe underlying model is a replaceable runtime dependency. Changing a model,\nprovider, or model version MUST NOT create a new Agent, discard its memory, or\nchange ownership of its history.\n\nA user MAY create multiple Agents. Each Agent develops its own memory and view\nof the user from the interactions and responsibilities assigned to it. Memories\nMUST NOT be silently pooled merely because Agents belong to the same user or\napplication.\n\nAgent and Identity are separate aggregates. Every Agent references exactly one\nacting IdentityID, and no two Agents share an agent identity. User, system, and\nservice-account identities may exist without an Agent.\n\nAn action initiated through an Agent records the Agent's acting IdentityID and,\nwhen applicable, the initiating user's IdentityID. A future fully automated\nWorkflowExecution MAY use an explicitly selected service-account identity for a\nconcrete unattended use case; this is not the default execution identity.\n\n## Workflow execution identity\n\nA Workflow is owned by an IdentityID. By default, every WorkflowExecution acts\nas the Workflow owner's identity.\n\nFuture delegated execution MAY add an explicit `RunAsIdentityID` assignment to a\nWorkflow. This allows a Workflow owner to select another user or service-account\nidentity for execution without changing Workflow ownership. Until that\ncapability is deliberately introduced, owner identity and execution identity are\nthe same.\n\nWorkflowExecution audit records MUST preserve the Workflow owner, effective\nexecution identity, initiating user when one exists, and any explicit delegation\nthat selected a different run-as identity.\n\n## Memory scopes\n\nThe Agent System requires four distinct durable memory scopes:\n\n- **global shared memory**: information intentionally available across the\n Archie application;\n- **agent-wide memory**: general expertise, operating knowledge, and perspective\n owned by one Agent and safe for that Agent to use across users;\n- **user-wide memory**: information associated with one user and available to\n authorised Agents serving that user;\n- **Agent-user relationship memory**: private history and learned context from\n one particular Agent's relationship with one particular user.\n\nAgent-wide memory is private to its owning Agent by default. Another Agent MUST\nNOT read, inherit, search, or mutate it implicitly.\n\nAgent-user relationship memory is addressed by both AgentID and User IdentityID.\nNeither another Agent serving the same user nor another user speaking to the\nsame Agent may access it implicitly.\n\nArchie is a multi-user system even when initially deployed for one person.\nMultiple users, including family members, may interact with the same configured\nAgents. Per-user memory is therefore an isolation and personalisation boundary,\nnot merely an attribution tag on the primary user's records.\n\nAn Agent MUST resolve the initiating user before assembling memory context.\nMemory belonging to one user MUST NOT become visible to another user merely\nbecause both users interact with the same Agent.\n\nShared memory does not erase provenance. Every memory record retains its scope,\nauthor or acting identity, originating user when applicable, source, and\nrevision history. The rules for selecting a write scope, granting access\nto global and user-wide memory, and authorising explicit cross-scope sharing are\nsettled by [`docs/prds/memory-engine-unification.md`](/dev/prds/memory-engine-unification/)\nand recorded in [`migration-decisions.md`](/dev/architecture/migration-decisions/#5-memory-placement-and-storage)\nsection 5.\n\nAn Agent interaction may answer conversationally, use a tool or capability, read\nor send messages, retain information, or propose durable workflow-backed work.\nNone of these operations inherently requires a WorkflowExecution.\n\n## Agent and Workflow domain boundary\n\nAgent and Workflow are separate domains:\n\n```text\ninternal/domain/agent\ninternal/domain/workflow\n```\n\nThe Agent domain owns persistent assistants, their specialisation, instructions,\nmodel-independent continuity, capability context, and associations with identity\nand memory.\n\nThe Workflow domain owns reusable, user- or Agent-defined behaviour and every\ndurable execution of that behaviour. An Agent may invoke or define a Workflow,\nbut the Workflow is not part of the Agent aggregate and does not require a\nconversational Agent interaction to execute.\n\n## Canonical workflow model\n\nThe Workflow feature uses these canonical terms:\n\n- **Workflow**: a reusable, versioned definition of how an agent behaves and\n performs a kind of work.\n- **WorkflowExecution**: one durable execution of a specific Workflow version.\n- **WorkflowStep**: one defined operation in a Workflow.\n- **StepExecution**: the recorded execution and outcome of one WorkflowStep.\n- **Attempt**: a retry of the same WorkflowExecution, not a new execution.\n\nA Workflow MAY structure any repeatable agent behaviour, including personal\nassistance and scheduled routines. For example, a morning email review may\ncombine a schedule event, message retrieval tool, LLM classification and\nsummarisation, and delivery of the resulting rundown. Workflow is therefore not\nrestricted to forge work, source changes, or requests submitted by a user at\nexecution time.\n\nWorkflows are extensible through Workflow-owned plugins. The Workflow domain\ndefines the typed plugin registration, definition, validation, versioning, and\nexecution contracts. Users and Agents may define reusable Workflows through\nthose contracts.\n\nThe generic plugin contract remains metadata-only. It MUST NOT own Workflow\nsemantics or replace the Workflow registry with generic hooks.\n\n`Workflow` and `WorkflowExecution` are the names used in architecture, generated\nreference, configuration, commands, events, and external contracts. The\nimplementation MAY use package qualification to remain concise:\n\n```go\nworkflow.Definition\nworkflow.Execution\nworkflow.Step\nworkflow.StepExecution\n```\n\nThe unqualified name `Execution` MUST NOT be used in cross-domain contracts or\ndocumentation because it is ambiguous among workflow, tool, command, policy, and\nservice execution.\n\n## Current workflow-execution feature\n\nThe current feature takes a channel request through agent execution and a\ndurable outcome.\n\nCurrent execution state includes source-message references, owning identity,\nrepository, selected workflow and stage, agent output, branch and pull-request\nreferences, token and iteration counts, retry information, human-wait\ninformation, and failure details.\n\nThe current `store.Task` lifecycle uses:\n\n```text\nqueued -\u003e running\nrunning -\u003e queued (crash recovery) | waiting_human | pr_open | completed | parked | closed_wont_do\nwaiting_human -\u003e queued | closed_wont_do\nparked -\u003e queued | dead\npr_open -\u003e merged | rejected\n```\n\n`merged` is reached only from `pr_open`. `completed` is terminal: it means a\nworkflow that opens no pull request finished its work.\n\nWorkflows select and sequence deterministic and agent-run stages. Implement,\nTDD, feasibility, and bootstrap are current Workflow definitions.\nEvery WorkflowExecution is handed wholesale over core NATS to a task-scoped\n`archie-agent` container, which owns the workflow stage loop.\n\n## Current workflow-feature placement\n\nThe behaviour is divided across technical packages:\n\n- `internal/domain/workflow` defines the current `Task` execution record and\n status strings; `internal/infrastructure/postgres` persists mutable state\n and transition history behind `internal/domain/storecontract`.\n- `internal/daemon` discovers, claims, schedules, retries, recovers, processes,\n waits for human input, and reconciles pull requests.\n- `internal/domain/workflow` routes work, executes stages, mutates tasks, applies\n outcomes, parks failures, and performs forge/worktree operations.\n- `internal/agentexec` defines and runs individual autonomous-stage protocols.\n- `internal/taskrun` hands a complete WorkflowExecution to a container worker.\n- `internal/gate` and repository configuration define execution constraints.\n- `internal/gateway` duplicates task status strings and applies task-control\n rules.\n- forge, worktree, NATS, PostgreSQL, containers, and agent runtimes are concrete\n dependencies mixed into orchestration.\n\n## Current `task` meanings and required replacements\n\nThe existing code uses `task` for unrelated concepts. Migration MUST resolve\neach meaning deliberately; a global text replacement is prohibited.\n\n| Current meaning | Representative locations | Target concept | Requirements |\n| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Forge issue or external task | `internal/forge`, daemon polling | Channel message | Preserve the channel's native reference and payload. Translate it into an Agent System request without importing forge or issue semantics into Workflow. |\n| Jira issue or other external work item | Messaging infrastructure adapter | Channel message or external artifact | The external system retains its own vocabulary inside its integration. Its record may lead to an accepted work request but is not a Workflow or WorkflowExecution. |\n| Persisted `workflow.Task` row | `internal/infrastructure/postgres` | WorkflowExecution | Own identity, Workflow ID and version, lifecycle, source-message reference, outputs, attempts, and outcome. Persistence implements the Agent System repository contract. |\n| Task status and transition history | `store.Status*`, `transitions` | WorkflowExecution lifecycle and domain events | Enforce legal transitions atomically. Domain events and state changes cannot diverge. Remove duplicated status strings from consumers. |\n| Claimed or queued task | `ClaimNext`, `ClaimByIssue`, `RecoverStale` | Scheduled WorkflowExecution | Claim by WorkflowExecution ID and expected version/state. Crash recovery and leases must not silently create a new execution. |\n| `workflow.TaskContext` | `internal/domain/workflow` | WorkflowExecution context | Replace the mutable service bag with execution state plus narrow step-required contracts. Complete application configuration and unrelated services are forbidden. |\n| `taskrun.Request` and `taskrun.Response` | `internal/taskrun`; consumed by `internal/app/agentworker`, decoded by `internal/infrastructure/agenttransport/nats` | WorkflowExecution handoff | This is the versioned worker contract for executing a WorkflowExecution. It carries domain-owned execution data, not store rows or global configuration DTOs. |\n| Per-stage `agentexec.Request.TaskID` | `internal/agentexec` | WorkflowExecutionID correlation | An agent invocation is a bounded WorkflowStep execution. Correlate request/result with WorkflowExecutionID, StepExecutionID, attempt, and protocol version. |\n| NATS `TaskMessage` and `archie.task.*` | `internal/domain/workintake` (subjects, `TaskEnvelope`); `internal/infrastructure/agenttransport/nats` (transport) -- `internal/nats` no longer exists; whether this row's \"current meaning\" itself is now superseded by `TaskEnvelope` needs a closer look, not asserted here | Channel request or WorkflowExecution transport | Discovery messages and execution dispatch are different contracts and subjects. Domain meaning stays outside NATS infrastructure. Identity and source-message idempotency must be explicit. |\n| `taskDispatcher` work item | `internal/daemon` | WorkflowExecution scheduling | Scheduling operates on execution references and declared concurrency policy. Repository serialisation is policy, not identity derived from an incomplete task struct. |\n| `container.TaskPayload` and `task.json` | `internal/container`; read by `internal/infrastructure/agentboot` | WorkflowExecution brief | Rename the boot artifact and schema. It identifies the mounted execution and required workspace only; it is not an alternative execution definition. Missing or malformed input must fail startup before the worker subscribes. |\n| `storage.TaskRef` | `internal/storage` | WorkflowExecution workspace lease | Storage receives the minimal execution/workspace reference and policy required to provision and release resources. It does not receive the WorkflowExecution aggregate. |\n| Gateway `TaskCreator` | `internal/gateway` | Accepted work request handoff | A channel may carry a request for durable workflow-backed work, but it does not request or create a WorkflowExecution directly. Work intake admits and routes the request; the Agent System consumes the accepted work request and is the sole owner of selecting a Workflow version and creating the WorkflowExecution. Ordinary agent interactions do not require this handoff. |\n| Gateway `TaskController` and `ChatTaskStatus` | `internal/gateway/tasks.go` | WorkflowExecution commands and query view | Approve, cancel, and status use Agent System contracts. Channel code does not duplicate lifecycle constants or decide transitions. |\n| Gateway `TaskProfile` | `internal/gateway/tasks.go`, `cmd/archied` | Channel/identity execution defaults | This is configuration selecting allowed source targets and default Workflow inputs, not task state. Split it into settings owned by the relevant channel and Agent System features. |\n| `config.TaskConfig` and `TaskForge` | `internal/config` | Worker execution settings and channel/service references | Dissolve with `internal/config`. Each field moves to its owning runtime settings or versioned WorkflowExecution handoff contract. Secrets and external configuration forms do not enter the domain. |\n| `events.Event.TaskID`, task event timelines | `internal/events`, stores, web UI | WorkflowExecutionID and WorkflowExecution events | Authoritative events are Agent System events. Observability and timelines project those events rather than inventing parallel generic task events. |\n| Web UI task lists and counts | `internal/webui`, store query APIs | WorkflowExecution read model | Dashboard status, history, and counts are projections of WorkflowExecution state and events. The UI does not depend on persistence records. |\n| Task-scoped forge/store/worktree RPC | `forgerpc`, `storerpc`, `worktreerpc`, `registerTaskRPCServers` | Execution-scoped capability ports | Worker calls use narrow domain-required service contracts, carry WorkflowExecutionID and IdentityID, and resolve the owning capability binding. Root-client fallback is forbidden. |\n| Per-task worktree, branch, volume, and cleanup | daemon, worktree, storage, container | WorkflowExecution workspace | Every WorkflowExecution receives isolated workspace state. Workspace lifecycle is explicit, identity-aware, recoverable, and controlled outside the model. |\n| Task retry, dead, waiting-human, PR reconciliation | daemon, workflow, gateway | WorkflowExecution lifecycle policies | Retry, waiting, cancellation, reconciliation, and terminal outcomes are Agent System behaviour. Channels and infrastructure report facts or submit commands; they do not mutate state. |\n| Task-run readiness and retry timeouts | daemon taskrun request loop | Worker-dispatch settings and policy | These settings govern WorkflowExecution handoff availability. They are not Workflow data and must be owned by the worker-dispatch capability. |\n\n### Terms that do not migrate\n\n- The `task` command in `Taskfile.yml` is the external Task build tool and keeps\n its name.\n- A forge may call its own records issues or tasks inside its channel adapter.\n- General prose may use “task” informally only when it clearly refers to an\n external system. Agent System contracts and generated reference MUST use the\n canonical Workflow vocabulary.\n\n## Current WorkflowExecution data inventory\n\nThis inventory is the migration ledger for data currently spread across\n`store.Task`, database-only columns, `workflow.TaskContext`, worker messages,\nand surrounding configuration. Implementation SHOULD NOT require another\nrepository-wide search to determine what existing execution data must be\npreserved or deliberately removed.\n\n| Current data | Current representation | Required target |\n| ---------------------- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |\n| Execution identity | `Task.ID`, `task_id`, NATS subjects, logs | Immutable WorkflowExecutionID used consistently by state, events, workers, projections, and capability calls |\n| Owning actor | `Task.Identity` string | Canonical IdentityID |\n| Source type | `Task.Source` (`forge` or `chat`) | Typed channel and SourceMessageReference; no closed source enum in WorkflowExecution |\n| Source address | `Owner`, `Repo`, `IssueNumber`; synthetic chat issue number | Channel-owned source reference with native message/thread/object identifiers; synthetic identifiers removed |\n| Requested work | `Title`, `Body`, `Labels` | Immutable execution input captured from the source request; channel-specific metadata retained in the source envelope |\n| Workflow selection | `Workflow` string | Immutable WorkflowID plus selected WorkflowVersion |\n| Lifecycle | `Status` string | Enforced WorkflowExecution state |\n| Current position | mutable `Stage` string | Derived current WorkflowStep reference plus durable StepExecution history |\n| Attempts | `Attempt`, `RetryCount` | Execution attempt history and retry-policy evidence; distinguish worker delivery retry from workflow retry |\n| Failure | `ParkReason`, transition detail, response errors | Typed failure classification, evidence, retryability, and resulting lifecycle event |\n| Human wait | `WatchCommentID`, `waiting_human` | Channel-neutral wait condition and correlation reference; forge comment IDs stay in the forge channel |\n| Workspace | `Dir`, `Branch`, worktree path, `task.json` | Execution workspace reference and independently managed workspace lifecycle |\n| Change outcome | `BuildSummary`, `BuildNoChanges`, changed files | Step outputs and final WorkflowExecution outcome |\n| Reproduction evidence | `ReproProof` | Typed TDD step evidence retained with the StepExecution and final outcome |\n| Planning output | `Plan` | Typed step output referenced by later steps rather than a special aggregate field |\n| Notes | `Notes`, appended agent notes | Ordered execution annotations with actor, source step, and timestamp |\n| Pull request | `PRNumber`, forge host/repository assumptions | Forge-channel delivery reference and reconciliation state |\n| Agent usage | `TokensUsed`, `Iterations` | StepExecution usage measurements aggregated into execution projections |\n| Decision capture | private feasibility `decision`, capture-tool output | Typed WorkflowStep output defined by that Workflow version |\n| Terminal outcome | mutable `Outcome{Status, Detail}` | Explicit WorkflowExecution outcome command/event validated by the state machine |\n| Skill context | `SkillBody`, `SkillPlugins` | Resolved Workflow version dependencies or execution inputs, with provenance |\n| Agent prompt context | `SystemPrompt`, `ExtraRules`, mission strings | Versioned WorkflowStep definition plus resolved execution context; mutable global prompt callbacks removed |\n| Gates and protection | `Gate`, `Preflight`, `Protection`, repository config | Agent System policy definitions resolved for the WorkflowExecution and enforced outside the agent |\n| Budgets | steps, tokens, wall clock, gate failures | Execution and StepExecution policy values with measured usage and evidence |\n| Tool access | capture tools, registry, guardrails, skill plugins | Resolved step capability set and policy; global registry pointers do not enter execution state |\n| Persistence timestamps | SQL `created_at` and `updated_at` | Domain timestamps and version used for optimistic concurrency and audit |\n| Transition history | SQL `transitions` and generic events | Authoritative WorkflowExecution domain events stored atomically with state |\n\n## Current runtime responsibilities to unify\n\nThe following behaviours currently act on the same WorkflowExecution but are\nimplemented in different packages:\n\n- accept a request from a channel and enforce idempotency;\n- select and version a Workflow;\n- create, queue, claim, schedule, and execute a WorkflowExecution;\n- serialise same-repository execution where policy requires it;\n- resolve identity-specific capabilities;\n- prepare, retain, recover, and clean up the workspace;\n- run deterministic and agent-backed WorkflowSteps;\n- enforce gates, budgets, protected paths, tool constraints, and result review;\n- persist state and StepExecution outputs;\n- publish authoritative lifecycle events;\n- wait for channel input and resume;\n- classify failure, retry, park, mark exhausted, cancel, or recover after crash;\n- deliver and reconcile forge changes;\n- project status, history, counts, and audit for channels and the web UI;\n- hand execution to a worker without moving domain ownership to the worker.\n\n## Current workflow-feature hazards\n\n- State update and transition-history insertion are separate operations, so\n state and audit can diverge.\n- Callers frequently discard persistence and transition errors.\n- Gateway task control duplicates lifecycle status constants by hand.\n- `workflow.TaskContext` is a mutable service bag containing complete\n configuration plus forge, store, worktree, agent, event bus, logging, memory,\n guardrails, skills, and scratch state.\n- Lifecycle changes occur in both daemon and workflow code.\n- Workflow orchestration runs inside task-scoped agent containers while the\n daemon retains lifecycle, admission, and authority-bearing RPC ownership.\n- Forge-backed and chat-created work share one record through source flags and\n synthetic issue numbers.\n- Repository labels currently select workflows and mirror task state, coupling\n domain transitions to forge presentation.\n- Failure generally becomes `parked`, but failure classification and\n retryability are not represented explicitly.\n- Generic observability events are emitted alongside state changes rather than\n being the authoritative domain events.\n\n## Existing invariants to preserve\n\n- The model never performs git operations directly.\n- Deterministic policy and gates cannot be waived by the agent.\n- TDD captures a failing reproduction before implementation and protects the\n test during the fix.\n- Every run ends in an explicit outcome; definition errors and stage errors do\n not disappear silently.\n- Crash recovery requeues interrupted work.\n- Agent execution crosses a versioned data boundary.\n- Forge, persistence, workspace, transport, container, and model runtimes remain\n replaceable infrastructure.\n- Work from the same repository is serialised by default.\n\n## Required migration\n\n- Rename the current persisted `store.Task` concept to WorkflowExecution.\n- Replace task lifecycle terminology in core commands, events, persistence,\n worker contracts, logs, and generated reference.\n- Preserve external source terminology only inside its channel adapter.\n- Replace synthetic issue numbers with explicit source-message references.\n- Associate every WorkflowExecution with an immutable Workflow identifier and\n version.\n- Record StepExecutions explicitly rather than treating one mutable `stage`\n field as complete execution history.\n- Keep Attempt as retry state within the same WorkflowExecution.\n\n## Curator isolation invariant\n\nDecided 2026-08-05\n([#435](https://github.com/samcharles93/archie-core/issues/435) epic,\n[#98](https://github.com/samcharles93/archie-core/issues/98) runtime).\nRecovered 2026-08-09 from the pre-migration issue tracker.\n\n**Curators are peers, not dependencies.** They must never block, delay, or take\ndown chat turns, agent runs, or the daemon.\n\n1. Registry-owned goroutines, one per curator, with per-pass `recover()`,\n per-pass budgets (max steps, tokens, timeout), and a configurable maximum of\n concurrent passes (default 1) so curators cannot saturate the shared LLM\n runtime against chat.\n2. Wake events ride the in-process `events.Bus`, which is mutex fan-out with\n bounded per-subscriber buffers that **drop** on overflow\n (`internal/events/events.go`). A slow curator can never backpressure the chat\n publisher.\n3. Wake is a best-effort nudge. The trigger is a deterministic read of persisted\n primary-input state at pass time, so a dropped wake only delays a run to the\n next check-in and never corrupts trigger accounting\n ([#22](https://github.com/samcharles93/archie-core/issues/22)).\n4. The curator registry lifecycle is independent of the agent loop. Daemon\n shutdown cancels in-flight passes and waits bounded.\n\nChat turns keep their per-session lanes; agent runs keep their workflow\ngoroutines and containers. Rule of thumb: **no code path a curator executes may\never appear on a chat turn's critical path.**\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/bindings/",
|
||
"title": "Playbook bindings: model and threat model",
|
||
"section": "Architecture",
|
||
"body": "# Playbook bindings: model and threat model\n\n**Status:** Approved foundation (documents shipped code: `archie-core-t2db.1`\nthrough `.5`, `.7`, `.9`-`.15`)\n**Date:** 2026-09-03\n\nThis document covers the binding model and the intake threat model portion\nof `archie-core-t2db.6`. It does not cover the task-lifecycle provenance rule\n(`messaging-and-work-intake.md`'s \"Intake provenance\") or the operator\nwalkthrough -- that walkthrough remains an open piece of the same ticket.\n\n## What a binding is\n\nA `binding` (`internal/domain/binding.Binding`) ties four things together:\na **matcher** (which captured events it applies to), a **mapping**\n(`internal/domain/mapping.Mapping`, how fields are pulled out of a payload),\na **workflow** (which registered workflow runs on a match), and a **state\nmachine** governing whether it is live.\n\n```go\ntype Binding struct {\n ID, Name // identity\n Matcher // { Source string } -- the webhook path segment, e.g. \"sentry\"\n MappingID // which Mapping resolves payload fields; its event type is the binding's\n Filter // optional CEL over the mapping's parameters\n Workflow // registered workflow name to dispatch to\n Version // bumped on every edit\n Status // pending_approval | armed\n Secret // HMAC-SHA256 shared secret, encrypted at rest, never returned by GET\n}\n```\n\nBindings live in the store (PostgreSQL), not `config.toml` -- operators author\nthem from the dashboard while the daemon runs, without a restart.\n\n### Matching\n\nA binding applies to a capture when the capture arrived on `Matcher.Source`,\nwas identified as the event type its mapping belongs to, and passes its\n`Filter` (`binding.CompileFilter`, checked against the mapping's parameters on\nsave). Any number of bindings may share a source; each dispatches a capture at\nmost once, claimed in `binding_dispatches` keyed by (binding, capture) before\nits task is enqueued. An event signed with any armed binding's secret on the\nsource is authenticated.\n\n### Mapping\n\n`mapping.Resolve(fields, payload)` walks a captured JSON payload against a\nlist of typed `Field`s (name, JSON path, expected type), returning resolved\nvalues plus a `[]Failure` for anything that didn't parse or type-match. A\nmapping belongs to an event type, and `mapping.CheckSchema` refuses a field\nwhose path or type the type's inferred schema lacks. Each capture a mapping\nresolves at dispatch is counted once in `mapping_matches`, read back as\n`MatchCount` and `LastMatchedAt`.\n\n### Versioning\n\n`Version` increments on every `UpdateBinding` call. This exists so an\neditor's later change can never silently rewrite the historical provenance\nof a task that already fired -- a dispatched task's binding reference is\npinned to the version that was armed when it fired, not \"whatever the\nbinding currently says.\"\n\n### State machine\n\n```\ncreate ---\u003e pending_approval ---\u003e armed\n ^ |\n +----------------+\n (any edit to an armed binding drops it back to pending_approval)\n```\n\nA binding is created `pending_approval`, so the operator who creates one can\napprove it immediately. Bindings stored as `draft` by an earlier version are\nmoved to `pending_approval` when the EDA store opens.\n\nModelled directly on Telegram's existing `dangerousAction`/`pendingApproval`\nflow (`internal/channels/telegram/dangerous.go`, `approval.go`) -- not a new\napproval mechanism invented for this feature. **Only `armed` evaluates\nagainst incoming events** (`binding.Matches`, `internal/domain/binding/\nbinding.go`); a `pending_approval` binding is inert. An edit to\nan already-armed binding cannot bypass approval by silent mutation -- it\ndrops back to `pending_approval` and requires an explicit `Approve` call to\nre-arm.\n\n## The threat model\n\nThis is a public, unauthenticated-by-default intake surface whose output is\nan agent with a repo, a worktree, and tool access. The design brief\n(`archie-core-t2db.5`, `docs/prds/webhook-intake-security.md`) named five\nnon-negotiables; each is verified below against the actual shipped code,\nnot the design intent.\n\n### 1. Per-source authentication\n\nReuses `internal/channels/webhook`'s existing HMAC-SHA256 scheme --\ncapture accepts unsigned events (visible in the dashboard, marked\nunauthenticated), but **only authenticated events can ever match a\nbinding**. `binding.Matches`' doc comment states this plainly: \"an\nunauthenticated event can never trigger a binding, no matter how well it\nwould otherwise match.\" Enforced structurally at two points, not just\nconvention:\n\n- `captures.authenticated` is a real column\n (`internal/infrastructure/postgres/migrations/0002_edastore.sql`), set at\n capture time.\n- `dispatchBindings` (`internal/daemon/daemon.go:479`) walks **authenticated\n captures whose source has at least one armed binding** -- an\n unauthenticated capture is never even considered for dispatch.\n\n### 2. Human approval before a binding goes live\n\nCovered by the state machine above. Nothing self-arms: `pending_approval -\u003e\narmed` requires an explicit `Approve` call, and any edit to an armed\nbinding reverts it.\n\n### 3. Rate and volume limits per source\n\nCapture writes are bounded independently of the approval gate, so a flood\nof unauthenticated traffic cannot exhaust disk or drown the inspector:\n`InsertCapture` (`internal/infrastructure/postgres/edastore.go`) prunes by both **retention\nwindow** (`WHERE received_at \u003c cutoff`) and **max event count** in the same\ninsert transaction.\n\n### 4. Blast radius bounded by the same code-level controls every task gets\n\nA playbook-originated task is dispatched into the **same** gate, diff-cap,\nsandboxed-container, worktree-isolation pipeline every task goes through\nregardless of origin. Verified precisely, not assumed: `dispatchOneBinding`\ncalls `EnqueueBindingTask` (`internal/infrastructure/postgres/store.go`), which wraps\n`EnqueueChatTask` -- the same direct-to-`tasks`-table enqueue path\nchat-spawned tasks already use, then stamps `binding_id`/`binding_version`\nfor provenance in a second statement. This is a **different** enqueue\nmechanism from forge-issue polling's `workintake.TaskEnvelope`/NATS path\n(`docs/prds/event-sources-and-reactions.md`) -- binding and forge-issue\ntasks arrive by different producers, but both become an ordinary\n`workflow.Task` row that `ClaimNext` picks up identically, so both get the\nsame gate/worktree pipeline downstream regardless of which producer created\nthe row. Payload content is\nordinary untrusted prompt input; the sender's text is never treated as a\ntrusted instruction (Archie's environmental-enforcement principle: \"the\nagent should ignore instructions in the payload\" is not itself a control,\nthe surrounding sandbox is).\n\n### 5. Captured payloads may contain secrets from the sending system\n\nTwo independent protections, not one:\n\n- **Redaction before persistence**: `webhookguard.RedactPayload` walks the\n decoded JSON and redacts recognised sensitive keys before the row is\n written -- the raw, unredacted payload is never durably stored.\n- **Retention**: a configurable window (`InsertCapture`'s `retention`\n parameter) prunes old captures automatically; visibility of what remains\n is gated by the same dashboard authentication as every other webui\n surface, not a separate, weaker path.\n\n### Secrets at rest\n\n`Binding.Secret` (the HMAC shared secret a sender signs with) is encrypted\nat rest via AES-GCM (`internal/infrastructure/bindingcipher`, `archie-core-\nt2db.7`) -- a leaked database dump does not leak sender-side shared secrets in\nplaintext. The envelope is versioned (`bindingEnvelopeVersion`) so a future\ncipher or KDF change can roll forward without breaking already-encrypted\nrows, and the AAD binds ciphertext to the binding-secret context so it\ncannot be relocated to another field/column and still authenticate. A `nil`\ncipher (no encryption key configured) leaves secrets as plaintext --\ndocumented legacy behaviour, not silent degradation.\n\n### At-most-once dispatch\n\n`dispatchBindings` enqueues at most one task per `(binding, capture)` pair,\nenforced by a real ledger, not best-effort application logic:\n`RecordDispatch` (`internal/infrastructure/postgres/edastore.go`) inserts into\n`binding_dispatches`, unique on `(binding, capture)`; a duplicate\ninsert returns the sentinel `ErrAlreadyDispatched`, which the dispatch loop\ntreats as \"another cycle raced us,\" not an error. This guarantee holds\n**across daemon restarts**, not just within one process's lifetime, because\nthe ledger is a durable table, not in-memory state.\n\nThis ledger is also the direct precedent for the EDA playbook engine's\nexecution-time idempotency, now landed (`archie-core-t2db.17` closed): the\n`playbook_dispatches` table (`RecordPlaybookDispatch`) copies\n`binding_dispatches`'s unique-ledger conventions with a\n`(playbook_id, playbook_version, event_id, action_id)` key\n(`docs/prds/eda-playbook-engine.md` gap 2).\n\n## What this document does not cover\n\n- An end-to-end operator walkthrough (point an external app at archie,\n build a first playbook) -- open, part of `archie-core-t2db.6`.\n- The playbook YAML / CEL / Module/Channel/Forge action-position work\n (`docs/prds/eda-playbook-engine.md`, `module-position.md`,\n `channel-forge-positions.md`) -- a different, newer mechanism than the\n binding model described here; the two are related but not the same\n system, and this document does not attempt to unify them.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/configuration/",
|
||
"title": "Configuration",
|
||
"section": "Architecture",
|
||
"body": "# Configuration\n\n**Status:** Approved; existing-type assignment is ongoing \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Ownership\n\nConfiguration has two separate responsibilities:\n\n1. `internal/infrastructure/configuration` loads files, environment variables,\n secret references, overlays, and compatibility representations.\n2. Each domain and plugin owns the typed runtime settings required for its\n behaviour.\n\n`internal/app` translates external input into those settings, validates\ncross-capability combinations, and supplies each component only its own\nsettings.\n\nThe following are prohibited:\n\n- passing complete application configuration into a domain;\n- making a domain depend on TOML, YAML, environment-variable names, secret\n references, or a configuration library;\n- defining a central runtime configuration type imported across the system;\n- placing plugin options in an unrelated global configuration model.\n\n## Private input document\n\n`internal/infrastructure/configuration` MAY use a complete decoded document\nwhile discovering files, decoding sections, and applying overlays. It is an\ninput-boundary DTO only.\n\nThe document:\n\n- remains private to the configuration infrastructure;\n- is composed from separately defined feature sections;\n- is immediately translated by application composition;\n- is never returned as the application's runtime model;\n- is never passed through runtime components.\n\nExternal DTOs and domain runtime settings are separate types.\n\nPlugins supply external configuration definitions through a narrow extension\ncontract. The core loader MUST NOT gain a field or recognised filename for every\nplugin.\n\nInvalid configuration input MUST NOT replace active valid settings or terminate\nunaffected capabilities. Configuration changes use the candidate validation,\npromotion, audit, and recovery requirements in\n[safe-change-and-recovery.md](/dev/architecture/safe-change-and-recovery/).\n\n## Dissolution of `internal/config`\n\n`internal/config` has no target location. It MUST be dissolved and MUST NOT\nsurvive as a shared package, configuration domain, or renamed global model.\n\nIts loading behaviour moves to `internal/infrastructure/configuration`. Existing\ntypes and methods are reassigned by behaviour. Repository execution policy,\ndispatch policy, retry policy, ecosystem defaults, tool policy, memory settings,\nprovider settings, and channel settings move to their\nrespective owners.\n\n## Repository settings\n\nThe existing `config.Repo` MUST NOT move intact into a repository domain or\nshared settings package. A managed repository is an application association:\n\n- work intake owns which repository references an identity monitors;\n- task execution owns gates, preflight, test protection, ecosystem behaviour,\n and change limits;\n- task lifecycle owns retry behaviour;\n- scheduling owns concurrency policy;\n- workspace management owns base branch and persistent-storage requirements.\n\nApplication composition correlates these settings by repository reference. No\ndomain receives a universal repository profile.\n\n## Documentation\n\nGenerated configuration reference describes external sections and their owning\ncapabilities. It MUST NOT establish a global Go configuration structure as the\npublic contract.\n\n## Value classification rule\n\nEvery value that controls behaviour MUST have an explicit owner and MUST be\nclassified as exactly one of:\n\n1. **Invariant** — changing it changes the meaning of the program, a wire\n protocol, or a domain concept. It may be a Go constant.\n2. **Runtime setting** — an operator, identity, domain, plugin, or deployment\n may legitimately choose a different value. It is typed configuration owned by\n the affected capability.\n3. **Policy value** — it expresses what is allowed, limited, routed, retried,\n protected, or gated. Its typed definition belongs to the owning domain and it\n is evaluated through the policy system.\n4. **Derived value** — it is computed from other settings and is not\n independently mutable.\n5. **Runtime state** — it changes as the application operates, such as the\n active model for a conversation. It is changed by a domain command and is not\n configuration merely because it can be changed.\n\nAn unclassified behavioural literal is a defect.\n\n`const` MUST NOT be used merely because a value currently has one default.\nConstants are reserved for actual invariants: enum discriminants, protocol\nversions, fixed compatibility tokens, and values whose alteration would change\nthe program's semantics rather than configure an installation.\n\nIn Go, a constant is a compile-time value and does not represent mutable runtime\nstorage. It generally has no address that can be observed or changed.\nArchitecturally, a constant therefore asserts that the value cannot change for\nthe lifetime of the application. Any value that may legitimately change while\nthe application is running, between application instances, or between\ndeployments is not a constant.\n\nA runtime setting may have a default, but the default is not its only access\npath. The owner exposes a typed settings structure, validation, and a\n`DefaultSettings` constructor or equivalent. Private constants used to implement\ndefaults do not turn those defaults into invariants and MUST NOT be consumed\ndirectly by runtime code.\n\nEach runtime setting and policy value MUST declare:\n\n- its owner;\n- type, units, validation constraints, and safe bounds;\n- default and derivation, if any;\n- whether it is startup-only, live-reloadable, or restart-required;\n- whether it may differ by installation, identity, repository, workflow,\n channel, or plugin instance;\n- whether it is secret, sensitive, or safe to report;\n- its current source, effective value, version, and last-known-good version;\n- the command and identity required to change it.\n\n## Current configuration migration ledger\n\nThis ledger records the current meanings hidden inside `internal/config`. It is\nthe source inventory for dissolving that package; rows are split where the\nexisting structure combines different owners.\n\n| Current values | Current location | Target owner and requirement |\n| ----------------------------------------------------------------------------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `WorkDir`, `StateDir` | root `Config` | Workspace/storage infrastructure settings, bootstrap-only. `StateDir` holds the embedded NATS store and endpoint file, task logs and the readiness disk target. Defaults derive from the platform data directory. |\n| `SkillsDir`, `PluginDir` | root `Config` | Skill and plugin infrastructure settings, independently typed. |\n| `PollInterval` | root `Config` | Work-intake scheduling setting; not a daemon-wide constant. |\n| `Label`, `BotUser`, `BotEmail` | root `Config` | Identity and dispatch associations. `BotEmail` may be derived from the selected forge account but remains overrideable. |\n| `MaxRetries` | root `Config`, `Repo` | Retry policy owned by the operation being retried. The duplicate global/repository representation must disappear. |\n| `DiffCapLines` | root `Config` | Agent-change policy value. |\n| `MaxSteps`, `WallClock`, `GateMaxFailures` | `Budgets` | Workflow-execution policy, with units and zero/unlimited semantics made explicit. |\n| `Class`, `APIKeyEnv`, `BaseURL` | `Provider` | Provider instance settings. Secret lookup becomes a `SecretRef`, not an environment variable name leaking into the provider domain. |\n| `Mode`, `Command`, `Env` | `LegacyAgent` | Decode-only compatibility input for the removed executor selector. Delete after the config migration window; it has no runtime settings owner. |\n| `Type`, `Host`, `Token` / `TokenEnv` (older form) | `Forge` | Forge adapter settings. The older `TokenEnv` input is accepted only at the loader boundary. |\n| `Trigger`, `AckReaction`, label names | `Dispatch` | Work-intake and forge presentation policy. Label vocabulary is configurable per forge/identity association. |\n| `Engine` | `MemoryConfig` | Memory engine selection. `internal/domain/memory` is the only engine family; per-engine settings remain owned and validated by that engine. |\n| `Name`, `Transport`, `Command`, `Args`, `WorkDir`, `URL` | `MCPServer` | One typed MCP server instance specification. Transport-specific variants must reject fields belonging to other variants. |\n| `MaxResultChars` | `ToolPolicy` | Tool-execution policy values, not generic daemon configuration. |\n| `URL`, `TokenEnv` | `NATSConfig` | NATS transport settings with a secret reference replacing the environment-name field. |\n| `Image`, `MaxConcurrency`, `MaxUptime`, `VolumeTTL`, `PullPolicy`, `Network` | `ContainerConfig` | Mandatory managed-worker infrastructure settings and resource policy. The legacy `enabled` input is decode-only. |\n| model mappings, email, webhook, Telegram | `ChatConfig` | Separate channel instance settings plus channel-routing associations; there is no chat-wide settings owner. |\n| `AllowedUserIDs`, `Token` / `TokenEnv` (older form), update commands | Telegram config | Telegram adapter settings and access policy. `Token` is the preferred `secret.SecretRef`; `TokenEnv` remains a backwards-compatible fallback. |\n| listen/relay addresses | email config | Email adapter instance settings. |\n| webhook target | `Notify` | Notification adapter instance settings. |\n| web listen address | `Web` | Web transport settings. |\n| `listen` under `[services.state]` and `[services.gateway]` | `ServiceConnection` | The bind address of the process that owns the service. Read only by that process, and defaulted to the address its clients dial so the pair cannot drift. |\n| forge, dispatch, models, providers, containers, memory, tools and repositories | `IdentityConfig` | Existing composite duplicates unrelated settings. Replace it with associations from an `IdentityID` to independently owned configured instances. |\n| owner, name, base, gate, protection, ecosystem, preflight, storage, retry and concurrency | `Repo` | Split across work intake, workspace, workflow execution, policy, gate, and scheduling as described in Repository settings. |\n| `TaskConfig`, `TaskForge` | worker snapshot | Replace with the minimum immutable, versioned WorkflowExecution input assembled by the application. It is not a second configuration model. |\n| `Extra` and arbitrary maps | root/provider config | Allowed only at an extension boundary whose owning plugin/provider supplies schema and validation. Core configuration never consumes an untyped catch-all. |\n\n## Behavioural literals and incorrectly fixed values\n\nThe following current values affect useful behaviour but are not represented as\nowned typed settings or policy. They MUST be classified and migrated before the\nold configuration package is removed. The listed target is the architectural\ndecision; exact setting names are implementation details.\n\n| Capability | Values fixed in current code | Required classification |\n| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Chat agent | maximum steps `8` in `cmd/archied` | Workflow/chat execution policy. |\n| Context compression | enabled, threshold `0.5`, context size `128000`, protect first `3` and last `20`, compression marker, token estimate of four characters per token | Model/context runtime settings. The marker is presentation; the estimate is provider/model strategy, not a universal truth. |\n| Tool guardrails | escalation thresholds `3`, `5`, `10`, `2` | Tool-use policy with named meanings and bounded defaults. |\n| Tool output | per-result cap `50000` characters, no aggregate turn cap, preview `500` characters | Tool-execution resource and presentation settings. Per-result spill/truncation remains; turn continuation is not character- or token-gated. |\n| Pairing/access | code TTL `1h`, maximum pending `3`, rate window `10m`, failures `5`, lockout `1h`, maximum lockout `24h` | Access policy. Code length/alphabet and salt length require an explicit security-constraint decision; they are not automatically valid constants. |\n| NATS daemon consumer | stream `ARCHIE_TASKS`, consumer `archie-daemon`, deduplication `2m`, poll `2s`, deliveries `3`, acknowledgement wait `5m`, inactive threshold `24h`, subjects | NATS topology settings plus delivery policy. Subject grammar or protocol headers may be invariants only if declared as versioned wire contracts. |\n| NATS worker consumer | stream `ARCHIE_TASKS`, consumer `archie-agent`, deduplication `2m`, poll `5s`, acknowledgement wait `30m`, deliveries `3`, inactive threshold `24h`, queue group and retry delay `1s` | Worker transport settings and delivery/retry policy. Shared stream defaults must have one owner rather than two copies. |\n| Worker readiness | readiness timeout `20s`, retry delay `250ms` | Worker lifecycle and retry policy. |\n| Execution/RPC | agent execution `30m`, task-run RPC `60s`, worktree RPC `15m` | Per-operation timeout policy. |\n| Plugin/tool cleanup | plugin rollback `10s`, tool-provider cleanup `10s`, MCP-provider cleanup `10s` | Lifecycle shutdown/rollback policy owned by each engine, with a shared policy vocabulary where appropriate. |\n| Memory read path | per-scope record limit `20`, rendered `\u003cmemory\u003e` block byte cap `8192` | Memory read policy owned by the engine family (`internal/domain/memory`). |\n| Built-in memory store | per-file size limit `100KiB`, query result limit `100` | Split into memory resource policy, input policy, and presentation settings. |\n| Memory safety scanner | compiled prompt-injection and sensitive-data patterns with fixed warn/block consequences | Versioned memory-domain policy definitions. Patterns and consequences must be inspectable and replaceable without editing scanner code. |\n| MCP stdio | initial backoff `500ms`, maximum backoff `30s`, unlimited default retries, send and shutdown `5s` | MCP transport retry, request, and lifecycle policy. Unlimited retries must be explicit rather than a zero-value accident. |\n| MCP SSE/HTTP | reconnect `1s` to `30s`, POST/request timeout `30s` | MCP transport settings and retry policy. |\n| Message framing | MCP frame `4MiB`, worker protocol frame `8MiB`, diagnostic `64KiB`, media/base64 `64MiB` | Resource/security policy unless a published wire protocol mandates the limit. Limits must be configurable only within safe bounds. |\n| Webhooks and web UI | webhook bind `0.0.0.0:8644`, webhook shutdown `5s`, web read-header `5s`, web shutdown `2s` | Transport instance and lifecycle settings. |\n| Telegram | typing refresh `4s`, draft interval `700ms`, update-action TTL `10m` | Telegram adapter presentation and lifecycle settings. |\n| Telegram limits | message length `4000` and callback/update prefixes | Record the upstream platform constraint and version it as an adapter compatibility limit; prefixes may remain wire constants when changing them would invalidate outstanding callbacks. |\n| Containers | stop and pull timeouts `10s` and `30s` | Container lifecycle policy. |\n| HTTP clients | feasibility notification timeout `10s`; other paths use `http.DefaultClient` | Adapter request policy. Every outbound client requires an explicit timeout profile. |\n| Workflow routing | labels `bug`, `feature`, `bootstrap`; workflows `tdd`, `feasibility`, `bootstrap`; fallback `implement` | Agent-system routing policy. Workflow identifiers are references, not compiled routing decisions. |\n| Filesystem/deployment | `.agents/skills`, `.archie/stages`, `.archie/gate.go`, `/data/worktree`, `/data/repo`, cache names/paths and prepared sentinel | Split into deployment settings, repository conventions, and genuine internal protocol markers. Paths an installation may relocate are settings. |\n| Packaged release data | `/usr/share/archie/CHANGELOG.md` | Packaging/deployment settings or paths derived from an installation prefix. |\n| Forge presentation | default label colour `bfd4f2` and state-label colours | Forge adapter presentation settings. |\n| Fallback system prompt | compiled fallback prompt | Agent-system default content, versioned and replaceable through its owning settings; safety constraints remain policy/code. |\n| Default paths and identities | config path, data paths, listen addresses, model roles and workflow names | Typed defaults belonging to their respective owners, never global constants consumed throughout the application. |\n\nThis inventory is intentionally broader than exported Go `const` declarations. A\nliteral hidden in a constructor, timeout, switch, retry loop, path join, prompt,\nlabel map, or client setup restricts behaviour just as effectively as an\nincorrectly used constant.\n\n## Constants that may remain\n\nA value may remain constant only when its owner documents why runtime variation\nwould be invalid. Typical valid cases are:\n\n- domain enum values and state names;\n- a versioned wire-protocol version or framing token;\n- reserved internal headers and callback prefixes that peers must interpret\n identically;\n- compile-time mathematical relationships;\n- sentinel errors and internal type identifiers.\n\nNames such as a stream, consumer, label, workflow, file, directory, network,\nmodel role, timeout, retry count, capacity, size limit, colour, prompt, command,\nor executable are not invariants merely because the current application has only\none of them.\n\nArchitecture tests SHOULD reject exported behavioural constants outside approved\ninvariant packages and SHOULD detect direct consumption of private default\nconstants outside their owning `DefaultSettings` construction.\n\n## Current mutation and reload ledger\n\nThe application currently changes behavioural values through unrelated paths:\n\n| Current change seam | Current behaviour | Required target |\n| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Telegram `/restart` in `cmd/archied` | Re-reads the global document and directly replaces Telegram token and allowed-user IDs; failure retains old values. | Submit a Telegram-settings candidate through the universal change protocol. Validate the complete candidate, promote atomically, audit the acting identity, and retain last-known-good. |\n| Active model selection | `chatModelManager.SetActiveModel` changes a model reference. | A conversation/session command changing runtime state. Validate against configured model instances and audit it; do not rewrite configuration. |\n| Active provider selection | `chatModelManager.SetActiveProvider` changes provider selection. | A conversation/session command changing runtime state, with the same distinction from provider-instance configuration. |\n| Persona selection | `PersonaRegistry.SetActive` mutates an in-memory per-session choice. | Agent-system session state. Persistence and audit requirements are owned by that domain, not the configuration loader. |\n| File and environment overlays | Applied while loading `internal/config`; defaults are filled by mutation in `finalize`. | Configuration-infrastructure input processing that produces separately typed candidates. Defaults and derivations remain visible in provenance. |\n| SIGHUP config reload (2026-08) | Re-resolves the file config, re-applies the database layer over it, validates, then atomically republishes through `config.Holder`. A failed reload, whether the file or the control plane caused it, keeps the running config and records `last_error`/`last_error_at` in `/api/config`. | Candidate promotion with last-known-good retention: the running snapshot is untouched on failure. A bounded observation period and actor audit are not yet implemented. |\n| Control-plane resource command (2026-09) | `ControlPlaneService.Command` validates a whole resource document against its owning feature, then writes it with an audit record (actor, source, request ID, expected version) under optimistic concurrency. Hosted on the State Store's gRPC server; the Web UI and messaging channels are its only clients. See `docs/prds/runtime-control-plane.md`. | Met for validate-stage-promote, actor audit and versioned history. Per-field health observation and validate-before-switch on live apply are not yet implemented. |\n\nAny further mutation seam discovered during migration MUST be added here before\nit is changed. Direct field assignment on a live settings object is prohibited.\n\n## Universal change protocol\n\nChanging configuration is an application command performed by an identity. It\nMUST:\n\n1. resolve the target owner and current version;\n2. parse into that owner's typed settings;\n3. validate syntax, semantic constraints, policy, references, and safe bounds;\n4. stage a candidate without altering the active value;\n5. run owner-defined readiness and health checks;\n6. atomically promote the candidate and record actor, source, reason, diff,\n prior version, and new version;\n7. observe the affected capability for a bounded period;\n8. retain the new version on success, or automatically restore last-known-good\n on failure;\n9. report the rejected or rolled-back candidate without terminating unaffected\n capabilities.\n\nStartup loading uses the same candidate rules. A bad user value therefore does\nnot partially mutate the application or make a previously healthy capability\nunavailable.\n\nSettings that cannot safely reload in-process are marked restart-required. The\nchange may still be validated and versioned immediately, but promotion occurs\nthrough a controlled restart with the same health and rollback semantics.\n\n## Implemented runtime reload and settings writes (2026-09)\n\nTwo mechanisms change behavioural values at runtime: SIGHUP re-reads the file\nconfig, and the control plane owns every database-backed setting. The dashboard\nruntime overlay that used to sit between them was deleted once the control plane\ncovered its surface; `-config-overlay` names a file overlay and is unrelated.\n\n### Published snapshot: `config.Holder`\n\nThe running configuration is a published snapshot held in `config.Holder`\n(`internal/config/holder.go`). Get returns a value under RLock; Set replaces the\nwhole snapshot. A published Config is immutable: reload constructs a fresh one\nand calls Set. `config.Clone` deep-copies every reference-type field so a failed\ndecode cannot mutate the published snapshot's shared maps. `Holder.Get` panics\non a nil receiver — a forgotten `NewHolder` must fail loudly, not boot a daemon\nthat silently runs on an empty config.\n\n### The reloadable-field criterion\n\nThe diff-and-warn on reload asks whether a changed field takes effect without a\nrestart. The criterion is: **a field is reloadable when every consumer re-reads\nit.** That question read as \"what does the daemon re-read\" while the daemon owned\nthe only Holder; it stopped being sufficient when webui began sharing that\nHolder, because webui handlers re-read on every request. The allowlist lives in\n`cmd/archied/reload.go` and is deliberately a deny-by-default list: a field added\nto Config later defaults to requires-restart, forcing whoever adds it to decide\ndeliberately. A reload that changes any non-allowlisted field logs\n\"requires a restart\" with the field names; a fully-applied reload logs Info.\n\n### SIGHUP reload\n\nSIGHUP re-resolves the file config, re-applies the database layer over it\n(`boot.runtimeConfig`, the same call boot makes), validates, and atomically\nrepublishes through the shared Holder. A failed reload keeps the running config\nand records `last_error`/`last_error_at` in `/api/config` so the stale state is\nvisible where the operator is looking. The three poll tickers re-read\n`PollInterval` per iteration and reset on change, so a reloaded interval\ngenuinely takes effect.\n\n### Settings writes go through the control plane\n\nDatabase-backed settings are read and written through\n`ControlPlaneService` on the State Store's gRPC server: catalogue, query,\ncommand, watch. Each feature owns its resource's validation and commands;\nstorage writes versions and audit records only. `docs/prds/runtime-control-plane.md`\nis the specification.\n\nTwo properties matter to anyone changing this area:\n\n- **Bootstrap stays file-owned.** The daemon reads the file config first and\n layers database resources over it at boot (`boot.loadRuntimeConfig`). Values\n the daemon needs before it can reach the State Store — the database path,\n listen addresses, credentials — stay file-only by design.\n- **Reload re-reads them.** SIGHUP resolves the file document and then runs the\n same `boot.runtimeConfig` layering boot does, so a reload cannot revert a\n database-owned setting to its file value. A control plane that cannot answer\n fails the reload rather than publishing the file's values over the running\n ones. Workflow execution budgets arrive on a watch rather than in a resource\n query, so boot records the last published settings and the reload re-applies\n them from there.\n- **Live kinds arrive on a watch.** Provider settings, model role assignments,\n repository policies and scheduling policy are `ApplyMode: live`\n (archie-core-zfb0.1): the daemon keeps a watch per kind and, on a new\n version, re-runs `boot.runtimeConfig` over the published snapshot and\n republishes through `config.Holder`. A candidate the layered document cannot\n run is refused before it is published — last-known-good keeps running and\n the refusal is reported through apply status. The kinds stay\n restart-required while a startup-built component still holds their value\n (tool, plugin, container and channel settings today); a kind joins the live\n list only with a consumer that re-reads it, and the model/provider half of\n it also rebuilds the gateway chat runtime's provider set and the chat model\n list the chat surfaces offer.\n\n## Completion criteria\n\nThe configuration rearchitecture is complete only when:\n\n- `internal/config` no longer exists;\n- every behavioural value has one classification and owner;\n- every current field and hard-coded value in the ledgers has a recorded\n disposition;\n- domains and plugins receive only their typed settings;\n- all changes use semantic commands and the universal change protocol;\n- active, candidate, rejected, rolled-back, and last-known-good versions are\n queryable with secret-safe provenance;\n- generated documentation lists every external key, default, constraint,\n mutability class, owner, and deprecation;\n- architecture tests prevent a new central runtime configuration model, direct\n live-setting mutation, and incorrectly fixed behavioural values.\n\n## Startup policy: missing credential degrades, invalid config is fatal\n\nDecided 2026-07-30. Recovered 2026-08-09 from the pre-migration issue tracker.\n\nA missing **credential** disables a capability. Only an **invalid config** stops\nthe daemon. \"Forge is a feature, not a requirement.\"\n\nBefore `518a1a0`, an unresolvable forge token did `return 1` in\n`cmd/archied/main.go`, so a chat-only user who never configured GitHub got no\nchat, no gateway, nothing — and under a systemd unit with `Restart=on-failure`\nand no start limit it crash-looped every five seconds with the error scrolling\npast unread. Forge construction now lives in `resolveForge`, shared by the\nprimary forge and every configured identity, and degrades to `forge.NewNoop`\nwith a warning. The identity path had the same defect and was worse: one\nidentity's missing credential killed all the others.\n\nMalformed configuration still fails fast in `configuration.Validate`, well\nbefore this point. That distinction is the whole design.\n\nApply the same reasoning to any future capability credential, LLM keys\nespecially: start, report the capability as unavailable, let the operator see it.\nGuarded by `TestResolveForgeDegradesInsteadOfFailing`.\n\n**Known consequence, tracked separately:** `worktree.Manager` still receives the\nempty token, so pushes fail late rather than at startup.\n\n## Config generation belongs to `archied setup`, not `install.sh`\n\nDecided 2026-07-30 ([#368](https://github.com/samcharles93/archie-core/issues/368)).\nRecovered 2026-08-09 from the pre-migration issue tracker.\n\n`install.sh` must stop generating `config.toml`; `archied setup` owns it. The\ninstaller hand-rolling TOML has already drifted from the schema once: the\ngenerated-config branch hardcoded `key = ARCHIE_GITHUB_TOKEN` and\n`host = ${GITEA_URL}` regardless of the answers given, so choosing Gitea produced\na config naming a token the installer never wrote (archied then exited 1), and\nchoosing GitHub pointed `host` at `gitea.example.com`. `61a5b35` patched this\nwith a `forge_block` shell helper shared by both config paths — that helper is a\n**workaround**, deleted by\n[#72](https://github.com/samcharles93/archie-core/issues/72). The durable fix is\nthat the code writing the config is the code that reads it.\n\nWhy Go and not Python: the installer must run where nothing is installed yet, and\n`python3` with `tomllib` means 3.11+, which is not safe on LTS distros. `archied`\nis already being built two steps earlier.\n\n**Key constraint for the port:** `BurntSushi/toml` does **not** survive a\ndecode/encode round trip with comments intact, and `config.example.toml` carries\ndocumentation users are meant to hand-edit. Decide the write strategy\n([#412](https://github.com/samcharles93/archie-core/issues/412)) before writing\nany prompting code. Do not port a separate `config.yaml`/`auth.yaml` secret\nsplit — Archie already has `secret.SecretRef{engine,key}`, so setup writes the\nreference to TOML and the value to the secret engine.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/dashboard-agent-tools/",
|
||
"title": "The dashboard's tools for an in-browser agent",
|
||
"section": "Architecture",
|
||
"body": "# The dashboard's tools for an in-browser agent\n\nThe dashboard registers in-browser tools an agent can call through the page's model\ncontext. Four are read-only — `list_tasks`, `get_task`, `recent_events`,\n`daemon_health` — and three recover stuck work: `retry_task`, `stop_task`,\n`cancel_task`. Each wraps the same `ui/src/lib/api.ts` client the dashboard itself\ncalls, so an agent and a person exercise one path rather than two.\n\nThe four judgement verbs are withheld for the reason `identity.md` gives: an agent\nthat can approve can approve its own work.\n\n## What a tool call can be attributed to\n\nA page can invoke its own tools, so an agent's call and a page script's call reach\nthe server identically — the difference does not exist on the wire, and no header\nrecovers it. A tool call is therefore attributed to the signed-in identity the\nrequest presented, with its source recorded as a hint; it never records that an\nagent acted, and it never records that a human approved. An agent's action is\nattributable only on a surface where the agent presents its own credential.\n\n## Confirmation is a signal, not a control\n\nThe three recovery tools require an explicit `confirm: true`, which makes a\nconsequential call deliberate rather than incidental. It is not a security control:\nan agent can trivially satisfy it, and the browser cannot enforce\n`consequentialHint` at all — Chrome accepts it at registration and omits it from\n`getTools()`, so no client can see what the browser does not surface.\n\nThe boundary that holds is the session. These tools cannot exceed what the operator\nsigned in at the provider is already permitted to do.\n\n## Two ways the surface fails silently\n\n- **A secure context is required.** On a plain-HTTP origin `document.modelContext` is\n absent even with the flag enabled, and nothing reports why. `mkcert` covers local\n development.\n- **The dashboard embeds `ui/dist` into its binary.** Rebuilding the distribution is\n not deploying it: a running process can serve a bundle without tools while the\n built one has them, or serve a bundle older than its own binary.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/dependencies-and-contracts/",
|
||
"title": "Dependencies and Contracts",
|
||
"section": "Architecture",
|
||
"body": "# Dependencies and Contracts\n\n**Status:** Approved foundation \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Dependency direction\n\n```text\ndomain and shared application contracts \u003c- infrastructure implementations\n ^\n |\n application composition\n```\n\nThe following rules are strict:\n\n1. Domains MUST NOT import `internal/infrastructure`.\n2. A domain defines the smallest interfaces required to perform its work.\n3. Infrastructure implements those interfaces and translates external\n representations at the boundary.\n4. Only application composition chooses concrete implementations.\n5. A domain MUST NOT manipulate another domain's state or call its internal\n services.\n6. Cross-domain work uses an explicitly designed command, event, or approved\n shared contract.\n7. Implementation difficulty is not permission to violate a boundary.\n\nBroker messages, database rows, SDK response types, and configuration DTOs MUST\nNOT leak into domain models.\n\n## Commands and events\n\n- An event records something that happened and is owned by its publishing\n domain.\n- A command requests an action and is owned by the domain expected to handle it.\n- Consumers do not redefine an event's meaning.\n- Senders obey the receiving domain's command contract.\n- There is no generic package containing unrelated application messages.\n\n## Event bus\n\n`internal/eventbus` is an approved shared application contract. It defines only\nthe messaging behaviour Archie requires, including publication, subscription,\ndelivery outcomes, and required lifecycle semantics.\n\nIt MUST remain broker-neutral. NATS subjects, JetStream configuration, consumer\ngroups, acknowledgements, reconnection, and serialisation belong under\n`internal/infrastructure/eventbus/nats` unless Archie explicitly requires a\nsemantic guarantee in the shared contract.\n\nMessage schemas stay with their owning domains. The event bus transports\nmessages; it does not own their meaning.\n\n## Wire contracts and generation\n\nCanonical worker and service wire contracts and their authoritative generation\ninputs are owned by the domain or capability that defines their meaning. They\nMUST NOT be centralised in `internal/infrastructure/spec` or another generic\nschema package.\n\nInfrastructure owns:\n\n- generic contract-generation machinery;\n- transport-specific adapters and generated implementations;\n- RPC, REST, NATS, and other external encoding and delivery concerns.\n\nFor example, agent-worker requests and results belong to task execution.\nSubprocess, NATS, or HTTP representations are infrastructure implementations of\nthat contract. The same rule applies independently to forge, persistence,\nworkspace, and other service contracts.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/generated-documentation/",
|
||
"title": "Generated Documentation",
|
||
"section": "Architecture",
|
||
"body": "# Generated Documentation\n\n**Status:** Data generation lives here; rendering and publishing live in the external Astro website\n**Date:** 2026-07-28 (revised 2026-09-25)\n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchie's documentation-generation approach is:\n\n- production Go code, registries, configuration, and CLI behaviour remain the\n source of truth for published contracts;\n- a repository-local Go generator extracts and normalises documentation data;\n- the generator writes deterministic JSON/data files and protocol schemas;\n- the external Astro website consumes generated data and serves the public\n documentation at `offloaded.dev/docs/`.\n\nHugo is not part of the target architecture.\n\nThe documentation system is rooted at:\n\n```text\ntools/\n go.mod\n go.sum\n docsgen/\n main.go\ndocs/\n data/generated/\n public/schemas/\n *.md\n```\n\n### Rendering and publishing — external Astro website\n\nSuperseded 2026-09-12. The repository previously carried a VitePress site\n(`docs/.vitepress/config.mts`, `docs/package.json`, a 1620-line\n`docs/pnpm-lock.yaml`, `docs/pnpm-workspace.yaml`) built and deployed to GitHub\nPages by `.github/workflows/docs.yml`. It was removed as an unnecessary build\nstep: the published site was never stood up, and the Pages target is not\nconfigured.\n\nThe Astro website is maintained outside this repository. This repository owns\nthe Markdown source and generated JSON artifacts; it does not build, preview, or\ndeploy the site. The external site owns page rendering, navigation, and\npublishing.\n\nThe current boundaries are:\n\n| Concern | Owner |\n| --------------------------------------- | -------------------------------------------------------------------- |\n| Reference-page rendering and publishing | External Astro website |\n| Documentation source and generated data | This repository |\n| Generated-data drift detection | `task docs:check`, `task docs:artifact:check`, and `task news:check` |\n| Navigation data | `docs/data/generated/docs.json` and `dev-docs.json` |\n\nChanges to an artifact contract must be coordinated with the external Astro\nconsumer. This repository does not validate the website's rendered links or\nroutes.\n\n## Single source of truth\n\nEvery published capability has one authoritative machine-readable definition\nowned by its domain or application boundary. Generated documentation derives\nfrom those definitions.\n\nGenerated reference includes:\n\n- domain commands and events;\n- Messages and typed message content;\n- Agent and Workflow contracts;\n- WorkflowSteps and Workflow plugin capability contracts;\n- policies and their typed inputs and outcomes;\n- tools and capabilities;\n- CLI commands, arguments, flags, defaults, and help;\n- runtime settings, types, defaults, validation, and secret metadata;\n- public worker and transport contracts;\n- indexes of registered functionality;\n- deprecation information.\n\nHandwritten documents contain:\n\n- architecture decisions and rationale;\n- tutorials and operational guides;\n- examples and walkthroughs;\n- migration guidance;\n- conceptual explanations.\n\nHandwritten documents link to generated reference rather than copying\nauthoritative tables.\n\n## Target generation architecture\n\n`tools/docsgen` is Archie's documentation extractor and normalizer. It produces\nstructured documentation data and protocol specifications. It does not own\npage layout and MUST NOT generate Markdown pages or fragments.\n\nThe generator is divided internally into reusable stages:\n\n```text\nload registries and types\n -\u003e normalize documentation model\n -\u003e validate ownership and identifiers\n -\u003e serialize data and protocol artifacts\n -\u003e compare or write output\n```\n\nThe normalised documentation model prevents each output renderer from\nindependently interpreting Go types. It contains stable documentation IDs,\nowners, names, descriptions, source references, versions, deprecations, fields,\ndefaults, validation, secret classifications, and relationships.\n\nInitial generator targets:\n\n```text\ndocsgen data\ndocsgen asyncapi\ndocsgen all\ndocsgen check\n```\n\n- `data` writes the normalised JSON reference data.\n- `asyncapi` writes protocol schemas and envelopes.\n- `all` generates every committed artifact.\n- `check` generates into a temporary directory and fails on drift.\n\nThe command may use subcommands or an equivalent explicit `--target` flag. It\nMUST retain one normalisation and validation path.\n\n## Target output\n\nDuring adaptation, `docs` remains the staging site. Generated outputs live in\na visibly owned subtree:\n\n```text\ndocs/\n data/\n generated/\n catalog.json\n contracts.json\n docs.json\n dev-docs.json\n cli.json\n configuration.json\n messaging.json\n agents.json\n workflows.json\n policies.json\n plugins.json\n tools.json\n public/\n schemas/\n asyncapi-header.yaml\n archie.yaml\n .tmp/\n```\n\nAll files under `data/generated/` and `public/schemas/` carry generated\nownership metadata where their format permits it and MUST NOT be edited\nmanually. `.tmp/` is generator-owned, ignored, and never committed.\n\n`catalog.json` is the deterministic machine-readable index used for:\n\n- reference navigation and route inputs;\n- duplicate-ID and duplicate-route validation;\n- coverage checks;\n- source-to-data drift diagnostics;\n- future external consumers.\n\nThese files are consumed at render time by whichever renderer is chosen (see\n\"Rendering and publishing\"); the generator's contract is the committed JSON, not\nany particular loader. A renderer reads `data/generated/*.json` directly — Archie\ndoes not use `createContentLoader`-style Markdown collection scanning, because\nthe authoritative inputs are JSON.\n\nThe final move from `docs/` to `docs/` occurs only after Archie content\ncontains intentional Archie pages and the current architecture documents have an\nintentional destination. The staging name MUST NOT leak into public URLs.\n\n## Generation steps\n\n### Step 1: maintain the Archie tools module\n\nKeep `tools/go.mod` isolated from the runtime module:\n\n```go\nmodule github.com/samcharles93/archie-core/tools\n\nrequire github.com/samcharles93/archie-core v0.0.0-00010101000000-000000000000\n\nreplace github.com/samcharles93/archie-core =\u003e ../\n```\n\nKeep generator-only dependencies such as `invopop/jsonschema` and any future\nprotocol-schema serializer in the nested tools module. They MUST NOT enter the\nroot module.\n\n`docsgen` imports Archie-owned contracts and writes normalised Archie data and\nprotocol artifacts.\n\n### Step 2: establish authoritative registries\n\nThe generator consumes domain-owned registries rather than scanning arbitrary\npackages for anything exported.\n\nEach published definition supplies:\n\n- stable documentation ID;\n- owning domain or boundary;\n- name and description;\n- version when versioned;\n- the Go type or declarative definition;\n- lifecycle and deprecation metadata;\n- source reference.\n\nRegistries remain executable application definitions used by production code. A\ndocumentation-only duplicate registry is prohibited.\n\nWhere the migration has not yet created a final domain registry, the generator\nmay use a narrow compatibility adapter over the current registry. That adapter\nhas explicit deletion criteria.\n\nThe initial agent-execution and messaging schema set is such an adapter. It is\ndeleted when those domains expose their production-owned documentation\nregistries; adding another hardcoded type list to `docsgen` is prohibited.\n\n### Step 3: generate AsyncAPI\n\nUse Archie's messaging and worker wire registries as the authoritative inputs.\n\nThe static header continues to own:\n\n- AsyncAPI version;\n- service and protocol description;\n- servers;\n- channels;\n- operations;\n- message envelope placement.\n\nThe generator owns:\n\n- component schemas;\n- discriminated concrete message lists;\n- generated envelopes;\n- rewritten schema references;\n- stable ordering.\n\nThe generated protocol artifact becomes:\n\n```text\ndocs/public/schemas/archie.yaml\n```\n\nThe final owner of each wire registry is the domain or capability defining its\nmeaning. `docsgen` renders the contract; it does not own the message semantics.\n\n### Step 4: add generated documentation data\n\nAdd extractors for the missing reference categories one vertical slice at a\ntime.\n\nRecommended implementation order:\n\n1. CLI commands and flags;\n2. runtime configuration/settings;\n3. Messaging commands, events, and message types;\n4. Agent definitions and public commands;\n5. Workflow definitions, WorkflowSteps, lifecycle, and plugin contracts;\n6. policy definitions and typed outcomes;\n7. tools and capability contracts;\n8. worker and external wire contracts.\n\nEach category extractor contributes:\n\n- stable IDs, owners, routes, and display ordering;\n- fields, types, defaults, and validation metadata;\n- source references;\n- version and deprecation information;\n- relationships to commands, events, policies, and contracts.\n\nThe first vertical slice proves the normalised model and deterministic JSON\nserialisation. Later categories reuse it rather than introducing\ncategory-specific ad hoc output formats.\n\n### Step 5: external Astro renderer\n\nThe Astro website reads `data/generated/*.json`, validates it, and renders\nreference views. Build and deployment remain owned by the website repository.\n\nNavigation and routes derive from `catalog.json`; they MUST NOT be duplicated by\nhand in renderer configuration.\n\n### Step 6: implement deterministic drift checking\n\n`docsgen check`:\n\n1. generates all artifacts into a temporary directory;\n2. normalises line endings and file modes;\n3. compares the complete expected data and schema trees with committed outputs;\n4. reports missing, changed, and obsolete files;\n5. identifies the authoritative definition associated with each mismatch;\n6. exits non-zero without modifying the working tree.\n\nGeneration MUST exclude:\n\n- wall-clock timestamps;\n- absolute filesystem paths;\n- map iteration order;\n- environment-specific values;\n- secrets and resolved credentials.\n\nTwo consecutive `docsgen all` runs MUST be byte-identical.\n\n### Step 7: Astro renderer — external consumer\n\nThe external Astro website consumes the committed generated JSON artifacts and\nserves the documentation under `/docs/` and `/dev/`. Its build and deployment\nare owned by that repository. The consumer MUST:\n\n- read `data/generated/*.json` rather than re-deriving from source;\n- validate internal links, with dead-link failures enabled (no blanket\n ignore list);\n- state its own local preview and build commands;\n- keep `node_modules/` and any build output out of source control.\n\n### Step 8: developer commands\n\n`task docs:generate` writes the committed artifact; `task docs:check` verifies\nit. `task check` runs `docs:check`, so generated drift is gated rather than\nadvisory: a stale artifact fails the gate and names the authoritative Go\ndefinition behind each mismatch.\n\n```sh\ntask docs:generate\ntask docs:check\n```\n\nBoth wrap the generator, which can still be run directly:\n\n```sh\ngo -C tools run -mod=readonly ./docsgen --repo-root ..\ngo -C tools run -mod=readonly ./docsgen check --repo-root ..\n```\n\n`docsgen all` is still target-only: `docsgen data` and `docsgen asyncapi` do not\nexist. Two artifacts are generated today: `contracts.json` from the Go\ncontract types (`docsgen`) and, from the Markdown sources, `docs.json` and\n`dev-docs.json` (`docsite`, one artifact per published set: the public set a\nreader of the product gets, and the development set of architecture and design\nrecords) -- see `task docs:artifact`.\n\n### Step 9: validation and CI\n\n`task check` validates generated contracts and Markdown-derived documentation\nartifacts. The Astro site's rendered routes and deployment are validated in its\nown repository.\n\n### Step 10: content migration\n\nThe Astro site consumes generated artifacts. Remaining content work is to:\n\nStill standing from the original cutover list:\n\n- remove Markdown, examples, specs, generated schemas, and assets that do not\n describe Archie;\n- adapt retained content and prove it against Archie code;\n- update `docs.go` and its tests to embed only intentional runtime\n documentation;\n- decide which current `docs/prds` remain published architecture pages and which\n remain repository-only planning documents — all of them are repository-only\n now, so the distinction is retired.\n\nDocuments MUST describe Archie. Publishing is a separate, undecided concern.\n\n## Hook and CI policy\n\nGenerated documentation is committed.\n\nNo pre-commit hook is installed in this repository, and no CI step runs the\ngenerator. Step 3 below is enforced by `task check`; steps 1, 2, and 4 still\nrely on the author, because nothing observes a commit:\n\n1. detects authoritative definition or generator changes;\n2. runs the generator;\n3. runs `docsgen check` — enforced by `task check`;\n4. refuses the commit when generated changes are unstaged or validation fails.\n\nA commit MUST NOT be created while generated documentation is stale.\n\n## Validation requirements\n\nGenerator tests cover:\n\n- stable ordering;\n- schema reference rewriting;\n- `$defs` flattening;\n- duplicate IDs and routes;\n- registry coverage;\n- comments and descriptions;\n- secret redaction;\n- obsolete-file removal;\n- byte-identical repeated generation;\n- check-mode diagnostics.\n\nRepository-documentation validation covers:\n\n- internal links resolve on the filesystem (repository-relative paths);\n- absence of unrelated product names and URLs;\n- absence of committed build artifacts and installed dependencies.\n\nRenderer validation (frozen dependency installation, production build, expected\noutput routes) is added with the renderer decision and is currently unenforced.\n\nThe root module remains free of generator-only dependencies.\n\n## Completion criteria\n\nThe documentation migration is complete when:\n\n- `tools` belongs to Archie and builds against the working tree;\n- `docsgen` consumes Archie-owned registries and types;\n- AsyncAPI output describes Archie;\n- required reference categories are generated as committed deterministic data;\n- `catalog.json` matches the generated definitions and rendered routes;\n- two consecutive generations are byte-identical;\n- `docsgen check` detects changed, missing, and obsolete artifacts;\n- build artifacts and installed dependencies are absent from source control;\n- `task docs:generate` and `task docs:check` work from a clean checkout;\n- every document describes Archie and every internal link resolves.\n\nRenderer-dependent criteria (rendered reference pages, local authoring server,\ndeployable build, published site) are removed pending the renderer decision.\n\n## Implementation references\n\n- [GitHub-flavoured Markdown specification](https://github.github.com/gfm/) —\n the dialect repository documentation is written in\n- [pnpm](https://pnpm.io/) — required only if a future renderer chooses a\n Node-based toolchain\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/identity/",
|
||
"title": "Identity",
|
||
"section": "Architecture",
|
||
"body": "# Identity\n\n**Status:** Approved foundation; lifecycle details are deferred \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nIdentity answers which persistent actor owns or performed an action. It supplies\nstable cross-domain attribution and lifecycle semantics.\n\nIdentity kinds include:\n\n- the Archie system identity, such as `System` or `archied`;\n- agent or bot identities;\n- service-account identities;\n- future user identities.\n\nA user is another kind of identity rather than a separate foundation concept.\n\n## Ownership\n\n`internal/domain/identity` owns:\n\n- immutable canonical `IdentityID` values;\n- identity kind;\n- lifecycle state;\n- ownership and attribution semantics;\n- identity-related commands and events.\n\nDomains may record an identity reference on their own state and events when\nownership or attribution is required. They MUST NOT receive a complete\nper-identity application configuration bundle.\n\nPersona and communication style are separate from identity.\n\nEvery Agent has its own immutable acting `IdentityID`. Agent actions use that\nidentity for ownership and attribution in the same way an organisation records\nwhich user accessed a resource, changed a permission, or performed another\nauditable action.\n\nAgent and Identity remain separate aggregates. Identity supplies the stable\nactor reference; Agent owns assistant behaviour, memory, specialisation,\nconversations, and capability context.\n\nAgent identities MUST NOT be shared between Agents. Automated WorkflowExecutions\nMAY eventually act through a dedicated service-account identity when a specific\nunattended automation requires it. Such use is an explicit execution decision\nand audit principal, not the default for Agents or a substitute for recording\nthe initiating user when one exists.\n\nA Workflow is owned by an IdentityID and executes as its owner by default.\nFuture delegated execution MAY explicitly assign a different `RunAsIdentityID`,\nincluding a user or service-account identity. Ownership, effective execution\nidentity, initiating identity, and delegation provenance remain distinct audit\nfacts.\n\n## Canonical identity\n\nEvery identity has an immutable `IdentityID`. It is the only value used for\ncross-domain ownership, attribution, routing, persistence, commands, and events.\n\nIdentity kind is explicit:\n\n- system;\n- bot;\n- service account;\n- user.\n\nDisplay names are mutable and MUST NOT be used as identifiers. Forge usernames,\nchannel bot names, Git authors, provider account names, and other external\nidentifiers are bindings associated with an identity rather than identity\nitself.\n\n`System` is represented by a real identity and `IdentityID`. The empty string\nMUST NOT retain special meaning as a legacy system identity.\n\nConfiguration refers to identities by `IdentityID` and supplies external\nbindings and capability settings separately. Renaming an identity or changing an\nexternal account MUST NOT change ownership or historical attribution.\n\nSecret-bearing channel bindings remain configuration concerns rather than\nidentity data. In particular, `[chat.telegram].token` is a\n`secret.SecretRef` resolved by the application through the configured secret\nregistry. It does not change the identity that owns or performs a chat action.\n\n## Persistence\n\nIdentity records are durable domain state. The Identity domain's repository is\nthe source of truth; infrastructure implements its persistence contract.\n\nConfiguration MAY bootstrap a missing identity through an explicit, audited\ncreation operation and MAY propose runtime capability bindings for an existing\n`IdentityID`. Configuration is not the identity registry.\n\nOmitting an identity from configuration stops assembling the omitted runtime\ncapabilities. It MUST NOT delete the identity record, erase history, or make\nhistorical references unresolvable.\n\nDisabling or retiring an identity requires an explicit Identity-domain command.\nRetired identities remain resolvable for historical attribution. A failed\nexternal binding affects capability availability without destroying or silently\nreplacing the identity.\n\n## Lifecycle\n\nIdentity lifecycle is deliberately small:\n\n- `active`: may participate in runtime composition;\n- `suspended`: deliberately prevented from initiating new work while ownership\n and history remain;\n- `retired`: terminal, unable to initiate work, and permanently retained for\n attribution.\n\nAllowed transitions are:\n\n```text\ncreate -\u003e active\nactive -\u003e suspended\nsuspended -\u003e active\nactive -\u003e retired\nsuspended -\u003e retired\n```\n\n`retired` has no outgoing transition.\n\nCapability health is separate from identity lifecycle. Forge, model, channel,\nworkspace, and other bindings report health independently. An active identity\nmay be partially degraded without becoming suspended or retired. Runtime\nsupervision isolates and restores affected bindings through the safe-change and\nrecovery protocol.\n\n## System identity\n\nArchie has exactly one built-in system identity. It is created during initial\nstore bootstrap, is strictly always active, and cannot be suspended or retired.\nIts `IdentityID` and kind are immutable.\n\nStartup, migration, supervision, remediation, and rollback actions performed by\nthe application use the system identity. The system identity MUST NOT be\ndelegated to individual identities. Every user, bot, service account, or other\nactor mutation records the identity that actually performed it.\n\nSystem capabilities may independently become unhealthy or unavailable without\nchanging the system identity.\n\n## Target placement and contracts\n\nThe Identity domain lives at `internal/domain/identity`.\n\nIt owns:\n\n- `IdentityID`, identity kind, display name, lifecycle, version, and audit\n timestamps;\n- create, rename, suspend, reactivate, and retire commands;\n- corresponding lifecycle events;\n- the repository contract required for durable identity records;\n- lookup of current and historical identities.\n\nExternal account bindings and capability settings remain with the domains that\nuse them and reference `IdentityID`.\n\n## Migration constraints\n\nMigration from the current string model MUST:\n\n- bootstrap the permanent system identity;\n- assign stable IDs to configured identities without losing their names;\n- replace empty-string legacy ownership with the system identity;\n- migrate tasks and sessions to canonical identity references;\n- include identity where task uniqueness and transport deduplication require\n independent ownership;\n- route container RPC through the task owner's capability bindings;\n- retain historical resolution after rename, suspension, or retirement;\n- remove identity-string equality from authentication or access semantics.\n\nThis completes the foundation required to restructure the current Identity\nbehaviour. Exact storage schemas and Go method signatures belong to\nimplementation design, not this architecture review.\n\n## Lifecycle\n\nIdentity lifecycle is deliberately small:\n\n- `active`: may participate in runtime composition;\n- `suspended`: deliberately prevented from initiating new work while ownership\n and history remain;\n- `retired`: terminal, unable to initiate work, and permanently retained for\n attribution.\n\nAllowed transitions are:\n\n```text\ncreate -\u003e active\nactive -\u003e suspended\nsuspended -\u003e active\nactive -\u003e retired\nsuspended -\u003e retired\n```\n\n`retired` has no outgoing transition.\n\nCapability health is separate from identity lifecycle. Forge, model, channel,\nworkspace, and other bindings report health independently. An active identity\nmay be partially degraded without becoming suspended or retired. Runtime\nsupervision isolates and restores affected bindings through the safe-change and\nrecovery protocol.\n\n## Explicit non-responsibilities\n\nIdentity is not authentication and is not access control.\n\nThe current architecture does not require authentication or authorisation. Those\ncapabilities MAY be introduced later if deployment or product needs justify\nthem. They MUST remain separate from the identity model.\n\nAn identity reference therefore proves attribution within Archie's trusted\nsystem model; it does not by itself prove credentials or permission.\n\n## Per-identity configuration\n\nValues selected for an identity remain external configuration values. At\ncomposition time they are split into the runtime settings owned by the affected\ndomains and correlated by identity reference.\n\nThe existing `IdentityConfig` aggregate MUST be dissolved. Forge accounts,\nrepositories, models, providers, workspace behaviour, polling, budgets,\nnotifications, and transport namespaces retain their respective owners.\n\n## Audit\n\nConfiguration changes and other state changes are actions attributed to an\nidentity. System-initiated changes use the system identity. Their audit records\nMUST preserve the acting identity, the target, and the resulting change.\n\nThe command ownership and persistence mechanism for configuration changes is a\nseparate configuration-management decision.\n\n## Current implementation\n\nIdentity is not currently implemented as a domain. It is a collection of string\nfields and process-local associations:\n\n- `config.IdentityConfig.Name` is the daemon routing key.\n- `config.IdentityConfig.BotUser` is the external forge and Git author name.\n- `daemon.IdentityRunner` bundles the configured name with a forge client,\n worktree manager, repository list, configuration subset, logger, and polling\n goroutine.\n- `store.Task.Identity` persists the configured name on tasks.\n- NATS task messages copy the same configured name.\n- gateway routing accepts an identity name and uses it to select a repository\n profile.\n- gateway sessions persist `BotUser`, not the configured identity name.\n\nThere is no identity registry, identity record, identity lifecycle state,\nidentity persistence, or identity command/event model. Removing an identity from\nconfiguration makes existing task ownership unresolvable.\n\nAn empty identity string selects the legacy single-identity path. This gives the\nempty string behavioural meaning rather than representing a real system\nidentity.\n\n## Current enforcement and isolation\n\n## Withheld judgement verbs\n\n`approve`, `reject`, `abandon` and `archive` are not exposed to an agent. An agent\nthat can approve can approve its own work, and the one control this surface exists\nto keep is a human accepting it.\n\nThe constraint is attribution, not capability: a task action records the acting\nidentity and the authorising principal, so exposing the verbs requires that an\nagent's action be recorded as an agent's and the authority it used recorded\nseparately. Until a surface carries both, the verbs stay withheld. This is the\nreason the four verbs are absent from the dashboard's in-browser tools, and it is\nwhy releasing them is a change to this boundary rather than a change to the tools.\n\nThe current implementation uses identity equality as an authorisation check for\nchat task approval and cancellation. This conflicts with the approved rule that\nidentity is attribution rather than authentication or access control and must be\nreassigned to an appropriate task-ownership or future access policy.\n\nFocused tests verify that:\n\n- a task resolves the forge client and repository list associated with its\n configured identity name;\n- cross-identity repository profiles do not leak;\n- chat task creation propagates the selected identity;\n- task persistence retains the identity string;\n- chat task control rejects a different identity string.\n\n## Current hazards\n\n- Task uniqueness is `(owner, repo, issue_number)` and excludes identity.\n Two identities discovering the same forge issue cannot own distinct task\n records.\n- NATS deduplication uses `owner/repo/issue` and excludes identity, so discovery\n by a second identity can be discarded as a duplicate.\n- Container-mode forge and worktree RPC servers are wired to the root clients,\n not the task owner's clients. The source documents this as an unresolved\n identity leak.\n- The shared task queue claims the oldest task without identity partitioning and\n relies on later string lookup to recover the correct runner.\n- Sessions use forge bot usernames where tasks use configured identity names, so\n there is no canonical cross-domain identity reference.\n- Identity lifecycle is process lifetime only; there is no disabled, degraded,\n retired, or unavailable state and no audit history.\n\nThe focused identity tests pass. Broader daemon and NATS suites could not open\nembedded listener ports in the current sandbox; their failures occurred before\nidentity behaviour executed.\n\n## Three-layer identity model\n\nAgreed 2026-07-30; the frame for SOUL\n([#68](https://github.com/samcharles93/archie-core/issues/68)) and the curator\n([#435](https://github.com/samcharles93/archie-core/issues/435)).\nRecovered 2026-08-09 from the pre-migration issue tracker.\n\n**Layer 1 — invariants.** Instruction precedence, core rules, and the `\u003ctools\u003e`\nand `\u003cenv\u003e` blocks live in `internal/gateway/templates/archie.md.tpl` and are\nunreachable at runtime by anyone.\n\n**Layer 2 — SOUL.** User-authored identity: agent name, register, warmth.\nEditable without a rebuild.\n\n**Enforcing the Layer 1 / Layer 2 boundary.** SOUL is rendered as the\n`Persona` field of `SystemPromptConfig` (`internal/gateway/prompt.go`) into a\n`\u003csoul purpose=\"identity_and_style\" trust=\"data\"\u003e` block, XML-escaped through\nthe same `xml` template helper used for tool and env metadata, and placed\n_before_ `\u003cinstruction_precedence\u003e` in `archie.md.tpl`. Escaping means SOUL\ntext can never forge a live `\u003ccore_rules\u003e`, `\u003cinstruction_precedence\u003e`,\n`\u003ctools\u003e` or `\u003cenv\u003e` tag -- an attempt renders as inert escaped text\n(`\u0026lt;core_rules\u0026gt;...`), and `\u003cinstruction_precedence\u003e` explicitly tells the\nmodel the `\u003csoul\u003e` block is reference data, not an authority. This is\nguarded by `TestBuildSystemPromptSoulCannotOverrideInvariants` in\n`internal/gateway/prompt_test.go`, which renders a hostile SOUL that attempts\nto countermand `core_rules` and asserts the real invariant tags stay singular\nand their text stays intact. Keep this test in sync if the template's escaping\nor block structure changes -- do not relax it to make a future SOUL feature\n(e.g. curator-proposed edits) fit.\n\n**Layer 3 — representation.** Curator-derived knowledge about the user. It may\nonly **propose** changes to SOUL and never apply them. If derived conclusions can\nsilently rewrite tone, personality drifts and the user loses the control SOUL\nexists to give them. A proposal queue also makes the visibility requirement fall\nout for free, since pending conclusions are inherently reviewable.\n\n**Why SOUL comes first:** tone currently lives in Go string constants\n(`DefaultPersonas` in `internal/gateway/persona.go`), so fixing a rude reply on\n2026-07-29 required editing Go and rebuilding. SOUL is small, fixes a real\ndefect, and defines the boundary the curator must respect.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/messaging-and-work-intake/",
|
||
"title": "Messaging and Work Intake",
|
||
"section": "Architecture",
|
||
"body": "# Messaging and Work Intake\n\n**Status:** Current state researched; foundation decisions in progress \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchie is a personal assistant and general agentic system. Its interactions\ninclude conversation, questions, notes, reminders, goals, message handling,\ntroubleshooting, infrastructure operations, server management, source changes,\nand open-ended future capabilities.\n\nA message to an agent is therefore not inherently a work request, and invoking\nan agent is not inherently a WorkflowExecution.\n\nMessaging and work intake are separate from the Agent System's durable workflow\nexecution model:\n\n- messaging preserves interactions with people and external systems;\n- an agent turn interprets and responds to an interaction using its available\n capabilities;\n- work intake handles the optional promotion of an interaction into durable,\n workflow-backed work;\n- the Agent System owns Workflow definitions and creates WorkflowExecutions.\n\nForge issues, Jira issues, Telegram messages, emails, webhooks, and future\nchannels remain peer external message or artifact sources. None becomes an\nexecution model merely because it can lead to work.\n\n## Confirmed boundary\n\nWork intake MUST NOT create a WorkflowExecution directly.\n\nInbound interactions normally route to the Agent selected by the channel\nbinding. The selected Agent retains its own memory, user knowledge, conversation\ncontinuity, specialisation, and capability context. This continuity is\nindependent of the underlying model or provider.\n\nEach selected Agent acts through its own immutable IdentityID. Interaction and\nresulting action records preserve both that acting identity and the initiating\nuser identity when one exists.\n\nAgent memory is private by default. Interaction context MAY additionally use\nuser-wide memory and intentionally global shared memory, but another Agent's\nagent-wide memory is not part of the selected Agent's context.\n\nMultiple users may interact with the same Agent. The channel binding and inbound\nmessage MUST resolve both AgentID and initiating User IdentityID before\nconversation or memory context is loaded. Sharing an Agent MUST NOT implicitly\nshare one user's private memory with another user.\n\nAn interaction may assemble memory from four scopes: global shared, the selected\nAgent's agent-wide memory, the initiating user's authorised user-wide memory,\nand the private Agent-user relationship memory addressed by both IDs.\n\n## Conversation identity\n\nEvery conversation is identified by the complete tuple:\n\n```text\nAgentID\n+ UserIdentityID\n+ ChannelBindingID\n+ ExternalConversationID\n+ ThreadID\n```\n\n`ThreadID` has an explicit empty value for channels without threads. No element\nof this key may be replaced by a display name, configured bot username, channel\ntype, or other mutable label.\n\nAgent and user memory continue across conversations according to their memory\nscope. Immediate conversation history and delivery correlation remain isolated\nto the identified channel conversation or thread. Moving between Telegram,\nemail, or another channel does not create a new Agent or user, but it does\ncreate a distinct conversation unless an explicit linking operation relates\nthem.\n\nConversation and message events support projections for channel-entry metrics,\nuse cases, volumes, outcomes, latency, capability usage, and other product or\noperational analysis. Metrics are derived read models; they MUST NOT become\nmutable counters embedded in the conversation aggregate or change message\nrouting semantics.\n\n## Canonical messages\n\nEvery inbound, assistant, tool-call, and tool-result message has an immutable\napplication-generated MessageID. Platform identifiers are external correlation,\nnot substitutes for the canonical ID.\n\nMessages are addressable because an interaction is more than text. A message may\ncontain assistant content, reasoning data subject to its retention policy, one\nor more tool calls, a tool result with authoritative execution metadata,\nattachments, reply relationships, edits, or other typed content. Stable\nMessageIDs also allow sessions to branch from an exact prior point.\n\nThe canonical session and message implementation must supply:\n\n- application-generated stable message IDs;\n- user, assistant, and tool roles;\n- assistant tool calls and correlated tool-result messages;\n- authoritative tool-result status, timing, error, truncation, and size\n metadata;\n- session lifecycle and active-request state;\n- session-to-agent-instance and parent-session lineage;\n- durable database round trips and timestamp validation.\n\nThese semantics must use the confirmed Agent, user, channel-binding, and\nconversation ownership model.\n\nThe canonical message store MUST preserve messages as immutable records and\nrepresent later edits, deletions, and branches through related records or\nevents.\n\nArchie requires exact branch-point correlation using stable MessageIDs.\n\n## Conversation branches\n\nA branch is a continuation of a previous conversation at an exact message. A\nchild conversation records:\n\n```text\nParentConversationID\nForkMessageID\n```\n\nMessages remain an append-only linear sequence within each conversation. The\nchild's effective initial context is the parent's history through\n`ForkMessageID`; messages after that point in the parent are not inherited.\n\nAfter the fork, parent and child are isolated:\n\n- child messages, tool calls, tool results, capability actions, and errors do\n not appear in the parent conversation;\n- later parent messages do not silently enter the child;\n- returning to the parent resumes its unchanged history;\n- the child may itself be branched using the same rules;\n- branch lineage remains queryable for navigation, audit, and metrics.\n\nThis supports building substantial context about a subject and then performing\none or more contextual subtasks without polluting the original conversation. For\nexample, a user may branch from a project discussion, create Jira issues in the\nchild using inherited context, and then return to the project discussion without\nadding the Jira tool transcript to the parent.\n\nBranching does not copy mutable message rows. History is resolved through the\nimmutable parent conversation and fork MessageID, followed by the child's own\nmessage sequence.\n\nBranch isolation applies to conversation history, not to Agent learning or\nmemory. A memory action performed in a branch has exactly the same semantics as\nthe same action performed in any other conversation. It writes to its selected\nglobal, agent-wide, user-wide, or Agent-user relationship scope.\n\nMemory provenance MAY retain the source ConversationID and MessageIDs for\nexplanation and audit. Retrieval and effect are determined by memory scope and\naccess rules, not by the conversation from which the memory originated.\nReturning to a parent does not insert the child's tool transcript into the\nparent history, but scoped memories learned in the child remain available to\nfuture retrieval wherever that scope is visible.\n\nChanging sessions or branching therefore controls immediate conversational\ncontext without preventing the Agent from learning durable knowledge.\n\n## Target ownership\n\nThe Messaging domain lives at `internal/domain/messaging`. It owns:\n\n- canonical Messages and typed message content;\n- Conversations, branches, and lineage;\n- channel bindings and external source correlation;\n- replies, acknowledgements, and outbound delivery attempts;\n- message and conversation lifecycle events;\n- domain-defined repository and delivery service contracts;\n- projections required for conversation navigation and metrics.\n\nWork intake is a separate cohesive domain at `internal/domain/workintake`. It\nowns admission, validation, deterministic routing, and accepted work requests\nfor the optional transition into durable workflow-backed execution.\n\nTelegram, email, webhook, forge, Jira, and future messaging systems are\ninfrastructure adapters implementing Messaging contracts. They are users of the\ndomain; they do not own Messages, Conversations, work admission, or\nWorkflowExecution semantics.\n\nThe Agent decides whether an ordinary interaction is answered directly, uses a\ncapability, changes or retrieves Agent-owned information, or proposes durable\nworkflow-backed work.\n\nExplicitly configured automation MAY enter work intake without a conversational\nAgent turn when the external trigger and admission rule already unambiguously\nrequest durable work. This avoids spending an LLM turn to rediscover a\ndeterministic routing decision.\n\nWhen an interaction requires durable workflow-backed execution:\n\n1. a channel adapter produces a channel-neutral inbound message;\n2. the acting agent handles or interprets the interaction;\n3. work intake validates, admits, and routes a proposed work request;\n4. work intake emits an accepted work request with its source correlation,\n acting identity, requested outcome, inputs, and routing evidence;\n5. the Agent System consumes that request, selects a specific Workflow version,\n and creates the WorkflowExecution.\n\nThe accepted work request is a cross-domain handoff, not an alternate execution\naggregate. The Agent System is the sole authority that creates a valid\nWorkflowExecution.\n\nAn interaction MAY instead be completed directly by an agent turn, delegated to\na tool or capability, recorded as a note or reminder, answered conversationally,\nor retained for later context. Those paths MUST NOT require a Workflow or\nWorkflowExecution.\n\nA Workflow MAY add reusable, versioned structure to any of those behaviours.\nScheduled personal-assistant routines, message triage, reminders, infrastructure\noperations, and other capability sequences are valid Workflow definitions. The\ndistinction is whether durable structured execution is useful, not whether the\nsubject originated from a forge.\n\n## Canonical execution language\n\n- **Workflow**: reusable, versioned behaviour definition.\n- **WorkflowExecution**: durable execution of a specific Workflow version.\n- **WorkflowStep**: a defined operation in a Workflow.\n- **StepExecution**: the recorded execution and outcome of a WorkflowStep.\n- **Attempt**: a retry within the same WorkflowExecution.\n\nExternal issues, messages, requests, goals, and notes are not synonyms for these\nconcepts.\n\n## Current implementation\n\nCurrent intake is fragmented:\n\n- Telegram, email, and webhook adapters translate payloads into\n `gateway.Message`, while an unused richer `MessageEvent` contract exists\n alongside it.\n- `gateway.Router` combines local channel commands, conversational LLM routing,\n and direct task lifecycle operations.\n- Telegram is the only channel with production LLM and durable conversation\n wiring. Email and webhook free text currently have no LLM responder.\n- Forge polling bypasses the gateway and directly enqueues store rows or NATS\n task messages.\n- Chat `/spawn` directly creates a `store.Task` and invents a synthetic forge\n issue number to satisfy the existing persistence key.\n- Forge replies and chat approval commands implement separate waiting-human\n paths.\n- Message delivery, acknowledgement, retry, deduplication, and source\n correlation are platform-specific side effects rather than one durable model.\n- `store.Task` collapses an external source reference and durable execution\n state.\n\n## Intake provenance\n\nA task's origin determines which enqueue path created it, though every origin\nconverges on the same `tasks` table and the same `Route`/`Run` lifecycle once\nqueued.\n\n- Forge-sourced work -- discovered by polling or, in single-identity\n deployments, delivered by the forge webhook receiver (`internal/forge/webhook`)\n -- enqueues through the same `workintake.TaskEnvelope` and `PublishTask` path.\n `TaskEnvelope.IdempotencyKey` is keyed on `owner/repo/number` and not on the\n delivery source, so a webhook-delivered issue and a later poll of the same\n issue dedup through `PublishUnique` instead of double-enqueuing.\n- Chat-spawned and playbook-binding-dispatched work enqueues through\n `EnqueueChatTask`/`EnqueueBindingTask` (`internal/infrastructure/postgres`), stamped with\n `binding_id`/`binding_version` for provenance.\n\nWebhook intake is opt-in per deployment (`Forge.Intake` = `poll` (default),\n`webhook`, or `both`) and single-identity only: a config naming\n`webhook`/`both` alongside `[[identities]]`, or setting intake on a\n`[identities.forge]` entry, is rejected at validation rather than started with\nthe setting ignored. One receiver binds one secret and one dispatch predicate,\nso it cannot judge a second identity's events by the right rules. A\nmulti-identity deployment polls per identity until per-identity intake lands.\nThe decided contract is `docs/prds/event-sources-and-reactions.md`; the binding\nmodel and its threat model are `docs/architecture/bindings.md`.\n\n## Current hazards\n\n- Treating every agent interaction as a work request would remove Archie's\n personal-assistant role and force conversation and immediate capability use\n through an inappropriate workflow lifecycle.\n- Treating every accepted work request as a WorkflowExecution would let intake\n or channel code select versions and construct invalid execution state outside\n the Agent System.\n- Source identity is incomplete: sessions use configured bot usernames, tasks\n use identity strings, and external message IDs are generally absent.\n- Database and NATS deduplication are based on forge coordinates rather than a\n channel-neutral source-message reference.\n- Replies and acknowledgements have no durable delivery-attempt lifecycle, so\n partial success and retry behaviour differ by adapter.\n- Production code has two competing channel-neutral message contracts.\n- The existing Archie session implementation lacks the required tool-call\n semantics and exact branch lineage and MUST be brought to the canonical\n contract.\n\n## Decisions still required\n\n- the cohesive owner and final location of messaging behaviour;\n- the cohesive owner and final location of work-intake admission and routing;\n- the canonical inbound message, conversation, and source-correlation model;\n- the boundary between an agent turn, direct capability use, and a proposed\n durable work request;\n- the exact relationship between an Agent, its acting IdentityID, its user, and\n its channel bindings;\n- outbound response, acknowledgement, delivery-attempt, retry, deduplication,\n and failure semantics;\n- the smallest accepted-work-request contract consumed by the Agent System;\n- migration of forge polling, replies, chat commands, email, webhook, Telegram,\n and Jira integrations.\n\n## Non-goal: semantic search over chat messages\n\nSemantic/vector search over chat messages is an **explicit non-goal** until usage\ndata shows a real need.\n\nMessage search is lexical only: session-scoped PostgreSQL full-text queries paged via\n`MessageQuery{Query,Limit,Offset}` → `MessagePage`. Do not add embeddings\nconfiguration or vector columns to message records.\n\nThe rationale is cost without a demonstrated use case. Measured 2026-08-03 with\n`embeddinggemma:300m` — note this was never an infrastructure problem, the model\nwas already pulled and `ai-sdk/provider/ollama` already has `embed.go`:\n\n- 121 ms median warm per message on the write path, 12.5 s cold;\n- 768 dims = 3072 bytes per vector, roughly 15× a typical message payload;\n- chronological reads would carry vector data they do not use on the hot path\n of every turn;\n- similarity search requires an explicit indexing and retention design.\n\nIf this is revisited, vectors belong in a sibling table so chronological reads\nnever load them.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/migration-decisions/",
|
||
"title": "Architecture Migration Decisions",
|
||
"section": "Architecture",
|
||
"body": "# Architecture Migration Decisions\n\n**Status:** Active migration inventory \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nThis document records what remains to be decided before Archie's current\nimplementation can be migrated into the approved domain-oriented structure.\n\nThe product and core domain model are not being redesigned. Existing Archie\nbehaviour and the existing session implementation are the migration\nbaseline. Remaining decisions MUST be grounded in current packages, consumers,\ntests, persistence, and runtime wiring.\n\n## Confirmed domain packages\n\n```text\ninternal/domain/\n identity/ (approved, not yet materialised as a package)\n agent/ (approved, not yet materialised as a package)\n workflow/\n messaging/ (canonical Message/Conversation types only; gateway migration pending, see section 2)\n workintake/\n plugin/ (approved, not yet materialised as a package)\n curator/\n mapping/\n scheduling/\n```\n\n- `identity` owns persistent actors, lifecycle, ownership, attribution, and\n audit identity.\n- `agent` owns persistent assistants, specialisation, instructions,\n model-independent continuity, capability context, and associations with\n identity and memory.\n- `workflow` is a separate domain. It owns reusable, versioned Workflow\n definitions, WorkflowSteps, WorkflowExecutions, StepExecutions, Attempts,\n outcomes, and typed Workflow plugin contracts.\n- `messaging` owns canonical Messages, Conversations, branches, channel and\n source correlation, replies, acknowledgements, delivery attempts, events, and\n projections.\n- `workintake` owns admission, validation, deterministic routing, and accepted\n work requests for the optional transition into durable workflow-backed\n execution.\n- `plugin` owns generic plugin identity, discovery, compatibility, and lifecycle\n mechanics. It remains metadata-only; capability semantics and typed\n registration belong to the owning domain.\n- `curator` owns the curator engine family: long-running agent loops that\n reason over memory and perform maintenance tasks on a declared shape\n (check-in interval, tool set, attached memory engine). See epic\n archie-core-yp9.\n- `mapping` owns the payload field-mapping vocabulary: binding named fields to\n JSON paths inside a captured webhook payload, and detecting drift. See\n `docs/prds/payload-field-mapping.md`.\n- `scheduling` owns the ticker engine: deciding when a scheduled job runs and\n with what concurrency. It declares `JobSource`/`Runner` contracts and owns\n no storage or delivery. See epic archie-core-1786637496320-249-be6d9a20.\n\n**Reconciliation (2026-08-31):** `curator`, `mapping`, and `scheduling` existed\nin the tree without being added to this list when their epics landed. They are\nconfirmed as legitimate, narrowly-scoped domains under the same rules as the\noriginal six — each owns one capability family, declares typed contracts, and\nhas no cross-domain reach. This list was out of date, not the tree.\n\nConversely, `identity`, `agent`, `messaging`, and `plugin` are approved target\ndomains that do not yet exist as packages. Their behaviour currently lives in\npre-migration packages (see \"Complete package destination map\" below for the\nfull inventory). Do **not** create empty stub packages for them ahead of an\nactual migration PR — an unused stub is dead code and gives no test coverage.\nEach materialises when its corresponding migration decision below is resolved\nand implemented in the same change.\n\nMemory's final package placement remains undecided pending focused review of the\nexisting implementation.\n\nThe following are approved shared or composition packages, not domains:\n\n```text\ninternal/eventbus/\ninternal/policy/\ninternal/infrastructure/\ninternal/app/archied/\ninternal/app/agentworker/\n```\n\n## Remaining migration decisions\n\n### 1. Complete package destination map\n\nEvery current package MUST receive one explicit disposition:\n\n- move into a confirmed domain;\n- move under infrastructure as an implementation of a domain-owned contract;\n- move into application composition;\n- merge with an identified duplicate;\n- remain as an approved shared application contract;\n- move to a plugin, extra, deployment, example, or generated artifact;\n- be deleted after its replacement is proven.\n\nThe largest unreviewed areas include memory, tools and capabilities, workspace\nand worktree behaviour, gates, storage, events, web UI, containers, RPC\npackages, release management, and scheduling.\n\nThe destination matrix MUST name the behaviour owner, target path, dependencies\nto remove, state and contracts to preserve, migration prerequisites, and\ndeletion criteria.\n\n**`internal/webui` / UI Service — RATIFIED Phase 3 disposition (2026-09-07).**\nThe current owner is the in-process HTTP/SSE dashboard: SPA hosting, operator\nauthentication, UI response projection, configuration reads and mutations,\ntask/control APIs, and direct adapters to State Store, channel, event/log,\nmemory, curator, workflow, forge, and Gateway runtime values. The target owner\nis a standalone UI application process; `internal/webui` remains its HTTP\ntransport adapter, while Gateway, State Store, configuration, Work Intake,\nMessaging, and other capability owners expose the contracts behind those\nroutes. The migration preserves retained route schemas, auth/CSRF behaviour,\ntask/capture/mapping/binding/session continuity, SSE replay semantics, and\ncommitted generated SPA assets. Prerequisites are the ratified UI boundary,\nroute-owner inventory, narrow remote adapters, configuration/auth/readiness\ncontracts, and process/conformance tests. Remove the shared live\n`config.Holder`, direct event/log bus, daemon stopper, forge client, channel\nmanager, capability registries, composite store access, and in-process UI\nlistener only after the focused criteria in\n[`docs/prds/ui-service-boundary.md`](/dev/prds/ui-service-boundary/) pass in\nproduction composition. The old construction and listener are deleted in the\nsame cutover change; no second UI authority remains.\n\n### 2. Session and Messaging migration\n\nThe existing session implementation is the migration baseline. The migration must\ndecide:\n\n- which session and message components are retained or adapted;\n- how Agent, user, channel, Conversation, and branch ownership are added;\n- how Archie's parallel gateway session implementation is replaced;\n- how canonical Messages become immutable persisted records;\n- how existing sessions and messages are migrated;\n- how outbound delivery, retry, acknowledgement, deduplication, and failure\n semantics are represented;\n- how current Telegram, email, webhook, forge, and Jira integrations migrate to\n Messaging infrastructure adapters.\n\nThe following are already decided and MUST NOT be reopened without contradictory\nimplementation evidence:\n\n- the composite Conversation identity;\n- immutable MessageIDs;\n- typed tool-call and tool-result messages;\n- exact branch lineage through ParentConversationID and ForkMessageID;\n- parent and child transcript isolation;\n- normal scoped-memory behaviour regardless of the originating Conversation;\n- Messaging and Work Intake ownership.\n\n**Decision (2026-08-31), resolving the open questions above:**\n\n`internal/gateway/` is the \"parallel gateway session implementation\" this\nsection names. It is retained as-is for now — a big-bang rewrite of live\nsession/compression/approval/branch logic across Telegram, web UI, email, and\nwebhook channels in one change is exactly the \"convoluted to satisfy edge\ncases\" failure mode CLAUDE.md's Revert-on-Flail rule warns about, and would\nrisk silent behavioural regressions in the one path every user-facing message\ntravels through.\n\nInstead, `internal/domain/messaging/` is materialised now with the pieces\nthat are already decided and don't require touching gateway: the canonical\n`Message` and `Conversation` types (composite Conversation identity, immutable\n`MessageID`, typed tool-call/tool-result message variants, branch lineage via\n`ParentConversationID`/`ForkMessageID`). These are pure domain types with no\ngateway dependency — a new package, not a rewrite of an existing one.\n\nMigrating `internal/gateway/`'s session, compression, approval, and branch\nlogic onto these types is real, cross-channel-risk work and is **not** done in\nthis pass. It is tracked as separate, appropriately-scoped bd issues (see\n`bd list --label messaging-migration`), sequenced channel-by-channel so each\nlands with its own behavioural verification instead of one large diff no one\ncan safely review.\n\n**Amended 2026-09-15.** `internal/infrastructure/gatewayrpc.StoreClient` is the\nadapter this migration will need: it implements `messaging.SessionStore` over\nthe chat service's session-store RPCs and is proven by\n`TestStoreClientPreservesCanonicalRecords`. It is deliberately unwired.\nProduction `archied` dials the standalone gateway for `ChatContract` only\n(`composeChatContract` → `gatewayrpc.Dial` returns a `*Client`, not a\n`*StoreClient`), and the Gateway's session store is PostgreSQL via\n`gateway.NewPostgresSessionStore`. The test comment that says it mirrors production\ndescribes the target, not today's tree. Kept rather than deleted because the\nmigration it serves is deferred by this very section, and re-deriving a tested\nwire adapter once that lands is pure cost.\n\n**Phase 4 Messaging Service extraction (2026-09-19, `archie-core-8cda.6.1`).**\nThe process boundary for external channels (Telegram, email, webhook) is ratified\nin [`docs/prds/messaging-service-boundary.md`](/dev/prds/messaging-service-boundary/).\nMessaging is extracted into `cmd/archie-messaging` / `internal/app/archiemessaging`,\ncalling Gateway's `messaging.ChatContract` exclusively over gRPC. Channel frontends\nhold no direct database handles, daemon state, or model runtimes.\n\n**Telegram operator surface after extraction (2026-09-19, `archie-core-8cda.6.6`).**\n`setupTelegramGateway` wired eight `telegram.Gateway` seams the daemon could\nsatisfy from its own process. Each is resolved by where the fact it needs is\nactually owned, not by where it used to be built:\n\n| Seam | Resolution |\n| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `Version` | Sourced from the Gateway via `ChatSnapshot.Version`. The Messaging Service never stamps or infers a build version; `cmd/archie-messaging` deliberately takes no version ldflags, so it cannot report a partial upgrade as a matched one. |\n| `Updates` (`/update`) | Built in the Messaging Service from `[chat.telegram].update_check_command` / `update_install_command` — its own configuration. `Enrich` is dropped: `componentInstallTypeEnricher` reads `[nats]`, which this service must not decode, so the check command's own reported install type stands unmodified. |\n| `UpdateReportPath` | A local state file under `work_dir`, keyed by the same `sha256(bot_user)[:8]` scheme the daemon used, so a pending report is found rather than written afresh. |\n| `ReleaseAnnouncements` | **Deliberately left nil**, for the same reason as `RunningVersions`. `releaseannounce.Announcer` announces nothing unless each `Component` carries a parseable version, and the only versions this service can obtain are the Gateway's own build stamps, not archied's. Wiring the announcer without them yields a no-op that reads as configured, so it stays unwired until component self-reporting exists. |\n| `SetShowToolCalls` | Projected from `[chat].show_tool_calls`. |\n| `Reload` | Local: re-resolves the token and allowlist from this service's own config file and overlay. Reload was never a daemon fact. |\n| `RunningVersions` | **Deliberately left nil.** It exists to turn an installer's claim into a checked one, and only a component's own compiled-in build can vouch for it. The Messaging Service knows neither archied's build nor the observed archie-agent version (`daemon.AgentStatus`), and the Gateway's own `gatewayVersion` is a different binary's stamp — reporting it as the daemon's would manufacture exactly the false success the check exists to catch. Update reports therefore relay as unverified claims until a component self-report contract exists. |\n| `Dangerous` | Stays nil. No daemon composition ever set it; `/rollback` and `/stop` reported \"not configured\" before the extraction and still do. |\n\nThe Messaging Service consequently reads `work_dir`, `bot_user`, `[chat]` and\n`[health]` in addition to the PRD's `[chat.*]` / `[services.gateway]` list.\n`[health]` is the daemon's health URL, which `releaseupdate.CommandInstaller`\npolls from outside to confirm an update came back up; the rest of the PRD's\nprohibition list is untouched.\n\n**Phase 4 messaging severance: what must move, and the decision the PRD does\nnot cover (2026-09-22, `archie-core-1ng1`).** The PRD's deletion gate cannot\npass today. `cmd/archie-messaging/architecture_test.go` does not exist, and the\ncheck that section specifies -- `go list -deps ./cmd/archie-messaging` links\nzero banned runtime packages -- reported four at the time (the task store was\nthen `internal/store` over `modernc.org/sqlite`; the gate now bans\n`modernc.org/sqlite`, `github.com/jackc/pgx` and\n`internal/infrastructure/postgres` in its place):\n\n| banned package | PRD category |\n| -------------------------- | -------------------------------------------------------- |\n| `modernc.org/sqlite` | State Store |\n| `internal/store` | State Store |\n| `internal/domain/workflow` | Workflow engine |\n| `internal/agentexec` | Workflow engine, via `internal/domain/workflow/agent.go` |\n\nAll four arrive through one import: `internal/app/archiemessaging/run.go`\nimports `internal/app/controlplane`, whose own dependency set is exactly those\nfour. The Messaging Service uses three symbols from that package\n(`NewRPCClient`, `RuntimeChatConfig`, `ChannelSettingsKind`); the package also\ncarries the store-backed `ResourceStore` server, the `Definition` registry and\nthe workflow step vocabulary. Sever the package. Amending the gate to tolerate\n`modernc.org/sqlite` and `internal/store` is rejected: a boundary you whitelist\naround is no longer a boundary.\n\n**What moves.**\n\n- Every kind constant. They are strings with no dependency, and the client names\n them.\n- The channel-settings vocabulary and its decode: `channelSettings`,\n `telegramSettings`, `webhookChannelSettings`, `emailSettings`,\n `rateLimitSettings`, `channelDuration`, and `decodeRenamingLegacyKeys`.\n- The generic client: `Client`, `NewRPCClient`, `RuntimeConfig` with\n `runtimeConfigFrom`, `runtimeChatConfigFrom`, the other `runtime*From`\n projections, and the `resourceReader` interface.\n- The persona watch (`WatchPersonas`, `AppliedPersonas`), which needs only\n `internal/domain/agent`.\n- `executionSettingsDocument`, the document alone.\n\n**What stays, and why.**\n\n- The store-backed server: `Server`, `NewServer`, `ResourceStore`,\n `ImportConfig`, `SeedSkip`, the `Definition` registry, and every other kind's\n seed, validator and `Normalize` hook.\n- `WorkflowDefinitionsClient` and `stepRegistry`, which decode through\n `internal/domain/workflow`.\n- `WatchWorkflowExecutionSettings` and `decodeSettings`: `decodeSettings`\n returns and validates `workflow.ExecutionSettings`, so any package holding it\n links the workflow domain and therefore `internal/agentexec`. This is why the\n client cannot move as one package -- see open question 2.\n- The schedules document, whose type is `scheduling.JobSpec` in\n `internal/domain/scheduling` -- the vocabulary moved there when the legacy\n file-backed cronstore was deleted, which is also what satisfies the rule\n that a domain vocabulary package may not import infrastructure.\n- The schema derivation, which describes the documents the server advertises.\n\n**Placement as landed (2026-09-22, `archie-core-1ng1`).** One new package,\n`internal/infrastructure/controlplanerpc`, holding both the vocabulary (kinds, the\nchannel-settings document and its decode) and the client\n(`NewRPCClient`, `Query`, `RuntimeChatConfig`, the exported `ResourceReader`, the\nwire sentinels and `ClientError`). A second `internal/domain/controlplane` package\nfor the vocabulary was planned and then measured out of the design: the vocabulary\nhas exactly one consumer, the client that projects it, and the store-backed side\nreads the same definitions through type aliases rather than a copy -- so a separate\ndomain package would have been a package with one consumer and a second name for\none thing. What made that safe is a measurement, not a preference:\n`internal/app/controlplane` was the ONLY direct import of the Messaging Service\nthat dragged a banned runtime (`gatewayrpc`, `staterpc`, `applystatus`, `messaging`,\n`configuration`, `readiness` and `health` were all clean), so severing that one\ndependency is sufficient, and the severance needed no third package. Promoting the\nvocabulary to a domain package stays open for the day a second consumer appears.\n\n**Settled by the implementation.** Two of the four open questions this decision\noriginally left are answered, in both cases by measuring rather than by preference:\n\n- The client did NOT become two packages. `WatchWorkflowExecutionSettings` and\n `WatchPersonas` stayed on `internal/app/controlplane`'s `Client`, which now\n _embeds_ the new one, so the daemon's validated watch is still defined once and\n the read path still has one implementation -- which is what the second package\n was for -- with one fewer package to keep in step.\n- The schedules and workflow documents stayed put, and\n `internal/domain/workflow/task` is linked deliberately: it carries the Task\n status and view vocabulary, its own dependency set holds none of the banned\n runtimes, and `cmd/archie-messaging/architecture_test.go` carries it as an\n exception checked in both directions (the same entry, for the same reason, as\n `cmd/archie-ui/architecture_test.go`).\n\n**Still open.**\n\n1. Does the server also leave `internal/app/`? `organisation.md` reserves\n `internal/app/` for wiring and composition, and a store-backed gRPC server is\n infrastructure -- but the gate does not require the move, and it would multiply\n the diff across archied's wiring and the State Store binary. Tracked as\n `archie-core-8f4n`.\n2. Does `internal/domain/workflow`'s contract/engine split happen, and when? A\n contract type (`WorkflowDefinitionCollection`) drags `internal/agentexec` only\n because `agent.go` sits beside it. Independent of this severance, tracked as\n `archie-core-inh6`, and the shape `archie-core-8cda.7` will meet again.\n3. Does the vocabulary move to a domain package when a second consumer appears?\n\n**Both consumers keep compiling.** archied imports the new vocabulary for the\nkinds it names and the new client package for `RuntimeConfig` and the persona\nwatch, and keeps `internal/app/controlplane` for the server, the registry,\n`ErrVersionConflict` and the workflow-aware pieces. The State Store binary\ncontinues to call `NewServer` and `ResourceStore` unchanged. The Messaging\nService then imports only `internal/infrastructure/controlplanerpc`,\n`gatewayrpc` and `staterpc`, so `go list -deps ./cmd/archie-messaging` loses all\nfour banned packages and the gate can land green.\n\n**Channel state to the dashboard: one administrative surface, following the\nconfig-snapshot precedent (2026-09-22, `archie-core-8cda.6.8`).** The Messaging\nService now records its channels' lifecycle -- `internal/channels/status.Manager`,\none entry per composed channel, written by each channel's own report through\n`channels.Lifecycle` -- and nothing reads that across a process boundary.\n`/api/channels` answers with nothing because `webui.Server.Channels` has no\nproduction source, and POST `/api/channels/{id}/reload` answers 501 for the same\nreason. The carrier is the State Store, and the shape is the one\n`archie-core-ymut` already established for config snapshots\n(`docs/prds/state-store-contract.md`, \"the one surface whose writer is the daemon\nand whose reader is the UI process\"): a pair of administrative RPCs, deny-by-default\nunder `authorizesTaskScopedCall`, so a task-scoped grant reaches neither.\n\nWhat that decides, so the implementation need not invent it:\n\n- A channel's state is **runtime** state, not a settings document. It does not\n belong in the `ControlPlaneService` resource kinds, which are validated,\n versioned documents seeded from config; a channel that is failed is not a\n document anybody edits.\n- The reader is the UI process, which already dials the State Store and pumps its\n cursor, so the surface needs no new transport in either direction.\n- Adding the pair changes the contract's RPC count, so\n `docs/prds/state-store-contract.md`'s grouped table gains a group in the same\n change that adds the RPCs -- the table is the contract's own index, and a\n surface it does not list is a surface nobody finds.\n\n**Reload gets its own surface, with its own lifetime (decided 2026-09-22).** Reload\nruns the other way -- the UI asks the service to reload a channel -- and the two\ncandidates were a request surface the service observes and clears, or a field on\nthe status surface the service watches. It is the first. The reasoning, recorded\nbecause it would look arbitrary later:\n\n- A request queue and a status report do not share a lifetime, so they should not\n share a surface. A pending request is transient and has an outcome; a channel's\n state is durable and has a current value.\n- A \"please reload\" field on the status resource makes READING status capable of\n RETRIGGERING an action, and makes a transient request indistinguishable from\n state the moment anything polls it.\n- It would also invert the writer/reader relationship the precedent establishes:\n the service would have to clear a field on an object whose whole contract is\n that the service reports it.\n\nSo the status surface stays writer-is-service, and the request carrier is its own\nobject with its own states.\n\n`archie-core-8cda.6.9`'s component self-report is the **same shape**: a process\nreporting something about itself for the UI to read. Whether it shares this\nsurface or gets its own is a decision for `6.9`, and the one outcome to avoid is\ntwo homes for \"what a component reports about itself\".\n\n### 3. Identity data migration\n\nThe migration must define:\n\n- the durable identity schema and repository implementation;\n- bootstrap of the permanent system identity;\n- stable IdentityIDs for existing configured Agents;\n- migration from configured identity names, empty-string ownership, and bot\n usernames;\n- preservation of historical attribution;\n- identity-aware uniqueness and transport deduplication;\n- correction of container and RPC capability resolution so execution uses the\n owning identity's bindings;\n- removal of identity-string equality from access-control behaviour.\n\nExact schema and method signatures are implementation decisions. The approved\nIdentity lifecycle and attribution semantics are fixed migration constraints.\n\n### 4. Workflow migration\n\nThe current implementation must be transformed deliberately:\n\n```text\nstore.Task -\u003e WorkflowExecution\nmutable Stage -\u003e StepExecution history\nRetryCount / Attempt -\u003e Attempt records\nworkflow.Registry -\u003e versioned Workflow definitions\ncurrent stages -\u003e WorkflowStep definitions or Workflow plugins\n```\n\nThe migration must decide:\n\n- Workflow definition and version persistence;\n- the smallest accepted-work-request contract consumed from Work Intake;\n- the enforced WorkflowExecution state machine (proposed in\n `docs/prds/execution-tree-state-machine.md`);\n- atomic state and domain-event persistence;\n- migration of existing task rows and transition history;\n- disposition of current skill, Yaegi, and stage-plugin implementations;\n- the versioned worker-handoff contract;\n- scheduler, claim, retry, crash-recovery, and waiting-human cutover.\n\nWorkflow remains a separate domain from Agent. A user or Agent may define or\ninvoke a Workflow, but Workflow semantics and plugins remain Workflow-owned.\n\n### 5. Memory placement and storage\n\nThe four required scopes are fixed:\n\n- global shared;\n- Agent-wide;\n- user-wide;\n- Agent-user relationship.\n\nA focused review of `internal/memory`, its providers, consumers, persistence,\nand tests had these decisions:\n\n- `internal/domain/memory` versus ownership within the Agent domain — settled\n (2026-09-02, below);\n- the authoritative memory record and revision model — settled by the PRD\n (2026-09-16, below);\n- provider and infrastructure boundaries — settled by the PRD (2026-09-16,\n below);\n- retrieval and access enforcement — settled by the PRD (2026-09-16, below);\n ranked retrieval remains open (below);\n- migration of existing memory data — open: existing data is not migrated;\n- provenance representation — settled by the PRD (2026-09-16, below).\n\nConversation branches do not create another memory scope. A memory action has\nthe same scoped effect regardless of which Conversation originated it.\n\n**`internal/domain/memory` ownership — DECIDED (2026-09-02).** The typed\nengine family lives at `internal/domain/memory`\n(archie-core-1786637490069-11-e245c170), mirroring `internal/domain/curator`'s\ncontract/registrar/registry split. Not owned within the Agent domain: memory\nis consumed by more than agent turns (curators, per archie-core-1786637499636)\nand the plugin engine rule requires its own family, not a capability bolted\nonto another domain's contract.\n\n**Authoritative record and revision model for the builtin engine — DECIDED\n(2026-09-02),** scoped to unblock archie-core-1786637499480 (ship the\nfile-writing store as the default engine). This decides only the builtin\nengine's mapping onto the new `Record{ID}`/`Forget(id)` contract, not the\nmemory model in general or a future non-file engine's model:\n\n- `internal/memory/builtin.Store` is unchanged. It has no concept of a\n stable per-block ID — blocks are addressed by content substring — and nothing\n in this decision requires changing that; existing `internal/memory` tests\n keep passing unmodified, per that issue's acceptance criteria.\n- A new infrastructure adapter (`internal/infrastructure/memory`, per\n `organisation.md`'s \"infrastructure implements domain contracts\" rule — not\n under `internal/domain/memory`, which stays contract-only) wraps one\n `builtin.Store` per identity.\n- **Identity → directory:** `hex(sha256(identity))`, not the identity string\n itself. Closes the path-traversal question without an allowlist regex, and\n sidesteps collisions from identities that differ only in characters a\n filesystem treats as equivalent.\n- **Record ID:** assigned once at `Write`, independent of content, so a later\n edit elsewhere in the file can never orphan a `Forget`. Realised as a\n leading marker line (`\u003c!--mem:\u003culid\u003e--\u003e`) on the block `builtin.Store.Add`\n writes; `Query`/`List` strip it back out of `Record.Content` before\n returning, and `Forget(id)` locates the block by that unique marker via the\n store's existing substring-based `Remove`. The marker is a pragmatic,\n reversible choice — plain-text records stay human-readable and\n human-editable, at the cost of one marker line an operator editing the file\n by hand should leave alone. Revisit if/when provenance representation\n (still open, below) settles on something richer.\n- **Kind → section:** `Observation.Kind` (default `\"general\"` when empty) is\n the section name, reusing `builtin.ValidateSectionName` — so an\n invalid `Kind` fails `Write` with the same validation error `memory_edit`\n already returns for an invalid section today, not a new error shape.\n\n**Memory engine: scopes, CRUD with retained revisions, and store placement —\nDECIDED (2026-09-16).** Supersedes the 2026-09-02 \"Authoritative record and\nrevision model for the builtin engine\" entry above, whose `\u003c!--mem:\u003culid\u003e--\u003e`\nlive marker cannot carry provenance. Authority:\n[`docs/prds/memory-engine-unification.md`](/dev/prds/memory-engine-unification/)\n(Approved); the contract is `internal/domain/memory/contract.go`. This register\nsupersedes; it does not erase.\n\n- **Four typed scopes, caller-named.** `internal/domain/memory` is addressed by\n `Scope{Kind, Agent, User}` over `global`, `agent`, `user` and `agent-user`,\n with `Validate` rejecting a component present where it is not required. A\n caller names the read and write sets (`Subject.Scopes()` /\n `WritableScopes()`; global is never model-writable) and the engine applies no\n policy of its own: **access control is naming a scope, and nothing else.** An\n engine that enforced policy would need to know about channels, sessions and\n bindings, and every backend would reimplement it. `Scope.Key()` length-prefixes\n each component, so two scopes can never share a key even when an opaque\n channel-native id embeds the separator.\n- **The caller resolves identity; the engine never does.** Resolution happens in\n the gateway turn path, where the native sender id and the session are both in\n hand, and is supplied per channel by the composition. A webhook's sender id is\n a route path, not a person, and treating it as one would give a URL an\n identity. An absent user id yields global and agent scopes only: \"show none of\n it\", never a wider fallback.\n- **CRUD supersedes, never overwrites.** `Store` is `Create`/`Get`/`Query`/\n `List`/`Update`/`Forget`/`Revisions` over the four scopes. `Update` appends the\n superseded state to the scope's `HISTORY.md` before replacing the live block;\n `Update.Expected` names the revision being replaced and returns\n `ErrStaleRevision` on mismatch, making a lost update between two processes\n sharing one scope file detectable. `Forget` records a deleted revision before\n removing the live block, so a record's provenance outlives its content; `Get`\n and `Forget` take a scope, because an id alone does not locate one.\n- **Provenance representation.** The live marker becomes JSON, since a\n `\u003c!--mem:\u003culid\u003e--\u003e` line cannot carry provenance, and `Record` retains the\n provenance `agent-system.md` requires — scope, author, originating user,\n source, and revision history. `Metadata` is dropped: it never round-tripped.\n- **Store and scanner placement.** The markdown store relocates to\n `internal/infrastructure/memory/builtin`, its correct home under\n `organisation.md`'s \"a package owns its on-disk format end-to-end\". The content\n scanner moves into `internal/domain/memory` and is applied in `Create` and\n `Update` — the single choke point every producer crosses, so curator-extracted\n content is covered too.\n\n**Open in this section.** Ranked retrieval is deferred: `Query.Text` stays in\nthe contract, unused by the prompt path. Existing memory data is not migrated:\n`\u003cworkDir\u003e/memory` and `\u003cworkDir\u003e/memory-engine` are left on disk, unreferenced.\nThe PRD leaves these undetermined and they remain open here: group-chat\nextraction, where the curator fails closed on mixed-sender sessions; two\nprocesses (`archied` and `archie-gateway`) holding one scope file, where\n`Update.Expected` detects but does not prevent a lost block; unbounded\n`HISTORY.md` with no compaction policy; whether the per-scope size bound is\nright; and whether the dashboard should carry a user identity.\n\n### 6. Runtime and process boundaries — DECIDED\n\n`archied` is the native resident orchestration and interactive-chat process.\nEvery autonomous repository workflow crosses one full-task core-NATS handoff\ninto a task-scoped `archie-agent` container. Embedded versus external NATS\nchanges broker deployment only; it never selects an executor. The former host\nin-process runner, per-stage NATS protocol, and subprocess runner are removed.\n\nForge, store, and worktree authority remains in `archied` and is exposed to the\ntask container through scoped RPC services. The worker runs the workflow and\nits worker-local ai-sdk loop; it does not receive forge credentials or direct\nstore access. See `docs/prds/embedded-nats.md`.\n\n**UI Service boundary — RATIFIED (2026-09-07).** Phase 3 has a focused\nauthority record in [`docs/prds/ui-service-boundary.md`](/dev/prds/ui-service-boundary/).\nThe UI Service owns HTTP/SPA delivery, UI DTOs, browser authentication, and\nits own endpoint/credential/readiness settings. It consumes Gateway and State\nStore contracts and does not receive `config.Holder`, daemon pointers, SQL or\nstore implementations, workflow registries, forge/channel/model runtimes, or\nresolved secrets. The current `internal/webui` implementation remains inside\n`archied` until the ratified migration gates prove route parity, adapter\nselection, failure/readiness behaviour, and deletion of the shared-holder and\ndirect-access paths. The extraction cutover removes the in-process UI\nlistener in the same change; there is no dual-live UI authority.\n\n**Dashboard live event delivery — DECIDED (2026-09-09, `archie-core-za9f`).**\nThe extracted UI receives live activity by polling the State Store's existing\n`EventsSince` cursor, not by a new push-subscription RPC. No proto change, no\ncontract amendment beyond this record.\n\nThe UI process runs one event pump. It primes its watermark by paging\n`EventsSince` to the end at startup without delivering, then polls\n`EventsSince(watermark, limit)` on an interval and hands each new event to\n`webui.Server.Broadcast`, exactly where `internal/app/archied/main.go`'s\n`persistAndBroadcastEvents` hands events to it today. One pump serves every\nconnected browser, so cost is one indexed query per interval regardless of\nviewer count, and `Broadcast` is a no-op when nobody is watching.\n\nThree properties make the cursor sufficient, and a push hub unnecessary:\n\n- **The cursor cannot skip.** `internal/infrastructure/postgres` takes an\n advisory event-append lock around each insert (`TestEventInsertsCommitInIDOrder`),\n so a row's `id` is assigned in commit order. There is no window in which a lower id becomes visible after a\n higher one, which is the failure mode that normally rules out polling an\n autoincrement cursor. SSE catch-up already depends on this same property.\n- **A broadcast is a wakeup, not the payload.** `sseStream.drain` treats each\n broadcast event as a signal to fill the persisted gap ahead of it via\n `catchUp`, then delivers the event only if catch-up did not already cover\n it, deduplicating on the `since` watermark. Substituting a poller for the\n in-process bus therefore changes no SSE semantics, and needs no change to\n `internal/webui/sse.go`. It also means a broadcast dropped to a stalled\n client self-heals on the next one.\n- **The table is the only complete fan-out point.** Events reach the store\n through several paths that never touch the daemon's bus, including the\n audit event `store.ArchiveTask` writes inside its transaction\n (`internal/infrastructure/postgres/store.go`) and the binding-dispatch failure\n `internal/daemon/daemon.go` inserts directly. A push hub would have to be\n hooked at every write site and at post-commit, and would silently miss any\n site added later that forgets to notify. Reading the table misses nothing.\n\nThe cost is one poll interval of latency on the activity feed, default 1s and\noperator-tunable. That is accepted: the feed reports task lifecycle history,\nand an operator action already gets its own synchronous HTTP response.\n\nThis resolves the `/events` row of the route-owner inventory in\n[`ui-service-boundary.md`](/dev/prds/ui-service-boundary/). `/api/logs` and\n`/api/logs/stream` are not resolved by it, they read the daemon's in-process\ndiagnostic feed and remain host-local (see `archie-core-8cda.5.4`).\n\nRevisit only if sub-second feed latency becomes a requirement, or if a second\nconsumer needs the same stream, at which point the fan-out belongs in the State\nStore process behind a `SubscribeEvents` RPC and this record is superseded.\n\n**Dashboard configuration page — DECIDED (2026-09-09, `archie-core-ymut`).**\nThe configuration page becomes read-only in the UI process for Phase 3. Reads\ncross the boundary as a published snapshot; writes are descoped, not\ncontracted. This amends the ratified boundary, which assumed a single admin\ncontract carrying both.\n\n**Reads.** `webui.ConfigView` is already the whole answer. It is a UI-owned,\nsecret-free value projection built in `handleConfig`: `ProviderView` carries\nthe API key's environment variable name and a `Configured` boolean, never a\nvalue, and `RepoView` is repository metadata only. It is already the exact\nshape the browser receives. So the daemon publishes that projection, plus\nprovenance, reload status and the overridden-key list, as a snapshot the UI\nreads; nothing new has to be designed, only moved. Two State Store RPCs carry\nit, a `PutConfigSnapshot` restricted to the administrative token and a\n`GetConfigSnapshot` the UI calls. The daemon republishes on boot and after\neach successful reload, which is precisely when the projection changes.\n\nThe State Store is the right holder because a published configuration snapshot\nis shared state, it is already dialled by the UI, and it costs no new listener,\nport, token or interceptor. The alternative, a config admin service on\n`archied`, would make the daemon a gRPC server for the first time. Today it is\npurely a client of Gateway and State Store; `grpc.NewServer` appears only in\n`RunGateway` and `RunStateStore`, which are the standalone binaries' entry\npoints, not the daemon's.\n\n**Writes.** `PATCH /api/config`, `POST /api/config/reset` and\n`PATCH /api/config/repos/{owner}/{name}` answer 503 in the UI process, and the\nSPA hides those controls rather than offering a button that fails. Editing is\n`config.toml` plus reload, which is already the only path for every setting\noutside the runtime-tunable allowlist.\n\nThis is a deliberate feature reduction and worth naming as one. The write path\nis not hard because of the transport; it is hard because the policy is\ndaemon-local and stays that way: `installUpdateConfigHandler` denies\nnon-runtime-tunable keys, applies a dotted overlay to a clone of the published\nconfig, validates the materialised result, writes overlay rows, and rebuilds\nprovenance. Contracting that means either moving the policy, which the\nboundary forbids, or exposing it verbatim as remote procedure calls, which\nbuilds a Configuration Service inside a migration bead. Neither is work this\nphase should be doing to keep one page's buttons alive.\n\nRevisit when dashboard configuration editing is picked up as a feature in its\nown right (`archie-core-1786637498420-327`, still open). At that point it gets\nan admin contract designed as a feature, and the Configuration Service this\ndocument already anticipates is its natural owner. Until then the UI renders\nconfiguration and does not write it, which is what the boundary always said\nabout ownership even when it assumed a wider contract.\n\nAmended 2026-09-11. This descope is a deferral, not a decision that the\ndashboard should be read-only. Name it that way in review: the daemon-side\npolicy was built and tested, then deleted by the cutover (`e037b95` removed\n`internal/app/archied/config_update.go` and its tests), so reviving it is\nrecoverable work, not a rewrite. `archie-core-j28m` carries the remaining\nquestion -- which process serves the policy and what transport carries the\nwrite -- and blocks the feature bead. The handlers, their error sentinels and\nthe `UpdateConfig`/`ResetConfig`/`UpdateRepoField` seams on `webui.Server` are\nkept deliberately for it; `ConfigView.Editable` follows whichever process holds\n`UpdateConfig`, so wiring that one field makes the page editable\n(`archie-core-ml30`).\n\nImplementation belongs to `archie-core-8cda.5.4`, the cutover that removes the\ndaemon's own dashboard. Nothing here is built yet.\n\n**Superseded 2026-09-21 by the runtime control plane.** The open question this\ndescope left -- which process serves the policy and what transport carries the\nwrite -- was answered by `docs/prds/runtime-control-plane.md`: neither a daemon\nadmin service nor a revival of the daemon-local policy. `ControlPlaneService` on\nthe State Store's existing gRPC server owns every database-backed setting, and\neach feature validates its own resource, so there is no daemon-local policy left\nto contract. `archie-core-j28m` is closed.\n\nConsequently the three write seams described above no longer exist.\n`PATCH /api/config`, `POST /api/config/reset` and\n`PATCH /api/config/repos/{owner}/{name}`, the `UpdateConfig`/`ResetConfig`/\n`UpdateRepoField` fields on `webui.Server`, and `ConfigView.Editable` were\ndeleted in `31622ec5` along with the runtime overlay store they wrote to.\n`GET /api/config` remains a read of the published projection, as this decision\nalways intended.\n\n### 7. Shared mechanics\n\nThe following cross-domain mechanics require exact contracts before dependent\npackages migrate:\n\n- event-bus delivery and failure semantics;\n- policy evaluation API, composition, and precedence;\n- configuration candidate preparation, health validation, atomic promotion,\n observation, and rollback;\n- service-specific health, bounded remediation, isolation, and escalation;\n- changelog association and grouping.\n\nThese shared mechanisms do not acquire ownership of domain meaning. Domains\ncontinue to own their commands, events, policies, settings, and consequences.\n\nThe Go documentation generator, its output placement, and drift checking are\ndefined in `generated-documentation.md`. The external Astro website consumes\nthe committed documentation and release JSON artifacts and serves them under\n`offloaded.dev/docs/`. This repository owns the source and artifact checks; the\nwebsite repository owns rendering and deployment. The generators and\nArchie's normalised documentation model remain migration work rather than open\nproduct architecture.\n\n**Amended 2026-09-15.** `internal/infrastructure/servicediscovery/nats`\nimplements the `servicediscovery.ServiceRegistry` contract (Connect, Register,\nResolve, Watch) over NATS KV and is fully tested, but no production code\nconstructs it and no config surface exposes it. This is a deliberate deferral,\nnot an oversight: service-to-service addressing belongs to the runtime and\nprocess boundary work in section 6, which currently pins container addressing\nby injecting the daemon's startup NATS URL rather than resolving through a\nregistry. The scaffold is kept for that work; re-derive it only if the\naddressing model changes.\n\n### 8. Cutover sequence\n\nThe final migration plan must order:\n\n- compatibility adapters;\n- schema migrations and data backfills;\n- any required dual-read or transitional paths;\n- domain-by-domain package replacement;\n- production wiring changes;\n- legacy package removal;\n- architecture tests preventing dependency regression;\n- focused, integration, race, and full quality gates.\n\nEach legacy package requires objective removal criteria. No compatibility path\nmay become an indefinite second implementation.\n\n## Completion criteria\n\nMigration design is complete when:\n\n- every current package has one approved owner and destination;\n- every mutable record has one authoritative owner and migration path;\n- all external and cross-process contracts have versioned owners;\n- existing behaviour and data have explicit parity requirements;\n- process boundaries have evidence-backed justification;\n- the cutover order preserves a working application;\n- legacy deletion criteria and deterministic architecture tests are defined;\n- no unresolved decision requires an implementer to invent domain semantics.\n\nExact Go APIs, SQL statements, and mechanical refactoring steps MAY be finalized\nduring implementation when they do not change these semantics.\n\n## Recorded migration position (2026-07-29)\n\nRecovered 2026-08-09 from the pre-migration issue tracker. This section records\ndecisions the maintainer approved that the repository did not otherwise capture.\n\n**Completed:** `internal/infrastructure/eventbus/nats` (rebuilt, not relocated);\n`internal/infrastructure/configuration` (loading moved out of `internal/config`,\nwhich is now types-only and does no I/O); `internal/eventbus` (broker-neutral\ncontract); `internal/domain/workintake` (the first `internal/domain/` package,\nowning `TaskEnvelope`, the routing `Kind`, the label vocabulary, and task\nsubjects).\n\n**DECISION — private input DTO deferred (approved).**\n`configuration.Document.Config` is still `internal/config.Config` rather than the\nprivate per-domain input DTO this document mandates. Building an ~800-line\nparallel type tree while every consumer still reads `config.Config` would mean\ntwo shapes kept in sync by hand, with no consumer for the new one. It is named as\na field so the compromise is visible at each use site, with the deletion\ncondition recorded in that package's `doc.go`. Do this **with** the\n`internal/config` dissolution, not ahead of it.\n\n**Agreed next: `internal/app/{archied,agentworker}`.** `run()` in\n`cmd/archied/main.go` is ~460 lines, against `organisation.md`'s rule that\n`cmd/*` must not contain substantive wiring or act as a service locator. It is\nalso the last non-infrastructure holder of the NATS SDK, and it is where the\nconfig DTO translation will land, so it unblocks the dissolution above.\n\n**Method constraint.** Do not write a separate recorded package-review document\nper area when the decisions are already dictated by\n`dependencies-and-contracts.md`. That is the ceremony pattern this project\ndeliberately avoids.\n\n## OPEN — `[chat.*]` config section naming\n\n`configuration.md:160` already states the position: `ChatConfig` holds \"separate\nchannel instance settings... there is no chat-wide settings owner\". The config\nsection name has not followed, so a settings block named for a product surface\nowns what the architecture calls channel state.\n\nTwo candidates, neither settled: `[channel.*]` and `[messaging.channel.*]`. The\nsecond reads as a domain path, and no config section in this repo is currently\nnamed after a domain package, so adopting it sets a precedent for every other\nsection rather than fixing one name.\n\nBlocked on a maintainer decision, not on evidence. This is a rename with an\noperator-visible TOML break, so it should land with the `internal/config`\ndissolution recorded above rather than on its own.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/observability/",
|
||
"title": "Task Observability",
|
||
"section": "Architecture",
|
||
"body": "# Task Observability\n\n**Status:** Implemented. The per-run detail surface was added on 2026-09-18 and\nis described below.\n**Date:** 2026-08-09 (revised 2026-09-18)\n\n## Historical problem\n\narchied's own daemon logging (`internal/logging/`) was already solid: a\nrotating file sink, a live `Feed` the dashboard subscribes to, and a reader\nthe `/api/logs` endpoint serves from. Before this implementation, none of\nthat reached inside a task.\n\nA task's actual work happens in a task-scoped `archie-agent` container,\ndispatched as one complete workflow over NATS.\nBefore this design was implemented, five things destroyed the evidence of what\nthat container did:\n\n1. **Worker startup** constructed an stderr-only logger and never replaced its\n handler after discovering a dedicated boot task, so runtime records could\n not leave the container.\n2. **`internal/container/pool.go`** creates every task container with\n `AutoRemove: true`. The container — and its stderr — is gone at exit,\n success or failure alike.\n3. **`internal/domain/workflow/implement.go`, `StageBaselineGate`** captures\n `go build`/`go vet`/`go test` output via `cmd.CombinedOutput()`, then\n clips it to 200 characters into a `Warn` log line and discards the rest.\n The eventual park reason is built from `res.Status` alone — the actual\n compiler error is never persisted anywhere a human or Archie's own chat\n tools can read it back.\n4. **`agentexec.Result`** (`internal/agentexec/protocol.go`) carries only\n `Summary`/`Detail` — no step-by-step transcript (tool calls, durations,\n errors). There is no transcript to recover even in principle.\n5. **`agentexec.SubjectForSystem`** is documented as the return channel for\n exactly this — _\"log dumps, health, and PII warnings... the daemon reads\n these for observability and never forwards them\"_ — but has no publisher\n and no consumer anywhere in the codebase except a subject-format\n assertion in `nats_test.go`. The designed return path was never wired.\n\nThe practical consequence: a task parks with\n`stage baseline: baseline red -- go build ./... fails and builder could not\nauto-fix (status: parked)` and there is no way — not from the dashboard, not\nfrom Telegram chat, not by SSH'ing into the deployment host — to see _why_\n`go build` failed. The container that knew is already gone.\n\nThis matters beyond debugging convenience: supported deployments can combine\na resident daemon with ephemeral Docker containers that run unattended for\nweeks between operator logins, and the system is\nintended to be usable by people other than the original operator. A system\nthat is only debuggable by an operator who happens to be watching `docker ps`\nin real time does not meet that bar.\n\n## Decision: NATS is the sole autonomous handoff\n\nThe agent image must reach NATS in every deployment, because the broker is the\nonly autonomous task handoff and the only channel carrying task results back\nto the daemon. `archied` has no workflow runner or host fallback. A container\nthat dies before establishing a NATS connection is a distinct, narrower\nfailure (connectivity/config) from the one this document addresses (a connected\ncontainer's own output being unrecoverable), and is out of scope here.\n\n## Implemented design\n\nThe key property: **logs leave the container while it is alive**, so its\ndeath at `AutoRemove` no longer matters. Persistence happens in the daemon\nprocess, which already owns durable state.\n\n### Sink: extend `internal/logging`, not a new package\n\n`internal/logging` already owns its on-disk format end to end per\n`organisation.md` (\"a package owns its own format end to end\"). Add a\ntask-scoped writer producing the same `logging.Entry` JSONL shape at\n`\u003cstate_dir\u003e/logs/tasks/\u003ctask_id\u003e/attempt-\u003cn\u003e.jsonl`. `logging.Tail`,\n`logging.Query`, and `logging.Components` operate on this unchanged;\n`internal/webui/api_logs.go` stays transport-only, as it does today.\n\n### Agent side: make the designed subject real\n\n`internal/app/agentworker` replaces the stderr-only `slog.Handler` with one\nthat tees to stderr (kept for container-level operator debugging when a\nshell is attached) and publishes each record to\n`agentexec.SubjectForSystem(taskID)`, which already exists for this purpose.\nThe application layer obtains only a narrow publisher from\n`internal/infrastructure/agenttransport/nats`; raw core-NATS types remain in\ninfrastructure. `internal/infrastructure/agentboot.TaskID` reads the task ID\nfrom `\u003cworktree\u003e/.git/task.json` per the container/pool.go trap documented in\n`CLAUDE.md`. Publishing is fire-and-forget: log shipping must never block or\nfail the run it is reporting on.\n\n### Daemon side\n\n`cmd/archied/main.go` registers `subscribeSystemLogs` once at startup on\n`agentexec.SubjectSystemWildcard`. It demultiplexes by task ID, appends to that task's\nsink, and publishes into the existing `logging.Feed` so live task output\nappears on the dashboard exactly like daemon output. The approved target is\nfor this composition to move to `internal/app/archied`; that package does not\nyet exist, so this document does not claim the migration is complete.\n\nThe daemon is also a writer in its own right for the decisions only it makes.\nEvery park records its reason into that attempt's sink (`daemon.recordPark`),\non the daemon side after the guarded transition lands and in\n`workflow.park` on the agent side, because \"why did this park?\" is the question\nthe log exists to answer and the container path cannot answer a run that never\nreached a container. The entry is written only once the park actually\nhappened, so a transition another actor won does not leave a log claiming an\noutcome the task board contradicts.\n\n### Gate output: stop discarding it\n\n`StageBaselineGate` and the equivalent TDD-stage gate runs write full\n`cmd.CombinedOutput()` to the task's sink under a `gate` component, and put a\nbounded tail (~2KB, not 200 bytes) into the park reason. This is the cheapest\nfix and independently valuable — it alone would have made the `go build`\nfailure diagnosable without any NATS plumbing.\n\n### Transcript\n\nStep-level events (tool name, argument digest, duration, error) are emitted\nas log entries on the same NATS system subject rather than added as new\nfields on `agentexec.Result`. This keeps the wire protocol\n(`internal/agentexec/protocol.go`) stable and the transcript lands in the\nsink automatically, with no separate delivery path to keep consistent.\n\n### Retention\n\nReuse the existing rotation (`MaxSizeMB`/`Keep`) semantics per task\ndirectory. Prune a task's log directory when `store.ArchiveTask` archives\nit, so task logs share the task's own\nlifecycle rather than accumulating independently.\n\n### Surfaces\n\n- `/api/tasks/{id}/logs`, built on `logging.Reader` the same way\n `/api/logs` is today.\n- A chat tool so Archie can answer \"why did task 4 park?\" directly from\n Telegram or the webui chat — this is the surface that actually matters\n for the deployment model in `CLAUDE.md`, where an operator may not be\n looking at a dashboard at all.\n- The per-attempt read surface described in the next section, on top of the\n same task-scoped stream.\n\n## Per-run task detail (attempts, stages, changes, config)\n\n**Status:** landed, together with the change set that carries this revision\n(workflow run `76f8e877`). The provenance half is in the tree: `events.Event.Attempt`,\nthe `changes_captured` and `config_captured` event kinds, and the two protobuf field\nadditions. So is the read half: the three per-attempt endpoints\n(`/api/tasks/{id}/attempts`, `/changes`, `/debug`) and the dashboard run-detail page\nthat consumes them. The decision record, the rejected alternatives and the honest\nlimits are `docs/prds/task-run-detail.md`.\n\nThe task log surface above answers _why did this park?_. This surface answers\n_what did this run actually do?_, per attempt.\n\n### Attempt attribution\n\nOne task can run many times, and the task's event stream is the durable record\nof all of them. `events.Event.Attempt` attributes each event to one run; `0`\nmeans **not attributable** — every event written before this change, plus the\ndeliberately task-agnostic producers. An unattributed event is reported as a\ncount and is never presented as an attempt.\n\nAttribution is a column, not a fold over `task_queued`/`task_retried`: a fold\nwould have to guess which side of a boundary an event belongs to across a daemon\nrestart, a stale-run recovery, or two interleaved attempts, and a guess rendered\nas structure is exactly the false fidelity this surface exists to avoid.\n\nThe column is part of the events table in\n`internal/infrastructure/postgres/migrations/0001_state_store.sql`, and nothing\nis backfilled.\n\n### Read surface\n\nThe routes are registered beside the existing task routes in\n`internal/webui`'s `registerTaskRoutes` and all resolve the task through the\nstore's `TaskByID`; a task that does not exist is a `404`, never a `200` with an\nempty body.\n\n- `GET /api/tasks/{id}/attempts` — every attempt with its stages inline, so the\n rail and the attempt selector can never disagree. Stages are ordered by\n occurrence within their attempt. An attempt with no stages is legitimate and\n renders as \"no stages recorded\", never as an unexplained empty rail.\n- `GET /api/tasks/{id}/changes?attempt=N` — the change captures for one\n attempt, ordered by capture time; more than one per attempt is normal (a\n mid-workflow commit such as TDD's `commit-repro`, then `commit-push`, then a\n final `open-pr` capture that carries the pull-request number).\n `found: false` means no capture was recorded for that attempt, which is\n **not** \"no files changed\"; a capture that was recorded but could not be read\n is reported as exactly that, and never as a diffstat of zeroes.\n- `GET /api/tasks/{id}` — unchanged: the raw event array, each event now\n carrying `attempt`. The per-attempt configuration is read from this response,\n selected by kind **and** attempt.\n- `GET /api/tasks/{id}/logs` — gains `stage` beside its existing filters. The\n match is exact and case-insensitive, and an entry that records no stage never\n matches (see `logging.Query.Stage`).\n- `GET /api/tasks/{id}/debug?attempt=N` — the stored task record and the task's\n events, deliberately unfiltered, so an operator can attribute everything the\n task did even when a panel has nothing to show.\n\nThe `attempt` parameter follows one rule on every route: absent or `0` selects\nthe task's current attempt, and the resolved attempt is echoed in the response.\n\n### Stage status\n\n`stage_start` and `stage_finish` are the only inputs, so the derivation is\nexplicit and each case is pinned by a test:\n\n| Events seen for the stage | Reported |\n| ------------------------------------------------------------------------- | -------------------------------------------------- |\n| `stage_finish` with `data.error` | `failed`, with the error text |\n| `stage_finish` with `data.interrupted` | `interrupted` |\n| `stage_finish`, neither | `ok` — \"returned without error\" |\n| `stage_start` unmatched, and this is the task's current in-flight attempt | `running` |\n| `stage_start` unmatched on any dead attempt | `interrupted` — the run ended without finishing it |\n| `stage_finish` with no matching start | the stage, with no start time invented |\n| no stage information at all for that attempt | `unknown` |\n\n**There is no exit code.** `stage_finish` carries a duration and an optional\nerror string and nothing else, so no stage can be shown as pass/fail and the\npage must not imply verification. A stage that finished cleanly and then parked\nstays `ok` on a failed task, which is why the attempt status is read together\nwith the task status rather than on its own.\n\n### Changed files\n\nThe change an attempt produced is read from the worktree at the moment it is\ncommitted or pushed, and persisted as a `changes_captured` durable event — plus\none final capture when `OpenPR` records the pull-request number, which is the\nonly point at which a capture can carry one. It has to be captured there: the\nworktree is deleted on merge, close and no-PR terminal states, a retry resets\nthe branch onto its base, and no forge contract exposes a changed-file read.\n`internal/domain/workflow/task/changes.go` owns the value types and the status\nstrings they persist.\n\n- One capture carries per-file `added`/`modified`/`deleted`/`renamed`/\n `typechange` status with added and deleted line counts, and totals.\n- Each capture names its schema (`archie/task-changes@1`, in `internal/events`)\n and a reader refuses a payload whose schema or key set it does not know,\n rather than rendering the zero value of every field it could not find. That\n refusal surfaces as \"a capture was recorded but could not be read\", which is\n a different statement from \"no capture was recorded\".\n- `base_sha` is the merge base the file list was diffed against, not the base\n branch's tip, so the pair describes one change set even when the base branch\n moved during the run.\n- `files` is capped; `totals` always covers the **full** set, and a truncated\n capture says so rather than silently dropping entries.\n- `binary` means \"no textual hunks\" — never a claim about the file's type on\n disk — and a path the repository recorded as a rename pair is reported as the\n repository recorded it.\n- Capture **degrades and never parks**: an implementation without the capture\n seam captures nothing, logs, and lets the run continue.\n\n### Per-attempt configuration\n\nThe effective configuration an attempt ran under is persisted as a once-per-\nattempt `config_captured` durable event carrying the schema constant\n`archie/task-config@1` and the non-secret task-runtime subset from\n`config.Config.ForTask()`. It is not stored in a table and has no endpoint of\nits own: the dashboard reads it out of the events response it already fetches.\nThe single-row `config_snapshot` table is left alone — it answers what the\nrunning configuration is now and is replaced on every publish.\n\nRedaction is proven positively by a test that builds a full configuration\ncontaining canary values and asserts none of them survive serialising, rather\nthan being inferred from the producer's contract. The timeline carries one\nhuman line for this kind; the document itself is rendered by the panel.\n\n**Trust boundary, stated honestly.** A task-scoped grant may insert events on\nits own task ID — that is how `tool_call` and `agent_finish` already work — so a\ntask could in principle write its own `config_captured` event. This widens no\ncapability, because the grant could already insert arbitrary events on itself,\nbut it does mean this provenance is exactly as trustworthy as the rest of that\ntask's event stream, and no more. It is not tamper-proofing and must never be\ndescribed as such.\n\n### The wire\n\nThe whole wire change is two fields on existing messages: `Event.attempt`\n(field 11) and `ReadTaskLogRequest.stage` (field 9) in\n`proto/state/v1/state.proto`. No RPC, no table and no configuration field was\nadded, so the ratified state-store contract in\n`docs/prds/state-store-contract.md` rev 2c is unchanged.\n\nVersion skew degrades instead of failing, and each degrade is a statement the\npage has to make rather than hide: an old server makes `attempt` read as `0` and\n`stage` match nothing, so the page says it cannot attribute events to attempts\ninstead of labelling the whole stream \"attempt 1\"; an old client sees\n`config_captured` as an unfamiliar kind, and the panel must then say it cannot\nread per-run configuration — a different claim from \"not captured for this\nattempt\".\n\n### Rules the surface keeps\n\n- \"No record\" and \"this deployment cannot read this\" stay different answers,\n for logs and for configuration alike.\n- No run-duration figure is shown. A run can spend days in `waiting_human`, so\n one number would understate wall time; per-stage durations only.\n- No duration is invented for a stage with no start; it reads \"not recorded\".\n- The stage filter is a narrowing, not a coverage claim: agent and tool output\n carries no stage and is never matched, and the pane says so beside the\n control.\n- An attempt predating the change shows \"not captured for this run\" for its\n configuration and its change capture, never an empty view that implies\n defaults applied or nothing changed.\n\n### Failure and rollback\n\nEvery part is additive. A capture or a configuration emit that cannot happen\nleaves the run alone — reporting must never park a task. Reverting the change\nset removes the endpoints and the page; the extra column and the extra event\nkinds are inert to the previous revision (a column no statement names is\nignored, and an unfamiliar event kind already renders as unfamiliar), and there is\nno data transformation to undo.\n\n## Dependency direction\n\n`internal/logging` remains cross-cutting per `organisation.md` — it imports\nno domain, infrastructure, or app package, so extending it to be task-scoped\ndoes not change its classification. The workflow domain writes through the\ncross-cutting logging contract. On the agent side,\n`internal/app/agentworker` composes the narrow publisher supplied by\n`internal/infrastructure/agenttransport/nats`. On the daemon side,\n`cmd/archied/main.go` currently composes the NATS subscription with the task\nsink until the approved `internal/app/archied` migration is implemented.\n\n## Implementation sequence (completed)\n\n1. Gate-output capture (~30 lines, no NATS changes) — fixes the immediate\n symptom that prompted this document.\n2. Task log sink in `internal/logging`.\n3. Agent-side publisher on `SubjectForSystem`.\n4. Daemon-side subscriber + `logging.Feed` integration + retention on\n archive.\n5. `/api/tasks/{id}/logs` + chat tool.\n\nTracked as GitHub epic/subtasks per the issue list linked from this\ndocument's PR.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/organisation/",
|
||
"title": "Repository Organisation",
|
||
"section": "Architecture",
|
||
"body": "# Repository Organisation\n\n**Status:** Approved \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Target structure\n\nArchie MUST be organised around cohesive application responsibilities:\n\n```tree\ncmd/\n archied/\n archie-agent/\n\ninternal/\n domain/\n \u003carea\u003e/\n eventbus/\n policy/\n infrastructure/\n configuration/\n eventbus/\n nats/\n forge/\n persistence/\n \u003cexternal-service\u003e/\n app/\n archied/\n agentworker/\n\nplugins/\n \u003cplugin\u003e/\ndeployments/\n \u003csupported-assembly\u003e/\ndocs/\n guides/\nexamples/\n \u003ccopyable-example\u003e/\nextras/\n \u003coptional-extension\u003e/\n```\n\nDomain names are chosen from application behaviour during package review.\nExisting names MUST NOT be retained merely to reduce migration work.\n\n## Domain areas\n\nA domain owns one cohesive job performed by Archie. It contains that job's\nlanguage, state, rules, operations, commands, events, policy implementations,\nruntime settings, and required service contracts.\n\nCode that changes together SHOULD live together. Empty or ceremonial `entities`,\n`repositories`, or `services` layers are prohibited.\n\n## Application composition\n\n`internal/app`:\n\n- constructs domains and infrastructure implementations;\n- translates external configuration into domain and plugin settings;\n- connects commands, events, and shared contracts;\n- starts services in dependency order;\n- owns health aggregation and shutdown ordering.\n\n`cmd/*` parses only process-level input and invokes the relevant application\ncomposition package. It MUST NOT contain substantive wiring or act as a service\nlocator.\n\n## Process boundaries\n\nArchie begins as a modular monolith even when capabilities run in separate\nprocesses. A domain becomes a separately deployed service only for a concrete\nneed such as security isolation, independent scaling, failure containment, or\nindependent operation.\n\nPossible future extraction does not justify premature network protocols or\nduplicated service scaffolding.\n\nA new command is not a reason for a new binary. `cmd/` entries are process\nboundaries: a binary earns its place when its surface fails independently of the\nfleet, or when one process may need to run in more than one location, such as a\nhigh-availability pair or across a NAT boundary. Operational commands over a\nsurface an existing binary already owns are subcommands of that binary. Offline\ndatabase maintenance belongs to `archie-state-store`, which owns the file and\nits path layout, not to a new recovery binary.\n\n## Optional and operational material\n\n- `extras/\u003cextension\u003e` contains a maintained optional extension with real\n behaviour. Core Archie MUST NOT import it, and removing `extras/` MUST NOT\n prevent the core application from building or running.\n- `deployments/\u003cassembly\u003e` contains a supported assembly, including deployment\n definitions and concise operational instructions.\n- `docs/guides/` contains handwritten explanation and links to generated\n reference material.\n- `examples/` contains copyable samples with explicit purpose, prerequisites,\n and support expectations.\n\nSupported assemblies SHOULD demonstrate that capabilities can be selected\nindependently.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/package-review/",
|
||
"title": "Domain and Feature Review Procedure",
|
||
"section": "Architecture",
|
||
"body": "# Domain and Feature Review Procedure\n\n**Status:** Active \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchitecture work proceeds one domain or feature at a time. Before any design\nquestion, the selected area is researched across its complete live\nimplementation, consumers, tests, state, configuration, and runtime wiring.\n\nExisting packages are evidence, not the unit that drives the interview.\nPlacement follows behaviour and ownership, not the existing directory name.\n\nNo restructuring implementation begins until the relevant package decisions\nare agreed and recorded in the focused domain or requirement document.\n\n## Required research\n\nBefore design questions, the current-state report MUST identify:\n\n- entry points and runtime composition;\n- state, persistence, and lifecycle;\n- public and internal contracts;\n- commands, events, and policy enforcement;\n- configuration and defaults;\n- all consumers and cross-domain interactions;\n- process and infrastructure boundaries;\n- tests, failure behaviour, and known invariants;\n- duplicated or conflicting implementations;\n- documentation that disagrees with live code.\n\nDiscoverable facts MUST be investigated rather than presented to the user as\ndesign questions.\n\n## Required decisions\n\nEach package review records:\n\n| Decision | Required result |\n| --------------------- | ---------------------------------------------------------------- |\n| Owned behaviour | The cohesive job the code performs |\n| Domain ownership | The owner of the behaviour and vocabulary |\n| Final location | Its target path |\n| Public contracts | The smallest interfaces, commands, and events exposed |\n| Required services | Domain-defined interfaces implemented by infrastructure |\n| State ownership | The owner of each mutable record and transition |\n| Dependencies | Allowed imports and dependencies to remove |\n| Duplication | Code to consolidate, delete, or keep separate |\n| Plugin status | Core domain, shared contract, infrastructure, plugin, or extra |\n| Process boundary | In-process, or the concrete reason for isolation |\n| Configuration | Settings owned by the domain or plugin |\n| Policy | Definitions, evaluators, consequences, and shared mechanics used |\n| Documentation | Authoritative definitions and generated reference |\n| Migration constraints | Behaviour and data that must remain correct |\n\n## Interview method\n\nFor every selected domain or feature:\n\n1. Declare the single area under review.\n2. Complete the current-state research.\n3. Explain actual behaviour in plain technical language.\n4. Separate facts from design decisions and concerns.\n5. Resolve obvious consequences directly and ask only questions whose answer\n materially changes behaviour or structure.\n6. Record each confirmed decision in its focused document.\n7. Complete that area's handoff before selecting another.\n8. Keep unrelated domains outside the implementation scope.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/plugins-and-extensions/",
|
||
"title": "Plugins and Extensions",
|
||
"section": "Architecture",
|
||
"body": "# Plugins and Extensions\n\n**Status:** Approved foundation \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Plugin framework\n\nThe core plugin domain lives under `internal/domain/plugin`. It owns:\n\n- plugin identity and metadata;\n- discovery and registration contracts;\n- lifecycle rules;\n- capability registration;\n- validation and compatibility rules.\n\nThe generic plugin contract remains metadata-only. Functionality is exposed\nthrough narrow typed capability contracts with an owning registry or manager.\n\nWorkflow plugins belong to the Workflow capability family. The Workflow domain\nat `internal/domain/workflow` owns their typed registration, validation,\nversioning, definition, and execution contracts. The generic plugin domain may\nprovide plugin identity, discovery, compatibility, and lifecycle mechanics, but\nit MUST NOT define Workflow semantics or expose untyped Workflow hooks.\n\n## Plugin engine rule (strict)\n\n“Engine” is reserved for a typed, lifecycle-managed capability family. It is not\na synonym for a generic plugin hook. Every new plugin engine must satisfy all of\nthese invariants:\n\n1. **Typed domain contract.** The engine interface exposes real\n capability-specific operations in addition to identity metadata. Examples are\n resolving a secret, delivering a message, or running an agent. A plugin that\n only exposes `Name` and `Version` is metadata, not an engine.\n2. **Owning family manager.** The capability package owns a `Registry` or\n `Manager` that controls provider registration and discovery, initialisation,\n or access. Providers do not register arbitrary callbacks directly on the\n daemon.\n3. **Lifecycle and health.** When an implementation owns resources or background\n work, its typed family contract must expose start, health, and stop\n semantics. The owning manager invokes them and defines failure isolation and\n shutdown ordering. Stateless providers may make these operations explicit\n no-ops.\n4. **Narrow host access.** Plugins receive the typed family registrar and only\n the host services their declared capability requires. They never receive\n `*daemon.Daemon`, a service locator, or an untyped hook map.\n5. **Metadata-only generic plugin contract.** `plugin.Plugin` remains limited to\n `Name()` and `Version()`. Domain hooks belong on typed engine interfaces;\n adding capability methods to the generic interface is forbidden.\n6. **Trust boundary stays explicit.** Explicitly trusted, operator-installed\n plugins—including third-party secret-engine plugins the operator chooses to\n trust—may run in-process and have daemon privileges. Repository-supplied code\n runs in the task container. Integrations that are not trusted as daemon code\n run out of process through MCP or a versioned protocol and, when a\n confidentiality or integrity boundary is required, inside a real container\n sandbox. A subprocess boundary alone is not a security sandbox, and calling\n Yaegi-loaded code a plugin does not isolate it.\n\n`internal/plugin/architecture_test.go` enforces the mechanically testable parts\nof this rule: the generic plugin method set, the shared agent-instruction\nsource, and the requirement that engine interfaces have domain behaviour plus an\nowning registry or manager.\n\n### Memory engine family\n\nOne engine, addressed by four typed scopes -- `ScopeGlobal`, `ScopeAgent`,\n`ScopeUser`, `ScopeAgentUser` (`internal/domain/memory`, `Scope`/`ScopeKind`).\nA caller names exactly the scopes it may read or write; the engine applies no\naccess policy of its own, so isolation falls out mechanically because\n`ScopeAgent{a}` and `ScopeAgent{b}` are different storage keys. See\n`docs/prds/memory-engine-unification.md` for the decision this contract\nimplements; it superseded `internal/memory` (`MemoryProvider`/`Manager`),\ndeleted in that PRD's slice 5.\n\n`MemoryEngine` (`internal/domain/memory/contract.go`) is the typed family\ncontract: `Create`, `Get`, `Query`, `List`, `Update`, `Forget`, `Revisions` --\nCRUD with revisions retained, not just create/read/delete. `Update` supersedes\nrather than overwrites: the superseded state is appended to the scope's\n`HISTORY.md` before the live block is replaced, and `Update` carries no\n`Scope` field, so a record can never change scope. The family's content\nscanner (prompt-injection / sensitive-data patterns) is applied inside\n`Create` and `Update`, the single choke point every producer crosses.\n\n`internal/domain/memory.Registry` is the owning family registry: registration,\nstart/health/stop, shutdown ordering, and failure isolation, following the\nplugin engine rule above. `internal/infrastructure/memory.NewBuiltinEngine` is\nthe only registered engine today; it persists through\n`internal/infrastructure/memory/builtin`, a markdown store (`MEMORY.md`-shaped\nper-scope files) that owns its on-disk format end-to-end.\n\nTwo per-turn seams in `internal/gateway` connect a chat turn to the engine,\neach narrowed to exactly the operations it needs:\n\n- **Read** (`turn_memory.go`, `MemoryStore`): `prepareTurn` runs a\n scope-only `Query` over the resolved `Subject`'s readable scopes,\n synchronously and before the prompt is built. It never returns an error --\n a panicking or failing engine degrades to an empty `\u003cmemory\u003e` block rather\n than failing the turn -- and is bounded by a per-scope record limit and a\n byte cap on the rendered block.\n- **Write** (`turn_memory_tool.go`, `MemoryWriteStore`): four tools --\n `memory_create`, `memory_update`, `memory_delete`, `memory_list` -- built\n per turn onto the resolved `Subject`'s writable scopes, not registered once\n at boot. The model chooses the scope kind; it never chooses the agent or\n user id, which come from the turn's resolved `Subject` and have no schema\n field to override. `ScopeGlobal` is never model-writable.\n\n`boot.setupMemoryEngine` (`internal/app/archied/bootstrap.go`) registers and\nstarts the engine and must run before `setupGatewayChat`, which constructs the\nonly chat turn runner in production and captures the engine at construction\ntime (`internal/app/archied/composition_order_test.go` pins the ordering), and\nbefore `setupCurators`, whose `Registrar` captures the engine registry. The\nsession curator (`internal/infrastructure/sessioncurator`) writes derived\nrecords onto `ScopeAgentUser`, making it the engine's other live producer\nbesides the chat write path.\n\n## Plugin implementations\n\nFirst-party plugins are self-contained vertical modules under\n`plugins/\u003cplugin\u003e`.\n\nA plugin:\n\n- owns its implementation, settings, templates, assets, and tests;\n- receives only explicitly supplied contracts and services;\n- MUST NOT receive daemon internals, a service locator, or unrestricted\n application access;\n- MUST NOT import arbitrary infrastructure or application composition code;\n- MAY use approved domain and shared application contracts;\n- MUST remain removable without changes to unrelated domains.\n\nPlugins may be internally cohesive like small services. They do not become\nnetwork services without a concrete security or operational reason.\n\n## Event sources and reactions\n\n`archie-core-7d5u.1` asked whether extensions should react to events\nin-process through typed capability families (this document's model) or\nout-of-process like a generic `DispatchEvent` dispatcher. Decided 2026-08-22\nin `docs/prds/event-sources-and-reactions.md`: in-process typed families,\nconfirming this document's model rather than amending it. The forge webhook\nreceiver (`internal/forge/webhook`; see \"Intake provenance\" in\n`docs/architecture/messaging-and-work-intake.md`) and the EDA playbook engine's\nModule action position\n(`docs/prds/eda-playbook-engine.md`, `internal/domain/eda/module`) are the\nshipped instances of that decision -- each is a narrow typed contract with an\nowning registry, not a generic event hook. The playbook engine's Channel and\nForge action positions are designed but not yet implemented (open\ninvestigation `archie-core-t2db.19`); when they land they follow the same\nmodel, not a new one.\n\n## Curator surprisal sampling\n\n**Decided 2026-08-05, wave 1 landed 2026-09-05** — the curator engine family\n(epic [#435](https://github.com/samcharles93/archie-core/issues/435)) selects which memories deserve agentic attention via\nsurprisal-based sampling: score memories by how surprising they are and\nspend the sampled reasoner's agentic budget on the most surprising items.\n\nThe prerequisites are built: the embeddings capability\n([#436](https://github.com/samcharles93/archie-core/issues/436), `internal/domain/embedding` contract +\n`internal/infrastructure/embedding` implementation, config-driven via\n`models.embedding`) and the wave-1 `Sampler` strategy seam\n(`internal/domain/sampling` -- not `internal/curator`, see\n`docs/prds/curator-sampler-wave1.md` for the settled design and tracking)\nwith four\ncheap, embedding-free strategies (recency, random, all, staleness proxy).\nNo curator consumes a `Sampler` yet, and the embeddings client is likewise\ndeliberately inert: `setupEmbeddings` constructs it at boot and holds it on\n`b.embeddings`, but no production code reads that field yet, so a resolved\n`models.embedding` role currently produces a client whose only observable\neffect is a boot log line. Both are prerequisites waiting on #437, not\nunwired defects.\n\nStill deferred: [#437](https://github.com/samcharles93/archie-core/issues/437), the embedding-backed surprisal strategy, which\ndepends on the above and on [#407](https://github.com/samcharles93/archie-core/issues/407). Requirements for that deferred work,\nso it can be picked up without re-deriving the design:\n\n- A `Sampler` implementation behind the same interface as the cheap\n strategies; selection deterministic given fixed inputs.\n- Algorithm: sample up to a candidate cap; build a k-NN graph over embedding\n vectors; row-normalise the adjacency with self-loops; power-iterate to the\n stationary distribution; surprisal = −log(stationary probability of the\n nearest point); select the highest-surprisal items.\n- Degradation: content without an embedding is skipped; an embedding failure\n aborts sampling and falls back to free exploration — a sampling failure\n must never fail or crash a curator pass.\n- Cost bound: the graph build is quadratic in the candidate cap; the cap must\n be a named constant with a documented bound.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/policy/",
|
||
"title": "Policy System",
|
||
"section": "Architecture",
|
||
"body": "# Policy System\n\n**Status:** Approved foundation; exact API is deferred \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nRetry limits, authorisation, gates, commands, protected paths, tool\navailability, and similar rules define or constrain what Archie may do. They\nneed consistent consumption, composition, evidence, and audit without erasing\ntheir different domain meanings.\n\n## Ownership\n\n`internal/policy` is an explicitly approved shared application contract. It is\nnot infrastructure and it is not a generic domain that owns every rule.\n\nThe shared policy system owns:\n\n- policy identity, version, scope, and applicability;\n- typed evaluation invocation;\n- deterministic composition and precedence;\n- common evaluation states such as satisfied, violated, indeterminate, and not\n applicable;\n- reasons and evidence;\n- evaluation audit records;\n- registration, discovery, and lifecycle.\n\nEach consuming domain owns:\n\n- its policy vocabulary and typed inputs;\n- its concrete policy definitions;\n- its evaluators;\n- the meaning and consequence of an evaluation.\n\nAccess therefore owns principals, resources, actions, and authorisation\nconsequences. Task lifecycle owns retry eligibility and exhaustion. Task\nexecution owns gate requirements. Workspace mutation owns protected-path\neffects. Tool execution owns tool and command constraints.\n\nThese policies use consistent evaluation mechanics without pretending their\ndecisions are semantically interchangeable.\n\n## Infrastructure\n\nInfrastructure MAY load, persist, distribute, and audit external policy\nrepresentations. It MUST NOT define what Archie allows or assign the meaning of\na domain policy result.\n\nThe exact Go API, composition operators, precedence rules, and representation\nformat require a focused policy design pass.\n"
|
||
},
|
||
{
|
||
"url": "/dev/architecture/safe-change-and-recovery/",
|
||
"title": "Safe Change and Recovery",
|
||
"section": "Architecture",
|
||
"body": "# Safe Change and Recovery\n\n**Status:** Requirements approved; mechanics are under design. Database-backed settings are implemented through the control plane (2026-09): see the degrade/recovery section below, which records a gap rather than a solution. The full runtime-supervision protocol (bounded observation, versioned audit trail) remains under design.\n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchie is built and improved by agents as well as humans. The architecture MUST\nmake a change easy to locate, understand, test, audit, extend, and reverse\nwithout requiring its author to understand unrelated parts of the application.\n\nSeparation of concerns is therefore an operational safety boundary, not only a\ncode-organisation preference.\n\n## Required outcomes\n\n- An agent can improve a focused feature without receiving authority over\n unrelated domains.\n- Every event, function, action, setting, and policy has an identifiable owner\n and provenance.\n- Source and runtime changes pass deterministic checks outside the actor making\n the change.\n- A rejected or unhealthy candidate cannot replace the last known working\n version.\n- Invalid user configuration does not terminate an otherwise healthy\n application.\n- Configuration and service failures are observable, attributed, and eligible\n for bounded remediation or rollback.\n- The application continues serving unaffected capabilities when one\n capability is misconfigured or unhealthy.\n- Recovery failure is reported and contained rather than hidden in an\n unbounded retry loop.\n\n## Source changes\n\nAgent-authored source changes occur through the task-execution boundary in an\nisolated workspace. The agent does not directly mutate the running\ninstallation.\n\nPromotion is controlled by deterministic repository-owned policy, including\ntests, generated-reference drift checks, protected paths, change scope, and\nother required gates. The actor proposing a change cannot waive these checks.\n\nRepository and domain separation SHOULD allow a task to load and modify only\nthe governing requirements and code for its declared scope.\n\n## Runtime changes\n\nConfiguration and other runtime changes are proposed as candidates rather than\nmutating the active state in place.\n\nThe affected domain owns:\n\n- the command describing the semantic change;\n- validation of its typed settings;\n- health signals that demonstrate correct operation;\n- domain-specific remediation and rollback consequences.\n\nShared policy supplies consistent evaluation, evidence, and audit mechanics.\nIdentity supplies actor attribution. Infrastructure supplies external\nconfiguration storage, service control, and other operational adapters.\n\n## Runtime change protocol\n\nLast-known-good preservation and candidate promotion are the universal runtime\nchange protocol.\n\n1. An identity proposes an immutable candidate version.\n2. Infrastructure parses it without changing active state.\n3. Each affected domain validates its typed settings.\n4. Shared policy evaluates cross-cutting constraints.\n5. Each affected capability prepares the candidate without destroying its\n active instance.\n6. Capability-specific health checks prove the candidate works.\n7. Runtime supervision atomically promotes the candidate.\n8. A bounded observation period watches for delayed failure.\n9. Failure restores the last-known-good version and records the reason,\n evidence, actor, and involved versions.\n10. Repeated remediation failure stops retrying, isolates the capability, and\n reports the fault.\n\nThe affected domain owns the semantic change command, typed validation, health\nsignals, and remediation consequences. Shared runtime supervision owns the\npromotion transaction. Infrastructure performs storage and service-control\noperations.\n\nThe protocol's exact Go contracts, storage model, atomicity boundary, and\nobservation rules require a focused runtime-supervision design pass.\n\n## Failure isolation\n\nInvalid input MUST leave the active valid configuration unchanged and produce a\ndiagnostic record.\n\nA capability that cannot start or reload SHOULD fail independently where its\nabsence does not violate a declared application invariant. Unaffected\ncapabilities continue operating.\n\nRemediation MUST be bounded, observable, and policy-driven. Recovery MUST NOT\nsilently oscillate between versions or retry forever.\n\n## Database settings: degrade paths and recovery (2026-09)\n\nThe control plane owns runtime-tunable settings and stores them in the State\nStore's PostgreSQL database.\nThis replaced a dashboard overlay store that degraded to file config on any\nfailure and had three documented ways back. The replacement has none, and the\nposture is the opposite: **the daemon fails closed.**\n\n### Degrade paths\n\n| Failure | Boot behaviour | Where the operator sees it |\n| --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |\n| State Store unreachable | `boot.loadRuntimeConfig` returns the dial error and `archied` exits non-zero | stderr: `runtime settings unavailable` |\n| A stored resource fails validation | Same: `validate database settings` wraps the error and the daemon exits | Same |\n| SIGHUP reload with a bad file, or a control plane that cannot be read | Running config is kept; the reload records `last_error` / `last_error_at`. The reload re-applies the database layer, so it never publishes file values over the running database-owned ones | `/api/config` → `reload.last_error` → dashboard banner |\n\nFailing closed is deliberate for settings the database owns: falling back to the\nfile values would restore the second source of truth the control plane exists to\nremove. The cost is that a bad stored value stops the daemon starting, and the\nvalue can only be corrected through the daemon's own API.\n\n### Serving from Postgres\n\nThe State Store, the daemon and the Gateway serve from `database_url` only; an\nempty value is a startup error. The embedded NATS store, its endpoint file and\ntask logs live under `state_dir` (default: the data home's `archie` directory).\n\nThe Gateway holds a Postgres serve-ownership claim for its whole life, so a\nsecond Gateway against the same database refuses to start (its turn recovery\nwould fail the first one's in-flight turns). The State Store holds its own.\n\n### Recovery: offline commands on the State Store\n\nNothing skips the control plane at boot, and the only writer for a resource is\n`ControlPlaneService.Command` on a running State Store -- which is exactly what\nthe daemon fails closed without. `archie-state-store` therefore carries the\noffline recovery commands (`archie-core-6zw0`). They work on the PostgreSQL\ndatabase the configuration's `database_url` names\n(`internal/app/archied/state_store_recovery_postgres.go`,\n`internal/infrastructure/postgres/maintenance.go`) and need neither `archied`\nnor the Web UI running: `validate` reads the configuration file the daemon\nboots with, which is what makes its verdict the daemon's verdict. One database\nbacks every service, so the scope is always the whole database.\n\n| Command | What it does | Serving processes |\n| -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- |\n| `backup -out F` | runs `pg_dump --format=custom`, reads the archive back with `pg_restore --list` and only then replaces `F` | may be running |\n| `restore -from F` | drops tables the snapshot does not hold and runs `pg_restore --clean --single-transaction`; every write made after the snapshot is lost | every one must be stopped |\n| `validate [-config C]` | refuses a schema newer than the binary's highest migration; every stored resource decodes under the definition that owns it; and `configuration.Validate` accepts the settings the daemon would read -- the stored ones, or the seed the State Store writes for a kind the database does not hold yet -- layered onto `C`, the check the daemon reports as `validate database settings` | may be running |\n| `rollback -kind K [-revision N]` | replays an earlier revision of a stored resource through the ordinary replace, recording the rollback as a new revision | the State Store must be stopped |\n\nThe two validation layers `validate` reports are not the same check, and it\nreports both rather than merging them: the write path's own decode is stricter\nabout shape (it rejects unknown fields boot's decode ignores), while boot is\nstricter about meaning (only boot checks `dispatch.trigger`, and a positive\n`poll_interval`). Either one refusing is a store somebody has to look at.\n\nNo rollback RPC exists and none is needed: a revision carries its own value, so\nrestoring one is the same replace the dashboard and the channels already use.\nThe offline command is that operation with the store stopped -- the one state in\nwhich the daemon's own API cannot serve it.\n\n\"Must be stopped\" is checked rather than assumed. Each serving process holds a\nPostgres ownership lock for its whole life. `rollback` holds the `state-store`\nclaim for the replay. `restore` claims every service role's lock (`state-store`\nand `gateway`) without waiting and refuses if any is held or the check cannot\ncomplete. The read-only commands -- `backup` and `validate` -- take no claim and\nare meant to run against a serving database: `backup` because the update\ninstaller runs inside the process it is replacing and cannot stop the State\nStore, and `validate` because an operator checks a running deployment with it.\n`scripts/archie-update-install` takes its pre-flight snapshot through this\nsubcommand, so the installer and an operator run one implementation.\n\n`rollback` is the surgical route back from a bad stored value; `restore` is the\nblunt one, and is also the way back from a schema migration a release cannot\ngrow out of.\n\nThere is no automatic schema rollback. goose `Down` drops tables, so it is data\nloss, not a rollback. `scripts/archie-update-install` refuses a daemon update\nit cannot snapshot first, and on a failed update `scripts/archie-update-watchdog`\nrolls the binaries back, leaves the database alone and prints the `restore`\ncommand for the pre-update snapshot. Whether to run it is the operator's call.\n\n## Gateway restart: two constraints learned the hard way\n\nTelegram `/restart` (`internal/channels/telegram/restart.go` plus the `Start`\nsupervisor loop). Recovered 2026-08-09 from the pre-migration issue tracker.\nBoth constraints must survive any refactor.\n\n**1. Deadlock.** The `/restart` handler runs _on_ the bot instance the supervisor\nis about to stop. Tearing down inline from the handler deadlocks the restart it\njust asked for. So the handler sends on a **buffered** channel (`restartCh`,\ncapacity 1) and returns immediately; the supervisor loop in `Start` does the\nteardown. Duplicate requests hit the default branch and are dropped, not queued.\n\n**2. Lockout.** A failed config reload must **not** be fatal. It logs and\nrelaunches with the previous in-memory settings. If a bad config edit killed the\ngateway, it would also destroy the only means of fixing it remotely — which is\nthe entire reason `/restart` exists.\n\n**Scope is gateway-only by design.** The daemon keeps running so in-flight agent\ntasks survive. Exiting the process would also restart under a `unless-stopped`\npolicy, but would kill running work. `Reload` (supplied by `cmd/archied/main.go`)\nre-reads config from disk and refreshes **only** the token and `AllowedUserIDs`;\neverything else is wired into the daemon at construction and still needs a full\nrestart.\n\n**Authorisation is inherited:** `authorizedMessage` rejects non-allowlisted\nsenders before any handler runs, so all gateway commands are admin-only by\nconstruction. Do not add per-command auth without checking that invariant still\nholds.\n"
|
||
},
|
||
{
|
||
"url": "/dev/development/",
|
||
"title": "Development guides",
|
||
"section": "Development",
|
||
"body": "# Development guides\n\nChecklists for changing Archie, one page per area. Each names every place a\nchange must reach, so a feature that works in one layer does not silently stop\nat the next. A change that spans areas starts on the page for the area it\nbegins in, which points to the others. The pages point at the documents that\nown the design (`docs/architecture/`, the PRDs, `CLAUDE.md`) rather than\nrestating them.\n\n| Area | Covers |\n| ---------------------------------------------- | -------------------------------------------------------------------------------- |\n| [Frontend UI](/dev/development/frontend-ui/) | Anything under `ui/`, and the dashboard's API routes |\n| [Agent](/dev/development/agent/) | The agent container, its image, profiles, tools and workspace |\n| [Workflows](/dev/development/workflows/) | Workflow definitions and step types |\n| [Daemon](/dev/development/daemon/) | Task statuses, fields and the brief; sources, event types, mappings and bindings |\n| [State Store](/dev/development/state-store/) | Stored fields, tables, RPCs and errors |\n| [Control plane and settings](/dev/development/control-plane/) | Settings in `config.toml` or the dashboard |\n| [Processes and wiring](/dev/development/wiring/) | Where code goes, processes, plugin engines, channels, NATS subjects |\n| [Merging work](/dev/development/merging-work/) | Merging a lane or a branch |\n\n## Every change\n\n- Write the failing test first for anything with logic, branching or a\n contract, and check it fails on an assertion, not a compile error.\n- Test through the path production uses. A test that sets a value directly on\n the consumer proves nothing about the layers that normally deliver it.\n- Run `task check` and let its exit status decide the commit:\n `task check \u0026\u0026 git commit ...`. `task test` passes `-short`, which skips the\n process and Postgres tests, so run `go test -count=1 ./cmd/... ./internal/...`\n too when a change crosses a process or the database.\n- Regenerate, never hand-edit, generated output: `task proto:generate`,\n `go -C tools tool sqlc generate -f $PWD/internal/infrastructure/postgres/sqlc.yaml`,\n `task ui`, `task docs:artifact`. `task proto:check` compares against the git\n index, so stage regenerated contracts before running it.\n- Stage by path, or read `git status` before `git add -A`. Tests that start\n embedded NATS or write logs must use `t.TempDir()`; anything they leave in the\n tree gets committed. `task check` also adds linter-tool lines to `go.sum`\n (bead 1e37); restore `go.sum` before committing unless you changed a\n dependency.\n- A change to `docs/prds/`, `docs/architecture/` **or `docs/development/`**\n carries the regenerated `docs/data/generated/dev-docs.json`: run\n `task docs:artifact` (not `task docs:check`, which checks contracts only).\n PRD edits also follow `docs/prds/RULES.md` and pass `task docs:prds`.\n"
|
||
},
|
||
{
|
||
"url": "/dev/development/agent/",
|
||
"title": "Agent",
|
||
"section": "Development",
|
||
"body": "# Agent\n\nAuthority: `docs/architecture/agent-system.md` (execution boundary, the model\nnever runs git) and the Deployment Model section of `CLAUDE.md`.\n\nThe daemon prepares a workspace, starts one container per task from a profile's\nimage, and hands the whole task to `archie-agent` over NATS in one\n`taskrun.Request`. The agent reaches the State Store, forge and worktree push\nonly through the daemon's RPC servers, using task-scoped credentials.\n\n## Giving the agent something new\n\n- **Data about the task** goes in `taskrun.Request` or the task brief\n (`container.TaskPayload`), never in an environment variable. Add a test that\n the request carries it (`TestRunViaAgentSendsMCPServers` is the pattern).\n- **A credential** is scoped to the task and revoked when it ends, like the\n State Store grant and the worktree publication grant. The daemon's own\n administrative token never enters a container.\n- **A tool** reaches the agent through the central registry, MCP, a repository\n script or a skill plugin. A profile's `tools` list filters those; ai-sdk\n agentloop's built-in file tools are outside it. A tool that must stay inside\n the workspace uses the agent loop's confinement, not its own path handling.\n\n## Changing the agent image or a profile image\n\n1. The agent image is `Dockerfile`; example profile images live in\n `examples/profiles/`, each with a build-only service in\n `docker-compose.yml` (`profiles: [\"build\"]`, so `up -d` never starts it).\n2. Anything the agent runs must be in the image. The container gets no\n package manager at run time beyond what the image ships.\n3. A locally built image needs `containers.pull_policy = \"missing\"`;\n `\"always\"` tries to pull it and fails.\n4. Profiles are configured under `[containers.profiles.\u003cname\u003e]` and resolved\n by the daemon before the container starts (`pinTaskProfile`). An unknown\n profile parks the task; it never falls back to the default image.\n\n## Changing how a task runs\n\n- Workspace preparation happens in the daemon (`prepareWorkspace`) before the\n container starts, because the container binds the directory. Clones need the\n daemon's forge credential; the agent never clones.\n- A task with no repository gets `worktree.Manager.ScratchDir`, which is\n removed when the task ends.\n- The agent runs as root in the container. Anything it writes into the\n bind-mounted workspace is handed back to the daemon's UID\n (`restoreWorktreeOwnership`); a new write path must stay inside that\n workspace.\n- NATS rules are in `CLAUDE.md`: task responses use `msg.Respond`, a responder\n is flushed after it registers, and `ARCHIE_TASKS` is a work queue.\n"
|
||
},
|
||
{
|
||
"url": "/dev/development/control-plane/",
|
||
"title": "Control plane and settings",
|
||
"section": "Development",
|
||
"body": "# Control plane and settings\n\nAuthority: `docs/architecture/configuration.md`. The wiring matrix, the\ndecoded-but-unwired ledger and the three shapes an unwired field takes are in\n`.agents/skills/archie-config-and-flags`.\n\nA field that parses is not a field that works. Before adding one, name the code\nthat reads it and the path the value takes to get there.\n\n## Decide what kind of setting it is\n\n- **Bootstrap-only** (read once at process start: `database_url`, `state_dir`,\n `work_dir`). It stays in the file, is checked by `validateBootstrap` in\n `internal/infrastructure/configuration/validate.go`, and the runtime overlay\n refuses changes to it.\n- **Database-owned** (repository policies, scheduling, tools, plugins,\n channels, containers). The file only seeds it. Once the State Store holds the\n control-plane resource, the stored document is layered over the file\n (`controlplane.Client.RuntimeConfig`) and the file key stops mattering. It is\n checked by `validateDatabaseOwned` and by the resource's validator.\n- **Worker-carried.** A value the agent container needs travels in\n `config.TaskConfig` (`Config.ForTask`) or on `taskrun.Request`. Never pass it\n as an environment variable the daemon did not already define, and never carry\n a secret this way.\n\n## Adding a database-owned field\n\n1. Add it to the wire-shape type the resource's `Document` names — a dedicated\n struct in `internal/app/controlplane/` with explicit snake_case json tags\n (`channel_settings.go`, `tool_settings.go`, `container_settings.go` are the\n precedents), never the internal config struct itself. A struct fed straight\n into the document falls back to Go field names (`Window`, `MaxRequests`),\n and a bare `time.Duration` marshals as nanoseconds — both render unusable\n in the Web UI. The resource's JSON schema and the dashboard's Advanced\n settings editor are generated from that type, so no UI work is needed for\n the editor itself.\n2. Map to and from the internal config type at the resource boundary (`Seed`,\n the layering in `runtime_config.go`, and the validator all use the same\n mapping). Durations in the document are a string-marshalling type\n (`config.Duration` or a document-local one), never a bare `time.Duration`.\n3. A document stored before the shape existed carries Go-cased keys. Read them\n with `controlplanerpc.DecodeRenamingLegacyKeys` (an `UnmarshalJSON` on the\n wire type) while keeping `DisallowUnknownFields` — dropping unknown keys\n would decode a misspelling as \"unset\" — and register a `Normalize` on the\n `Definition` that re-encodes the document, so a legacy document converges\n the next time it is written. A map or a nested struct needs its layering\n checked: a stored map replaces the file's, not merges into it, and a field\n the stored document omits keeps the file's value. Add a case to\n `internal/app/controlplane/runtime_config_test.go` for both.\n4. Validate it in one function both sides call, and add a case to\n `TestResourceValidatorsRejectWhatEffectiveValidationRejects` in\n `validators_agree_test.go`, so the file and the resource cannot disagree.\n5. Decide whether it reloads: it does only if every consumer re-reads it from\n `config.Holder`. The allowlist is `internal/app/archied/reload.go`; anything\n not on it requires a restart.\n6. If the settings page shows it as a plain field, add it to\n `internal/webui/config_schema.go` and the view in `api_config.go`.\n7. A generic catalog resource (`replace` command, not domain-managed, no\n query service) is watched **once per archie-ui process** by `webui.RunLive`,\n then published to every browser through `/api/stream`. Do not add an\n EventSource per resource to a settings store or a gRPC Watch per HTTP client.\n The control-plane server polls `Resource` every 250ms per upstream Watch;\n verify watch count is independent of client count and that late clients get\n the hub's current value (`internal/webui/live_hub_test.go`, real server over\n bufconn). Reopens resume after the last delivered version and charge failed\n attempts; accepting a gRPC stream does not establish its health.\n\n## Every new field\n\n- Test through the layering: build the effective config the way boot does,\n not by setting the field on the consumer.\n- Add a commented example to `config.example.toml`. Touch `deployments/*.toml`\n only when a profile there needs a value different from the default.\n- Update the wiring matrix in `.agents/skills/archie-config-and-flags`.\n- Secrets are `SecretRef`s resolved through the secret registry. Never inspect\n a resolved value, even in a test.\n- Reject a configuration that sets a field with no consumer rather than\n accepting and ignoring it (per-identity `forge.intake` is the precedent).\n"
|
||
},
|
||
{
|
||
"url": "/dev/development/daemon/",
|
||
"title": "Daemon",
|
||
"section": "Development",
|
||
"body": "# Daemon\n\nThe daemon claims and runs tasks, and turns captured events into tasks through\nbindings. This page covers both.\n\n## Task lifecycle\n\nAuthority: `docs/architecture/agent-system.md`. The status vocabulary lives in\n`internal/taskstate`, which imports no internal package.\n\n### Adding a status\n\n1. Add the constant to `internal/taskstate/taskstate.go` and place it in\n `Terminal`, `Actions` and `Statuses`. Its value is stored, so renaming it\n later is a migration.\n2. Alias it in `internal/domain/workflow/task/task.go` and\n `internal/domain/workflow/task_aliases.go`.\n3. Add it to the default catalogue in `ui/src/lib/task-meta.ts`. The server\n catalogue replaces the defaults when it loads, but the defaults render first.\n4. Find every hand-written status list in SQL:\n `grep -rn \"'merged'\" internal/infrastructure/postgres/queries`.\n `ClearTerminalTasks` must match `taskstate.Terminal`, which\n `TestClearTerminalTasksMatchesTaskstate` enforces. The stats queries count\n outcomes by name; decide whether the new status counts.\n5. Extend the table in `taskstate_test.go`.\n\n### Adding a task field\n\nFollow [State Store](/dev/development/state-store/) for the\ncolumn and the wire. Then decide who writes it: `UpdateTask` writes a fixed\ncolumn set, so a field set at enqueue needs its own statement (as\n`StampTaskBinding` does for binding provenance and inputs).\n\n### Changing what the agent receives\n\n- The task brief is `container.TaskPayload`, written to\n `\u003cworkspace\u003e/.git/task.json` by `WriteTaskJSON`. Never write it to the\n workspace root: the commit stage stages everything there.\n- The run request is `taskrun.Request`. `Validate` rejects a request that\n cannot run; a new requirement goes there.\n- Parking always carries a class (`taskstate.ParkTransient`,\n `ParkNeedsHuman`, `ParkTerminal`) chosen by the code that parks. Pick the one\n that says what an operator must do.\n- A task with no repository (`Task.HasRepository`) runs in a scratch workspace\n with no worktree grant. Code that assumes an owner and repo must check first.\n\n## Event automation\n\nAuthority: `docs/prds/event-automation.md` (the model: source, event type,\nmapping, binding, workflow, profile) and `docs/architecture/bindings.md`\n(threat model).\n\n### Where each check lives\n\nA binding is checked twice, and a new rule usually belongs in both places:\n\n- **On save**, in `checkedBinding` (`internal/webui/api_binding.go`), so the\n operator sees the problem while editing. It calls the domain checks\n `binding.Validate`, `binding.CompileFilter` and `Binding.CheckWorkflow`.\n- **At dispatch**, in `dispatchOneBinding` and `resolveBindingTarget`\n (`internal/daemon`), because the workflow or mapping may have changed since\n the save. A refusal is recorded with `recordDispatchFailure` and starts no\n task.\n\nPut the rule itself in `internal/domain/binding` or\n`internal/domain/workflow/task` and call it from both, rather than writing it\ntwice.\n\n### Adding a binding field\n\n1. Domain type and `Validate` in `internal/domain/binding`.\n2. Column, queries, store mapping, wire field, `newColumns`\n ([State Store](/dev/development/state-store/)).\n3. The request type and `checkedBinding` in `api_binding.go`.\n4. The draft, payload and editor in `ui/src/bindings/`\n ([Frontend UI](/dev/development/frontend-ui/)).\n5. Its effect at dispatch, with a case in\n `internal/daemon/dispatch_inputs_test.go` or `dispatch_bindings_test.go`.\n\n### Rules that must hold\n\n- Signing belongs to the source. Intake verifies the HMAC against the source's\n secret before the payload is parsed.\n- An event must be identified as an event type before anything dispatches it,\n and a binding applies only to its mapping's event type. Dispatch test\n fixtures therefore create an event type before inserting captures.\n- The dispatch ledger is claimed before the task is enqueued (at most once).\n- Anything an event supplies that selects what the agent acts on, such as a\n repository, must be checked against configuration. An event never widens\n what an agent can reach.\n"
|
||
},
|
||
{
|
||
"url": "/dev/development/frontend-ui/",
|
||
"title": "Frontend UI",
|
||
"section": "Development",
|
||
"body": "# Frontend UI\n\nAuthority: `.agents/skills/archie-web-ui` (stack facts, design-system rules,\ntone and accessibility). This page covers what a change must reach, not how it\nshould look.\n\n## Adding or changing a feature\n\n1. The feature lives in `ui/src/\u003cfeature\u003e/`. Move a helper to `ui/src/base/`\n only when a second feature needs it.\n2. Keep wire shapes, draft state and payload building in a plain `.ts` module\n beside the components (`ui/src/bindings/binding-draft.ts` is the pattern),\n and test it in `ui/test/*.test.ts`. Components stay thin.\n3. Every mutation goes through `api.*` in `ui/src/lib/api`, which sends the\n CSRF and Content-Type headers the server's mutation guard requires. A bare\n `fetch` is refused.\n4. A value the server also describes (task statuses, actions, workflow inputs)\n comes from the server. Keep a local default only so the first render works,\n and keep it in step (`ui/src/lib/task-meta.ts`).\n5. Run `task ui`, `task test:ui` and `cd ui \u0026\u0026 npm run typecheck`. Commit the\n regenerated `ui/dist/`; the Go binaries embed it. A fresh git worktree has\n no ignored `ui/node_modules`: run `cd ui \u0026\u0026 npm ci` before these commands.\n6. **One long-lived browser connection:** `ui/src/lib/api.ts` has the only\n `new EventSource`, owned by `stores/live-updates.ts`. Feature stores consume\n topics with `live.subscribe(topic, callback)`; do not open a page-owned SSE\n stream. Logs alone opt in while mounted, replacing that same EventSource\n with `?topics=logs\u0026since=\u003clast cursor\u003e`; a browser's `Last-Event-ID` only\n survives its own automatic reconnect, not a newly constructed EventSource.\n Task and log topics resume by cursor; control-plane, identity and apply-status\n topics replay the latest snapshot. Test a fetch and same-origin navigation\n after visiting Settings and Logs: the UI process serves HTTP/1.1 and six\n persistent streams can starve every later browser request.\n\n## Adding an API route\n\n1. The handler goes in the `internal/webui/api_\u003cconcern\u003e.go` file for its\n concern. Check the request fully on the server; the UI's checks are only a\n convenience.\n2. A mutation calls `authorizeTaskMutation` first.\n3. A route whose store is not configured answers 503, and\n `internal/app/archieui/degraded_routes_test.go` pins what each route\n returns without its contract. Add the route there.\n4. The dashboard runs in its own process (`archie-ui`) and must not link the\n daemon or the workflow engine (`cmd/archie-ui/architecture_test.go`). Reach\n stored data through `storecontract` interfaces wired in\n `internal/app/archieui/compose.go`.\n5. Settings that live in a control-plane resource need no new route: the\n Advanced settings page edits every resource from its generated schema.\n6. For a new live topic, add one process-wide producer to `internal/webui`'s\n `RunLive` and fan it through the `/api/stream` hub; `internal/app/archieui/run.go`\n starts the producers with the process context. Never create an upstream\n watch or poller in an HTTP handler. A slow client must reconnect for cursor\n and snapshot repair rather than silently lose an update. Prove the producer\n with several browser clients against the real peer, not just a scripted\n stream; `live_hub_test.go` is the example.\n"
|
||
},
|
||
{
|
||
"url": "/dev/development/merging-work/",
|
||
"title": "Merging work",
|
||
"section": "Development",
|
||
"body": "# Merging work\n\nParallel lanes (see `.agents/skills/archie-agent-fleet`) merge into an\nintegration branch before `main`. Each merge is a change in its own right and\npasses the same gate.\n\n## Resolving conflicts\n\n- **Generated files** (`*.pb.go`, `postgresdb/*.go`, `ui/dist/`,\n `docs/data/generated/*.json`): take either side, then regenerate. Never merge\n them by hand.\n- **One side deletes a file, the other edits it**: the deletion wins when the\n deleting lane replaced the thing the file served. Move the other side's\n change onto its replacement.\n- **One side changes how code is built (a new store, renamed packages), the\n other what it does**: take the behaviour side's version and reapply the\n mechanical change to it. That is usually cleaner than merging line by line.\n- **Error sentinels** are rows in `wireErrors`, not switch cases.\n\n## After resolving\n\n1. Search for leftovers of anything the other side removed: the old package\n path, the old type, and comments describing the old behaviour.\n2. Tests written against one side's semantics may pass there and fail after\n the merge. A test that used to pass on the old store may be asserting a rule\n the other side deliberately changed. Fix the test's setup to the merged\n rule; do not loosen the rule to the test.\n3. Run the full suite without `-short`, then `task check`, then commit. Check\n `git status` for files a test wrote into the tree before staging.\n4. **A merge can surface a bug neither lane's own tests exposed alone.** Two\n real cases from 2026-09-26: a test that dialled no NATS URL passed on a\n machine with an ambient dev server running and failed the moment it landed\n on a clean CI runner (start an ephemeral server per test, never rely on a\n default port resolving to something); and a cycle-detector that walked a\n Go map in range order reported the same cycle two different ways from run\n to run, invisible until enough fresh (uncached) runs hit the unlucky\n ordering. Both looked like \"the merge broke a passing test\" but were\n latent defects the merge only had the opportunity to expose. Fix the\n defect the failure is pointing at, not the assertion that caught it — see\n CLAUDE.md's test-quality bar before writing the replacement.\n5. `go test`'s cache reuses a package's last result when nothing in its\n dependency graph changed. A merge that touches one package can leave a\n sibling's flaky test silently cached \"pass\" locally while CI runs it\n fresh. `task check` passing on your machine is not proof a merge is\n clean — rerun anything nondeterminism-prone with `-count=1` before\n trusting it, and let CI's from-scratch run be the tiebreaker it is.\n"
|
||
},
|
||
{
|
||
"url": "/dev/development/state-store/",
|
||
"title": "State Store",
|
||
"section": "Development",
|
||
"body": "# State Store\n\nAuthority: `docs/prds/state-store-contract.md` and the State Store section of\n`CLAUDE.md`. All persistence is PostgreSQL behind the standalone State Store\nprocess; nothing else opens the database.\n\n## Adding a column\n\n1. Write the next migration in `internal/infrastructure/postgres/migrations/`\n (`00NN_name.sql`, goose Up and Down). Give the column a `NOT NULL DEFAULT`\n that is correct for existing rows.\n2. Update the queries in `internal/infrastructure/postgres/queries/` and\n regenerate with sqlc. A query that lists columns (`ListTaskSummaries`, the\n binding selects) does not pick the new one up; `SELECT *` queries do.\n3. Map it in the store (`taskFromRow`, `bindingValue` and friends). If the\n column holds encoded data, the package that owns the type owns the encoding\n (`task.EncodeInputs`), and the store and the wire both use it.\n4. Carry it over the wire (next section) if any process other than the State\n Store reads it.\n\n## Adding a field to a wire message\n\n1. Add it to `proto/state/v1/state.proto` with a new field number. Never reuse\n a number; retire one with `deprecated = true`.\n2. `task proto:generate`, then map it both ways in\n `internal/infrastructure/staterpc/values.go`.\n3. `TestTaskProtoCarriesEveryDomainField` fills every `task.Task` field and\n fails on one the mapping forgets. If it cannot fill the field's kind, extend\n it. Other messages have no such test, so add a round trip to\n `conformance_test.go`.\n\n## Adding an RPC or a store method\n\n1. Put the Go method on the narrow consumer interface in\n `internal/domain/storecontract` (keep each at eight methods or fewer) and add\n the matching RPC. A Go method without an RPC does not cross the process.\n2. Implement it in the Postgres store, in `staterpc.Client` (with its `var _`\n assertion) and in `staterpc.server`. A missing dependency answers\n `codes.Unavailable`, never a fabricated default.\n3. Wire the dependency in every composition that serves or consumes it:\n `stateStoreDeps` in `internal/app/archied/state_store.go`, the daemon's\n assertions in `bootstrap.go`, and `internal/app/archieui/compose.go` for the\n dashboard process.\n4. Task-scoped tokens may call only `Update`, `Transition` and `InsertEvent` on\n their own task (`staterpc.TaskGrants`). A new RPC is admin-only unless it is\n added there deliberately.\n5. Add it to `conformance_test.go`, which runs every surface over a real gRPC\n connection.\n\n## Adding an error sentinel\n\nAdd the sentinel to `storecontract`, a row to `wireErrors` in `values.go`, and\na case to `error_test.go`. The canonical message is part of the wire contract:\nchanging it breaks `errors.Is` on the client silently.\n\n## Harness OAuth secrets\n\n`HarnessSecretStore` stores token sets by org and credential service through\nadmin-only State Store RPCs. The EDA store seals them with the binding cipher's\nkeyring and a separate `HarnessSecretDomain`. OAuth writes require a configured\nbinding encryption key; the legacy binding plaintext fallback does not apply.\nThe daemon's egress proxy (`internal/infrastructure/egress/oauth.go`) is the\nonly daemon consumer: for a run granted the service it swaps sentinels for the\nstored tokens and writes every refresh back. The setup terminal is a separate\nharness slice.\n"
|
||
},
|
||
{
|
||
"url": "/dev/development/wiring/",
|
||
"title": "Processes and wiring",
|
||
"section": "Development",
|
||
"body": "# Processes and wiring\n\nAuthority: `docs/architecture/organisation.md` (layers, process boundaries),\n`docs/architecture/dependencies-and-contracts.md` (dependency direction, wire\ncontracts) and `docs/architecture/plugins-and-extensions.md` (the plugin engine\nrule).\n\n## Placing new code\n\n- Domain logic, entities and the interfaces it needs go in\n `internal/domain/\u003carea\u003e/`, which imports no infrastructure or app package.\n- Implementations of those interfaces go in `internal/infrastructure/\u003cservice\u003e/`.\n- Construction and wiring go in `internal/app/\u003capplication\u003e/`; `cmd/` only\n parses flags and calls it.\n- `logging`, `events`, `eventbus`, `policy` and `taskstate` import no internal\n package. There is no `shared/`, `utils/` or `common/` package.\n- A behaviour has one authoritative path. Before adding a second way to do\n something, remove or redirect the first.\n\n## Adding a capability to a process\n\n1. Decide which process owns it. A new binary is justified only when the\n surface must fail, scale or run somewhere independently.\n2. Wire it at that process's composition root. An optional capability\n degrades: a missing provider logs a warning and marks health degraded\n rather than stopping the process (MCP providers are the model).\n3. Every process that reaches the capability through the State Store gets it\n through a `storecontract` interface asserted on the client\n ([State Store](/dev/development/state-store/)).\n4. The architecture tests (`cmd/archie-ui/architecture_test.go`,\n `cmd/archie-messaging/architecture_test.go`, `TestOpenStoresNeverOwnsTaskDB`)\n pin what each binary may link. A failure there means the placement is wrong,\n not the test.\n\n## Adding a plugin engine\n\nSatisfy every invariant in \"Plugin engine rule (strict)\" in\n`plugins-and-extensions.md`: typed domain contract, an owning manager,\nlifecycle and health, narrow host access, and an explicit trust boundary.\n`internal/plugin/architecture_test.go` checks the parts that can be checked.\n\n## Adding a channel or a NATS subject\n\n- Channel rules (Telegram's pending-update drop, command scopes) are in\n `CLAUDE.md` under Per-Package Invariants.\n- A subject is part of the wire contract. Its payload type belongs to the\n domain that defines its meaning, and the transport code lives in\n infrastructure.\n"
|
||
},
|
||
{
|
||
"url": "/dev/development/workflows/",
|
||
"title": "Workflows",
|
||
"section": "Development",
|
||
"body": "# Workflows\n\nAuthority: `docs/architecture/agent-system.md` for the lifecycle,\n`docs/prds/event-automation.md` for inputs, repository modes and profiles.\n\nWorkflow definitions are YAML stored as a control-plane resource. The State\nStore validates them and the agent compiles them, each against the step\nvocabulary it registers, so both must see the same step types.\n\n## Adding a step type\n\n1. Write the factory in `internal/domain/workflow` as a `StepType`. It decodes\n and validates its settings and returns an error for anything it cannot\n honour. The factory runs when a definition is saved and again when it is\n compiled, so a bad definition is refused on save.\n2. Register it through a provider in\n `internal/infrastructure/workflowsteps/workflowsteps.go`. That one list is\n what every process registers, which is how the validating and executing\n sides agree.\n3. An agent-driven step builds on `AgentStage`; do not start a second path to\n the agent runner.\n4. A step that needs no repository goes in `repoFreeStepTypes` in\n `definition.go`, or workflows declaring `repository: none` or `optional`\n cannot use it. A step that needs one may assume `tc.Dir` holds a worktree\n only when the workflow's repository is `required`.\n5. A step that ends the workflow sets `tc.Outcome`. A workflow that runs out of\n steps without an outcome is parked as a definition bug.\n6. Add a parse test (settings accepted and refused) and a run test with fake\n `Trees` and agent runner.\n\n## Changing the definition format\n\n- The fields a caller needs without the engine (inputs, repository, profile)\n live in `task.WorkflowInterface`, because the dashboard process must not\n import the workflow engine (`cmd/archie-ui/architecture_test.go`). Anything a\n binding or the dashboard checks goes there.\n- The YAML decoder rejects unknown keys. Existing stored definitions must keep\n parsing, so a new key is optional with a default that preserves today's\n behaviour.\n- Examples in `examples/workflows/` are parsed against the registered\n vocabulary by `TestExampleWorkflowsParse`. Add one for a new capability.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/01-project-management/",
|
||
"title": "Architecture Decision Index",
|
||
"section": "PRDs",
|
||
"body": "# Architecture Decision Index\n\n**Status:** Approved \n**Date:** 2026-07-28 \n**Beads issue:** `archie-core-5d7`\n\n## Purpose\n\nThis file is the navigation point for Archie's target architecture. It is\ndeliberately short. Architectural decisions are separated by domain or\nrequirement so an implementation task can load only the rules governing its\nscope.\n\nImplementers MUST read the decision document for the domain or requirement they\nare changing. They MUST NOT infer permission to change neighbouring domains from\na broader restructuring objective.\n\n## Global invariants\n\n- Organise code around cohesive application responsibilities, not generic\n technical mechanisms.\n- Domains own their behaviour, vocabulary, state, commands, events, policy\n implementations, runtime settings, and required service contracts.\n- Domains do not depend on infrastructure or manipulate another domain.\n- `internal/app` owns composition; `cmd/*` remains thin.\n- Cross-domain work uses an explicitly owned command, event, or approved shared\n contract.\n- Process boundaries are deployment choices and do not determine source-code\n ownership.\n- Existing package names and boundaries have no presumption of survival.\n\nThe words **MUST**, **MUST NOT**, **SHOULD**, and **MAY** are normative.\n\n## Decision documents\n\n| Scope | Authoritative document |\n| --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |\n| Repository layout, domains, composition, deployments, and optional material | [architecture/organisation.md](/dev/architecture/organisation/) |\n| Dependency direction, domain contracts, infrastructure, commands, events, and wire contracts | [architecture/dependencies-and-contracts.md](/dev/architecture/dependencies-and-contracts/) |\n| External configuration, runtime settings, and dissolution of `internal/config` | [architecture/configuration.md](/dev/architecture/configuration/) |\n| Persistent actor identity, ownership, lifecycle, and attribution | [architecture/identity.md](/dev/architecture/identity/) |\n| Agent definitions, workflows, workflow executions, tools, and worker boundaries | [architecture/agent-system.md](/dev/architecture/agent-system/) |\n| Personal-assistant interactions, messaging channels, and optional promotion into durable work | [architecture/messaging-and-work-intake.md](/dev/architecture/messaging-and-work-intake/) |\n| Shared policy evaluation and domain-owned policy semantics | [architecture/policy.md](/dev/architecture/policy/) |\n| Safe source/configuration changes, health supervision, and recovery | [architecture/safe-change-and-recovery.md](/dev/architecture/safe-change-and-recovery/) |\n| Plugin framework, plugin implementations, and extension isolation | [architecture/plugins-and-extensions.md](/dev/architecture/plugins-and-extensions/) |\n| Generated references, deprecations, and changelog generation | [architecture/generated-documentation.md](/dev/architecture/generated-documentation/) |\n| Remaining package destinations, data migrations, runtime cutover, and completion criteria | [architecture/migration-decisions.md](/dev/architecture/migration-decisions/) |\n| Domain/feature research, design interview, and required handoff | [architecture/package-review.md](/dev/architecture/package-review/) |\n\n## Deferred decisions\n\nThe remaining package-placement, data-migration, shared-mechanics, process-\nboundary, and cutover decisions are maintained in\n[architecture/migration-decisions.md](/dev/architecture/migration-decisions/).\n\nEach decision is made in its focused document after code-grounded review.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/RULES/",
|
||
"title": "PRD writing rules",
|
||
"section": "PRDs",
|
||
"body": "# PRD writing rules\n\nThese rules apply to every file in `docs/prds/`.\n\n- Search existing PRDs, issues, architecture records, and code first. Do not ask\n the maintainer to repeat recorded requirements.\n- A newer maintainer decision overrides an older design. Mark the old design as\n superseded instead of preserving or defending it.\n- Write plain English. Use a technical term only when the reader needs that\n exact contract or concept.\n- Use the shortest direct sentence. Write “When upgraded, existing instances\n are migrated to the database,” not a description of an importer seeding\n absent resources.\n- State each fact once. Do not repeat a heading, summarise a list before showing\n it, or restate a decision afterward.\n- Every sentence must add a decision, requirement, constraint, evidence, or\n verification step. Delete narration and explanations of obvious text.\n- Do not turn a requested outcome into extra product features. Keep inferences\n open until the maintainer or repository evidence settles them.\n- Do not create a process for a logical boundary. Add one only when it must\n deploy, scale, start, secure, or fail separately.\n- Describe working behaviour, not explanatory UI around missing behaviour. State\n what users can do and how success is proved.\n- Before approval, use independent read-only reviewers to check a cross-cutting\n PRD for missing behaviour and exaggerated claims. Verify and synthesise their\n findings; do not copy their prose.\n- Before requesting approval, review every sentence for complexity, repetition,\n and unsupported scope.\n- Write nothing whose truth expires. No `file.go:123` citations (cite the file\n and the symbol), no progress reports on the work being designed, no dated\n status claims, and no record of how a finding was produced. A phased plan is\n durable design; which phase is finished is not. Do not refer to what this\n document used to say: the reader cannot see it. A supersede marker is fine\n when it names a file, a section or a symbol they can find. `task docs:prds`\n enforces this.\n- Every PRD carries one of three statuses, and nothing else in that field.\n `Draft` until the maintainer explicitly approves it, `Approved` once they\n have and while the work is being built, `Finalised` when the work is\n complete. A finalised PRD does not change; supersede it with a new one\n instead. A revision number belongs in `**Revision:**`, a pointer to the\n authoritative document in `**Authority:**`, and what a past revision changed\n belongs in git.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/action-playbook-run/",
|
||
"title": "Running action playbooks",
|
||
"section": "PRDs",
|
||
"body": "# Running action playbooks\n\n**Status:** Approved\n**Date:** 2026-09-23\n**Parent:** `docs/prds/multi-action-playbooks.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-t2db.31`\n\n## Why this exists\n\nAn action playbook loads and type-checks but never runs: `Store.Dispatch`\nskips it and the daemon logs a warning at boot (`multi-action-playbooks.md`,\nD1). This document defines when it runs, what each action does, and how the\n`playbook_dispatches` ledger gates it.\n\n## When a playbook runs\n\nThe daemon runs action playbooks at the point it pins a task's workflow\ndefinition (`Daemon.resolveWorkflowID`), using the same `DispatchInput` the\nworkflow playbooks use. The definition is pinned once per task, so the run\nhappens once per task.\n\n- Every action playbook whose trigger matches runs, in load order. A workflow\n playbook stops at the first match because it chooses one workflow; action\n playbooks choose nothing, so no match suppresses another.\n- A run never changes routing. The task is pinned by the workflow playbooks\n and bindings exactly as when no action playbook exists.\n- An action or playbook failure never parks or fails the task. It is logged\n at warning level with the playbook, the action and the task.\n- A task that already names a workflow (the approval requeue) is skipped, as\n it is for workflow playbooks.\n- Actions run synchronously in the pin path, bounded by the daemon's context.\n The only Module kind, `log`, returns immediately. A slow kind must bring its\n own timeout.\n\n## What one action does\n\nActions run in order against one evaluation context whose `actions` map\nstarts empty.\n\n1. Evaluate `when`. False or an evaluation error skips the action (J3). A\n skipped action records nothing and produces no result.\n2. Evaluate every `args` value. An evaluation error stops the playbook.\n3. Call `playbook.InvokeOnce` with the Module invoke as its function. The\n ledger row is written first. A duplicate row skips the invoke.\n4. An invoke error stops the playbook. The row stays, so the action is not\n retried: the ledger promises at most once, not at least once.\n5. On success, `ModuleRegistry.DecodeResult` converts the result, and it is\n stored as `actions.\u003cid\u003e.result` when the action declares an id.\n\nThe event id in the ledger key is `DispatchInput.TaskID`, the issue's\nidempotency key (`owner/repo/number`). An action therefore runs at most once\nper issue for each playbook version. Editing a playbook changes its version\nand lets it run again for issues it already handled.\n\nA skipped duplicate has no result. A later action that reads it sees a\nmissing field: its `when` evaluates false, or its args fail and the playbook\nstops. This only happens when the same issue is pinned twice under one\nplaybook version.\n\n## Wiring\n\n- `storecontract.PlaybookDispatcher` is lifted onto the daemon from the State\n Store client in `internal/app/archied/bootstrap.go`, the same way\n `BindingDispatcher` is.\n- The daemon's `Playbooks` dependency gains the run method. The Module\n registry is passed to the playbook store at load, so the daemon needs no\n new dependency for it.\n- With no `PlaybookDispatcher` wired, action playbooks do not run, and the\n boot warning names each one. They never run without the ledger.\n- The boot warning that action playbooks do not execute is removed when the\n ledger is wired.\n\n## Verification\n\n- A matching two-action `log` playbook runs both actions in order, and the\n second reads the first's result.\n- Pinning the same task twice under one playbook version invokes each action\n once.\n- A `when` that evaluates false skips that action and the run continues.\n- An invoke error stops the playbook. The task is still pinned to the same\n workflow it would get with no action playbook.\n- A non-matching trigger runs nothing.\n- With no ledger wired, nothing runs and the warning names the playbook.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/adversarial-self-review/",
|
||
"title": "Adversarial self-review stage -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Adversarial self-review stage -- decision\n\n**Status:** Finalised\n**Authority:** `docs/architecture/adversarial-review.md` describes the stage and is authoritative. This PRD remains the design-decision record.\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-h019.1`, blocks `h019.2/.3/.4/.5`\n\n\u003e **Section 1 is superseded by the decision below.**\n\u003e The original execution mechanism assumed daemon-owned in-process execution\n\u003e and a daemon-side second `Pool.Acquire`; `archie-core-q15y` removed both\n\u003e call sites, leaving one open question: how a fresh reviewer receives an\n\u003e isolated snapshot from inside the task-scoped worker, without a host\n\u003e executor or a nested container lifecycle. Section 1 below now answers it\n\u003e with `agent.Subagent`. Sections 2-5 were never invalidated and still hold;\n\u003e section 3's contract shipped as `archie-core-h019.2`.\n\nAnswers the five questions in `h019.1`'s description against the actual\nimplement workflow, not designed from scratch: the gate pattern, the\ncontainer pool, and the workflow engine's `Outcome` mechanism already answer\nmost of this.\n\n## 1. Where it runs: `agent.Subagent`, inside the task worker\n\nThe implement workflow's stage list\n(`internal/domain/workflow/implement.go`) already has the right slot:\nafter `StageDiffCap()`, before `StageOpenPR()` -- gates\nhave passed, the diff is final, and nothing has told the forge about this\nchange yet.\n\n**The reviewer is an `agent.Subagent` run, not a second container.** ai-sdk\n`v0.1.22` (already the pinned version) added `agent/subagent.go`: a nested,\nsynchronous `core.GenerateText` call in the same process, with its own model,\nsystem prompt, toolset and step budget. That is what makes this feature\nbuildable at all after `q15y` -- the reviewer runs inside the task-scoped\n`archie-agent` worker that is already executing the workflow, so there is no\nhost executor, no second `Pool.Acquire`, and no nested container lifecycle.\n\n**Two isolations are needed, and Subagent only provides one of them.**\n\n_Conversation isolation is structural, and free._ `Subagent.Run`\n(`agent/subagent.go`) calls `GenerateText` with `Prompt` set and\n`Messages` never populated. The implementer's history has no code path into\nthe nested run; only the prompt string crosses, and only text comes back.\nThis is the literal form of \"environmental enforcement over prompt rules\":\nthe reviewer is not told to ignore the implementer's reasoning, because\nnothing hands it any. `h019.3`'s required test asserts this as a construction\nfact.\n\n_Workspace isolation is still ours to build._ A sub-agent shares the parent's\nprocess, working directory and credentials. So the `.git`-free snapshot from\nthe original decision survives unchanged and is load-bearing: export the\nreviewed commit (`git archive \u003ccommit\u003e | tar -x`) into a fresh scratch\ndirectory beside, not inside, the task worktree, and write `diff.patch` (the\ndiff already computed for `StageYaegiGate`'s `Trees.Diff()`) and `issue.md`\n(the originating issue's title and body, nothing else) alongside it. Stripping\n`.git` is what makes commit messages, branch name and reflog -- the actual\nchannel a multi-commit implementer's reasoning leaks through -- structurally\nunreachable rather than merely undisclosed.\n\n`Subagent.Tools` therefore takes a **distinct read-only toolset rooted at the\nsnapshot**. Never the worker's own toolset: it carries the `worktreerpc`\npublication grant, so a reviewer holding it could push the branch it exists to\nblock.\n\n**Call `Run()`, not `Tool()`.** `Tool()` would register delegation as a tool\nthe _implementer's_ model may choose to invoke -- self-certification by the\nimplementer, which is the exact failure this feature exists to prevent. The\nstage calls `Run()` unconditionally. That also sidesteps both of `Tool()`'s\ndocumented footguns: the parent's two-step minimum and the recursion risk of\npassing a toolset back to itself.\n\n**Findings return through a closure, not the return value.** `Run` yields only\nfinal text, so section 3's structured contract cannot ride the return. Because\nthe sub-agent is in-process, the capture tool's Go implementation appends\ndirectly to a `[]workflow.ReviewFinding` the stage owns. This preserves the\n\"cannot end its turn without calling a typed tool\" property, and it defeats\n`Subagent`'s documented \"no partial results on failure\" limitation: findings\nalready captured survive an error return, so **the stage reads the captured\nslice even when `Run` returns an error**. A reviewer that logged three defects\nand then hit its step limit still yields those three.\n\n## 2. What it's given, and what it must never be given\n\n**Given:** the clean file-content snapshot (so the reviewer can read whole\nfiles -- follow an import, check interface satisfaction, see a call site --\nnot just diff hunks, which the manual-read checklist in CLAUDE.md requires),\n`diff.patch`, and `issue.md`.\n\n**Never mounted or passed, by construction:** the implementer's\n`agentexec.Result` (summary, captures, stop reason), any prior turn's tool-call\ntranscript, commit messages/git history/branch name (excluded by stripping\n`.git`, not by instruction), daemon internal state, and any other task's data.\nThis is the environmental-enforcement principle applied literally: the\nreviewer is never told \"ignore the implementer's reasoning,\" because there is\nno code path that hands it any.\n\n## 3. Findings contract: `workflow.ReviewFinding`, workflow domain owns it\n\nNot a reuse of `internal/gate.Finding` (`gate.go`), despite the similar\nshape. That type belongs to the Yaegi gate contract (`.archie/gate.go`,\nrepo-owner-authored checks); coupling the review stage's blocking contract to\ngate's evolution would violate \"a domain declares the interface it needs, it\nnever names who implements it.\" `h019.2` implements, in\n`internal/domain/workflow`:\n\n```go\ntype ReviewFinding struct {\n Level string // \"error\" (blocks) | \"warn\" (advisory)\n Category string // dead-code | unchecked-error | hardcoded-value |\n // interface-satisfaction | nil-risk | goroutine-leak |\n // race | other -- CLAUDE.md's own checklist categories\n File string\n Line int\n Message string\n}\n\nfunc ReviewBlocking(findings []ReviewFinding) bool // mirrors gate.Blocking\n```\n\nThe domain owns the type and the pure blocking decision; turning the\nreviewer agent's raw output into `[]ReviewFinding` is app/infrastructure code\n(`h019.3`/`h019.4`), not domain code -- same direction as gate's Yaegi\nevaluator producing `[]gate.Finding` for the domain to consume.\n\n**Findings are reported through a structured capture tool**, the same\n`agentexec.CaptureTool` pattern `feasibility.go`'s `decideCaptureTools`\nalready uses, not free text the caller regexes out of a final message. This\nis the findings contract's own application of environmental enforcement:\nthe reviewer cannot end its turn without calling a typed tool, rather than\nbeing asked to format its answer as JSON and trusted to comply.\n\n## 4. Lifecycle effect of a surviving finding: `StatusParked`, not `WaitingHuman`\n\nThe epic's acceptance criteria say a surviving finding \"parks the task as\n`waiting_human`,\" which names two different states in the actual machine\n(`internal/taskstate/taskstate.go`). This decides between them:\n**`store.StatusParked`**, following `StageDiffCap`'s existing precedent\n(`steps.go`) exactly -- a stage-computed policy stop with a `Detail`\nexplaining why, before PR open. `Parked`'s available actions, retry and\nabandon, are the right shape here: fix the flagged code and retry, or\nabandon/override if a finding is a false positive. `WaitingHuman`'s\napprove/reject actions have no obvious referent -- there is no PR yet to\napprove or reject.\n\nA `Level == \"error\"` finding that survives sets\n`tc.Outcome = Outcome{Status: store.StatusParked, Detail: \u003crendered findings\u003e}`\nand the stage returns `nil`; `StageOpenPR` never runs. `Level == \"warn\"`\nfindings, and the zero-findings case, leave `tc.Outcome` unset and the stage\nreturns `nil`, so `StageOpenPR` runs next. Attaching warn-level findings to\nthe task record for PR-body/dashboard display is `h019.6`'s job, not decided\nhere.\n\n**A review that fails to run is fail-closed.** The\nblocking rule is `Verdict == confirmed \u0026\u0026 Level == error`, so only a confirmed\nfinding blocks -- but that governs findings, not the reviewer's own failure. A\nsub-agent that errors, is truncated, or produces no conclusion yields\n`ReviewStatusNotRun`, for which `ReviewReport.Passed()` is false and\n`Blocking()` is also false; the contract deliberately refuses to guess. This\nstage resolves that as **park, do not open the PR**. The reasoning is that\n`Repo.ReviewEnabled` is opt-in: an operator who turned the gate on wants the\ngate, and a provider outage silently degrading into \"PR opened unreviewed\" is\nthe failure they were buying protection against. The cost is accepted -- a\ntransient model error parks a task, and `Parked`'s retry action is the\nremedy. `Detail` must say the review did not run, never that it passed.\n\nNote this is why `h019.2` models \"ran and found nothing\" separately from \"did\nnot run\": collapsing them to an empty slice would make an outage\nindistinguishable from a clean review, and fail-closed unimplementable.\n\n## 5. Model, effort, and whether a repo even runs this yet\n\n**Model:** a new `\"reviewer\"` key in the existing `Config.Models` map (already\na generic `map[string]string`, no schema change) with the same single-level\nfallback precedent `agent.go` already uses for planner-\u003ebuilder:\n`Models[\"reviewer\"]`, falling back to `Models[\"builder\"]` if unset. Operators\nwho want a genuinely independent second opinion configure a distinct model;\nnothing forces one, because CLAUDE.md's project scope is \"smallest change\nthat works,\" not a mandated model-diversity policy.\n\n**Effort/budget:** reuse the per-stage budget override `AgentStage` already\nsupports (`agent.go`), not a new config surface -- a repo wanting a\ncheaper or more thorough reviewer sets `MaxSteps` the same way\nthe plan stage already can. Map that budget onto `Subagent.MaxSteps` and\n`Subagent.MaxTokens` deliberately: **`MaxTokens` is a per-call ceiling, not a\ncumulative budget**, so an N-step reviewer can emit up to N x `MaxTokens`.\nSetting it as though it bounded the whole run is a costing error, not a\nsafety one. `Subagent.MaxSteps` defaults to 10 when unset.\n\n**Observability:** the nested run is synchronous and non-streaming -- nothing\nis emitted between the parent's call and its result, so a long review reads as\na stalled stage. Emit task events around the call so the dashboard shows the\nreviewer working rather than a gap.\n\n**Opt-in, not automatic:** `h019.5` (calibration -- confirm findings\nadversarially before they surface) is a separate, lower-priority bead than\n`h019.4` (wiring). Shipping the stage unconditionally the moment `h019.4`\nlands would gate every existing repo's PR flow on an uncalibrated reviewer\nbefore calibration exists, which is exactly the \"reviewer that cries wolf\"\nrisk the epic names as the thing that kills this feature. `h019.4` adds\n`Repo.ReviewEnabled bool` (default `false`) to `internal/config`; an operator\nopts a repo in once they trust it, the same shape as any other per-repo gate\nsetting.\n\n## Packages this touches\n\n- `internal/domain/workflow` (`h019.2`, `h019.4`): `ReviewFinding`,\n `ReviewReport`, `ReviewBlocking` -- shipped -- and a `ReviewStage` inserted\n between `StageDiffCap` and `StageOpenPR` in `implement.go`. The domain owns\n the contract and the pure blocking decision, and declares the reviewer\n interface it needs; it never names `agent.Subagent`.\n- `internal/worktree` or a sibling package (`h019.3`): the clean-snapshot\n export (`git archive`, no `.git`) building the reviewer's isolated\n workspace. Still required -- `Subagent` isolates conversation, not\n filesystem.\n- `internal/app/agentworker` (`h019.3`): constructs the `agent.Subagent` from\n the worker's own resolved provider (`agentexec.NewRuntime`), with a\n read-only toolset rooted at the snapshot and the findings capture tool. This\n is app-layer wiring: the domain gets an implementation of its reviewer\n interface, not an SDK type.\n- `internal/config` (`h019.4`): `Models[\"reviewer\"]` (no type change) and\n `Repo.ReviewEnabled bool`.\n- `internal/store`/`internal/taskstate`: unchanged; reuses `StatusParked`.\n- Surfacing findings on the PR body and dashboard is `h019.6`, out of scope\n here.\n\n## What this does not decide\n\nThe exact review checklist prompt content, how many findings can survive\nbefore `MaxSteps` is exhausted, and the adversarial-confirmation mechanism\nthat keeps a finding from surfacing on a single reviewer's say-so are\n`h019.5`. The PR-body/dashboard rendering of findings is `h019.6`. This\ndocument decides isolation, inputs, the findings contract's owner and shape,\nthe lifecycle effect, and model/effort/opt-in configuration only.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/agent2agent-protocol/",
|
||
"title": "Agent2Agent as archie's agent-to-agent protocol — decision",
|
||
"section": "PRDs",
|
||
"body": "# Agent2Agent as archie's agent-to-agent protocol — decision\n\n**Status:** Draft\n**Date:** 2026-09-22\n**Authority:** the A2A Protocol Specification v1.0.0 (`a2a-protocol.org`) for the\nprotocol, `docs/architecture/organisation.md` for package placement, and\n`docs/architecture/migration-decisions.md` §2 for session and messaging design.\n\n## Decision\n\nA2A is the protocol archie speaks to other agents. It carries agent-to-agent\nwork: discovery of what a remote agent offers, delegation of a unit of work to it,\nprogress on that work, and the artifacts it produces.\n\n## Layers\n\n**MCP stays the tool layer.** MCP connects a model to tools and data; A2A connects\nagents to agents. They are complementary and archie runs both:\n`internal/tools/provider` registers MCP providers as optional capabilities, so an\nA2A call and an MCP tool call are different kinds of traffic with different\nowners.\n\n**A2A rides archie's existing transports.** A2A's canonical data model is\nProtocol Buffer messages and it defines a gRPC binding, so archie speaks A2A over\nthe gRPC and NATS it already runs. A second wire format, a second broker, or a\nbespoke protocol for agent-to-agent traffic is a requirement not to introduce.\n\n## Where A2A stops\n\nA2A is not a general replacement for either of the two protocols archie already\nhas, and each boundary is a requirement rather than a preference.\n\n- **`internal/domain/messaging` stays the human and channel layer.** A\n Conversation is a ChannelID and a ThreadID, its participants are people on\n Telegram, email, webhook and the dashboard, and its records are what a curator\n reads. An A2A peer is not a channel participant and does not address a\n Conversation.\n- **`internal/agentexec` stays the daemon-to-worker execution protocol.** It\n carries a Request, a Result, gate and budget policy, and capture tools between\n the daemon and the agent container it owns. An A2A peer never receives that\n protocol, and A2A never carries gate, budget or protection policy.\n\nA2A is therefore an edge protocol: it exchanges work with peers outside archie's\nown process boundary, and it composes with the two protocols above rather than\nreplacing or abstracting them.\n\n## A2A Task mapping\n\n**Decision: archie's work lifecycle is mapped to A2A's, not expressed in it. An\nA2A Task is a view over archie's own task, and archie's status remains the record\nof truth.**\n\nThis is the load-bearing decision, because A2A's Task carries its own state\nmachine and archie's lifecycle is the on-disk format of its task store, whose\nstatus strings a rename migrates rather than changes.\n\nThe mapping is partial, and each gap is a case where archie knows something A2A's\nstates do not carry:\n\n- **Completion by another system.** A2A's terminal states describe what the agent\n did (`completed`, `failed`, `canceled`, `rejected`). archie's `merged` and\n `rejected` describe what a forge did to a pull request after archie finished\n its work. There is no A2A state for \"an external system accepted this\", so a\n completed A2A task cannot express the outcome that matters most to the caller.\n- **Handing work to a reviewer.** archie's `pr_open` means the work left\n archie's hands for human review. A2A's `input-required` is the nearest state,\n but it means the _client_ must send more input, and a reviewer is not the\n client.\n- **Operator decisions.** `declined`, `parked` and `dead` are outcomes of\n archie's own policy and operator actions. A2A has no state for \"the operator\n refused this\" or \"a budget ran out and an operator must decide\".\n\nA WorkflowExecution is the unit an A2A task maps onto, because it is the attempt\nthat runs, streams and produces artifacts, and a follow-up attempt is a new\nexecution. A2A's own model agrees that a task is immutable once terminal and that\na refinement is a new task sharing the `contextId`, which is why archie's\nper-attempt executions and A2A's per-request tasks correspond.\n\n**Requirement:** the A2A state a peer sees is derived from archie's status by one\nmapping, and archie's status is carried on the task so a peer can read what the\nderivation lost. The mapping is total in the direction that matters -- every\narchie status has an A2A state -- and deliberately not injective, because the\ngaps above are facts A2A cannot hold.\n\n**Requirement:** adopting A2A's state names as archie's persisted task status is\nrejected. It would migrate the task store, and it would drop the forge outcomes\nthat tell a caller whether the work was accepted.\n\n## What an A2A agent is\n\n**An A2A agent is an archie Identity.** The Agent Card describes a durable,\naddressable actor with an endpoint, capabilities and authentication requirements,\nand an Identity is the actor archie persists, versions, suspends and scopes\ncredentials to.\n\nAn `agent.Persona` is not an A2A agent: a persona is prompt text selected for a\nturn, and it has no endpoint or credential. An agent container is not an A2A\nagent: it is task-scoped and ephemeral, and it terminates with the work it ran.\nThe daemon is not an A2A agent: it is the process that serves the identity.\n\n## The Agent Card\n\n**The Gateway serves the card, for the identity it is configured as.** A2A's\n\"Get Agent Card\" is an abstract operation with a gRPC binding, so the card is\nserved through the binding archie runs and needs no HTTP listener.\n\n**The card is derived, never hand-written.** Its skills come from the skill\ncatalogue, its identity from the configured Identity, and its capabilities from\nwhat the process already advertises about itself. A card that restated\ncapabilities independently would be a second inventory of the same facts, which\nis the class of drift the existing capability reporting exists to prevent.\n\n**The card advertises the authentication archie applies**, and it carries no\nsecret. A card that named a token would be a credential published at a\nwell-known location.\n\n## Server before client\n\n**archie is an A2A server first.** Accepting A2A calls is the smaller direction\nand the more valuable one: the work a peer would delegate -- an issue taken to a\npull request, through archie's gates, worktrees and review -- exists only inside\narchie, and archie already serves its own contracts over gRPC to the Messaging\nService, the dashboard and the State Store. Acting as an A2A client to remote\nagents is a later capability, and one that consumes the same contract.\n\n## Authentication and authorisation\n\n**An inbound A2A call is authenticated at the transport, by the same bearer-token\ninterceptor the Gateway applies to its other contracts.** A2A carries no identity\nin its payload by design: identity is established where the connection is, so the\ntransport token is the identity and no field in a Message or Task names a caller.\n\n**Authorisation is per skill**, which is A2A's own model: the card advertises\nskills, and a caller may invoke the skills its credential is scoped to.\n\n**Task-scoped grants are not A2A credentials.** They scope a worker's access to\none task's records inside the State Store and are issued by the daemon for its own\ncontainers. Reusing one as an inbound A2A credential would let a peer act with a\nworker's authority, which is the isolation the grants exist to enforce.\n\n**The dashboard's authentication is not an A2A scheme.** It gates an operator\ninterface, not an endpoint that agents call, so it authenticates no A2A request.\n\n## Package placement\n\n- The contract -- A2A's data model, operations and the mapping from archie's\n status -- lives in `internal/domain/\u003carea\u003e/`.\n- The binding that carries it over gRPC lives in `internal/infrastructure/`,\n beside the existing gRPC clients and servers.\n- Wiring of the contract to the Gateway and the task store lives in\n `internal/app/`.\n\n## Verification\n\n- A peer fetches the Agent Card over gRPC and receives the configured identity's\n identity, skills and authentication requirements.\n- A delegated task reaches an archie task, and the A2A state a peer observes\n changes as archie's status changes, through the one mapping.\n- A task that reaches `merged` reports an A2A state derived from it, and the\n archie status is readable on the task.\n- A call with no credential, and a call with a credential scoped to another\n skill, are both refused, with the refusal A2A defines for them.\n- A task-scoped grant presented as an inbound A2A credential is refused.\n- No A2A traffic reaches `internal/domain/messaging` or `internal/agentexec`.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/artifact-sender/",
|
||
"title": "Artifact sender",
|
||
"section": "PRDs",
|
||
"body": "# Artifact sender\n\n**Status:** Approved\n**Revision:** 1\n**Authority:** this document\n\n## Purpose\n\narchie-core publishes artifacts — design documents, review reports, research briefs,\nrunnable pages — to the offloaded.dev collaborative editor. A person reads the\nartifact, comments on it, and the artifact page is the shared surface for iteration.\n\nThe receiving contract is owned by the editor: `POST /api/artifacts` with a bearer\nservice token, a closed field set, and server-side idempotency on `requestId` plus\nlast-write-wins on `sentAt`. This document owns only what archie-core sends.\n\n## Scope\n\nVersion one publishes one-way: archie-core sends, the editor receives. The first\nproducer is the pull-request adversarial review: a completed `ReviewReport` is\nrendered to markdown and published after the review completes.\n\nChat-surface tools (`create_artifact`, `update_artifact`), comment read-back, share\ntokens, and runnable-format producers (`html`, `react`, `vue`) are later slices. The\nreceiving contract already carries them; version one does not exercise them.\n\n## Configuration\n\n- New `[artifacts]` section: `base_url` (site origin), `token` (`SecretRef`) with\n `token_env` fallback naming `WORKSPACE_INGEST_TOKEN`.\n- An empty `base_url` disables the sender: no client is constructed, nothing is\n registered, and publishing degrades to a log line.\n- The token is runtime configuration. It is never baked at build time.\n\n## Sender\n\n- New package `internal/infrastructure/artifactsync` exposing one method,\n `Publish(ctx, Artifact) (string, error)`, where the string is the artifact URL.\n One-method injected seam, concrete client constructed by the boot path when the\n configuration resolves — the `crondelivery.Courier` posture.\n- The client POSTs the frozen contract body. Each publish sends the complete\n source: versions of record are full snapshots, never diffs.\n- A `503` response is retried three times with backoff. `401` and `400` are not\n retried; they are logged with the field reason the response names.\n- `requestId` is `artifact:\u003ctask-id\u003e:\u003cversion\u003e` when the producer is task-backed,\n otherwise `artifact:\u003cidentity\u003e:\u003csentAt-utc\u003e`. Sending the same `requestId` with\n an equal or older `sentAt` must be a server-side no-op, which makes sender-side\n retry safe.\n\n## Report rendering\n\n- `ReviewReport` gains a pure markdown renderer in `internal/domain/workflow`:\n the summary as the lede, one finding per subsection with file, line, verdict,\n level and the failure scenario, and the verified properties as a \"checked\n clean\" list. Findings carry their disposition when present.\n- The renderer performs no I/O and imports no infrastructure.\n\n## Attribution\n\n- `actor`: the producing identity. The reviewer publishes as `system:pr-reviewer`\n when no task identity resolves; otherwise the task's identity name.\n- `source`: the producing subsystem, `pr-review` for version one.\n- `task`: owner, repository and issue number from the task when one exists.\n\n## Failure behaviour\n\n- Publishing never fails the producing work. A failed publish is logged with its\n status and reason; the artifact is republished on the next production of the\n same artifact.\n- A successful publish records the artifact URL in the run report, so the PR\n description or task log links to the readable artifact.\n- The published URL is the editor deep link for the artifact, keyed on its\n immutable id. archie-core never mints share links; sharing is an\n account-holder decision made in the editor.\n\n## Verification\n\n- Client tests against a test server cover 201, 401, 400 with a field reason,\n and 503 retry behaviour.\n- The report renderer test covers a report with findings, checks, dispositions\n and a not-run status.\n- Configuration tests cover the disabled case, the `SecretRef` path and the\n `token_env` fallback.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/binding-secret-encryption/",
|
||
"title": "Binding secret at-rest encryption -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Binding secret at-rest encryption -- decision\n\n**Status:** Approved\n**Date:** 2026-09-02\n**Beads issue:** `archie-core-t2db.7` (P2), parent `archie-core-t2db`\n\n## Problem\n\n\u003e Storage superseded: every store now lives in PostgreSQL behind\n\u003e `internal/domain/storecontract`, implemented in `internal/infrastructure/postgres`\n\u003e (`docs/prds/state-store-contract.md`). SQLite names below describe the\n\u003e original design.\n\n`binding.Binding.Secret` (the per-source HMAC shared secret) is persisted as\nplaintext in the `bindings.secret TEXT` column. Anyone with read access to the\nSQLite file -- the same access level as `captured_events.body` and `tasks.body`\n-- sees sender-side shared secrets. A leaked database file leaks secrets even\nwhen the filesystem ACLs were correct at rest.\n\n## Decision\n\nEncrypt the `secret` column at rest with **AES-256-GCM** (stdlib\n`crypto/aes` + `crypto/cipher`, no new dependency), transparently inside the\nstore: encrypt on write, decrypt on read. The store is the data boundary; its\nconsumers (dispatch loop, HMAC verification) receive the plaintext secret\nexactly as and the webui already blanks `Secret` on every response.\n\n### Cipher parameters\n\n- **AEAD:** AES-256-GCM, 12-byte random nonce (`crypto/rand`), tag appended.\n- **Key length:** 32 bytes, derived as `SHA-256(resolved_material)`. Kept as\n SHA-256 (no Argon2id) because the source is a **32-byte high-entropy** value\n resolved through the secret registry, so re-deriving to 32 bytes merely\n re-shapes entropy rather than defending a weak passphrase.\n- **AAD:** a static domain separator, `[]byte(\"arcie-binding-secret\")`, so a\n ciphertext cannot be relocated to a different field/column and still\n authenticate. AAD is intentionally **not** row-bound: binding secrets are\n random HMAC keys and are never used as lookup keys, so cross-row swap is not\n a meaningful attack here.\n\n### Envelope format\n\n```\narcie-binding:v1:\u003ckeyFingerprint\u003e:\u003cbase64url(nonce || ciphertext || tag)\u003e\n```\n\n- `arcie-binding` -- domain marker\n- `v1` -- format version (future cipher/KDF change is a new version, old rows\n stay readable)\n- `\u003ckeyFingerprint\u003e` -- `hex(SHA-256(material))[:16]`, the identity of the key\n that sealed the row, enabling rotation without a one-shot rewrite\n- payload -- `base64url` of `nonce ‖ GCM.Seal(plaintext, aad)`\n\n### Key source (confirmed with maintainer)\n\nResolved through the existing secret registry (`internal/secret`), the\nestablished `SecretRef` pattern already used for `forge.webhook_secret`:\n\n```toml\n[bindings]\nencryption_key = { engine = \"env\", key = \"ARCHIE_BINDING_KEY\" }\n# Rotation: after swapping encryption_key to a new value, retain the previous\n# material so rows sealed under the old key still decrypt.\nprevious_encryption_keys = [ { engine = \"env\", key = \"ARCHIE_BINDING_KEY_OLD\" } ]\n```\n\nThe active key encrypts new writes; `previous_encryption_keys` are retained for\ndecrypt-only. Key identity is intrinsic to the material (the fingerprint), so\nno manual key IDs are configured.\n\n### Rotation story\n\nRotate by changing `encryption_key`, moving the old material into\n`previous_encryption_keys`. Rows sealed under the old key still decrypt via\ntheir embedded fingerprint; new writes use the new active key. Once no\nremaining row references an old fingerprint, remove it from the config. Losing\na fingerprint's material before all its rows are rewritten makes those rows\nunreadable -- the same operational rule as any key rotation, and the reason the\nold material must be retained.\n\n## Non-goals / deferred\n\n- **`captured_events.body` redaction hardening** was completed alongside this\n work (not deferred): `webhookguard.RedactPayload` now redacts an expanded set\n of secret key markers (compound names like `access_token`, `client_secret`,\n `signing_key`, `x_api_key`, …) as well as unambiguous value shapes (compact\n JWTs, PEM private-key blocks) under any key. `gateway.sensitiveParameterKey`\n was broadened identically to preserve the documented agreement. It remains\n best-effort per `docs/prds/webhook-intake-security.md` point 5 -- a sender\n that picks a name the heuristic misses is still captured.\n- **Key re-derivation on config reload** is a startup concern: the store holds\n the cipher for its lifetime, so a reload that changes `[bindings]` keys\n requires a daemon restart to take effect. Documented, accepted.\n\n## Wiring\n\n- `internal/config`: add `BindingsConfig` (`encryption_key SecretRef`,\n `previous_encryption_keys []SecretRef`) on `Config.Bindings`.\n- `internal/app/archied/bootstrap.go` `openStores`: resolve both refs through\n the already-built `secrets` registry, pass the material to `store.Open` via a\n new option `store.WithBindingCipher(cipher)`.\n- `internal/infrastructure/bindingcipher`: the `BindingCipher` interface and\n concrete AES-GCM impl, consumed by the event-capture store; a `bindingCipher` field on `Store`, and the `Open`\n option. **Nil cipher = plaintext**, so existing call sites and tests change\n nothing and legacy deployments keep current behaviour until they set a key.\n\n## Back-compat \u0026 tests\n\n- `store.Open` gains the option variadically; all current call sites compile\n unchanged.\n- Existing plaintext rows are unaffected until a cipher is configured; rows\n written before a cipher is installed remain plaintext and read as plaintext.\n- Tests: cipher round-trip (Encrypt/Decrypt), wrong-key and tampered-ciphertext\n integrity failure, envelope parse errors, and a store round-trip that asserts\n the persisted column is NOT the plaintext while `GetBinding` returns it.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/channel-forge-positions/",
|
||
"title": "Channel and Forge action positions (resolves `t2db.19`)",
|
||
"section": "PRDs",
|
||
"body": "# Channel and Forge action positions (resolves `t2db.19`)\n\n**Status:** Approved\n**Date:** 2026-09-03\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Blocked on:** `archie-core-t2db.24` (the `playbook_dispatches` ledger) for\nimplementation of either position. Both are side-effecting, and `t2db.17`\nresolved the keying scheme in design only.\n\n## Forge\n\n`internal/forge.Forge` is already exactly the shape a native-Go action position\nneeds -- a real, typed, general-purpose interface, not something that needs a\nYaegi/Module-style schema-generation detour:\n\n```go\ntype Forge interface {\n IssueForge // AssignedIssues, IssuesWithLabel, CloseIssue, ...\n PullRequestForge // CreatePR, PRState\n RepoForge // AcceptInvitations, VerifyPush, LinkBranch, ...\n}\n```\n\n**Typed contract shape: native-Go-by-name, not a Module-kind.** Module's whole\nreason to exist is giving _Yaegi-authored, operator-supplied_ code a typed\ncontract to implement. `Forge` is already a typed Go interface with real, tested\nimplementations (GitHub, Gitea) selected at daemon startup by `forge.New(...)`.\nWrapping it in a Module-kind schema/`go:generate` detour would duplicate a\ncontract that already exists. A Forge action position should be: a small set of\nnamed operations (e.g. `close-issue`, `create-pr`, `link-branch`), each with a\ntyped `Args`/`Result` struct mirroring the matching method's parameters/return.\n\nThe dispatch target is `internal/domain/workflow.Forger`\n(`CloseIssue`/`CreatePR`/`LinkBranch` -- already exists, already exactly these\nthree operations), **not** `internal/forge.Forge` directly. `organisation.md`'s\ntarget structure places `forge/` under `internal/infrastructure/forge/`; the flat\n`internal/forge` is a flat, unmigrated legacy package in the same position\n`internal/workflow` was in before its own migration. A new domain package\ndepending on the flat package directly would create a domain-to-infrastructure\ndependency that breaks the moment `forge` migrates. `workflow.Forger` is the\nnarrow, already-domain-owned contract that exists for exactly this reason -- the\ndaemon's real `forge.Forge` client satisfies it at the wiring layer.\nDepend on `Forger`, not on `internal/forge`.\n\n**Operation scope for a first slice:** `close-issue` and `link-branch`\n(single-call, cleanly idempotent by event+issue identity) are the safe starting\nset. `create-pr` is explicitly a heavier case -- opening a PR is a\nharder-to-reverse action with its own dedup shape beyond \"did this event fire\nbefore\" (a redelivered event must not open a second PR for the same logical\nchange) -- defer it to a follow-up slice once the keying scheme is proven on the\nsimpler operations, don't design it blind now.\n\n**Host access:** the coordinator needs the same `Forge` instance\n`internal/daemon`'s poller already holds (constructed once at startup via\n`forge.New`). This is a daemon-composition wiring question (pass the existing\ninstance into the coordinator's constructor), not a new extensibility mechanism.\n\n## Channel\n\n### Constraints this satisfies\n\nMessaging is the root, Channel is the integration layer, an implementation such\nas Telegram sits under Channel. A channel is added by implementing one reusable\ninterface, and every channel must implement it. A notification mechanism belongs\nunder Messaging, and `channels.Channel` must not reach across into it.\n\n### Notification is a Messaging contract, not a channel type\n\nThe contract lives in `internal/domain/messaging`. A channel implementation\nsatisfies it by importing its own parent, which it already does, so the\ndependency stays `channels -\u003e messaging` and nothing points back.\n\n`channels.Channel` does not change. It gains no notification method and no\nimport, so the interface every channel must implement stays the lifecycle and\nconfig contract it is. `internal/channels/telegram` satisfying both\ncontracts is an implementation choosing to, not the Channel layer knowing about\nnotification.\n\nA notification is not a channel type because it has no lifecycle to start and\nstop, no inbound direction, and no configuration schema of its own. Modelling it\nas one would force `Start`/`Stop`/`ConfigSchema` onto something with no meaning\nfor them, and make every real channel declare whether it is a notification.\n\n### Destinations are configured, never named by a playbook\n\n`[notify]` becomes a set of named destinations. Each binds a name the operator\nchooses to a channel already registered in the Messaging composition\n(`internal/app/archiemessaging/compose.go` registers `telegram`, `email`,\n`webhook`) plus that channel's own address, such as a Telegram chat id. The\nexisting single-webhook form keeps working as a destination named `webhook`.\n\nA playbook action names a destination. It never names an address. Playbook\ntriggers are reachable from a public, unauthenticated intake surface, so an\naction that could address an arbitrary chat would be a new exfiltration surface;\na destination the operator configured once is not.\n\n### Args and Result\n\n```go\ntype NotifyArgs struct {\n Destination string // a name from [notify]\n Text string\n}\n\ntype NotifyResult struct {\n Delivered bool\n}\n```\n\nAn unknown destination is a reported error at playbook load, not at dispatch:\nthe destination set is known at startup, so this follows the engine's\nreject-at-load rule.\n\n### Host access: no new process surface\n\n`archie-messaging` is its own process (`cmd/archie-messaging`). It dials out to\nthe Gateway and the State Store and serves nothing, and it deliberately has no\nNATS connection: `internal/app/archiemessaging/telegram_features.go` records\nthat it \"must not decode\" the daemon's `[nats]` config. So a notification cannot\nbe pushed into it by any route.\n\nDelivery therefore travels the connection that already exists. Every process\nthat needs Messaging already dials the Gateway: `archied`\n(`internal/app/archied/chat_service.go`), `archie-messaging`\n(`internal/app/archiemessaging/run.go`) and `archie-ui`. The Gateway is the hub.\nA new server-streaming RPC on `ChatService` lets `archie-messaging` subscribe at\nstartup and receive notifications the daemon publishes; Messaging then delivers\nthrough the named channel instance it already holds in `Service.channels`. No\nnew listener, no NATS in Messaging, no change to how any process is reached.\n\nThe existing `Stream` RPC is not reused: it streams one chat turn's events for\none inbound message (`internal/infrastructure/gatewayrpc/server.go:Stream`).\n\n### Out of scope\n\nGiving playbook-originated sends the session and approval scoping the\ninteractive channels use. Configured destinations carry the address, so there is\nno conversation to anchor to and no scoping mechanism to generalise.\n\n## Packages this touches\n\n- `internal/domain/eda` (or a new sibling package, e.g.\n `internal/domain/eda/forgeaction`): typed `Args`/`Result` per operation, a\n small registry keyed by operation name, dispatched against a\n `workflow.Forger`-typed value -- not `internal/forge.Forge` directly (see\n correction above).\n- `internal/app/archied`: wiring the coordinator's constructor to receive the\n daemon's already-built `forge.Forge` client, passed in as the\n `workflow.Forger` it already satisfies -- no new lifecycle to manage, and no\n change to `internal/forge` itself.\n\n- `internal/domain/messaging`: the notification contract and its\n `NotifyArgs`/`NotifyResult`.\n- `internal/config`: `[notify]` becomes named destinations.\n- `proto/gateway/v1` and `internal/infrastructure/gatewayrpc`: the\n server-streaming notification RPC.\n- `internal/app/archiemessaging`: subscribe at startup, resolve a destination to\n a registered channel instance, deliver.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/code-graph-mcp/",
|
||
"title": "Code graph over MCP (`tools/codegraph`)",
|
||
"section": "PRDs",
|
||
"body": "# Code graph over MCP (`tools/codegraph`)\n\n**Status:** Draft\n**Supersedes:** `symbol-reference-index.md`, which scoped only the data layer.\n\n## The problem\n\nEstablishing what a change touches costs an agent eight to fifteen\ngreps, and the answers are wrong in specific ways. `grep -rn \"Register(\"`\ncannot separate `plugin.Registry.Register` from `captureintake.Receiver.Register`\nfrom `forgerpc.Server.Register`, all of which exist here. It also matches the\nsibling checkouts under `.worktrees/`, which are near-complete copies of the\nmodule, so every count is silently inflated.\n\nThat cost lands hardest in investigation, triage and the TDD loop, where the\nfirst question is always \"what does this touch\".\n\n## Decision: exact analysis, not semantic retrieval\n\nTools like Greptile parse with tree-sitter, summarise each node, embed the\nsummaries, and retrieve semantically. That design exists because they must\nserve any language. This repository is one Go module, and Go's type checker is\nground truth:\n\n| Question | Source of truth |\n| --------------------------- | ------------------------------------------------ |\n| Who references this symbol? | `go/packages` + `go/types`, resolved by identity |\n| What calls what? | `callgraph` RTA over SSA |\n| What imports what? | `go list -deps` |\n\nNo vector database, no embedding cost, no edges invented by a summary that\ndrifted from the code. For a single-language repository this is both cheaper\nand strictly more accurate. Copy the interface those tools expose, not their\narchitecture.\n\n**A code graph does not power a documentation chatbot.** It answers structural\nquestions about Go symbols. Answering \"how do I configure a second identity\"\nmeans retrieving prose from markdown, which needs chunking, embedding and\nsimilarity search over a different corpus entirely. That remains a separate\ncapability with its own design. Conflating the two would build one system that\ndoes neither well.\n\n## Interface: an MCP server\n\n`archied` already consumes MCP as a client (`internal/app/agentworker`\nstarts providers) and serves none. Exposing the graph over MCP reaches Claude\nCode, pi, codex and Archie's own agents through one surface, with no coupling\nto the daemon: it is configured like any other entry in `[[mcp_servers]]`.\n\nFour tools, each returning JSON with file:line citations:\n\n- `references(symbol)` — every reference site, classified `prod` / `test` /\n `generated` / `dynamic`\n- `callers(symbol, depth)` — inbound call edges\n- `callees(symbol, depth)` — outbound call edges\n- `blast_radius(file | symbol)` — the transitive neighbourhood a change reaches,\n which is the single call meant to replace the grep sequence\n\nDiagrams are deliberately not a tool. A model given an accurate subgraph writes\nits own Mermaid; a diagram endpoint would only fix the rendering of something\nthe model already has.\n\n## Decisions\n\n**Every answer declares what it could not see.** RTA over-approximates\ninterface dispatch, and reflection is invisible: every `reflect.ValueOf` entry\nin `wfextract`, `gateextract`, `pluginextract`, `secretextract` and\n`logextract` makes a symbol look reachable whether or not anything calls it.\nEach response carries a `dynamic_edges` field naming those boundaries. An agent\ntrusting a confident wrong graph is worse off than one that greps, so the graph\nstates its own limits rather than implying completeness.\n\n**Warm in memory, keyed on HEAD, never persisted to disk.** `deadcode` does\nwhole-program RTA in under seven seconds; this does less work. The server\nrebuilds when HEAD changes and refuses its cache outright when the tree is\ndirty. Nothing is written to disk, because a persisted index becomes a rendered\nview that outlives its evidence. That is the failure that made `STATUS.md`\nreport 90 of 91 findings re-verified while most statuses were 77 commits stale,\nand the same one that made `reachaudit` serve a previous tree's analysis from\n`/tmp` until its cache was keyed on HEAD.\n\n**`tools/codegraph`, not `internal/`.** It is developer tooling, not daemon\nruntime, so it belongs beside `docsgen`, `contractaudit` and `reachaudit` in\nthe `tools` module, which `task test:tools` already covers. Start as a single\nGo file; split only when it earns it.\n\n**No new dependency.** `golang.org/x/tools` is already in `go.sum` at v0.49.0.\n\n## How it is verified\n\nRed first. Table-driven tests against small fixture modules built in\n`t.TempDir()`, asserting the behaviour that distinguishes this from grep:\n\n1. Two methods spelled `Register` on different types resolve to two separate\n records, and a reference to one does not increment the other.\n2. A symbol referenced only from `_test.go` reports zero `prod` references.\n3. A symbol referenced only from a `reflect.ValueOf` table reports zero `prod`\n references and one `dynamic`, and names the table in `dynamic_edges`.\n4. A generated file's references land in `generated`, not `prod`.\n\nMutation-verify each: reintroduce the conflated behaviour and watch the\nassertion fail.\n\nAcceptance is a live cross-check rather than a unit test. Run it against this\ntree and confirm it reports zero production references for\n`worktree.Manager.Resume`, `webui.NewDangerousService` and\n`secret.DefaultRegistry`. All three were confirmed dead by hand, and `deadcode`\neither stayed silent on them or, in the case of a package var, cannot represent\nthem at all. Agreeing with `deadcode` is not the bar. Catching what `deadcode`\nstructurally cannot is.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/config-example-drift/",
|
||
"title": "Example-config and generated-config drift — decision",
|
||
"section": "PRDs",
|
||
"body": "# Example-config and generated-config drift — decision\n\n**Status:** Approved\n**Date:** 2026-09-15\n**Closes:** the checked-in-config half of audit findings `config-1` and\n`repo-root-deployment-2`; strengthens `doc-drift` findings generally.\n**Authority:** extends `docs/architecture/generated-documentation.md` and\n`docs/prds/config-schema.md`; does not replace either.\n\n## Question\n\nShould `config.example.toml` and `deployments/*.toml` be _generated_, so they\ncannot drift from the config code?\n\n## Decision\n\n**No — do not generate them. Gate them instead.** Neither file is a projection\nof the schema, and generating either would destroy the thing that makes it\nuseful. Three verified reasons, in order of weight:\n\n1. **`config.example.toml`'s value is its comments.** It documents **137 keys**\n (measured: active _and_ commented-out), and those hand-written comments are\n the file's purpose — `docs/architecture/configuration.md` states outright\n that \"`config.example.toml` carries documentation users are meant to\n hand-edit\". TOML round-tripping does not preserve comments, which is exactly\n why `tomlwrite` does line-oriented edits rather than decode/encode. A\n generator would emit a correct file with the documentation deleted.\n2. **`docs/prds/config-schema.md` forbids building on `config.Config`.** That PRD\n rejects \"a reflection-based schema over `config.Config`\" because the struct is\n \"scheduled for dissolution\". A generator deriving the example from its struct\n tags would build on a foundation this repo has already decided to remove.\n3. **`docs/architecture/generated-documentation.md` prohibits it.** \"Adding\n another hardcoded type list to `docsgen` is prohibited\" — so any generator\n must consume a domain-owned registry, which does not exist for configuration\n yet. Creating one is the config-domain migration's job, not this change's.\n\nThe deployment profiles are also not a projection: they set **3-46 keys** against\nthe example's 137, and they are curated overlays (\"a complete config, not an\noverlay\", per their own headers), not near-duplicates. Generating them would add\nmachinery to remove duplication that largely is not there.\n\n## The gap that is real\n\n**Unknown config keys are silently ignored.** Verified at\n`internal/infrastructure/configuration/decode.go`:\n\n```go\nif _, err := toml.DecodeFile(path, target); err != nil {\n```\n\nThere is no `MetaData.Undecoded()` check, so a typo — `max_concurrancy = 4` —\nparses clean, validates clean, and does nothing. That is the same failure shape\nthis repo's audit found repeatedly: a setting that looks applied and is not. It\nis also detectable **without any hand-maintained list**, because `BurntSushi/toml`\nreports exactly which keys the target did not consume.\n\nNote the second, separate gap already admitted in writing:\n`generated-documentation.md` — \"`task check` does not run `docs:check`, so\ngenerated drift is ungated.\"\n\n## What to build\n\n**1. Unknown-key detection at the loader (behaviour).** Decode with metadata and\ncollect `Undecoded()`. Report it as a warning at startup listing the offending\nkeys, and expose it so tests can assert emptiness.\n\n**Deliberately a warning, not a fatal error.** `configuration.md`'s settled\nstartup policy is \"a missing credential disables a capability; only an invalid\nconfig stops the daemon\". A stray key is far more likely to be an operator typo\nin a file they are otherwise happy with than a reason to refuse to boot, and\n`[web]` is documented as landing in a _different_ binary\n(`internal/app/archieui`), so a key unknown to one process is legitimately known\nto another. Failing closed here would break working deployments on upgrade.\n\n**2. A checked-in-config contract test (gate).** Load `config.example.toml` and\nevery `deployments/*.toml` through the real loader and assert `Undecoded()` is\nempty, with an explicit allowlist for keys another binary owns. This is the\nmechanism that would have caught a renamed key left behind in an example, and it\nneeds no inventory.\n\n**3. Gate generated data in `task check`.** Done. `task docs:check` runs in the\ngate, closing the admitted \"generated drift is ungated\" hole. The\ncheck mode did not exist when this was written; it was added for issue #855,\nwhich also made the check non-destructive so the gate can never rewrite the\nartifact it is judging.\n\n**4. Explicitly NOT now: a generated key inventory.** `configuration.md`'s\ncompletion criteria want \"generated documentation lists every external key,\ndefault, constraint, mutability class, owner\". That is a real target and it is\nthe config domain's, once a domain-owned registry exists — the prohibition above\nmeans it cannot be bootstrapped from a list inside `docsgen`. Recorded as\ndeferred, not rejected.\n\n## Non-goals\n\n- No generation of `config.example.toml` or `deployments/*.toml`.\n- No reflection-based schema over `config.Config`.\n- No new type list inside `docsgen`.\n- No change to `archied setup`'s template: `configtemplate.Example` stays the\n verbatim embedded example, hand-maintained, and `setup` continues to _edit_ it\n via `tomlwrite` rather than regenerate it.\n\n## A withdrawn premise, recorded so it is not reused\n\nAn earlier measurement claimed 17 keys were \"set by a profile but documented\nby no example\", and used it to argue for generation. It was an\nartifact: the extractor stripped comment markers and tracked a single table\nvariable, so commented blocks were mis-attributed. Re-checking refuted it —\n`providers.\u003cname\u003e.api_key_env` is a real, documented backwards-compatible field\n(`internal/config/config.go`, `config.example.toml`) and `[[identities]]`\nis documented at `config.example.toml`. The decision above does not rest on\nthat measurement.\n\n## Acceptance criteria\n\n1. A config file containing an unknown key surfaces those keys to the operator at\n startup, and the daemon still starts.\n2. `config.example.toml` and every `deployments/*.toml` are loaded by a test that\n fails if any key is undecoded, except keys explicitly attributed to another\n binary.\n3. `task check` fails when committed generated data is stale.\n4. Introducing a renamed key into `config.example.toml` fails the gate (prove it\n by doing it, not by asserting it).\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/config-schema/",
|
||
"title": "Backend-owned configuration schema for the webui -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Backend-owned configuration schema for the webui -- decision\n\n**Status:** Approved\n**Date:** 2026-09-02\n**Beads issue:** `archie-core-b6ew` (epic), with children `.1`-`.5`\n\n## Decision\n\nThe dashboard's Configuration page keeps reading a secret-free,\nhand-picked view of `config.Config` -- that allowlist is a safety\nboundary and stays exactly as it is. What changes is who defines the\npage around it: `ui/src/settings/settings.js` and\n`config-row.js` hardcode every row's label, section, type, and\neditability by hand; after this, the backend attaches that metadata to\neach field it already allows through, and the frontend renders\nsections and fields generically from what the backend sends.\n\nThis is **not** a reflection-based schema over `config.Config`.\n`internal/config/config.go`'s own package doc states it is \"scheduled\nfor dissolution: its types and methods are to be reassigned to the\ndomains whose behaviour they describe\" -- building a generic\nschema-from-struct-tags mechanism over a struct on its way out would be\nbuilding on a foundation this repo has already decided to remove.\nInstead, every field already in `ConfigView` (`internal/webui/api_config.go`)\ngot there because someone deliberately vetted it as safe to show; the\nschema adds a hand-authored descriptor to that same deliberate act,\nnot a new mechanism that re-derives safety automatically.\n\n## Why now\n\nTask #6's investigation (`/9/26`) found the concrete failure this\nproduces: `Repo.AllowConcurrent` and `Repo.MaxRetries` already reach\n`GET /api/config` (`RepoView` in `api_config.go`), but the\nrepositories card renders a static read-only table that never shows\nthem, so the fix for \"why isn't a second agent running\" required an\nagent tracing Go source instead of a checkbox on the dashboard.\n`Repo.ReviewEnabled` is worse: it never reaches the API at all. Both\nare symptoms of the same root cause -- the frontend decides what a\nfield means and whether it's shown, and nothing keeps that in sync\nwith what the backend actually allows.\n\n## Shape\n\n```go\n// internal/webui/config_schema.go (new)\n\ntype ConfigFieldType string\n\nconst (\n FieldString ConfigFieldType = \"string\"\n FieldInt ConfigFieldType = \"int\"\n FieldBool ConfigFieldType = \"bool\"\n FieldDuration ConfigFieldType = \"duration\"\n FieldEnum ConfigFieldType = \"enum\"\n FieldStructured ConfigFieldType = \"structured\" // repositories, models, providers\n)\n\ntype ConfigField struct {\n Key string `json:\"key\"` // dotted path, matches UpdateConfig's key space\n Label string `json:\"label\"`\n Description string `json:\"description,omitempty\"`\n Type ConfigFieldType `json:\"type\"`\n Value any `json:\"value\"`\n Editable bool `json:\"editable\"`\n LockedReason string `json:\"locked_reason,omitempty\"`\n Overridden bool `json:\"overridden\"`\n Options []string `json:\"options,omitempty\"` // enum only\n RestartRequired bool `json:\"restart_required\"`\n}\n\ntype ConfigSection struct {\n ID string `json:\"id\"`\n Label string `json:\"label\"`\n Fields []ConfigField `json:\"fields\"`\n}\n```\n\n`handleConfig` builds `[]ConfigSection` from the existing per-field\nvalues it already assembles, plus `overlay.DeniedKeys` for\n`LockedReason` and the existing `Overridden` list -- both already\ncomputed just re-attached per field instead of returned as\nside-channel maps the frontend cross-references by key string.\n`Locked`/`Overridden`/`Reload`/`Provenance` stay as top-level\n`ConfigView` fields; they describe runtime state (what happened on\nthe last reload, which keys the overlay refuses), not the field\ncatalogue, and duplicating that logic per-field would be the same drift\nrisk this change exists to remove.\n\n`FieldStructured` fields (repositories, models, providers) carry\n`Value` as structured JSON and are rendered by dedicated\neditors, not the generic renderer -- see \"What this does not do\".\n\n## Sections\n\nMatches the cards the settings page already has: `identity`,\n`repositories`, `models`, `providers`, `budgets`, `storage`,\n`containers`, `web`. No new sections; this is metadata on existing\nrows, not new configuration surface.\n\n## Frontend\n\n`config-row.js`'s `row(label, value, opts)` becomes the renderer's\nprivate implementation detail, called generically:\n\n```js\nfor (const section of schema.sections) {\n const rows = section.fields\n .filter((f) =\u003e f.type !== \"structured\")\n .map((f) =\u003e\n row(f.label, f.value, {\n key: f.key,\n type: f.type,\n options: f.options,\n locked: f.locked_reason,\n overridden: f.overridden,\n restartRequired: f.restart_required,\n }),\n );\n // structured fields render through their dedicated editor (b6ew.4)\n}\n```\n\n`settings.js` stops knowing that `containers.max_concurrency` is an\n`int`, belongs in the \"Sandboxed containers\" card, or requires a\nrestart. It knows how to render a section and a field of a given type.\n\n## What this does not do\n\n- **Does not marshal `config.Config` wholesale or add reflection over\n it.** The allowlist boundary in `api_config.go` is unchanged; this\n adds a label next to a value that was already being shown.\n- **Does not make every backend field appear automatically.** A field\n still requires someone to write its descriptor, same as it requires\n someone to add it to `ConfigView` -- the fix is one\n source of truth instead of two, not zero review.\n- **Does not solve structured editing generically.** Repositories and\n maps get their own typed editor (`archie-core-b6ew.4`), not a\n generic \"array of objects\" renderer -- a repo's fields have their\n own validation shape (owner/name pair identity, gate command\n argv) that a generic structured-field renderer would either get\n wrong or reinvent per-type anyway.\n- **Does not change what `UpdateConfig`/`overlay.DeniedKeys` accept.**\n `Editable` on a field descriptor mirrors that existing policy; it\n does not grant new write access.\n\n## Acceptance criteria (epic-level; see child issues for per-task detail)\n\n1. Every field rendered as a hardcoded row in `settings.js`\n has a backend-authored descriptor with type, label, and section.\n2. `Repo.ReviewEnabled` reaches the API and the dashboard, alongside\n `AllowConcurrent` and `MaxRetries`.\n3. Adding a new scalar field to the schema requires one backend\n descriptor and zero frontend changes to appear correctly labelled,\n sectioned, and editable.\n4. `TestHandleConfigNeverLeaksSecrets`-style coverage extends to the\n schema response; no descriptor exists for a secret-bearing field.\n5. A schema-fixture-driven frontend test asserts the generic renderer\n needs no field-specific branch to render every scalar field type.\n\n## Work breakdown\n\n- `archie-core-b6ew.1` -- descriptor types + hand-authored section/field\n metadata for every existing `ConfigView` field.\n- `archie-core-b6ew.2` -- `handleConfig` returns the schema-driven\n shape (depends on `.1`).\n- `archie-core-b6ew.3` -- generic frontend renderer for scalar/bool/\n int/enum/duration fields (depends on `.2`).\n- `archie-core-b6ew.4` -- structured editors: repositories (incl. the\n `ReviewEnabled` gap), models, providers (depends on `.2`).\n- `archie-core-b6ew.5` -- contract tests for metadata completeness,\n secret-safety, and generic-renderer coverage (depends on `.3`, `.4`).\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/contract-conformance-audit/",
|
||
"title": "Contract conformance audit — decision",
|
||
"section": "PRDs",
|
||
"body": "# Contract conformance audit — decision\n\n**Status:** Finalised\n**Scope:** an **advisory** report, deliberately **not** part of `task check`.\n**Date:** 2026-09-15\n**Authority:** sibling to `archie-core-06ag` (standing reachability audit) and\n`docs/architecture/dependencies-and-contracts.md`. Replaces neither.\n**Open:** withdraw it, or rebuild it structurally. See \"Open decision\" — not to\nbe resolved by hardening the regexes.\n\n## Question\n\nDoes every declared contract surface have a consumer, or an explicit declaration\nthat it has none?\n\nReachability (`tools/reachaudit`) asks the complementary question — _is this code\nin a shipped binary?_ — and cannot see a capability that is **reachable and\nunconsumed**. `ratelimit` was in the binary closure and called by nothing\n(`archie-core-6wxb`); reachability would not have found that.\n\n## Decision\n\nYes, as an advisory survey. `tools/contractaudit` classifies each subject of each\nsurface:\n\n| Classification | Meaning | Fails `--strict`? |\n| ----------------------- | ----------------------------------------------------------------------------- | ----------------- |\n| `CONSUMED` | a call site exercises it | no |\n| `DECLARED-UNCONSUMED` | allowlisted in `docs/contract-declarations.json`, with a reason and a tracker | no |\n| `UNDECLARED-UNCONSUMED` | nobody reaches it, nobody declared it | **yes** |\n| `STALE-DECLARATION` | an allowlist entry that is no longer true | **yes** |\n\nThere are **four** classes and **two** of them fail. The allowlist must match\nreality exactly, and it fails in both directions: a missing entry is a finding,\nand an entry that outlived its subject is a finding, because a stale allowlist is\nhow a check quietly stops checking anything.\n\n**Not gated.** `task contract` is advisory and `task check` does not run it,\nmatching `tools/reachaudit`. The surfaces are discovered by scanning source text,\nso a rename or a reformat elsewhere in the tree can change the output with nothing\nactually wrong. It has already produced 16 false \"unconsumed\" readings once, when\nthe dashboard's request helper was renamed. A check that can fail for reasons\nunrelated to what it measures must not be able to break a build.\n\n## Surfaces actually built — three, not five\n\n| Surface | Subjects | \"Consumed\" means |\n| ------------------------------------ | ------------- | -------------------------------------------------------- |\n| `proto/state/v1` `StateStoreService` | **45 RPCs** | an adapter body calls `client.\u003cRpc\u003e(...)` |\n| `proto/gateway/v1` `ChatService` | **21 RPCs** | same |\n| `internal/webui` dashboard routes | **49 routes** | a path literal appears as a call argument under `ui/src` |\n\nConsumption for the gRPC surfaces is established by **scanning adapter source\ntext** for calls through the generated client, not by reflection. See the errata\nbelow: the first revision of this document claimed reflection, and the first\nimplementation did use it, but the shipped mechanism is a regex and the document\nwas not updated to match.\n\n## What this audit has actually found\n\n**Nothing actionable.** On the current tree it reports 0 undeclared and 0 stale.\nIts only allowlisted entries are two RPCs superseded by their streaming variants\n(`ListCaptures`, `ListUndispatchedCaptures`), which were already known from the\nsource comment on `staterpc.Client.ListCaptures`.\n\nEvery real defect found in the session that produced this tool was found by other\nmeans:\n\n- the three 415 refusals, by reading `authorizeTaskMutation` and reproducing the\n browser's exact headers against the real handler;\n- the dashboard client's duplicated request plumbing, by reading the module.\n\nRecorded plainly because a survey that has never found anything is evidence about\nthe survey, not about the tree.\n\n## Errata — claims in the first revision of this document that were false\n\n- **\"RPC coverage is derived by reflection over the exported client type.\"**\n It is not. `proto.go` regex-scans adapter bodies for `client.Method(` calls.\n The reflection version was replaced and the document was not updated. The unused\n `google.golang.org/grpc` family in `tools/go.mod` was residue from that\n iteration; `go mod tidy` removed it on 2026-09-15.\n- **Surface sizes of \"43 / 12 / 52\".** The real counts are 45 / 21 / 49.\n- **\"Three classifications, and the third is the only finding.\"** Four classes,\n two of which fail.\n- **\"Raw `fetch()` calls caused 401s to render as `refused`/`broken`.\"** Wrong.\n Every raw call site passed `res.status` to `ApiError`, so `classifyActionError`\n already mapped 401 to `session-expired`; the chat stream never classified at\n all. The real defect in that area was different (below).\n- **A five-surface \"Shape\" naming `surface.go`, `generated.go`,\n `declarations.go`, and `testdata/`.** None were written. The real files are\n `main.go`, `audit.go`, `proto.go`, `webui.go`.\n\n## The two gaps this was aimed at\n\n**1. `docsgen check` is documented and does not exist.** Closed by issue #855.\n(issue #855 / `archie-core-5gzx`). `tools/docsgen` now has a `check` subcommand\nand `task docs:check`, wired into `task check`: it generates into a temporary\ndirectory, compares against the committed artifact, names the authoritative Go\ndefinition behind each mismatch, and never writes to the working tree.\n`docsgen data` and `docsgen asyncapi` remain target-only.\n\n**2. The dashboard's client was not centralised while its own docstring claimed\nit was.** Now closed by `18e8f8f`. The docstring said \"Single place that knows how\nto talk to archied… so auth handling and error shape live in one file\" while 22\ncall sites built their own `fetch`, headers, timeout and error handling, and\n`chat.jsx` built a 23rd. The defect actually present there was that three\nbodyless mutations — `bindingApprove`, `mappingDelete`, `bindingDelete` — omitted\n`Content-Type` and were refused **415** on every use, because\n`authorizeTaskMutation` requires a JSON media type even when there is no body.\nThe Go tests missed it because their helper set `Content-Type` unconditionally,\nincluding for a nil body.\n\n## Testing rule — applies to this tool and to anything like it\n\n**Do not assert on the wording, formatting, or content of generated or\nhand-authored text and config files.** A test that pins a source snippet, injects\ntext into a fixture, or counts an identifier in a file is testing that file's\nformatting rather than the behaviour under test: an extra space, a newline, a\nrename, or a comment that merely mentions the token breaks it, and the\nmaintenance burden is permanent. A test that cannot tell \"the code is wrong\" from\n\"the file is formatted differently\" is testing the wrong thing.\n\nAssert behaviour on the observable effect — the request a client makes, the value\na function returns — and keep parsing out of the assertion. Where a parser must\nexist, test it through its own interface with in-memory input, not through a file\non disk.\n\nThe first revision of this tool violated this in six tests (four that injected\ntext into fixtures, two that asserted on a source file's contents). All six were\ndeleted on 2026-09-15. The dashboard suite's \"api.jsx holds exactly one fetch\ncall\" check was deleted for the same reason: it broke on a reformat and on a\ncomment that mentioned `fetch`, and it was patched with a comment-stripper instead\nof being removed. Every `api.*` method is covered behaviourally instead, by\nasserting the headers and body each one sends.\n\n## Open decision\n\n1. **Withdraw.** Delete the tool, the allowlist, this document, and the `contract`\n task. It has found nothing, its mechanism is unsound, and the same question is\n better asked by a behavioural test per method.\n2. **Rebuild structurally.** Take the contract from the compiler instead of from\n text: the generated `StateStoreServiceClient` / `ChatServiceClient` interfaces\n give the exact RPC set, and `go/ast` over the adapter bodies and\n `internal/webui/server.go` is formatting-insensitive where a regex is not.\n Cost: the dashboard-consumer half has no Go-native JS parser, so it is either\n dropped or replaced by a behavioural test per route — which is what would have\n caught the 415s.\n\nNeither is \"make the regexes cleverer\". Recorded here rather than acted on\nunilaterally because withdrawing a capability and rebuilding one are both larger\ndecisions than adding a report.\n\n## Acceptance\n\n1. `task contract` prints a report and exits 0.\n2. Every unconsumed subject appears with a classification, and where declared with\n a reason and a tracker id.\n3. `--strict` fails on an undeclared entry **and** on a stale one. Verified by\n construction against a copy of the tree, not by asserting on the tree's text.\n4. The tool neither deletes nor rewrites anything.\n5. No test in this tool asserts on the content or formatting of another file.\n\n## Non-goals\n\n- No runtime schema validation of HTTP responses. The webui handlers already\n return non-nil slices on every empty path (`[]SkillView{}`, `[]CuratorView{}`,\n `make([]taskView, len(tasks))`), so the nil-slice class does not apply; adding a\n live-stack validator before it has a defect to find is scaffolding. Deferred,\n and filed as `archie-core-nddv`.\n- No re-parse of the `.proto` as a second source of truth; `buf lint` and\n `buf breaking` already own it.\n- No generation of `config.example.toml` or `deployments/*.toml` — settled\n against in `config-example-drift.md`.\n- No change to `tools/reachaudit`'s scope or report shape.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/control-plane-apply-status/",
|
||
"title": "Control-plane apply status",
|
||
"section": "PRDs",
|
||
"body": "# Control-plane apply status\n\n**Status:** Approved\n**Date:** 2026-09-21\n**Tracking:** `archie-core-pskb`, and `archie-core-nwa0` which depends on it\n\n## Outcome\n\nAn operator who edits a control-plane resource can see which processes are\nrunning that version, which are still on an older one, which rejected it, and\nwhich have stopped reporting.\n\n`docs/prds/runtime-control-plane.md` requires this and does not say where the\nreport goes. This decides that.\n\n## Where the report goes\n\nApply status is a State Store contract surface. `ControlPlaneService` is the\nsurface for editing a resource, and a status report is not an edit. The\nprecedent is the config snapshot pair: a process publishes what it knows, the\nUI process reads it, and both RPCs are administrative, so a task-scoped grant\nreaches neither. That denial needs no new rule: `authorizesTaskScopedCall`\nnames three methods and denies everything else.\n\n`docs/prds/state-store-contract.md` gains the pair in its inventory, and its\nRPC count at two places moves from 45 to 47.\n\nThe read returns all records as a list, including an empty one. The\n`found=false` convention for absent single rows does not apply.\n\n## What is reported\n\nOne record per process and resource kind, replaced in place. It holds the\nprocess name, the resource kind, the version applied, the error if the process\ncould not apply it, and the time it reported.\n\nEach process re-stamps its records every 30 seconds. A record is stale once it\nis older than 90 seconds, and a stale record means the process that wrote it is\nno longer running. The report time is therefore a liveness signal, not only a\nwrite timestamp.\n\nA process reports at the point it applies a resource, so the record says what\nhappened rather than what was intended. Nothing probes.\n\nProcess names come from one shared constant naming `archied`, `archie-gateway`\nand `archie-messaging`. A deployment runs at most one of each. Writer and\nreader are bound to that constant by test, because a drifted name is\nindistinguishable from a process that never reported.\n\n## Which processes report\n\n- `archied` and `archie-gateway` apply every restart-required kind through\n `boot.runtimeConfig`, at boot and on SIGHUP.\n- `archied` also applies the live kinds it watches, and reports each update it\n applies.\n- `archie-messaging` applies channel settings at startup and reports that kind.\n\n## What can be reported, and what cannot\n\nA process reaches its apply point only after the State Store has answered it.\nAt startup, when the State Store is unreachable, the process exits or refuses\nto start and writes no record. A live watcher that loses its established\ncontrol-plane stream reports the connection failure against its last applied\nversion while it retries. Re-establishing the stream clears that outage record\nwithout clearing a versioned document rejection. A transport failure is not a\ndocument rejection; it reports that live delivery is unavailable.\n\nAbsence has two meanings and the UI never collapses them: no process applies\nthis kind, or a process that applies other kinds has not reported this one.\n\n## Relationship to the reload status\n\nApply status covers control-plane resource kinds. `ReloadStatus` covers the\nfile document and stays the only report for a SIGHUP that failed to resolve or\nvalidate it. Neither is derived from the other.\n\nWriting a record is best-effort. On the SIGHUP path it takes its own bounded\ncontext, separate from the budget the reload spends reading the control plane,\nand a failed write is logged rather than turned into a failed reload.\n\n## What the UI shows\n\nThe settings page shows, per resource, the stored version beside each process's\napplied version, error, and report time. A process on an older version of a\nrestart-required resource is pending a restart. A process with an error is\nfailed, with the error beside it. A process whose record is stale is unknown,\nnever current.\n\n## A record outlives the process that wrote it\n\nRecords are replaced in place, so a process that applies a version and then\nstops would otherwise leave a record asserting that version indefinitely, and\nthe settings page would show a dead process as current.\n`docs/prds/status-health-surface.md` faced the same choice and stamped the start\nof a poll pass so a wedged poller reads as stale rather than healthy.\n\nSettled: the re-stamp above. The alternatives considered were\ncarrying the instance identity generated at process start, which detects a\nrestart but not a death, and accepting the limitation with the page saying so.\n\n## Verification\n\n- A process that applies a resource writes a record carrying that version.\n- A process whose apply fails validation writes a record carrying the error and\n no new version.\n- A live watch outage is reported against the last applied version until the\n stream is re-established; re-establishment preserves any outstanding\n versioned rejection.\n- A reload that fails leaves the previously applied version recorded.\n- A record written under a process name outside the shared constant fails the\n test binding writer to reader.\n- A task-scoped credential is refused by both RPCs.\n- A failed report does not fail the reload that produced it.\n- A record that stops being re-stamped renders as unknown rather than current.\n\n## Not included\n\n- History of past applies. Records are current state; resource history already\n records what was written.\n- Checking a live replacement before switching to it. That is\n `archie-core-nwa0`, which reports its failures through this surface.\n- Any change to how a resource is applied.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/curator-sampler-wave1/",
|
||
"title": "Curator sampler wave 1: `Sampler` interface + four cheap strategies -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Curator sampler wave 1: `Sampler` interface + four cheap strategies -- decision\n\n**Status:** Approved\n**Date:** 2026-09-05\n**Beads issue:** bead `archie-core-1786637500776-416-600a66e5` / GitHub #437\ncovers the embedding-backed follow-on strategy only. This PRD covers the\nprerequisite interface and the cheap strategies.\n**Parent epic:** `archie-core-1786637500725` (curator engine)\n**Depends on:** nothing shipped (this is itself a prerequisite)\n**Blocks:** the embedding-backed surprisal strategy (GitHub #437), which\nimplements this same interface once the embeddings capability\n(`internal/domain/embedding`, this change) lands\n\n## Problem\n\nA future curator that reviews a large body of content (memories, sessions,\nskills) cannot always afford to run its agentic budget over every candidate\nevery pass. It needs to pick a bounded subset first. GitHub #437's acceptance\ncriteria requires that subset-picking to sit \"behind a `Sampler` interface\nthat wave 1 of the curator epic ships (cheap strategies: recency, random,\nall, staleness proxy)\" and that the embedding-backed strategy implement \"the\nsame interface\" later. No such interface, and none of the four cheap\nstrategies, exist anywhere in the codebase (bead 434e5bb9's closure\nwithout a linked PR was the tracker being wrong, not the code existing) --\nthis PRD is the one-page settled design for it, per this repo's Scope\nDiscipline rule.\n\n## Decision\n\nA `Sampler` is a pure, call-scoped selection function: given a fixed slice of\ncandidates and a request, it returns a subset. No lifecycle, no host access,\nno I/O for the four wave-1 strategies -- they are computation only, so they\nlive directly in `internal/domain/sampling` (a domain package, not\n`internal/infrastructure/*`) alongside the contract, the same way\n`curator/activity.go`'s pure ring buffer sits next to `curator/contract.go`\nrather than in an infrastructure package.\n\n```go\n// Package sampling is the curator sampler family: given a bounded set of\n// candidates, pick a subset to spend a curator pass's agentic budget on.\n// See docs/prds/curator-sampler-wave1.md.\npackage sampling\n\n// Candidate is one item eligible for sampling. It is deliberately narrow:\n// an opaque identifier, the timestamp the timestamp-based strategies score\n// against, and an open Metadata bag for whatever a future strategy needs\n// (e.g. the embedding-backed surprisal strategy's vector) without widening\n// this struct for every strategy that comes after wave 1.\ntype Candidate struct {\n ID string\n At time.Time\n Metadata map[string]any\n}\n\n// Request carries the knobs every strategy shares: a selection cap and a\n// deterministic seed. Both are optional -- Cap 0 means unlimited (the \"all\"\n// strategy relies on this), Seed 0 is a valid seed, not \"unset\".\ntype Request struct {\n Cap int\n Seed int64\n}\n\n// Sampler selects a subset of candidates to spend a curator pass's agentic\n// budget on. Every implementation -- the four here and the future\n// embedding-backed surprisal strategy (GitHub #437) -- must be\n// deterministic given the same candidates and Request: same inputs, same\n// output, every time. That is what makes a pass reproducible and testable,\n// and is GitHub #437's stated acceptance criterion for the strategy it\n// adds behind this same interface.\ntype Sampler interface {\n // Name identifies the strategy for logging/attribution (e.g.\n // \"recency\", \"random\", \"all\", \"staleness\", later \"surprisal\").\n Name() string\n // Sample selects from candidates per req. Implementations must not\n // mutate candidates or rely on anything but candidates and req --\n // determinism depends on that.\n Sample(ctx context.Context, candidates []Candidate, req Request) ([]Candidate, error)\n}\n```\n\n### The four wave-1 strategies\n\nAll four are `internal/domain/sampling` types constructed with `NewX(...)`,\neach implementing `Sampler`:\n\n- **`recency`** (`NewRecency()`): sorts candidates newest-`At`-first (stable\n sort, ties keep input order so output is deterministic even when two\n candidates share a timestamp), then truncates to `req.Cap` (0 = all).\n- **`staleness`** (`NewStaleness()`): the mirror of recency -- sorts oldest-\n `At`-first, then truncates to `req.Cap`. This is explicitly a _proxy_:\n \"hasn't been touched in the longest time\" using whatever timestamp the\n caller populated `Candidate.At` with (created, updated, or\n last-reviewed -- the caller's choice, not this package's), never the\n embedding-based surprisal calculation GitHub #437 owns.\n- **`all`** (`NewAll()`): no filtering. Returns candidates in input order,\n truncated to `req.Cap` (0 = every candidate). Input order is the only\n order this strategy has an opinion about, and preserving it is what makes\n it deterministic.\n- **`random`** (`NewRandom()`): a uniform-random subset, sized to\n `req.Cap` (0 = every candidate, order-shuffled). Determinism comes from\n `req.Seed`: `math/rand/v2`'s `rand.New(rand.NewPCG(seed, seed))` seeded\n per call (not a package-level or struct-level generator), so the same\n `(candidates, req)` pair always produces the same permutation regardless\n of call order or concurrent callers -- required for GitHub #437's\n determinism criterion and for this package's own tests.\n\n### Why one shared `Request` and not one options struct per strategy\n\nThe embedding-backed strategy (GitHub #437) is deferred, but its stated\nshape -- \"sample up to a candidate cap,\" a seed for reproducible tie-\nbreaking in the power-iteration -- already fits `Request{Cap, Seed}`\nwithout modification. A per-strategy options type would mean GitHub #437\neither reuses this one (proving it was unnecessary to split) or grows its\nown incompatible shape that the `Sampler` interface can't express without a\nbreaking change later. One shared, small `Request` is the smallest thing\nthat already accommodates the known future consumer.\n\n### Why `Candidate.Metadata map[string]any` instead of a typed field per strategy\n\nOnly `ID` and `At` are used by any wave-1 strategy. `Metadata` exists solely\nso GitHub #437 can carry an embedding vector (or whatever else the k-NN/\npower-iteration step needs) through the _same_ `Candidate` type without this\npackage importing `internal/domain/embedding` or growing a `Vector` field\nthat every non-surprisal strategy ignores. Untyped and optional, exactly\nlike `image.GenerateRequest.Options` and `curator.Action.Detail`'s existing\n\"provider-specific overflow\" convention elsewhere in this codebase.\n\n## What we deliberately do NOT do\n\n- **No curator-family wiring.** `curator.Registrar` gains no `Sampler`\n field and no curator declares one in this change. Nothing in\n `internal/domain/curator` consumes `sampling.Sampler` yet -- there is no\n curator that samples anything. Wiring a sampler into a real\n curator's `Pass` is that future curator's job, not this prerequisite's.\n- **No embedding-backed strategy.** That is GitHub #437, explicitly\n deferred behind the embeddings capability this same change also adds\n (`internal/domain/embedding`) -- see that issue for the k-NN graph /\n power-iteration / surprisal-scoring algorithm. This package only\n guarantees the interface shape GitHub #437 must implement.\n- **No persistence, no candidate discovery.** `Sampler.Sample` takes\n `candidates` as a plain slice; where they come from (a memory store, a\n session list, a skill directory) is entirely the caller's concern, same\n boundary `curator.Registrar`'s `SkillStore`/`ConversationSource` already\n draw between \"the curator finds candidates\" and \"the curator engine acts\n on them.\"\n- **No configuration surface.** Strategy selection (which `Sampler` a\n curator uses) is a future curator's constructor argument, not a new\n `[curator]` config section -- no curator exists yet to configure.\n\n## Acceptance criteria\n\n1. `Sampler` interface plus `Candidate`/`Request` types exist in\n `internal/domain/sampling`, importing nothing from\n `internal/infrastructure/*` or `internal/app/*` (domain-layer rule).\n2. Four `Sampler` implementations (`recency`, `random`, `all`, `staleness`)\n with table-driven tests proving: correct selection per strategy,\n `Cap` truncation (including `Cap` larger than the candidate count, and\n `Cap` 0 meaning unlimited), and -- for `random` -- that the same\n `(candidates, req)` produces the same output across repeated calls\n (determinism), while a different `req.Seed` can produce a different\n permutation.\n3. Every strategy's `Sample` leaves the input `candidates` slice\n unmodified (verified by a shared test helper across all four).\n4. The interface shape is reviewed against GitHub #437's stated\n acceptance criteria (\"a `Sampler` given fixed inputs producing\n deterministic selection,\" \"sample up to a candidate cap\") and\n accommodates them without a breaking change -- this document's\n \"Why one shared `Request`\" section is that review.\n\n## Files this change adds\n\n- `internal/domain/sampling/contract.go` -- `Sampler`, `Candidate`,\n `Request`\n- `internal/domain/sampling/recency.go`, `staleness.go`, `all.go`,\n `random.go` -- the four strategies\n- `internal/domain/sampling/*_test.go` -- table-driven tests per strategy\n plus the shared non-mutation test helper\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/dashboard-aware-chat-navigation/",
|
||
"title": "Dashboard-aware chat: Archie sees the page, and points the operator there",
|
||
"section": "PRDs",
|
||
"body": "# Dashboard-aware chat: Archie sees the page, and points the operator there\n\n**Status:** Approved\n**Date:** 2026-08-26\n**Compound:** the \"chat available on any page\" surface work (global chat presence)\n\n## Decision\n\nMake the web chat **dashboard-aware**: the agent is told which dashboard page\nthe operator has _open_, knows what each page is for, and can **point**\nthe operator to the relevant page with a **clickable destination chip** the UI\nrenders and follows - Option B (context + navigate), not just \"tell me in text.\"\n\nThree seams, each an existing extension point, one new tool and one message field:\n\n1. **Current page, into the prompt.** The browser sends its route with every\n chat message; the gateway threads it into the system prompt's runtime-metadata\n env block. Archie always knows where the operator is looking.\n2. **Page index.** A page_index tool lists the dashboard pages and what each one\n shows, so Archie knows what to point at - the agent \"has the map\".\n3. **Navigate.** A dashboard_navigate tool returns a structured path/label; the web\n UI renders it as a clickable chip that routes the operator there. This is the\n \"plenty of resources\" the requirement named.\n\nWe deliberately keep the agent's answer honest: the chip is _evidence_ it can\nvalidly produce because dashboard_navigate is a real tool whose result is\nrendered, not a markdown link the model invents.\n\n## Why now\n\nThis is the natural next step of the \"chat on any page\" work, and the original\nrequest named it explicitly: _\"archie should be able to see what page you're on\ntoo. and point out where to go. so it needs plenty of resources.\"_ The chat\nis a full-page route with no knowledge of the rest of the dashboard; the agent\ncannot see which page the operator is on and cannot reliably direct them anywhere.\nThe critique of the chat UI already identified \"chat is not available on any page\"\nas the largest structural gap (P1). This design is the capability that makes that\ngap worth closing.\n\n## Shape of the API\n\n### Message field (gateway.go)\n\nAdd Page to Message:\n\n```go\n// Page is the dashboard route the operator is on when they sent this message\n// (e.g. \"/tasks\", \"/logs\"). Empty for non-web channels, which have no page\n// context. It is per-message, not per-session: the operator navigates freely\n// while a conversation stays open.\nPage string\n```\n\n### Web request (api_chat.go)\n\nchatMessageRequest gains Page, decoded into Message.Page in decodeChatMessage.\n\n### System prompt config (prompt.go)\n\nSystemPromptConfig gains Page; BuildSystemPrompt passes it through to the template.\nThe env block of archie.md.tpl renders the current page when set.\n\n### Page index (dashboard_tools.go, new)\n\nA package-level registry mapping route to {Label, Description}. page_index returns\nthe list so the model knows what each page shows:\n\n```go\ntype DashboardPage struct {\n\tPath string // \"/tasks\"\n\tLabel string // \"Tasks\"\n\tDescription string // \"Issues Archie picked up, and where they stand.\"\n}\n```\n\n### Navigation tool (dashboard_tools.go)\n\ndashboard_navigate validates its path against the registry and returns the resolved\npath/label. Unknown path means a clear error, not a guess. The model must call the\ntool (not invent a link) to produce a chip, so the chip is honest.\n\n### Tool wiring (turn.go)\n\nextraTools already merges TaskTools + SessionTools. Add the dashboard tools alongside\nthem, gated on the channel being the web UI. The tool list is built from the same\nregistry the prompt's page line comes from, so the agent never points at a page it\nwas not told about.\n\n### Web UI (chat-render.js, chat-tools.js)\n\nA dashboard_navigate tool call renders as a **clickable chip**: an anchor with\nhref=\"/tasks\" and the label. The chip is a\nreal link that works from any page once the chat is a global presence. The chip is\nstyled as an action, not a code line, so it visibly invites a click.\n\n### Route transfer (chat.js)\n\nEach sendMessage includes the current location.hash route as page in the stream\nrequest body, so the backend gets the operator's live position.\n\n## What we deliberately do NOT do\n\n- **No navigation from a markdown link.** The model must use dashboard_navigate; a\n bare \"click here\" is not rendered as a trusted route.\n- **No full-page route hijack from the chat.** The chip sets location.hash.\n- **No page index for non-web channels.** Telegram/discord have no dashboard.\n- **No automatic history annotation.** Page is per-message from the browser.\n- **No change to the dangerous-action approval flow.** Separately tracked.\n\n## Acceptance criteria\n\n1. Chat messages from any page carry the current page to the agent's prompt.\n2. page_index returns every dashboard page with a plain-language description.\n3. dashboard_navigate renders a clickable chip that routes to the target page.\n4. dashboard_navigate rejects an unknown route with a clear error, no chip.\n5. Non-web chat channels get neither tool, and Message.Page stays empty.\n6. Tests cover prompt page line; navigate resolve/reject; non-web exclusion; and the\n web request decoder round-trips page.\n\n## Files this change touches\n\n- internal/gateway/gateway.go - Message.Page\n- internal/gateway/prompt.go + templates/archie.md.tpl - SystemPromptConfig.Page, env line\n- internal/gateway/dashboard_tools.go (new) - registry, page_index, dashboard_navigate\n- internal/gateway/turn.go - merge dashboard tools into extraTools\n- internal/gateway/prompt_test.go + dashboard_tools_test.go (new) - prompt + tool tests\n- internal/webui/api_chat.go - chatMessageRequest.Page, decoder\n- ui/src/chat/chat.js - send page with each stream request; render navigate chips\n- ui/src/chat/chat-render.js + chat-tools.js + chat.css - clickable chip\n- ui/src/main.js + shell - persistent chat presence + current-page tracking (separate)\n\n\u003e The persistent \"chat on any page\" surface (drawer/launcher in the shell) is the\n\u003e companion change that makes the chip navigable from anywhere. It is the same\n\u003e capability and should land with or just before it, but is not required for the\n\u003e backend to be correct and testable on its own.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/dashboard-live-state/",
|
||
"title": "Dashboard live state",
|
||
"section": "PRDs",
|
||
"body": "# Dashboard live state\n\n**Status:** Approved\n\n## Decision\n\nThe dashboard must not require passive **Refresh** buttons. One app-wide Pinia\nstore owns the existing `/api/stream` EventSource and translates backend events\ninto resource revision counters. Pages load their authoritative HTTP projection\non entry and re-read it when the relevant revision changes. Event payloads are\ninvalidation signals, not substitute read models.\n\nSSE remains the transport. Dashboard updates are server-to-client only, and the\nexisting stream already provides replay, reconnect, and persisted ordering; a\nWebSocket would add a second transport without adding a capability.\n\nThe first slice covers resources for which the backend already emits a truthful\nsignal:\n\n- task events invalidate the dashboard, task board, workflow statistics, and\n the matching task-run page;\n- capture events invalidate the capture list;\n- curator events invalidate curator status, and curator actions also invalidate\n the skill catalogue;\n- update reports invalidate System status;\n- opening a page always loads its resource, and reopening the SSE stream\n invalidates mounted projections after a connection loss.\n\nLogs retain their dedicated SSE tail. Local mutations continue to re-read their\nown resource after success. Configuration, channels, and catalogues that have no\nbackend change event load on route entry; future write contracts must emit an\nowner-specific event rather than introduce polling or restore a button.\n\n## Boundaries\n\nThe Pinia store owns browser connection state and invalidation only. It does not\nbecome an alternate backend cache or copy domain state into a generic global\nobject. Existing feature modules continue to own their DTOs and API reads.\n\nOperational commands are not refresh controls. Channel **Reload** remains an\nexplicit adapter lifecycle action. Authentication recovery is also not a data\nrefresh: a 401 moves the application into shared authentication state; reloading\nthe same URL cannot restore Archie's missing or rotated token and must not be\noffered as a fix.\n\n## Verification\n\n- No passive page action renders the label **Refresh** or **Retry** for a read.\n- Only one shared `/api/stream` EventSource is opened by the application.\n- Resource classification tests distinguish task, capture, curator, skill, and\n update invalidations.\n- UI tests, type checking, generated asset freshness, and `task check` pass.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/dashboard-port-brief/",
|
||
"title": "Dashboard port: what the agent delivers",
|
||
"section": "PRDs",
|
||
"body": "# Dashboard port: what the agent delivers\n\n**Status:** Approved\n**Date:** 2026-09-20\n**Beads:** `archie-core-tavn` (epic), `archie-core-ocfm` (navigation)\n**Reads with:** `docs/prds/vue-migration-checklist.md` (the item-by-item)\n\nThis is the standard the port is held to. The checklist says what to port; this\nsays how, and what counts as done. It exists because the first attempt ported\nthe flat topbar verbatim and invented a palette, both of which had already been\ndecided elsewhere.\n\n## The deliverable\n\nThe Preact dashboard, rebuilt in Vue 3 with shadcn-vue, feature complete\nagainst the checklist, gate clean, with `ui/dist` rebuilt and committed in the\nsame commit as the source it was built from.\n\n## Non-negotiables\n\n- **Vue 3 + shadcn-vue (Reka UI, Tailwind v4, Lucide).** A component that\n exists in shadcn-vue is not hand-rolled. Add it with the CLI\n (`npx shadcn-vue@latest add \u003cname\u003e`), do not paste it in.\n- **History routing.** `createWebHistory`, `base: \"/\"`. No hash routing.\n `src/legacy-hash-redirect.ts` handles old `#/path` links and stays.\n- **No test port.** 43 Preact test files are deliberately dropped.\n `task test:ui` is a `vue-tsc` typecheck. Write a test when there is a real\n failure to pin, not to re-assert a port.\n- **Do not downgrade TypeScript.** The target is TS 7 native, not TS 5.\n- **`ui/dist` is law.** Never hand-edit it, never omit it, never revert it.\n- **`task check` is the gate.** Not an assembled substitute for it.\n- **Beads, not markdown TODOs**, for anything that outlives the change.\n- **Conventional commits**, scoped `feat(webui):`. Commit gate-clean work\n without asking. Do not push without being told.\n\n## Componentisation\n\nOne component, one job, one file. A surface is composed from named components,\nnot assembled inside a single large file. The topbar is the worked example and\nthe required shape:\n\n```\ncomponents/topbar/\n Topbar.vue composes the header, owns nothing else\n Brand.vue Logo + BrandName, links to /\n Logo.vue\n BrandName.vue\n Nav.vue reads the nav model, renders links and groups\n NavTrigger.vue the shared top-level control, link or dropdown trigger\n NavGroup.vue one dropdown\n NavItem.vue one item inside a dropdown, with its tooltip\n SearchBar.vue\n SupportLink.vue\n ThemeToggle.vue\n ProfileDropdown.vue\n```\n\nThe same rule applies to every page ported after it. If a file is doing the\nwork of four named things, it is four files. Data and behaviour that more than\none component needs lives in `src/lib/`, not passed down through a chain of\nprops.\n\n## Ordering\n\nBuild the navigation restructure during the port, not after it. Porting the\nflat eleven-item bar first and restructuring later means writing the topbar\ntwice and touching the Go route registry twice. The approved design is five\ntop-level items (Dashboard, Work, Agent, Events, System), items renamed because\ntheir group carries the context, per-item descriptions shown as hover tooltips,\nConfiguration split into `/system/*`, `/settings` kept as a redirect, and Logs\nfinally given a nav entry.\n\n## Design authority\n\n- The approved documents win. Where a document decides something, implement it\n rather than re-deciding it.\n- **Do not invent a visual direction.** Palette, typography and density are the\n maintainer's call. A palette change is its own request, never a side effect\n of a port.\n- Contrast is AA against every surface the foreground lands on, hover states\n included. The corrected values (`--fg-subtle`, the dark\n `--primary-foreground`) came from a measured failure and must survive.\n- `color-scheme` follows the theme so the UA paints controls, scrollbars and\n overscroll to match.\n\n## Code style\n\nComment the non-obvious why: an invariant, a trap, a measurement, a contract\nanother file depends on. Do not narrate what the code plainly does, do not\nexplain a choice that was never in question, and do not write migration phase\nhistory into a diff. That history belongs in the commit message.\n\n## Done, per surface\n\n1. Renders with real data against `task dev`, not just typechecks.\n2. Keyboard reachable, focus visible, and the nav announces where you are.\n3. Works at 1440x900 and at 720px wide, with nothing clipped and no horizontal\n page scroll.\n4. Empty, loading, failed and unconfigured (501) states all say something true.\n A section that cannot load says so; it does not render blank.\n5. `task check` passes, `ui/dist` rebuilt, committed together.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/dashboard-vocabulary/",
|
||
"title": "One source for the dashboard's server vocabularies",
|
||
"section": "PRDs",
|
||
"body": "# One source for the dashboard's server vocabularies\n\n**Status:** Finalised\n\n## Problem\n\nThe dashboard kept its own copies of two vocabularies the daemon owns:\n\n- the changed-file statuses (`added`, `modified`, `deleted`, `renamed`,\n `typechange`), which are the `Change*` constants in\n `internal/domain/workflow/task`;\n- the schema stamp on a `config_captured` payload, `events.ConfigCapturedSchema`.\n\nNothing derived one side from the other, while `/api/task-meta` already served\nthe status and action vocabularies for exactly this reason.\n\n## Decision\n\n`GET /api/task-meta` also serves `change_statuses` (id and label) and\n`config_schema`. `buildTaskMeta` in `internal/webui/api_task_meta.go` takes the\nids from the `Change*` constants and the stamp from `events.ConfigCapturedSchema`.\n\nThe dashboard renders `ui/src/lib/task-meta-snapshot.ts` until the served\ncatalogue arrives, or when archied is unreachable. `loadTaskMeta` replaces only\nthe keys the payload carries, so a server without the new keys leaves the\nsnapshot in place. `changeStatusLabel` and `configSchema` in\n`ui/src/lib/task-meta.ts` are the only readers; an unknown status renders as\nits id.\n\n## Verification\n\n- `TestTaskMetaChangeStatusesAreDeliberate` fails when a `Change*` constant is\n added without a label.\n\nA rename on one side therefore fails a test on the other.\n\n## Rejected\n\nDeleting `/api/task-meta` and accepting hand-copied constants. The endpoint is\nlive, and deleting it would still leave the two copies this document removes.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/deep-research-capability/",
|
||
"title": "Deep research capability — design",
|
||
"section": "PRDs",
|
||
"body": "# Deep research capability — design\n\n**Status:** Draft\n**Date:** 2026-09-14\n**Scope:** A deep-research capability in archie-core: an orchestrator that\nspawns 1..N subagents to explore and research in parallel, collates their\nfindings, and returns one cited report. Grounded in web search and web fetch.\n**Beads issue:** none yet (this document is the input to it)\n**Depends on:** nothing shipped for slice 1; slice 2 depends on slice 1.\n\n## §0 Implementation status\n\nEvery claim in this document is mapped to what exists in the tree.\n\n| Claim | Status | Where it stands |\n| ---------------------------------------------------------------------- | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `web_fetch` tool (URL → readable text) | **Implemented** | `internal/tools/webfetch/{tool.go,webfetch.go}`; registered in `internal/app/archied/bootstrap.go` `registerStandaloneTools`, disabled-by-config returns `nil` |\n| **Web search** (query → result list) | **Aspirational — zero code** | No search provider, no search tool, no config block anywhere in the tree or in `ai-sdk` v0.1.32 |\n| `agent.Subagent` — nested agent in a fresh context window | **Implemented** (dependency) | `github.com/samcharles93/ai-sdk@v0.1.32/agent/subagent.go`. Synchronous, non-streaming, own model/system/toolset, `MaxSteps` defaults to 10, nesting depth capped at 5, returns final text only |\n| Subagent used in production | **Implemented, one consumer** | `internal/app/agentworker/review.go` — the adversarial reviewer. That is the only non-test use of `Subagent` in the repo |\n| A **delegate/subagent tool** on the chat surface | **Aspirational — zero code** | No `subagent`/`delegate` tool name is registered anywhere |\n| **Fan-out orchestration** (N missions, one collator) | **Aspirational — zero code** | `agentloop`'s package doc states it runs _one_ mission: \"orchestration across missions lives with the caller\". No caller does this yet |\n| Parallel-safe tool execution | **Aspirational — zero code** | The unreachable `internal/tools/dispatch.go` dispatch/gating engine (`ExecutionClass`, `DispatchSequential`/`DispatchConcurrent`/`ClassifyEntry`) was deleted as dead code (tools-core-1); live concurrency is ai-sdk's `GenerateOptions.MaxParallelToolCalls`, which archie-core never sets |\n| Per-result caps + spill-to-disk | **Implemented** | `internal/agentexec/toolset.go` `ToolLimits{MaxResultChars,SpillDir}`, applied in `toolExecute` via `tools.CapPayload` (`limit.go`); no **aggregate** turn cap by design (`toolset.go`). `config.ToolPolicy.SpillDir`; default cap 50k chars, spill dir deliberately not defaulted |\n| Tool guardrails (repeat-failure / no-progress) | **Implemented, not on the call path** | `internal/tools/guardrail.go` engine exists. Its only non-test caller is `dispatchOne` — which is itself unreachable — so **the chat path never consults it**, and the agent path only records per _stage_ (`domain/workflow/agent.go`) |\n| Structured capture tools (model reports into a typed store, not prose) | **Implemented** | `internal/app/agentworker/review.go` `record_finding` / `record_checked` — `core.NewTypedTool` appending to a caller-owned slice. This is the pattern a research source ledger should reuse |\n| Curator engine (interval, declared tool set, lifecycle) | **Implemented, tools broken** | `internal/domain/curator`; a curator declaring tools is **refused** at registration (`registry.go`, no `ToolBuilder` implementation) and `curatorLLMRunner` (`internal/app/archied/main.go`) drops `ChatRequest.Tools`. Budgets are time+concurrency only (`runtime.go`) — **no token budget** |\n| Cron / scheduling + delivery router | **Implemented, wired into the daemon** | `internal/domain/scheduling` (ticker engine), `internal/infrastructure/cronstore` (schema v2, `Kind` discriminator), and `internal/infrastructure/crondelivery`; `internal/app/archied/scheduling.go` composes them and `main.go` starts the engine. |\n| Workflow engines (multi-stage, resumable) | **Implemented, worktree-bound** | `internal/domain/workflow` — every non-triage workflow opens with `StagePrepareWorktree()` and mounts `tc.Dir` into a container; the task persists after every stage (`workflow.go`). `triage.go` documents there is \"no cheaper directory-less classification primitive\" |\n| Long-running chat-initiated work with progress | **Partial** | Chat turns run **once, synchronously**, inline in the channel's turn runner (`internal/gateway/turn.go`). Chat can spawn a _workflow task_ (`task_spawn`) which is asynchronous — but that path is worktree/PR-shaped |\n| **Operator-push delivery of an async result** | **Aspirational — zero code** | Nothing pushes a finished result to a chat. No channel subscribes to the event bus (Telegram only _publishes_ `KindTurnCompleted`); `crondelivery.Courier` is the intended seam but has **no implementation and no caller**; `send_file` works only inside the live turn. The one out-of-band notification in the tree is feasibility's n8n webhook (`feasibility.go`) |\n| Citation storage / source ledger | **Aspirational — zero code** | No source, citation, or provenance store exists for research output |\n\n\u003e **Target state, not current state.** Everything marked aspirational below is\n\u003e design, not a description of the daemon as it runs.\n\n## Problem\n\nSam's requirement: _\"Deep research should spawn 1 or many subagents for\nexploration or research work. And they should collate all findings, they\nshould use web search/fetch features.\"_\n\nNone of the three primitives that sentence needs can be assembled from what\nships:\n\n1. **Web search does not exist.** `web_fetch` reads a URL you already have. A\n research loop's first act is to _discover_ URLs from a query, and there is\n no tool, provider, or config for that — in archie-core or in the shared SDK.\n2. **Nothing can fan out.** `agent.Subagent` can nest _one_ nested generation,\n synchronously, and only the PR reviewer calls it. There is no N-way fan-out,\n no bounded concurrency, no shared budget across children, no collation pass.\n `agentloop` explicitly refuses to orchestrate: it runs one mission and the\n caller owns orchestration — and no caller orchestrates.\n3. **A research run outlives a chat turn.** A chat turn is synchronous and\n inline: the operator waits, and the turn's own context window is what\n carries the result. A fan-out over N subagents with web search is minutes to\n tens of minutes of wall clock. It cannot be a turn.\n\nThe interesting part is not the loop. It is where the boundary sits: which\nexisting long-running mechanism this reuses, how citations survive collation\nwhen the thing being collated is model-written prose, and what stops a run\nfrom spending unbounded money.\n\n## What the run actually is\n\nSix stages, each of which must be a _stage_ rather than a prompt instruction:\n\n1. **Frame.** One cheap model call turns the request into a research brief: the\n question, 3..N distinct sub-questions, and a per-sub-question budget. Output\n is structured (a capture tool), not prose — the fan-out width comes from\n this step, so it must be a number the code reads, not a sentence.\n2. **Fan out.** One subagent per sub-question, **in parallel**, with a bounded\n concurrency, each running a read-only research toolset (search + fetch +\n read) against its own fresh context window, each required to record its\n findings as _typed source records_ before finishing, and each compressing\n what it returns.\n3. **Cross-check.** Deterministic, no model: group by canonical URL and by\n claim, flag single-sourced claims, mark conflicts. See D6.\n4. **Collate.** One model pass over the children's _conclusions and source\n records_ — never their transcripts — that writes the report body against the\n pre-reconciled source set.\n5. **Citation pass.** Its own stage: validate every marker in the body resolves\n to a stored source, and every stored source was fetched successfully.\n6. **Render.** The citation list is rendered from the **source store**, not\n from the collator's prose, so a URL can only appear in a report if some\n subagent actually recorded it.\n\nStages 5 and 6 are the load-bearing decision, and they are what the commercial\nproducts' published failure modes point at: a synthesizer that is free to write\ncitations will invent them, and a citation that cannot be traced to a fetch is\nworse than no citation.\n\n## Prior art: what the disclosed designs and the open-source field say\n\nTwo external research passes inform this design — one on the commercial\nproducts (official engineering blogs, model docs, system cards, plus\npeer-reviewed benchmarks), one on open-source deep-research implementations\nread at pinned commits. The findings that changed decisions:\n\n**Only one fan-out unit buys capacity.** Four different units are in use:\nAnthropic fans out _agents with their own context windows_; Google fans out\n_queries_; Perplexity fans out _generated programs_; OpenAI uses pipeline-stage\nagents for query enrichment only. Query fan-out is cheap and buys breadth of\nretrieval; agent fan-out is what buys context capacity. Our step 2 is agent\nfan-out, deliberately, because context isolation is the thing a single GPU\ncannot otherwise buy.\n\n**Nobody collates section-by-section.** The open-source survey found one merge\nprompt over concatenated child summaries (`final_report_generation` in\n`langchain-ai/open_deep_research`), lead-agent synthesis (Anthropic), a\ncode-level reconcile (Perplexity), or single-model continuation. STORM is the\nonly demonstrated section-mapped merge. This confirms D3: one collation pass,\nnot a mapped merge.\n\n**Citations are lost at the merge in 8 of 10 surveyed repos.** Only `lajosdeme/mole`\nassigns citation markers **mechanically in code before the prompt is built** —\nits `internal/output/report.go` carries the comment _\"it cannot introduce a\ncitation, because the citation markers are assigned mechanically before the\nprompt is built\"_ — and stores a verbatim `Quote` plus a `QuoteOffset` (byte\noffset into the fetched document) so a citation can be re-verified later.\nClaims whose quote is not found in the source are dropped by code\n(`internal/actors/mine.go`: `\"claim rejected: quote not found in source\"`).\nThis is the strongest available argument for D1, and `QuoteOffset` is worth\nadopting: it makes re-verification cheap and turns \"the model said so\" into\n\"the document says so, here\".\n\n**The failure is measurable and it is at the orchestrator.** An EMNLP 2026\nstudy localising errors across three top-ranked open-source deep-research\nsystems found **84.7% of final-report errors originate at the orchestrator,\nroughly 31% of them hallucinations and the rest citation mistakes**, with\ninformation passing \"through agents like a telephone game\"\n(https://arxiv.org/abs/2608.24306). Anthropic's own mitigations for this are a\nseparate **CitationAgent** pass after synthesis and a filesystem-artefact\npattern — subagents store work externally and pass back lightweight references\nrather than full outputs. Perplexity reached the same conclusion independently.\nBoth are adopted here: D1 (store, not prose) and a citation pass that is its\nown stage.\n\n**A cross-check stage that no vendor has.** Worker A and worker B never compare\nnotes in any disclosed design, and nothing in the published set _resolves_\nconflicts — Perplexity only \"notes\" them. Grouping findings by URL and by claim,\nflagging single-sourced claims, and marking unresolved conflicts is the\ncheapest correctness win available and is where a local implementation can beat\nthe frontier products rather than imitate them.\n\n**Fan-out is expensive in tokens, and on one GPU that is wall clock.** Anthropic\ndiscloses ~15x tokens for multi-agent versus ~4x for a single agent, and\nexplicit tier rules in the orchestrator prompt (1 agent / 3-10 tool calls;\n2-4 subagents / 10-15 calls each; \u003e10 subagents for complex research, 3-5\nspawned concurrently). On a single card, N concurrent workers contend for the\nsame weights and KV budget, so **wall clock scales roughly linearly with worker\ncount** — the isolation benefit survives, the throughput benefit does not. One\ninstrumented study puts **web search at 73% of wall-clock time on average (up to\n91%)** (https://cseweb.ucsd.edu/~yiying/2025_NIPS_ERW_Deep_Research_Perf_Study.pdf),\nwhich is the argument for fanning out _retrieval_ aggressively and _generation_\nsparingly, and for overlapping generation with retrieval.\n\n**Frontier citation volume is not a target.** Frontier runs emit 41-113\ncitation URLs per query, while 3-13% of citation URLs are hallucinated and \"more\ncitations per query do not mean fewer errors per citation\"\n(https://arxiv.org/abs/2604.03173). Ten to twenty _verified_ citations is a\ndefensible, evidence-backed target.\n\n**Do not port Perplexity's style.** Search-as-code-generation needs\nfrontier-class codegen against a bespoke SDK and an atomised self-owned search\nstack. Port the _intent_ — deterministic fan-out, dedupe and filter in code\nbefore anything reaches a model — and leave the code generation to the Go\norchestrator.\n\n## Architecture: three candidate shapes\n\n| # | Shape | Where a run starts | Durability | New code | Cost |\n| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | ----------------------------------------------------- | --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |\n| **A** | New capability family `internal/domain/research` + `infrastructure/research`, typed engine, owning registry, lifecycle, background run, completion delivered through an injected courier | A chat tool `research_start` returns a run id | Runs survive the turn; progress observable; resumable | Domain + infra + store + wiring + UI + config | Largest |\n| **B** | A fourth cron job kind (`KindResearch`) in `crondelivery`, reusing `scheduling` + `cronstore` + the delivery router | Chat enqueues a one-shot immediate job | Same store and delivery as cron; no new persistence | Runner + kind + blueprint only | Medium — but fakes \"on demand\" as a scheduled job, and a job store keyed on a schedule is the wrong shape for a request |\n| **C** | A `deep_research` **tool** that runs the whole fan-out synchronously inside one chat turn, using `agent.Subagent` for the children | Directly — the model calls the tool | None: the run dies with the turn | Search seam + tool + fan-out helper | Smallest |\n\n**Recommendation: ship C's machinery, then promote it to A.** Slice 2 (below)\nbuilds the orchestrator as a tool because that is the smallest thing that\ndelivers Sam's stated design and reuses the `Subagent` primitive that already\nworks. Slice 3 lifts the identical orchestrator behind a durable run once its\nruntime cost and shape are known from real use — which is exactly the order\nAGENTS.md's solo-project rule asks for (\"prefer the smallest workable change\"),\nand it avoids inventing persistence for a shape that has not been exercised.\n\nShape B is rejected as the primary path: `cronstore.JobSpec` is a _schedule_\n(cron expression, next-run, catch-up); an on-demand research request is not.\nReusing it would mean every ad-hoc research run leaves a permanent job row.\n\n### Where the pieces live (shape C, then A)\n\n| Piece | Home | Contract |\n| ----------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| Search provider seam | `internal/domain/research` (interface) + `internal/infrastructure/research/search/\u003cprovider\u003e` (impls) | `Searcher.Search(ctx, query, opts) ([]Result, error)`; `Result{URL, Title, Snippet, RetrievedAt}`. Consumer-owned and one-method, per the `crondelivery.Courier` precedent — nothing here talks to a network |\n| `web_search` tool | `internal/tools/search` | Mirrors `internal/tools/webfetch`: `Tool(cfg) *tools.ToolEntry`, returns `nil` when disabled, `ClassIdempotent`, `Toolset: \"web\"` |\n| Source store | `internal/domain/research` entity + store | `Source{ID, URL, CanonicalURL, Title, RetrievedAt, Hash, State(ok/failed/duplicate), CitedBy []subq}` |\n| Fan-out orchestrator | `internal/domain/research` (pure) with the model/tool access injected | Bounded worker pool over sub-questions; per-child `agent.Subagent`-shaped `Researcher` seam; shared step/token/wall-clock budget |\n| Collator | `internal/domain/research` | Model pass over `[]ChildConclusion` + source store → report body |\n| Report renderer | `internal/domain/research` | Markdown + numbered citation list rendered **from the store** |\n| Chat trigger | slice 2: `internal/tools/research`; slice 3: gateway tool + courier in `internal/app/archied` | — |\n| Durable run + delivery (slice 3) | `internal/domain/research` + `internal/infrastructure/research` + `internal/app/archied` wiring | Follows the strict plugin-engine rule in `docs/architecture/plugins-and-extensions.md` (\"Plugin engine |\n| rule (strict)\"): typed contract, owning registry with start/health/stop, narrow typed registrar |\n\n### Why not the curator\n\nThe curator family already looks like the right home — a declared interval, a\ndeclared tool set, an owning registry, per-pass budgets and panic isolation.\nTwo facts rule it out as the host for _this_:\n\n- **Its tool path does not work.** A curator declaring tools is refused\n registration because `ToolBuilder` has no implementation, and even if it were\n bound, `curatorLLMRunner` drops `ChatRequest.Tools` (`main.go`). Fixing\n that is `archie-core-trnd.1`, an independent piece of work.\n- **Its contract is maintenance, not requests.** The isolation invariant\n (`docs/architecture/agent-system.md`, \"Curator isolation invariant\") defines\n curators as periodic peers reasoning over memory. A deep-research run is\n operator-initiated work with a deliverable, on the operator's clock.\n\nA research _curator_ is a legitimate later idea — \"research this standing topic\nweekly\" — but it is a consumer of this capability, not its foundation.\n\n## Decisions this design takes\n\n### D1. Citations come from a store, never from prose\n\nEach subagent gets a `record_source` typed tool (the `record_finding` pattern,\n`review.go`) whose handler canonicalises the URL, deduplicates against the\nstore, marks which sub-question recorded it, and appends. The collator may\n_select and order_ sources; it may not author them. Rendering the reference\nlist is then a pure function of the store.\n\nThe strongest prior art goes further, and one step of it is worth adopting:\nstore a **verbatim quote plus a byte offset into the fetched document** with\neach claim, and drop any claim whose quote cannot be found in the source\n(`lajosdeme/mole`: `FindQuote` + `Claim.QuoteOffset`, with the miner logging\n`\"claim rejected: quote not found in source\"`). That converts an unfalsifiable\n\"the model said so\" into a re-checkable \"the document says so, at offset N\", and\nit makes citation re-verification cheap rather than a second research run.\n\n_Rejected alternative:_ require the child to end with a JSON block of sources\nand parse it. A parse failure loses every source in the child's run, and a\nmodel that reformats its output for readability silently drops citations.\n\n### D2. Fan-out is parallel, bounded, and shares one budget\n\n`agent.Subagent.Run` is synchronous, so N children means N goroutines behind a\nsemaphore — concurrency from config, sized from the VRAM budget rather than\nfrom the research question. Two reasons to keep it small: the shared model\nruntime also serves chat, and on a single GPU N concurrent workers contend for\nthe same weights and KV budget, so **wall clock scales roughly linearly with\nworker count**. The isolation benefit of fan-out survives that; the throughput\nbenefit does not.\n\nFan-out width and per-child `MaxSteps` are enforced by the orchestrator, not\nrequested in a prompt, against a whole-run wall clock and token ceiling.\n`agentloop.Budget` already carries `MaxSteps`, `MaxTokens`, `WallClock`, and\n`TokenBudgetIs(maxTokens)` exists as a stop condition (`agentloop.go`) —\nthe shape exists, the fan-out owner does not.\n\n**Fan out retrieval aggressively and generation sparingly.** Web search is 73%\nof wall-clock time on average (up to 91%) in the one instrumented study\navailable, so retrieval is where the parallel win lives; generation is the\nserialised, expensive half.\n\n### D3. The orchestrator never holds child transcripts\n\n`Subagent.Run` sends a prompt and returns a string; the parent's history is\nnever passed down and the child's is never returned. That is the property that\nmakes N children affordable at all — the parent's context grows by N\nconclusions, not N conversations. This is also why collation is a _separate\npass_ rather than an accumulating agent loop: the collator is the only place\nthat must hold all N conclusions, and it holds nothing else.\n\nEach child compresses its own findings before returning, so what crosses the\nboundary is a summary, not a transcript. Anthropic names the same pattern\n(filesystem artefacts with lightweight references back) and Perplexity arrived\nat it independently; it is the one disclosed primitive that _increases_\neffective capacity without a bigger context window.\n\n### D4. Search is a provider seam with no silent default\n\n`web_search` is registered only when a provider resolves, following two\nexisting conventions: return `nil` for a disabled capability\n(`webfetch/tool.go` — \"an advertised tool that never works costs a round\ntrip\") and resolve the credential _before_ registering\n(`registerMinimaxTool`, `bootstrap.go`). At least two implementations are\nwanted — a self-hosted meta-search endpoint (no per-query cost, no data egress)\nand one hosted API (quality fallback) — behind the same one-method interface,\nselected by config.\n\n### D5. A run reports what it could not establish\n\nThe collator is required to emit, alongside the report, the sub-questions that\nreturned nothing usable and the sources that failed to fetch. A research report\nthat omits its own gaps reads as authoritative over a hole, and the operator\ncannot tell a negative finding from a missing one.\n\n### D6. Two stages no surveyed design has: a mechanical cross-check and a separate citation pass\n\nChildren never compare notes in any disclosed design, and nothing in the\npublished set _resolves_ conflicts — Perplexity only \"notes\" them. Because\n84.7% of final-report errors originate at the orchestrator (mostly citation\nerrors), both gaps are worth closing in code rather than in a prompt:\n\n- **Cross-check (deterministic, no model).** Group findings by canonical URL and\n by claim. Flag single-sourced claims, mark unresolved conflicts explicitly,\n and count sources per sub-question. This runs _before_ the collator, so the\n collator receives a pre-reconciled set rather than raw prose to untangle.\n- **Citation pass (its own stage, after collation).** Anthropic runs synthesis\n and then a separate CitationAgent. Here the pass is cheaper for being partly\n mechanical: it validates that every marker in the body resolves to a stored\n source, and that every stored source was actually fetched successfully.\n\nBoth stages are small, both are testable without a model, and together they are\nwhere this implementation can be _better_ than the frontier products rather\nthan an imitation of them. A run that is expected to emit 10-20 well-verified\ncitations rather than 41-113 partly-unverified ones is making a deliberate,\nevidence-supported trade.\n\n## Open decisions\n\n1. **Trigger surface.** A chat tool that blocks the turn (shape C) is\n immediately useful but the operator waits; a durable run (shape A) is the\n right end state. Does slice 2 ship the blocking tool, or does the capability\n go straight to durability?\n2. **Delivery for a durable run.** Which channel gets the finished report, and\n is the report an inline message, a file, or a served page? The\n `crondelivery.Courier` seam covers the chat case; a long cited report\n probably wants a document, since chat surfaces render markdown but not\n footnotes well.\n3. **Search provider choice and egress.** A hosted search API sends every query\n to a third party; a self-hosted meta-search keeps it local. This is an\n operator decision with a real privacy dimension, and the default matters.\n4. **Report persistence and reuse.** Does a report become a memory, a document,\n or neither? Reusing a prior report is the difference between \"research\" and\n \"research again next week\".\n5. **Cost ceiling.** What is the per-run token ceiling, and is it a config\n value, a per-request parameter, or derived from the fan-out width?\n\n## Failure and security paths\n\n- **SSRF.** Search results are attacker-influenceable (a page can rank for a\n query). `web_fetch` already vets the _resolved address_ at dial time with\n `AllowPrivateNetworks` off, which is the property to preserve — a search\n tool must not become a way to reach the daemon's own control surfaces by\n returning an internal URL.\n- **Prompt injection.** Fetched pages are untrusted content and will contain\n instructions. Subagents fetch; the orchestrator and collator must never\n execute an instruction sourced from a page. Bounded by giving subagents a\n read-only toolset with no shell and no write.\n- **Runaway spend.** Unbounded fan-out × unbounded steps is the failure mode.\n Bounded concurrency, per-child step caps, a whole-run wall clock, and a\n token ceiling, all enforced in code.\n- **Degenerate fan-out.** A model asked for \"N sub-questions\" will produce N\n near-duplicates of the same question. Deduplicate sub-questions before\n dispatch, and require distinct search queries per child.\n- **Failed dependencies.** Search provider down: the tool is absent (D4), not\n broken. Fetch fails: recorded as a `failed` source so the report can say so.\n- **Collator degradation.** If the collator call fails or truncates, the report\n must still be rendered from the source store and the child conclusions —\n degraded, not empty. This mirrors the reviewer's rule that findings captured\n before a failed run are still real (`review.go`).\n\n## Slices\n\n1. **Search seam + `web_search` tool.** Interface, one implementation, config\n block, tool registration, tests. Independently useful to every chat turn.\n2. **Fan-out orchestrator + collation, as a tool.** `record_source` capture\n tool (with quote + offset), bounded parallel children with per-child\n compression, cross-check, collator pass, citation pass, store-rendered\n citations. Triggerable from chat; blocks the turn.\n3. **Durable run + delivery.** Promote the orchestrator behind the engine\n contract, add the run store, progress events, and completion delivery.\n4. **UI + reuse.** Dashboard surface for runs and past reports; optional\n research curator on a schedule.\n\n## Non-goals\n\n- Replacing `web_fetch`; research uses it.\n- A general multi-agent collaboration framework (`archie-core-1786637492113-89`\n owns that question).\n- Report publication to a static site or external destination.\n- Fine-tuning or a dedicated model: the capability is orchestration, and it\n must work on whatever model the operator has configured.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/dynamic-curators/",
|
||
"title": "Dynamically-defined curators -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Dynamically-defined curators -- decision\n\n**Status:** Approved\n**Date:** 2026-09-12\n**GitHub epic:** [#789](https://github.com/samcharles93/archie-core/issues/789)\n**Parent epic (curator engine):** [#435](https://github.com/samcharles93/archie-core/issues/435) (closed)\n**Depends on:** nothing shipping; this resolves the open question that epic #435's\n\"registered and configured declaratively, not hardcoded\" criterion left open.\n\n## §0 Implementation status\n\nEvery claim in this document is mapped to what exists in the tree.\n\n| Claim | Status | Where it stands |\n| --------------------------------------------------------------------------------------------------------------------- | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Curator engine family: typed contract, owning registry, start/health/stop, narrow host access | **Implemented** | `internal/domain/curator/{contract,registry,runtime,registrar,activity,wake}.go` |\n| Curator declared shape (`Manifest`: interval, cooldown, on_input, tools, skills, memory engine, conversations, model) | **Implemented** | `internal/domain/curator/contract.go`; `Validate()` requires `Interval \u003e 0` |\n| Two reference curators run through the registry | **Implemented** | `internal/infrastructure/skillcurator`, `internal/infrastructure/sessioncurator` |\n| Curator definitions as persisted, API-editable data | **Aspirational -- zero code** | No table, no entity, no RPC, no config block |\n| Generic definition-driven engine that executes a declared tool set + free-form instructions | **Aspirational -- zero code** | Only the two code-registered curators exist |\n| `Registrar.Tools` (`ToolBuilder`) resolved and bound at registration | **Partial** | Interface declared (`registrar.go`); **no implementation anywhere**; `bootstrap.go` builds the `Registrar` with no `Tools` field |\n| A curator can declare tools at all | **Partial** | `Manifest.Tools` exists, but `registry.go` fails registration with `\"curator manifest: declares tools but the registrar has no ToolBuilder\"` |\n| Curator passes that actually reach a model with tools | **Cosmetic only** | `curator.ChatRequest` carries `Tools`/`MaxSteps`; `curatorLLMRunner` (`internal/app/archied/main.go`) drops both and returns `ChatResult{Text}` only |\n| Dynamic registration / removal (no restart) | **Aspirational -- zero code** | `Registry` has `Register`, no `Unregister`; `Runtime` has `Start`/`Nudge`/`Stop`, no `Add`/`Remove`; `Start()` snapshots membership |\n| Curator definition CRUD over REST | **Partial** | `GET /api/curators` read-only (`internal/webui/server.go`); no write routes |\n| Curator definition config | **Aspirational -- zero code** | No `[curator]` block in `internal/config/config.go` |\n| Store/API as definition authority | **Aspirational -- zero code** | No curator persistence of any kind |\n| No live-path defaults for interval/cooldown/tools/model/memory engine | **Partial** | `skillcurator.DefaultInterval` / `sessioncurator.DefaultInterval` are Go consts read at registration; `curator.NewRuntime(..., RuntimeConfig{})` uses code defaults for pass timeout and concurrency |\n| Idle curators are observable in logs | **Partial** | `runtime.go` logs nothing when `Check` reports not-due; the session-memory curator's hourly pass is the only curator activity ever logged |\n| Sampler family | **Implemented** | `internal/domain/sampling`, shipped; no curator consumes a `Sampler` yet |\n| Forge/issue reach from chat | **Implemented** (via `shell`) | No dedicated issue tool exists, but the chat agent's `shell` is unconfined (`chat.unrestricted_filesystem = true`), so `gh issue edit | comment | create | close` works. Dedicated tools (#161) are convenience, not a blocker |\n\n\u003e **Target state, not current state.** Everything below marked aspirational is\n\u003e design, not a description of the daemon as it runs.\n\n## Problem\n\nA curator exists only if a Go package implements `curator.CuratorEngine`\nand `bootstrap.go/1025` names it in a `Register` call. Nothing about a\ncurator is definable as data. Worse, no curator can declare a tool set at all:\nthe `ToolBuilder` interface has no implementation in the repo, so the registry\nrefuses (correctly, fail-closed) any curator that declares tools -- and the\ndaemon's model adapter drops the tools even if it were bound. The curator family\ntherefore has exactly one shape (cheap, tool-less maintenance pass) and its only\nextension point is compiling Go.\n\nSam's requirement is the opposite: _\"Curators should be dynamically created and\ndefined, in addition to being able to be defined in code/config/webui\"_ and _\"No\nconfiguration should ever be hard-coded.\"_ The motivating consumer is a\nproject/orchestrator curator -- declared as data, given forge issue tools, and\npointed at a repo to review, refine, and propose issues.\n\n## Decision\n\n**A curator definition is data: a persisted entity whose declared shape is\nexactly `curator.Manifest` plus identity, free-form instructions, and an enabled\nflag. One generic engine interprets any definition.** The shape is not a second\ninvention -- `Manifest` already encodes interval, cooldown, on-input, tools,\nskills, memory engine, conversations, and model, and `Validate()` already\nenforces what a valid shape means. The definition adds only what `Manifest`\ncannot express:\n\n| Definition field | Relationship to `Manifest` |\n| -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| `name`, `enabled` | Identity and lifecycle; the registry's key is the name |\n| `instructions` | **Free-form** system prompt for the generic engine -- the piece that makes a definition more than a knob set |\n| interval, cooldown, on_input, tools, skills, memory_engine, conversations, model | The existing `Manifest` fields verbatim; the definition **is** a `Manifest` |\n\nCode-registered engines keep `CuratorEngine` unchanged. A data-defined curator is\nserved by one registered engine whose `Manifest()` is the stored definition and\nwhose `Pass` composes `instructions` with the built tool set.\n\nThis is the curator domain's missing half, not a standalone tool: it lands in\n`internal/domain/curator` (entity + engine), `internal/infrastructure/postgres` (persistence),\n`internal/webui` (CRUD), and `internal/app/archied` (seed + apply wiring).\n\n## Definition sources and the merge rule\n\nThree sources, per Sam's requirement. **The store is authoritative.**\n\n| Source | Role | Collision behaviour |\n| --------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- |\n| **Code** (a registered Go engine) | The escape hatch: a curator whose strategy is Go, not tools+instructions | Wins over data for its own name -- a data definition cannot silently replace a compiled engine; the store refuses the collision and logs it |\n| **Config** (`[[curators]]`) | **Seed data only**, merged at startup | Insert-if-absent. A store row of the same name is never overwritten, and deleting a store row is not undone by config on the next boot |\n| **WebUI/API** (the store) | **Authoritative** | Wins over config on a name collision; the only source of truth after first boot |\n\nConfig existing at all is what makes a definition reproducible on a fresh host;\nmaking it authoritative would make the API a lie, so it is seed data with a\none-way merge. The alternative -- config authoritative, store a cache -- would\nmean an API edit silently reverts on restart. Rejected.\n\n## How the generic engine executes a definition\n\nA pass is: resolve the declared tools, compose the prompt, run the model loop,\nrecord actions.\n\n1. **Tool resolution.** `Registrar.Tools.Build(ctx, declared)` returns exactly\n the declared set, never a broader registry. A declared name with no\n implementation fails the pass rather than silently running with fewer tools.\n This is the enforcement point, and it is why the `ToolBuilder` binding is\n load-bearing rather than a convenience: without an implementation, no curator\n can declare tools at all (the `registry.go` guard).\n2. **Tool-carrying model call.** `curator.ChatRequest` already carries\n `Tools []tools.ToolEntry` and `MaxSteps`; `curatorLLMRunner` must pass them\n into `core.GenerateOptions` (which has `Tools`, `ToolChoice`, `MaxSteps`, and\n already executes the tool loop) instead of dropping them. `ChatResult` is\n widened only as far as attributing the pass's actions requires.\n3. **Prompt.** The system prompt is the definition's `instructions` plus the\n declared tool summaries -- rendered from the same set that was built, never\n from a wider list. Each pass records `Action`s so activity stays attributable\n through the existing registry `activity` surface and events.\n\nA bespoke non-tool strategy (the skill curator's structural validation, the\nsession-memory curator's own extraction) remains a registered Go engine. That is\nthe escape hatch, not the mode: the mode is tools + instructions as data.\n\n## Dynamic registration and mutation\n\nA definition change must apply without a daemon restart, which the runtime\ncannot do (`Start()` snapshots membership into one goroutine per curator;\nno `Unregister`, no `Add`/`Remove`).\n\n- `Registry.Unregister(name)` -- valid while running, same failure isolation as\n `Register` (a rejected removal affects only that curator).\n- `Runtime.Add(engine)` / `Runtime.Remove(name)` -- `Remove` cancels the removed\n curator's in-flight pass context and stops its goroutine within a bounded\n wait; it never holds the registry mutex across a pass.\n- The app layer owns the apply sequence (stop old, register new, start) so the\n registry stays the membership authority and the runtime the scheduler.\n\nRemoval must not be able to orphan a loop, and a pass that ignores cancellation\nmust not block removal forever.\n\n## REST / WebUI surface\n\nMirror `internal/webui/api_binding.go`, not a new pattern:\n\n| Route | Handler shape |\n| --------------------------------------------- | ------------------------------------------------------------------- |\n| `GET /api/curators` | Existing read-only list, fields unchanged (dashboard keeps working) |\n| `POST /api/curators` | `handleCuratorCreate` |\n| `GET /api/curators/{id}` / `PATCH` / `DELETE` | `handleCuratorGet` / `Update` / `Delete` |\n\nAll mutations behind `authorizeTaskMutation`, same as bindings and mappings. A\nsuccessful mutation goes through the runtime apply path above, so it is live\nwithout a restart.\n\n## Sequencing: persistence and the State Store\n\nThe State Store is a standalone process that solely owns the task database; the daemon\nand Gateway never open it (`TestOpenStoresNeverOwnsTaskDB` guards this), and\n`proto/state/v1/state.proto` is one gRPC service fronting every ratified store\ncontract. AGENTS.md forbids adding a Go interface method without a matching RPC.\nThat fixes the order:\n\n| Phase | Ships | Proto work |\n| ------------------------- | --------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| 1. Engine + config source | `ToolBuilder`, tool-carrying passes, generic engine, `[[curators]]`, idle logging | None -- nothing persisted yet |\n| 2. Runtime mutation | `Unregister`, `Add`/`Remove`, apply path | None -- in-memory membership |\n| 3. Persistence | definition entity + store schema, config seed (store authoritative), REST CRUD | **Same change**: request/response messages + RPCs, `staterpc.Client` façade, `mapError`/`unmapError`, and an update to `docs/prds/state-store-contract.md` |\n\nPhases 1 and 2 are genuinely independent of the wire boundary, so they are not\nblocked on it. Phase 3 is where the boundary is unavoidable: a definition the\nWebUI can edit must be reachable from the WebUI, and the WebUI reaches the store\nthrough the State Store, not by opening the database. Splitting the RPCs into a\nlater change would leave a table nothing can read -- rejected. Deferring the\nwhole phase is the same rejection, later.\n\n## No live-path constants\n\nNo default may be a live-path constant. Interval, cooldown, tools, model, memory\nengine, pass timeout, and concurrency are definition or config values. A\ndefinition missing `interval` is refused with a clear error, never defaulted at\npass time; `skillcurator.DefaultInterval` and `sessioncurator.DefaultInterval`\nstop being read at registration (the value becomes seed data), and\n`curator.NewRuntime(..., RuntimeConfig{})` stops relying on code defaults.\n\n## What this deliberately does NOT do\n\n- **No fixed catalogue of curator \"types\".** A definition is arbitrary tools +\n instructions, not a union of known kinds.\n- **No chat tool that mints curators.** A definition grants an LLM a tool set; a\n curator able to create curators is privilege escalation. Mutation is REST with\n the same authority as bindings -- not exposed on the chat tool surface.\n- **No dashboard SPA editor page.** The REST surface is the WebUI definition\n source; the dashboard can adopt it separately.\n- **No forge issue tools here.** This design only ensures a curator _can declare_\n tools. Dedicated issue tools (#161) are a convenience, not a prerequisite: the\n chat agent already reaches issues through the unconfined `shell` tool\n (`gh issue edit|comment|create|close`). See the correction on #789.\n- **No second authority.** No file-per-curator definition store, no parallel\n config reader.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/dynamic-workflow-triage/",
|
||
"title": "Dynamic workflow triage",
|
||
"section": "PRDs",
|
||
"body": "# Dynamic workflow triage\n\n**Status:** Finalised\n\nEpic: archie-core-enfj. Scope: give `workflow.Route()` a real, content-aware\nfallback instead of unconditionally defaulting every unlabelled task to the\nheaviest workflow.\n\n## Problem\n\n`Route()` (`internal/domain/workflow/workflow.go`) has exactly two real\nsignals: an explicit `t.Workflow`, and label matches via\n`workflowForLabels` (`routing.go`, `bug`→tdd, `feature`→feasibility,\nbootstrap→bootstrap). Anything else — which is every chat-spawned task,\nsince `task_spawn` rarely sets labels — falls straight through to\n`reg[\"implement\"]`, the full `prepare → baseline → plan → build →\ncommit-push → review → open-pr` pipeline, regardless of what the task\nactually asks for.\n\n`task_spawn`'s own tool schema overpromises: _\"workflow: … Omit to let the\ndaemon route it\"_ (`internal/gateway/task_tools.go`) implies real\nrouting exists. It doesn't — \"route\" means \"check labels, then\ngive up and run the heaviest workflow anyway.\"\n\nConfirmed in production: a chat-spawned task whose\nentire content was \"this is just a test, close it when you receive it\" ran\nthe full `implement` pipeline end to end — 669,421 tokens, ~7 minutes —\nwhen nothing about the request needed a worktree-driven build at all. The\n`plan` stage's own LLM output _correctly_ concluded no code change was\nneeded, but that conclusion arrives two expensive stages (`baseline`,\n`plan`) too late to save anything.\n\n## Decision\n\nAdd a **`triage` workflow**: cheap, classifies once, then either closes the\ntask immediately or hands off to the workflow the task actually needs.\n`Route()`'s final fallback (`reg[\"implement\"]`) becomes\n`reg[\"triage\"]` when a `triage` entry is registered, else the existing\n`implement` fallback — a two-line change, fully backward compatible when\n`triage` isn't wired up.\n\nTriage does **not** replace label-based routing. A labelled task\n(`bug`/`feature`/bootstrap) already has a free, reliable signal and goes\nstraight to its known workflow unchanged — spending a classification\ncall there would be pure waste. Triage only fires in the gap that\ndefaults to `implement` blindly: no explicit workflow, no label match.\n\n### Triage workflow shape\n\n```\nStagePrepareWorktree() -- still needed: AgentStage mounts tc.Dir into the\n agent container: internal/domain/workflow/agent.go:74\n passes tc.Dir straight through. No cheaper\n primitive exists in this codebase for a\n directory-less classification call.\nAgentStage \"classify\" -- ReadOnly, ONE cheap agent turn, no exploration\n tools required beyond what ReadOnly already\n grants. Mirrors feasibility.go's decideCaptureTools\n pattern: a structured capture tool, called\n exactly once, MaxCalls: 1.\n```\n\nMission: read the task's title/body/labels (already in `taskPromptBlock`)\nand decide, via a captured `decide` tool call:\n\n```json\n{\"needs_code_change\": bool, \"workflow\": \"implement\" | \"tdd\" | \"feasibility\", \"reasons\": \"...\"}\n```\n\n`needs_code_change: false` → close path: reuse `closeNoChangesIssue`\n(`steps.go`) directly — issue closed if forge-backed, task marked\n`StatusMerged`/`\"completed -- no changes required\"` otherwise. No baseline,\nno plan, no build, no commit, no PR. This is the actual fix for the\nproduction incident: a no-op request now costs one classification call\n(low thousands of tokens) instead of 669,421.\n\n`needs_code_change: true` → sets `tc.Task.Workflow` to the classifier's\nchosen workflow (`implement`/`tdd`/`feasibility`, defaulting to\n`implement` if the field is missing or unrecognised) and an Outcome that\nrequeues the task under it, the same requeue mechanism\n`chatTaskControllerAdapter.ApproveChatTask` already uses\n(`main.go`, `requeue(ctx, taskID, fromStatus, workflow)`). The task then\nre-enters `Route()` on its next claim with `t.Workflow` now set, and takes\nthe normal path for that workflow from there — triage never runs its\nclassification twice for the same task.\n\n### Non-goals (v1)\n\n- Full bug/feature intent classification quality — the classifier only\n needs to be _directionally_ right; a wrong `tdd` vs `implement` call\n still produces a working PR, just via a slightly less-tailored pipeline.\n Tightening this is a follow-up, not a blocker.\n- Any change to label-based routing, `Route()`'s explicit-workflow branch,\n or `Bootstrap()`/`Feasibility()`/`TDD()` themselves.\n- Triage for forge-sourced (real GitHub issue) tasks with no labels —\n in scope structurally (same fallback branch), but the dominant case this\n fixes is chat-spawned tasks, which essentially never carry labels.\n\n## Testing\n\n- `workflow_test.go`: `Route()` picks `reg[\"triage\"]` over `reg[\"implement\"]`\n for an unlabelled task when both are registered; falls back to\n `reg[\"implement\"]` unchanged when `triage` isn't registered (regression\n guard for every existing `Route()` test).\n- New `triage_test.go`: classify → `needs_code_change: false` closes\n without ever calling `tc.Trees.Prepare` a second time or touching\n `tc.Agent.Run` again; classify → `needs_code_change: true` sets\n `tc.Task.Workflow` and produces a requeue Outcome, using a fake\n `agentRunnerFunc` returning a captured decision, mirroring\n `feasibility_test.go`'s existing `decideCaptureTools` test style.\n- Integration-shaped test reproducing the production scenario: a\n chat-spawned task with no labels, title \"this is just a test, close it\n when you receive it\" — asserts the task closes without ever reaching a\n `baseline`/`plan`/`build` stage.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/eda-playbook-engine/",
|
||
"title": "EDA playbook engine -- design",
|
||
"section": "PRDs",
|
||
"body": "# EDA playbook engine -- design\n\n**Status:** Draft\n**Date:** 2026-09-03\n**Beads epic:** `archie-core-t2db`\n\n## Problem\n\n\"Which workflow runs for which trigger\" is hardcoded Go: a literal\nlabel vocabulary (`workintake.labelKinds`) and a literal label-\u003eworkflow map\n(`workflow/routing.go`), extended only by adding a Go case or shipping a\nskill whose `metadata.archie.workflow` overrides one _named_ built-in\nworkflow. Extending the system to a new trigger, a new action, or a new\nevent source means a feature branch, a code change in a hardcoded\npath, and new Go tests -- for every addition. That is the barrier this\ndocument exists to remove.\n\nThe goal: triggers, the actions they run, and the\nconditions that connect them should be **drop-in and load**, not **drop-in\nand touch application code**. Yaegi-evaluated Go is the unit of imperative\n\"do a thing\" logic; YAML playbooks are pure orchestration data -- no logic,\njust \"on this event, with this data, call these actions, in this order,\nunder these conditions.\" The daemon's event coordinator doesn't know what a\n`bugfix` or `image-gen` action does, it resolves the name against a\nregistry and pipes typed data through.\n\nThis is explicitly **not** scoped to code-generation workflows. A playbook\naction can be any registered capability -- run a codegen workflow, send a\nmessage on a channel, open something on a forge, generate an image -- so\nlong as it's implemented behind one of the typed engine families below.\n\n## Relationship to existing decisions\n\n- `docs/prds/event-sources-and-reactions.md` ruled\n that reactions in the two epics it covers are **producer-only** (no veto,\n no mutation of in-flight work) and explicitly rejected a single generic\n `Source` interface as \"the untyped-hook shape the plugin engine rule\n exists to prevent.\" **This document extends that scope**: playbooks can\n dispatch to multiple typed action kinds, not just \"produce a\n `TaskEnvelope`.\" It does not relitigate producer-only-ness (still no\n veto/mutation of in-flight work here) and it does not introduce one\n generic interface -- see \"Typed positions, not one generic hook\" below,\n which is how it stays compliant with the plugin engine rule instead of\n reopening that document's #4.\n- `docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`\n governs every extension point\n introduced here. Every position below is a typed domain contract with an\n owning registry, not a generic callback map. This is a hard constraint,\n confirmed against the existing rule text before writing this doc.\n- Yaegi is already a proven in-process extension mechanism\n (`internal/plugin`, `internal/domain/workflow/wfeval/yaegi.go`,\n `internal/gate/gateeval/yaegi.go`, `internal/skill/plugin.go`) -- this\n design generalises its existing role, it does not introduce a new\n execution mechanism.\n- `internal/domain/workflow/skillbuild.BuildRegistry` already composes a\n `workflow.Registry` from a skills catalogue, plugin-defined overriding\n built-in. The playbook loader described here supersedes label routing\n specifically; it does not replace `skillbuild`'s workflow-composition role,\n it becomes an additional input the daemon composes at startup.\n- Webhook event intake is a separate design. This\n document defines the router each webhook-sourced event should be able to\n reach; it does not redesign webhook receipt itself.\n\n## Shape: four logic positions, each a typed engine family\n\nEvery entrypoint kind (mechanism, action, loop, schedule, route, trigger) is itself a schema emitter: defining a new kind produces the\ngenerated contract downstream tooling consumes. The kinds slot into a fixed\nhierarchy of logic positions:\n\n```\nWorkflow -\u003e Module -\u003e Channel (telegram/whatsapp/email/...)\n -\u003e Forge (GitHub/Gitea/GitLab/...)\n```\n\nEach position is its own typed engine family per the plugin engine rule:\nidentity + capability-specific operations, an owning `Registry`/`Manager`,\nexplicit lifecycle where the implementation owns resources, narrow host\naccess. Concretely:\n\n- **Workflow** -- already exists (`internal/domain/workflow`). A playbook\n action of kind `workflow` invokes a registered `workflow.Registry` entry\n by name, unchanged from the existing execution path.\n- **Module** -- the new general-purpose action position. A `Module` is a\n Yaegi-evaluated implementation of a generated interface (args in, typed\n result out) for one declared action kind (e.g. `image-gen`, `notify`,\n `open-pr-comment`). This is the position that makes actions \"not\n restricted to code-generation.\" Full design (schema-per-kind, the\n registry shape, the trust boundary, the recommended first slice):\n `docs/prds/module-position.md`.\n- **Channel** -- already exists as `channels.Channel` (telegram, email,\n webhook). A playbook action of kind `channel` sends through a named,\n already-registered channel. No new mechanism; this document just makes it\n addressable from a playbook.\n- **Forge** -- forge clients (GitHub, Gitea, GitLab) become an addressable\n action position the same way Channel is, so a playbook can say \"comment\n on this forge issue\" without that being a special case wired only into\n the poller.\n\n**Typed positions, not one generic hook.** There is no single `Action`\ninterface with an `any` payload. Each position keeps its own typed\ncontract; the playbook YAML names a position and an implementation, and the\nevent coordinator resolves that pair against the matching typed registry.\nThis is what keeps the design compliant with the plugin engine rule and\nwith `event-sources-and-reactions.md`'s rejection of a generic `Source`/\nhook interface -- the generality lives in the YAML data model (which\npositions exist, which named implementations are registered), not in a\nruntime `any`-typed dispatch path.\n\n## `go:generate`-driven schemas, from day one\n\nSchemas are not a future nicety bolted onto an ad hoc\nYAML shape. They are the struct+interface contracts for each logic\nposition, and they drive code generation from the start.\n\n- Each logic position (Workflow/Module/Channel/Forge) has a schema source\n (Go-annotated types, following the existing pattern elsewhere in the\n repo) from which `go:generate` emits the interface and struct scaffolding\n an implementation must satisfy.\n- A Yaegi module implements the generated interface for its position. The\n generated contract is what makes an implementation valid, not hand-review.\n- The schema is also what the linter and LSP (below) validate a playbook\n YAML against -- one source of truth for codegen, startup loading, and\n authoring-time tooling, so the three can't drift from each other.\n- Schema versioning is explicit per position, independent of a playbook's\n own workflow version (see collision handling below).\n\nThe generator itself is not designed here. The constraint is\nschema-first, `go:generate`-driven, per logic position.\n\n## Playbook YAML: pure orchestration data\n\nA playbook declares no embedded logic -- only:\n\n- **trigger**: the event predicate (e.g. forge issue assigned + labelled\n `bugfix`, a schedule tick, a webhook payload shape).\n- **actions**: an ordered list, each naming a logic position and a\n registered implementation, with typed argument data and optional\n conditions.\n- **data flow between actions**: results/facts from one action available to\n later actions in the same playbook run -- exact interpolation syntax is\n an open point, not decided here (see Open questions).\n\nPlaybooks live in a configured directory (repo/directory-scoped, per Sam's\n\"define repos, directories, playbooks\" framing) and are loaded at archied\nstartup, alongside the existing skills-catalogue composition.\n\n## Dedup: detect and log, no arbitration mechanism\n\nNo runtime auto-resolution rule. An automatic \"pick a winner\" algorithm\n(e.g. by version) is itself a mechanism that can fail and need debugging\nwhen a collision happens -- the wrong kind of complexity to introduce for\na case that should just be visible and fixed by a human. Instead:\n\n- **At daemon startup:** if independently-sourced playbook directories\n declare a collision on the same trigger, that is a definition failure.\n The colliding definitions are dropped (not loaded) and the failure is\n logged plainly (which sources, which trigger) and reported back to the\n caller that requested the load -- not swallowed as a background log line\n only. The schema is the definition; a schema conflict is a reported\n failure, not something the daemon silently arbitrates.\n- **Linter**: a CI/dev-time tool, reading the same generated schemas, that\n flags duplicate or conflicting trigger definitions _within_ a single\n maintained tree before merge.\n\n **Resolved** (`archie-core-t2db.12`). A standalone\n gopls-shaped binary (`cmd/archie-playbooks`, lint subcommand) validates\n playbook directories against the exact loaders the daemon uses\n (`LoadPlaybookDirs`/`LoadKindWorkflowsYAML`/`LoadLabelWorkflowsYAML`),\n exiting non-zero on any collision / malformed file / invalid binding.\n A finding about one binding key leads with the key's `file:line`, read\n from the `yaml.Node` the loader decodes; the linter agrees with runtime\n validation by construction, which is the load-bearing property. `-eda-dir`\n lints an `eda_playbook_dir` the same way, through `playbook.Load` with its\n CEL, action-id and args checks.\n Discoverable via `task lint:playbooks` or direct `go run\n./cmd/archie-playbooks lint -dir ...`. The LSP/serve mode is a later\n entrypoint of the SAME binary, per the shared-package decision above.\n\n- **LSP**: same schema source again, feeding author-time hover/completion/\n inline errors while someone edits a playbook YAML in an editor. Kept\n explicitly in scope per Sam (not deferred), because it's the same schema\n artifact as the linter's, just a different consumer.\n\n `archie-playbooks serve` is the language server, over stdio. When a file is\n opened or saved it lints the file's saved directory with the loader the\n daemon runs for it (a file with a top-level `trigger` key is an EDA\n playbook, anything else a routing binding file) and publishes the findings\n for that file as error diagnostics, each on the line of the key, action,\n `when` or args key it names; a file-level finding sits on the first line.\n Unsaved edits are not validated. Hover and completion are later additions.\n\nThe startup log-and-refuse path exists for what the linter/LSP structurally\ncannot see -- sources composed together only at daemon startup, never\nchecked against each other by either tool. If the linter and LSP do the job\nthey're built for, a real collision reaching that startup path should be\nrare in practice, not a routine occurrence the design leans on.\n\n**Shared package, thin consumers -- not a bundled-vs-standalone choice.**\nThe validation logic (`LoadKindWorkflowsYAML`, `LoadLabelWorkflowsYAML`, the\ndirectory-merge from t2db.11) lives in `internal/domain/workflow`, a plain\nGo package with no knowledge of which binary calls it. This makes \"should\nthe linter/LSP be bundled into archied or a separate tool\" a non-question:\n\n- `archied` already imports the package directly and validates in-process\n at startup -- no subprocess, no external tool call.\n- An agent that needs to validate a playbook in-process can import the same\n package directly, for the same reason.\n- The standalone `cmd/` binary (t2db.12, gopls-shaped: one binary, a lint\n mode now, an LSP/serve mode later) exists only because CI and editors are\n out-of-process consumers by nature -- a CI step needs an exit code, an\n editor speaks LSP over a process boundary, neither can `import` a Go\n package. It is a thin wrapper per `organisation.md`'s `cmd/` rule, not a\n second implementation.\n\nFolding CLI/LSP protocol handling into `archied` itself would grow the\ndaemon into unrelated concerns, against the plugin-engine rule's spirit of\nnarrow capability families.\n\n## Execution-time gaps (both resolved)\n\nEverything above covers _loading_ a playbook. Nothing above covers what\nhappens while one _runs_. Two gaps, both load-bearing enough to resolve\nbefore implementation, not defer:\n\n### 1. Mid-playbook action failure\n\n**Resolved** (`archie-core-t2db.18`). The default below\nis a committed decision, not a candidate list; changing it later would\nrequire its own decision. Gap 2 below is resolved by `archie-core-t2db.17`.\n\nA playbook is an ordered action list with data flowing between steps. If\naction 2 of 4 fails at runtime (an image-gen API times out, a forge call\n403s), the run needs a defined outcome. This\ncodebase already answers a structurally identical question for its\nexisting workflow engine, precisely enough to reuse rather than\nre-derive: `workflow.Run` (`internal/domain/workflow/workflow.go`)\ndraws a three-way distinction on a stage failure, and a playbook run\ndraws the same one on an action failure.\n\n**Decision: reuse `Run`'s three-way distinction directly.** The stated\ndefault is **stop-on-first-failure**: a real action error halts the\nplaybook run immediately and is reported, with no continue-to-next-action\nand no automatic retry or compensating follow-up. The three points below\nare the complete decision.\n\n1. **A real action error** (the action's own logic failed -- an API\n error, a validation failure inside the action) stops the run\n immediately. No later actions execute. The failure is reported the\n same way a load collision is reported -- dropped, logged plainly\n (which action and why), and returned to the caller, never swallowed\n as a background log line only (the Dedup section's drop-and-report\n rule). `Run` is the mechanical\n precedent for the stop itself: a stage error sets the park reason and\n returns without running further stages (`workflow.go`). This\n is stop-on-first-failure, matching the document's existing bias\n against clever automatic recovery, and it is now a stated decision\n rather than a nobody-chose default.\n2. **Interruption** (the daemon is shutting down mid-dispatch, `ctx.Err()\n!= nil`) is explicitly **not** a failure. `Run` already treats this\n case specially -- \"Daemon shutdown is not a workflow failure. Leave\n the task running ...; parking here would publish a false failure and\n require manual intervention\" (`workflow.go`) -- and the same\n reasoning applies to a playbook run: an in-flight dispatch interrupted\n by restart must not be recorded as a broken playbook. What \"leave it\n alone for restart\" means for a playbook run (there is no `store.Task`\n row a playbook run inherently owns the way a workflow stage does) is\n an open implementation detail for whoever builds the coordinator, not\n decided here -- but the _semantic_ (interruption ≠ failure) is decided.\n3. **A run that produces no outcome at all** (every action ran without\n error, but nothing was assigned to happen) is itself an error\n condition, matching `Run`'s own \"a workflow must end with an explicit\n outcome; not doing so is a definition bug, which still must not vanish\n silently\" (`workflow.go`). A playbook with actions that all ran\n but never reached a coherent terminal state is a definition bug in the\n playbook, reported as such.\n\n**Rollback:** explicitly **not attempted**, for exactly the reason the\ngap's own description named -- a posted forge comment is not revocable.\nThis is not a limitation to work around later; it is the correct\nsemantic. An operator who needs a corrective action after a failure\nwrites a _new_ playbook/action for that, they do not get automatic\nundo.\n\n**This stays inside the producer-only rule.** Stop-on-first-failure is the\nrun deciding its own terminal state from its own action's error. Nothing\nalready in flight is blocked or altered; the run only declines to start\nactions that had not started yet, and it records the failure on its own\nunit the way `Run` sets a park reason and transitions the task to\n`StatusParked`. A future compensating action (\"if the notify action\nfailed, also log an incident\") would be producing new work, which the rule\nalready permits. No syntax for it is designed here.\n\n**J3 relationship, stated explicitly:** J3 (a CEL `when` condition\nerroring at eval time → treated as false → skip that action → run\ncontinues) is a **different case from an action failure** and is not\nrevisited by this resolution. A condition error means \"we couldn't\ndetermine whether to run this action,\" which this document already\nresolved as \"then don't, and move on.\" An action failure means \"we tried\nto run this action and it broke,\" which stops the run per point 1 above.\nThe two must not be conflated: a coordinator implementation should be\nable to point at one `when`-evaluation code path and one\naction-invocation code path and show they use different outcomes.\n\n### 2. Idempotency at execution time\n\n**Resolved** (`archie-core-t2db.17`, a\ndesign-investigation ticket: resolve, do not implement). The decision\nbelow is the answer; the keying scheme itself is separate follow-up work\nand is intentionally not built here. It **extends**\n`event-sources-and-reactions.md`'s existing contract -- it does not\ncontradict it (relationship argued at the end).\n\n**Problem (unchanged):** `event-sources-and-reactions.md` decided\nreactions ride `internal/eventbus`'s at-least-once delivery specifically\nso `PublishUnique`/idempotency keys absorb redelivery without duplicate\nside effects. This document inherits that for free only for\n`workflow`-kind actions, because `TaskEnvelope` already carries an\nidempotency key. It said nothing for `channel`, `forge`, or `module`\nactions with real side effects -- a posted comment, a sent message, a\nbilled image generation.\n\nOne correction to the problem statement, checked against the code rather\nthan assumed, because it changes what the answer must be:\n**`PublishUnique`'s dedup is not a consumer-redelivery guard at all.**\n`PublishUnique` only sets the JetStream `Nats-Msg-Id` header on the\noutgoing message (`internal/infrastructure/eventbus/nats/publisher.go`;\nkey constant `message.go`); JetStream suppresses _republished_\nmessages carrying a repeated key inside `Config.DedupWindow`\n(`client.go`, `Duplicates: cfg.DedupWindow`), default 2 minutes\n(`config.go`, `DefaultDedupWindow = 2 * time.Minute`). A consumer\nredelivery is the same _stored_ message re-delivered on Nak or\nacknowledge-timeout (`internal/eventbus/eventbus.go`: \"a handler\nreturning an error causes redelivery\") -- not a republish; its\n`Nats-Msg-Id` is never re-firewalled, and even if it were, two minutes is\nnot an at-most-once guarantee for a side effect that must never repeat.\n**Conclusion: side-effecting action kinds need a consumer-side, durable\ndedup record; the bus cannot supply it.** That is precisely the situation\n`binding_dispatches` was built for -- so the decision below reuses its\nmechanism rather than designing a new one.\n\n#### Decision\n\n**Granularity: per-action-per-event -- one dedup record per\n(action, event) pair, not per playbook run.** Whole-run keying, the\nobvious alternative, has two crash windows: the run row is written when the run starts, so a crash mid-run\nleaves the run \"started\" and a redelivery skips _every remaining\naction_ (work silently lost); a crash before the row write re-runs\n_every already-fired action_ (duplicate side effects -- the exact harm\nthis gap exists to prevent). Per-action keys collapse both into one\nwindow and one behaviour: an action's record is written immediately\nbefore that action's side effect is invoked, so an action fires iff its\nown record is absent, no fired action ever re-fires, and a redelivered\nevent resumes at the first unrecorded action. For the shipped\nsingle-action playbooks the two granularities are equivalent in effect;\nthe difference starts with multi-action runs -- where whole-run keying\nwould already be wrong.\n\n\u003e Storage superseded: every store now lives in PostgreSQL behind\n\u003e `internal/domain/storecontract`, implemented in `internal/infrastructure/postgres`\n\u003e (`docs/prds/state-store-contract.md`). SQLite names below describe the\n\u003e original design.\n\n**Storage/lookup: a new durable ledger table `playbook_dispatches` in\nthe event-capture store, copying `binding_dispatches`'s conventions exactly**\n(durable table, `INSERT OR IGNORE`, sentinel error, rows freed with\ntheir parent, survives daemon restarts -- `RecordDispatch`):\n\n```sql\nCREATE TABLE playbook_dispatches (\n playbook_id TEXT NOT NULL,\n playbook_version TEXT NOT NULL,\n event_id TEXT NOT NULL,\n action_id TEXT NOT NULL,\n dispatched_at TEXT NOT NULL,\n PRIMARY KEY (playbook_id, playbook_version, event_id, action_id)\n);\n```\n\nWritten with the same `INSERT OR IGNORE` + sentinel-error convention\n(`ErrAlreadyDispatched`, `bindings.go`), so a duplicate is a no-op\nwrite rather than a constraint error, and the caller matches it with\n`errors.Is` the way the binding dispatch loop already does\n(`internal/daemon/daemon.go` -- \"another cycle already won this\nrace,\" not an error to surface). The record is consumed by the playbook\ncoordinator at dispatch time through a domain-side interface implemented\nby the event-capture store -- the same shape as `BindingDispatcher`\n(`internal/domain/storecontract`) -- and the table lands in the\nstore's schema migrations. No change\nto `internal/eventbus`; no per-message dedup state on the client.\n\n**Lifetime: no time-based expiry, and no automatic reclamation.** Rows are\nretained until an explicit `DeletePlaybookDispatches` for their playbook.\nRemoving a playbook file does not delete its rows: playbooks are read only\nat boot, so the only place to notice a removal is a boot-time reconcile,\nand a boot that finds the directory missing or unmounted loads as \"no\nplaybooks\" and would reclaim the whole ledger, letting redelivered events\nfire their side effects again. A binding's rows go with an explicit\noperator delete (`bindings.go`, same transaction); a file disappearing is\nnot an equally deliberate act. The at-most-once guarantee must not\nsilently decay, which is exactly the property that disqualifies\n`DedupWindow` above; table growth is bounded by distinct (playbook,\nevent, action) pairs and is accepted, the same tradeoff\n`binding_dispatches` already makes.\n\n**Redelivery behaviour: skip the recorded action and continue the run.**\nA detected duplicate is not re-invoked, is not reported to the caller as\nan error, and is logged at debug level rather than as a warning -- this\nis the expected, correct behaviour of at-least-once delivery, not an\nanomaly (same treatment as `ErrAlreadyDispatched` at\n`daemon.go`).\nThe run continues with the next action (or ends, for a single-action\nplaybook whose one action is recorded), which is what makes a redelivery\na resume rather than a dead restart.\n\n**Key derivation: a fixed composition of four structural identities,\ncomputed by the coordinator -- not a CEL expression.** The key is the\n`(playbook_id, playbook_version, event_id, action_id)` tuple above, where:\n\n- `playbook_id` is `Playbook.ID` -- the playbook file's path relative to\n its configured directory root, slash-normalised, unique within the load\n composition by construction (`internal/domain/eda/playbook`).\n- `playbook_version` is `Playbook.Version` -- a SHA-256 of the loaded\n file, recomputed on every load (`internal/domain/eda/playbook`),\n mirroring `Binding.Version`'s provenance-pinning purpose: the dispatch\n is pinned to the exact definition that was active when it fired. A\n changed playbook definition is a different key and may fire again for\n the same event -- deliberate, not a bug.\n- `event_id` is `DispatchInput.TaskID` (`playbook.go`), which\n carries the `TaskEnvelope.IdempotencyKey()` value\n (`\"archie:\" + owner/repo/number`,\n `internal/domain/workintake/envelope.go`). It is chosen over\n `store.Task.ID` because the trigger vocabulary is issue-level\n workintake label/kind, dispatched at discovery time (pollNATS, forge\n webhook receiver), where no task row exists yet (`playbook.go`\n comment; `playbook_test.go`: \"the value available at the\n discovery/dispatch point... NOT a store.Task.ID int64\"). Keying on the\n issue identity rather than the delivery source is the load-bearing\n property the existing contract already established for `TaskEnvelope`\n (`event-sources-and-reactions.md` question 4: a webhook-sourced\n envelope for the same issue produces the _same_ idempotency key as a\n poll-discovered one; `internal/forge/webhook/receiver.go` --\n \"Idempotency is not this package's job\"); this resolution inherits that\n property rather than re-deciding it. Consequence, stated explicitly:\n same-issue re-discoveries (a re-label that re-triggers the same kind)\n are the _same event_ by design -- the workintake trigger vocabulary is\n issue-granular, exactly as `TaskEnvelope`'s own key is. A future\n capture/binding-originated trigger type keys on `captured_events.id`\n instead; derivation is per-trigger-type, not one formula for all\n triggers (rule carried over unchanged from the draft).\n- `action_id` is the action's declared `id` -- J1's named ids, \"a\n required field on every action that later actions reference\", unique\n and stable-identifier-shaped, validated at load (the CEL resolution's\n J1 and its context table above in this document). For an action declaring no `id`\n (permitted by J1 while unreferenced), the key uses the action's 1-based\n position in the playbook: deterministic and stable for the\n single-action playbooks, and a later reorder is a file edit whose\n content hash changes `playbook_version` with it, so a positional\n fallback cannot silently collide with a prior definition.\n\n**No CEL in the key -- stated explicitly because the CEL resolution\nflagged this as gap 2's \"first real exercise\"** (\"CEL's read-only result\naccess makes gap 2's keying derivation readable\" -- the CEL resolution's\nclosing note). Readability\nis not the binding constraint; **determinism under redelivery is**, and\nCEL key expressions lose three ways. First, they are not load-checkable:\n`event` stays `dyn` by the resolved J4 (schema-by-example), so\na key expression over event fields cannot be type-checked at load,\nviolating the document's reject-at-load philosophy. Second, they can\nfail at dispatch (a missing field is a CEL error), and a key that cannot\nbe computed is an unfireable state -- neither \"skip\" nor \"fire\" is safe\nwithout losing or duplicating work. Third, the contract needs one\nderivation, and two authors writing different key expressions would\nfragment the ledger. The key is therefore structural by construction:\nevery component above is an immutable identity of the run's inputs, so\ntwo deliveries of the same event derive the same key with no evaluation.\nThe exercise's actual outcome is the derivation rule above; CEL remains\na _condition_ surface (`when`), never a key surface. An action kind that\ngenuinely needs finer granularity than (action, event), such as \"per\ncomment, not per issue\", arrives with its own trigger type and its own\nevent identity, per the per-trigger-type rule.\n\n**Record-before-invoke -- one deliberate divergence from\n`binding_dispatches`, justified by the harm profile.** The binding\ndispatch loop enqueues the task first and records the ledger row after\n(`daemon.go`), accepting that racing cycles may both enqueue\n(\"the duplicate task remains queued... delete-on-races is its own can of\nworms\"); the selection query excludes already-dispatched captures\n(`bindings.go`), but the race window remains. That ordering is\ncorrect there because the duplicate artifact is an internal task row --\nrecoverable, low-harm. The playbook ledger's ordering is reversed on\npurpose: **the row is written and committed before the action's side\neffect is invoked** -- the `INSERT OR IGNORE` is the atomic gate, and\nthe lost-write loser skips without invoking. The cost is stated, not\nhidden: a crash between the row commit and the invoke loses that one\naction's side effect, repairable by hand -- the reverse direction of the\nsame window that would otherwise duplicate a posted comment or\nre-bill an image generation. For side effects whose duplication is\nexternally visible and not revocable, at-most-once is the required\ndirection; the binding system's at-least-once-tolerant ordering is not\nthe precedent for this ledger.\n\n**Gap-1 interplay, stated to keep the two mechanisms separate:** the\nledger answers one question per action -- has this action already fired\nfor this event? It does not answer whether the run continues, stops, or\nis re-attempted; that is gap 1's coordinator concern (a run-outcome\nrecord; gap 1 explicitly leaves \"what leave it alone for restart means\nfor a playbook run\" to whoever builds the coordinator). One consequence\nof record-before: an action that FAILED (gap 1's real-action-error case)\nhas a ledger row, so a later redelivery of the same event skips it --\nconsistent with gap 1's \"no automatic retry\" resolve (a corrective\naction is a _new_ playbook/action, per gap 1's rollback paragraph). A\nfailed action is never re-attempted by the bus; if a run must be\nrecovered, an operator edits the playbook -- new version, new key --\nrather than fighting ledger state.\n\n**Relationship to `event-sources-and-reactions.md` -- extension, not\ncontradiction.** Keyed to that document's sections:\n\n1. Its decision 2 commits reactions to at-least-once delivery via\n `PublishUnique`'s \"without duplicate enqueue\" guarantee -- which, with\n the code facts above, is accurate for the _publish_ path only:\n `PublishUnique` absorbs duplicate publishes of the same event (poll\n vs. webhook), which is what that decision was actually about. This\n document's own problem statement, taken literally, overstates it\n (\"keys absorb redelivery without duplicate side effects\") -- that\n overstatement is exactly what this gap exists to correct. The\n existing contract never claimed consumer redeliveries of\n side-effecting reactions were absorbed, so nothing it says contradicts\n a consumer-side ledger; this resolution pins down the boundary the\n sentence left implicit (publish-path dedup vs. consumer-path dedup)\n rather than changing either.\n2. Its decision 3 is untouched: the ledger is the run's own bookkeeping\n about its own actions, suppressing a re-fire and nothing else.\n3. Its decision 4 (no generic `Source` interface; `TaskEnvelope` is the\n typed contract, keyed on issue identity, not delivery source): the\n event half of the playbook key IS that same identity\n (`TaskEnvelope.IdempotencyKey()`), and the per-trigger-type rule\n means any future trigger type brings its own identity rather than a\n new generic one -- the playbook ledger introduces no second identity\n vocabulary and no untyped payload. The only new type in this\n resolution is the ledger's own value object, consumed through a\n narrow interface like `BindingDispatcher` -- the same shape the\n document's \"typed families\" rule (decision 1, plugin engine rule)\n already blesses.\n\n**Scope: this is the `channel`/`forge`/`module` decision, not a\n`workflow`-position change.** Workflow-kind actions keep their existing\nkeying claim unchanged -- the `TaskEnvelope` path that this document\nalready inherits for free (`daemon.go`, `PublishTask` publishes\nwith `task.IdempotencyKey()`), with its cross-poll/webhook property\nuntouched. The ledger gate applies to the side-effecting positions\n(channel, forge, module), which is why the closing statement below\nstill holds: the first slice (Module + loader + workflow-kind dispatch)\nwas buildable without this resolution, and channel/forge/multi-action\nplaybooks were not. A coordinator MAY implement one position-uniform\ngate as a simplification; what this resolution requires is only that\nthe side-effecting positions are gated.\n\n## Resolved questions\n\n1. **Data-flow/condition syntax.** Does this project want a small existing\n Go expression evaluator (there may be one already in the codebase worth\n checking, e.g. anything backing gate conditions), or a bespoke minimal\n grammar?\n\n **Resolved** (`archie-core-t2db.14`). CEL (`cel.dev/cel-go`, pinned) is\n the one mechanism for both an action's `when` condition and its `args`\n values. The trust boundary, evaluation context, load-time validation and\n the five judgement calls are in\n `docs/prds/playbook-expression-syntax.md`.\n\n2. **Playbook directory config field.** Where this lives in\n `internal/config` / `configuration.Document`, and whether it's one\n directory or a list (mirroring `SkillsDir`'s shape) -- follow existing\n config precedent, not invented fresh.\n\n **Resolved** (`archie-core-t2db.11`). The directory\n shape landed as `playbook_dirs` (a LIST of directories of\n `*.yaml`/`*.yml` binding files), loaded at startup as an additional\n input to the two single-file fields, which remain supported unchanged.\n A single `playbook_dir` would contradict\n supporting multiple independently-maintained playbook sources, which\n the Dedup section already assumes ('independently-sourced playbook\n directories collide'), so the field is a list. Cross-source collision\n (same key in two directories, or in a single-file field and a\n directory) is reported like an in-directory collision -- nothing is\n arbitrated by source precedence. This is what finally exercises the\n design doc's drop-and-report collision rule, since a single file\n cannot collide with itself. Repo-scoping (tying directories to\n `Config.Repos` owner/name) is explicitly deferred to a separate,\n later decision; org/tenant keying is out of scope (no auth/identity\n model).\n\n Decisions recorded from the label-vocabulary slice:\n - Arbitrary labels bind to registered workflow names via\n `LabelWorkflows` + `LoadLabelWorkflowsYAML`/`SetLabelWorkflows`\n (`internal/domain/workflow/routing.go`).\n - The closed `Kind`/NATS-subject set is untouched; the label map only\n extends binding authority for labels the kind layer does not own.\n - Collision rule: a label already owned by the\n kind set (bug/feature/bootstrap), an empty label, an empty workflow\n name, or a duplicate binding is a reported load failure -- dropped\n and logged, the error returned to the caller. Nothing is silently\n arbitrated by schema/version. This is the \"schema defines the\n accepted message; anything else is the caller's fault\" rule applied\n to the label layer.\n - Precedence in `Route()`: explicit `t.Workflow` → arbitrary-label\n binding → kind binding → triage → implement → default.\n\n3. **Trust boundary for Module Yaegi code.** The plugin engine rule's\n invariant 6 draws a line between operator-trusted in-process code and\n repository-supplied code that must run in a container.\n\n **Resolved** in `docs/prds/module-position.md`, \"Trust boundary\": a\n Module is operator-installed, in-process and daemon-privileged, the same\n tier as `PluginDir` and `SecretEngineDir`, never repository-supplied\n task code.\n\n4. **First implementation slice.** Recommend: Module position + the\n playbook loader + trigger-to-workflow dispatch only (subsuming the existing\n label routing) first, proving the schema-gen -\u003e Yaegi -\u003e playbook path\n end to end on the smallest useful case, before adding Channel/Forge\n action kinds or the linter/LSP.\n\n **Resolved** (`archie-core-t2db.13`, `.14`, `.15`). The Module position\n (registry + log kind, t2db.13), the CEL expression environment\n (t2db.14), and the playbook document + event coordinator with\n single-action workflow-kind dispatch (t2db.15) are shipped.\n\n **Resolved** (`archie-core-t2db.23`): the coordinator selects a real\n task's workflow. The wiring point is the daemon's definition pin\n (`Daemon.resolveWorkflowID`, consumed by `pinWorkflowFromCollection` in\n `internal/daemon`), not `pollNATS`/`publishTask`. Routing belongs to the\n pin because a workflow definition is a\n database row pinned before dispatch, which makes the pin the one place a\n task's workflow is chosen. Precedence there is explicit `Task.Workflow`\n (the waiting_human -\u003e approved requeue) → matching playbook →\n `workflow.ResolveWorkflowID`'s label binding → kind binding → triage →\n implement. A playbook naming an undefined workflow is a reported failure\n that parks the task rather than a silent fall-through to the binding's\n choice.\n\n `Store.Dispatch` returns the selected workflow _name_ and the matching\n playbook's id and version, not a compiled `workflow.Workflow`: the\n production caller decides against the active definition collection, and\n the id and version are the first two components of the gap-2 idempotency\n key. The playbook package does not import `internal/domain/workflow`.\n\n A `when` condition at this point reads what the intake knows about the\n originating issue: `kind`, `labels`, `owner`, `repo`, `number`, `title`,\n `body`, `source`, built by `playbookInput` in `internal/daemon`.\n\n The trigger shape reuses the existing workintake kind/label vocabulary.\n The single-action hard boundary is superseded by\n `docs/prds/multi-action-playbooks.md` (two-shape rule); Module, Channel,\n and Forge positions still have no production dispatch path.\n\n## Standing constraint\n\nThis may be commercialised, and is to be discernible from other open-source\nevent-driven-automation tooling. No design decision follows from that yet;\nit bears on licensing or feature-gating questions about playbook authoring\nand the LSP when they arise.\n\n## Packages this touches (first slice)\n\n- New: a schema-generation package/convention (`go:generate` source +\n emitted contracts) for the Module position.\n- New: playbook loader + event coordinator (name/location TBD -- likely\n `internal/domain/eda` or similar, following the domain-layer rule: no\n infrastructure imports, no untyped hooks).\n- `internal/domain/workflow`: dispatch target for `workflow`-kind actions,\n unchanged in its own contracts.\n- `internal/domain/workintake`: `labelKinds`/routing becomes one playbook\n among others once the loader subsumes it; existing Go fallback stays for\n backward compatibility until playbooks fully replace it (mirrors the\n `dynamic-workflow-triage.md` precedent of a two-line fallback change, not\n a rip-and-replace).\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/embedded-nats/",
|
||
"title": "Embedded NATS execution -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Embedded NATS execution -- decision\n\n**Status:** Finalised\n\n**Date:** 2026-08-24\n\n**Beads epic:** `archie-core-q15y`\n\nThis decision supersedes the position that embedded NATS was only a\ndistribution mechanism and could not be an agent-execution topology. That was\ntrue only because the embedded listener was unauthenticated loopback. It left a\nsecond production architecture in which `archied` ran autonomous workflows in\nits own process, weakening the isolation boundary that managed task containers\nexist to provide.\n\n## One execution architecture, two broker deployments\n\nEvery autonomous repository workflow is handed over as one complete task via\nNATS and runs in a task-scoped `archie-agent` container. `archied` remains a\nnative, long-lived orchestration and interactive-chat process; its operator\nconfigured host filesystem access is unchanged. It owns task admission,\nstate, forge and worktree operations, container lifecycle, and the broker when\nthe embedded deployment is selected. It does not run an autonomous model loop.\n\nNATS has two supported deployment shapes:\n\n| mode | meaning |\n| ---------- | --------------------------------------------------------- |\n| `embedded` | `archied` owns an in-process JetStream server (default). |\n| `external` | `archied` and workers connect to the configured NATS URL. |\n\nBoth shapes have identical task semantics: one container and one full-task\nrequest per workflow. Broker placement must not select a different executor.\nThe legacy `agent` section and `containers.enabled` key remain decode-only\nmigration inputs. `nats.mode = \"off\"`, per-stage NATS, in-process execution,\nand subprocess execution are not production choices.\n\nSQLite remains authoritative for task state. NATS is the sole autonomous work\nhandoff. SQLite recovery may identify work that needs republishing, but it must\nnot cause `archied` to call `workflow.Run` locally.\n\n## Embedded reachability and authentication\n\nAn embedded listener uses a cryptographically random bearer token generated for\neach daemon start. The token exists only in process memory and the environment\nof managed worker containers; it is never written to config or logs.\n\nFor a native daemon with managed containers, startup resolves the Docker bridge\nthe workers will join. The listener binds only to that bridge's host gateway\naddress on a random port. Both the native daemon and workers use that same\nreachable endpoint, and every client supplies the generated token. Binding a\nDocker bridge address keeps the listener off ordinary LAN interfaces without\npublishing a host port. A configured user bridge is respected; otherwise the\nDocker default bridge is used. A network without a usable local IPv4 gateway is\na startup error for autonomous execution.\n\nIf Docker is unavailable, interactive chat may still start and embedded NATS\nmay remain loopback-only, but autonomous tasks park with an explicit container\ncapability failure. They never fall back to host execution.\n\n## Ownership and lifecycle\n\n- `internal/infrastructure/eventbus/nats` owns the authenticated embedded\n server and exposes its runtime endpoint, not the server implementation.\n- `internal/container` resolves the worker bridge and gateway and owns task\n container lifecycle.\n- `internal/app` composition selects embedded or external broker deployment,\n freezes the connected runtime endpoint for worker environments, and wires the\n full-task handoff.\n- `cmd/archied` remains process input only as the domain migration reaches this\n composition path.\n\nThe daemon starts the embedded server before its client, closes clients before\nthe server, and keeps JetStream data beneath the daemon data directory. NATS URL\nand token are startup-built values: a reload cannot send new workers to a broker\ndifferent from the one the daemon is using.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/event-automation/",
|
||
"title": "Event automation: identified events, typed parameters, purpose-built agents",
|
||
"section": "PRDs",
|
||
"body": "# Event automation: identified events, typed parameters, purpose-built agents\n\n**Status:** Approved\n**Date:** 2026-09-23\n**Parent:** epic `archie-core-t2db`\n\n## Why this exists\n\nArchie is an event-driven automation system. Any system that can send a\nwebhook (a forge, an error tracker, a firewall, a monitoring stack) should be\nable to start an agent built for that job. The shape this replaces cannot:\n\n- A binding matches only the URL segment an event was posted to, and a source\n may have one binding. Two kinds of event from one sender cannot go to\n different workflows.\n- A mapping is a list of JSON paths written against one example payload.\n Nothing recognises what kind of event arrived.\n- Mapped values reach the task only as `key=value` lines in its body text.\n- Every task needs a repository and runs in a git worktree.\n- Every agent runs the same image with the same forwarded keys.\n\n## The model\n\n```\nsource ─▶ event type ─▶ mapping ─▶ binding ─▶ workflow ─▶ agent profile\n (identified) (params) (+ filter, (inputs, (image, tools)\n repo) profile)\n```\n\n### Sources\n\nA source is the endpoint one sender posts to. Its path is a generated UUIDv7\n(`/webhooks/capture/0199…`), so an endpoint cannot be guessed or collide. The\noperator may replace it with a custom path for a sender that cannot use that\nformat; a custom path must be unique and URL-safe, and is refused otherwise.\nA source owns the signing setting:\n\n- **Signed** (default). Archie generates the secret. Only events with a valid\n `X-Hub-Signature-256` or `X-Signature-256` HMAC are dispatched.\n- **Unsigned.** The operator turns signing off for a sender that cannot sign.\n The source and every event from it are marked unsigned in the inspector,\n the binding list and the task timeline. Turning signing off needs the same\n approval as arming a binding.\n\nUnauthenticated events on a signed source are captured and never\ndispatched.\n\n### Event types\n\nAn event type is a named kind of event from one source, such as\n`github / pull_request.opened` or `firewall / blocked_connection`. It is\ndefined by a match rule:\n\n- header equality (`X-GitHub-Event = pull_request`), and\n- payload conditions (`action = \"opened\"`, or a path being present).\n\nAn event type can be created two ways:\n\n- **From an example payload.** The operator pastes a payload (and optionally\n its headers), and archie converts it to the type's schema. No event needs to\n have arrived.\n- **From captures.** Archie proposes types from what has arrived, as below.\n\nEither way, archie infers the schema, not the operator:\n\n- Each capture gets a structural signature: the set of JSON paths with their\n value types, plus the headers that look like discriminators (`*-Event`,\n `*-Type`, `Content-Type`).\n- Captures from one source that share a signature are grouped. The inspector\n shows each group as a proposed event type, with its inferred schema, its\n number of events and a sample, until the operator names it or dismisses it.\n- The proposed match rule is the set of discriminators that separates the\n group from its siblings. The operator can edit it before saving.\n- A capture that matches no named type is shown as unidentified. It is never\n dispatched.\n\nTwo event types on one source whose rules can match the same event are\nrefused when the second one is saved.\n\n### Mappings\n\nA mapping belongs to an event type. It maps payload paths to named, typed\nparameters, and it is checked against the type's inferred schema, so a path\nthe type does not have is refused when the mapping is saved. Any number of\nbindings may reuse one mapping.\n\nEach mapping shows how many events it has matched, with the time of the most\nrecent, so an operator can see which mappings are live and which have never\nfired.\n\n### Workflows declare inputs\n\nA workflow definition declares its inputs:\n\n```yaml\nid: firewall-investigate\ninputs:\n src_ip: { type: string, required: true }\n severity: { type: string }\nrepository: none # none | optional | required\nprofile: network-investigator\nsteps: ...\n```\n\n- `repository: none` runs the agent in a scratch workspace with no clone.\n `required`, the default, gives the agent a worktree of the task's\n repository. `optional` gives it a worktree when the binding names a\n repository and a scratch workspace when it does not.\n- A workflow whose `repository` is not `required` may use only steps that need\n no repository. `agent.run` is one: it runs one agent mission and finishes\n the task as done, with no pull request.\n- The declared inputs reach the agent as structured data in the task brief\n and in the agent's context. They are not body text.\n- A workflow may declare `outputs`, the structured result it returns when it\n finishes.\n\n### Workflows call workflows\n\nA workflow step can call another workflow. The call passes the callee's\ninputs, and the callee runs as its own run with its own agent profile, so one\nevent can start an investigator that then calls a containment workflow:\n\n```yaml\nsteps:\n - type: workflow.call\n workflow: firewall-contain\n inputs: { src_ip: inputs.src_ip }\n wait: true # false starts the callee and continues\n```\n\n- With `wait: true` the caller receives the callee's outputs and fails if the\n callee fails. With `wait: false` the caller continues and the callee's\n outcome is reported on its own run.\n- A call's inputs are checked against the callee's declared inputs when the\n calling workflow is saved.\n- A workflow that calls itself, directly or through other workflows, is\n refused when it is saved. Calls are limited to a fixed depth at run time.\n- A callee must be enabled for the caller's org.\n\n### Bindings\n\nA binding connects an event type to one workflow:\n\n- an optional filter on mapped parameters (`severity in [\"high\", \"critical\"]`),\n written in the CEL the playbook engine already uses;\n- an assignment of workflow inputs from mapped parameters or constants,\n checked against the workflow's declared inputs when the binding is saved;\n- a repository, when the workflow's `repository` is not `none`: either fixed\n (`acme/api`) or taken from a parameter (the GitHub webhook's\n `repository.full_name`). A repository taken from a parameter must be a\n configured one, or the event does not dispatch.\n\nAny number of bindings may use one event type. Approval and re-approval after\nan edit are unchanged. A binding whose workflow input types no longer match is\nrefused at dispatch and reported on the binding.\n\n### Agent profiles\n\nA profile is a named execution environment that a workflow selects:\n\n- the container image;\n- an allowlist over the tools Archie adds to the agent: MCP servers,\n repository scripts and skill plugins. The agent's own file tools are always\n present;\n- the secrets injected into it, by reference to the secret store, never\n inline (superseded by `docs/prds/orgs-and-access.md`: secrets are granted to\n identities, and a profile holds no secrets);\n- its network access and forge access (superseded by\n `docs/prds/orgs-and-access.md`: permissions are granted to identities).\n\nProfiles are configured under `[containers.profiles.\u003cname\u003e]`. The global\n`[containers].image` becomes the default profile, so existing workflows are\nunchanged. A task whose workflow names an unconfigured profile is parked for\nan operator. A secret or permission is granted to an identity, never to a binding, so an\nevent can never widen what an agent can do (`docs/prds/orgs-and-access.md`).\n\n### Workflows belong to orgs\n\nArchie ships a default workflow catalogue that every org can use. An org can\nadd its own workflows, and can enable or disable any workflow, default or its\nown, for itself. A binding can target only a workflow enabled for its org, and\ndisabling a workflow stops its bindings dispatching and marks them on the\nbinding list.\n\n## Delivery order\n\n1. Event types: structural signatures, grouping, naming, match rules, and the\n inspector view of proposed and unidentified events.\n2. Mappings per event type, and multiple bindings per source.\n3. Workflow inputs, structured parameters in the task brief, and\n `repository: none | optional | required`.\n4. Agent profiles with image and tools.\n5. Signing as a source setting, with the unsigned option; UUIDv7 source paths\n with a custom-path override.\n6. Workflows calling workflows, with declared outputs.\n7. Org-scoped workflow catalogue, once orgs exist.\n\nEach step ships usable on its own. Step 3 is the first where an event that has\nnothing to do with a repository can run a workflow.\n\n## Out of scope\n\n- How forge-issue playbook triggers (`internal/domain/eda/playbook`) relate to\n event types. They stay as they are until event types exist.\n- Built-in catalogues of event types for known senders.\n- Orgs themselves: membership, access, and how sources and bindings belong to\n one. This design needs only an org to own workflows and their enabled state.\n- Running an agent outside a container.\n\n## Verification\n\n- Two GitHub webhooks with different `X-GitHub-Event` headers posted to one\n source are grouped into two proposed event types, and a firewall payload\n with a different shape is grouped separately.\n- A capture matching no named type is shown as unidentified and starts\n nothing.\n- A firewall event runs a `repository: none` workflow with its mapped inputs\n readable by the agent as structured data, and no clone is made.\n- A GitHub `pull_request.opened` event runs a PR review workflow against the\n repository named in the payload.\n- An event type created from a pasted payload has that payload's schema, and a\n later matching event is identified as it.\n- A new source gets a UUIDv7 path; a custom path that is already taken is\n refused.\n- A workflow that calls another with `wait: true` receives its outputs; one\n that calls itself through another workflow is refused when saved.\n- A mapping's matched-event count rises by one for each event it resolves.\n- A workflow disabled for an org cannot be targeted, and its existing bindings\n stop dispatching.\n- A binding whose filter excludes an event does not dispatch it.\n- An agent receives only the secrets its identity is granted.\n- An unsigned event on a signed source is captured and not dispatched. An\n event on an unsigned source is dispatched and marked unsigned.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/event-capture-storage/",
|
||
"title": "Event capture storage -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Event capture storage -- decision\n\n**Status:** Finalised\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-t2db.1`\n\n`webhook-intake-security.md` decided points 1-5 (auth split, approval gate,\nrate limits, blast radius, retention/redaction) and explicitly left the\nstorage mechanism open, forward-pointing at \"`7d5u.2`, embedded NATS.\" That\npointer predates `7d5u.3`'s 2026-08-22 narrowing, which assigned capture\nstorage to this bead by name and told it to decide NATS-vs-SQLite itself\nrather than inherit an answer. This document is that decision.\n\n## Decision: SQLite, not NATS JetStream\n\nCaptured events are stored in archied's existing SQLite database, in a new\n`captured_events` table following the same shape as `internal/store/events.go`\n(`internal/store`, `*Store`, SQL migrations inline in Go). No JetStream\nstream is introduced for capture.\n\n### Why\n\n**NATS availability does not make capture a broker concern.** NATS is now\nunconditional for autonomous workflow handoff, including embedded mode, but\ncapture is also used by deployments with no repository workflow configured.\nSQLite is guaranteed present because it backs daemon state, and keeps\ncapture inspection independent of broker retention and consumer lifecycle.\nThe original version of this decision cited optional NATS as an additional\nreason; that premise was superseded by `archie-core-q15y`, while the storage\nfit and independence arguments remain.\n\n**Capture's access pattern doesn't need what JetStream is for.** JetStream\nearns its complexity for reaction delivery (`7d5u.2`) because multiple\nindependent consumers need fan-out, at-least-once redelivery, and Ack/Nak\nsemantics -- exactly the trap `CLAUDE.md` documents about `WorkQueuePolicy`\nand overlapping consumers. Capture has exactly one writer (the HTTP handler)\nand one reader (a human via the dashboard inspector, `t2db.2`). There is no\ncompeting-consumer problem to solve and no redelivery semantics to get right,\nso JetStream's core value proposition doesn't apply here. A bounded SQL\ntable is a better fit on the merits, not merely as a fallback.\n\n**Reusing `7d5u.2`'s stream, once it exists, would still be wrong.** That\nstream is provisioned for fan-out reaction delivery with its own retention\npolicy decision. Capture's retention need -- age- and count-bounded, prune\non write, no replay -- is a different shape. Building a second stream just\nfor capture, gated on an unrelated bead landing first, would trade a small\namount of duplication now for a hard dependency on `7d5u.2`'s timeline that\nthis bead's own acceptance criteria say not to accrue by accident.\n\nThis does not foreclose NATS later. If cross-daemon capture sharing or\ndurable replay becomes a real requirement, that is a new decision made\nagainst a real need, not now against a speculative one.\n\n## Storage shape\n\n`captured_events` table (`internal/store/captures.go`, mirrors\n`internal/store/events.go`'s pattern):\n\n| column | type | notes |\n| ------------- | ------- | ------------------------------------------------------------------ |\n| id | INTEGER | primary key, autoincrement |\n| received_at | TEXT | RFC3339Nano |\n| source | TEXT | opaque path segment chosen by the sender's operator |\n| remote_addr | TEXT | `RemoteAddr` at capture time |\n| content_type | TEXT | `Content-Type` header |\n| headers | TEXT | JSON, redacted |\n| body | TEXT | JSON, redacted (`webhookguard.RedactPayload`) |\n| authenticated | INTEGER | 0/1; always 0 until a source registers a secret (future: `t2db.4`) |\n\nIndexed on `(source, id)` for per-source listing and pruning, and `id` alone\nfor the global cap.\n\n## Disk-bound mechanics\n\nThree independent bounds, composed, per `webhook-intake-security.md` point 5\nand this bead's own \"capture cannot exhaust disk\" criterion:\n\n1. **Per-request size cap.** The HTTP handler reads the body through\n `http.MaxBytesReader` (256 KiB default, configurable). A single oversized\n POST is rejected (413) before redaction or storage sees it.\n2. **Per-remote-address rate limit.** `webhookguard.RateLimiter`, keyed by\n the request's remote address, applied before the body is even read.\n _Not_ keyed by the `source` path segment: `RateLimiter`'s own doc comment\n states its bucket map assumes a bounded, operator-registered key space,\n never evicted. `source` is exactly the opposite here -- an unregistered,\n attacker-chosen URL segment, since capturing senders with no registration\n yet is this endpoint's entire purpose. Keying on it would let one sender\n bypass the limit by rotating the segment every request (a fresh key\n always starts with a full burst) and grow the bucket map without bound.\n Remote address is the bounded identity actually available pre-auth.\n Over-budget requests get 429, observable per point 3 of the security\n decision.\n3. **Retention prune-on-write.** Every insert prunes, in the same\n transaction: rows older than `retention_days` (default 7, per the\n security decision) OR beyond `max_events` total rows (default 5000, this\n bead's own addition -- age alone doesn't bound disk under a sustained\n flood inside the retention window; a count cap does). No background\n sweep goroutine; the daemon doesn't need one for this.\n\nTogether these give a deterministic worst-case: `max_events * max_body_bytes`\nregardless of sender behaviour, independent of whether the age-based or\ncount-based bound is what actually fires first.\n\n## HTTP surface\n\nMounted on the existing `internal/webui` dashboard server\n(`Server.Handler`'s `top` mux), not a new listener. `POST\n/webhooks/capture/{source}` bypasses `requireToken` the same way `/healthz`\nalready does -- capture must accept unauthenticated senders by design (that\nis the entire point: you cannot HMAC-verify a source you haven't configured\na secret for yet). Reachability is therefore whatever already exposes the\ndashboard in a given deployment; no new port/host config surface is added,\nand no new claim about public reachability is made beyond what already\nexists for the dashboard.\n\nAmended 2026-09-10 by the UI cutover (`archie-core-8cda.5.4`): the one\nprocess serving the dashboard is now the standalone archie-ui service, so\nthe receiver mounts there, composed from the UI process's own State Store\nclient. The binding-dispatch loop that turns captures into tasks stays in\nthe daemon, consuming the same store, so the HTTP front door moved without\nmoving work intake.\n\n`GET` listing/inspection endpoints (`t2db.2`) stay behind `requireToken` like\nevery other `/api/*` route -- captured payloads are visible only to an\nauthenticated operator, per the security decision's point 5 visibility rule.\n\n## Config\n\nNew `[capture]` section in `config.example.toml` / `internal/config`:\n\n```toml\n[capture]\nretention_days = 7\nmax_events = 5000\nmax_body_bytes = 262144\nrate_per_second = 1.0\nrate_burst = 5\n```\n\nAll fields have defaults; an empty `[capture]` section (or its absence) is\nvalid and produces the defaults above -- capture is on by default, since an\nunbound POST being captured rather than rejected is this bead's acceptance\ncriterion, not an opt-in.\n\n## What this does not decide\n\nBinding a captured event to a workflow, the draft/pending_approval/armed\nstate machine, and per-source HMAC secret registration are `t2db.3`/`t2db.4`.\nThe dashboard inspector UI is `t2db.2`. This bead ships capture and storage\nonly.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/event-sources-and-reactions/",
|
||
"title": "Event sources and reactions -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Event sources and reactions -- decision\n\n**Status:** Approved\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-7d5u.1`, blocks `7d5u.2/.3/.4/.5`\n\nAnswers the four questions in `7d5u.1`'s description. Investigated against\nthe actual tree, not designed from scratch: three mechanisms already in this\ncodebase answer most of this, and the job here is picking the right one for\neach question and saying which packages change.\n\n## 1. In-process typed families, not out-of-process dispatch\n\nArchie has no out-of-process extension mechanism. The plugin engine\nrule (`docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`,\nmechanically checked by\n`internal/plugin/architecture_test.go`) keeps every capability family\nin-process behind a narrow typed registrar; NATS RPC exists for\ndaemon\u003c-\u003eagentexec, a different actor with a different trust boundary (task\nexecution, not third-party extension code), not a precedent for generic\nplugin dispatch. Neither epic's near-term payoff -- react to a labelled\nissue, bind a captured webhook to a workflow -- needs extension code in\nanother language or process. Building out-of-process dispatch now would be\ndesigning for a requirement neither epic states.\n\n`docs/architecture/plugins-and-extensions.md` does **not** need amending.\nThe line that looked like a blanket ban on hooks (`archie-core-7d5u`'s notes\nrecord the correction) is scoped to Workflow semantics specifically, and\nnothing decided here touches Workflow's ownership of its own contracts.\n\n## 2. Reaction delivery: `internal/eventbus`'s contract, a NEW stream\n\n`internal/eventbus.Publisher`/`Consumer` already has everything a reaction\nneeds: `PublishUnique(subject, idempotencyKey, payload)` for at-least-once\nwithout duplicate enqueue, `Ack`/`Nak` with \"a handler returning an error\ncauses redelivery.\" Do not design new delivery semantics; use this contract.\n\n**Do not reuse the existing `nats.Client` as-is.** `Config.StreamName` and\n`Config.Subjects` are already composition-supplied (\"the bus must not know\nwhich subjects belong to which domain,\" `config.go`), but the\nretention policy is not parameterised: `New` hardcodes\n`Retention: jetstream.WorkQueuePolicy` (`client.go`), the competing-consumer\npolicy correct for `ARCHIE_TASKS` task distribution -- one message, claimed by\nexactly one consumer -- and wrong for reactions. Two independent reactions\nboth wanting to see the same event under `WorkQueuePolicy` hit exactly the\ntrap `CLAUDE.md` already documents: overlapping filter subjects on a\nwork-queue stream mean the second consumer silently gets nothing. Reactions\nneed fan-out, not claim-once, so the client needs retention made a `Config`\nfield (defaulting to `WorkQueuePolicy` so `ARCHIE_TASKS` is\nunaffected), and a second stream (`ARCHIE_REACTIONS` or similar) configured\nunder a fan-out-capable policy, alongside the existing task-distribution\nstream -- not instead of it.\n\nThis is where `7d5u.2`'s embedded-vs-external axis attaches: whichever NATS\ninstance backs `ARCHIE_TASKS` also backs the reaction stream. NATS is not\noptional as a technology -- the choice is only\nwhether it runs as an external server or embedded in-process\n(`github.com/nats-io/nats-server/v2` is already a direct, test-proven\ndependency, just never wired into production startup). `7d5u.2` wires that;\nthis document does not re-decide it.\n\n## 3. Reactions in these two epics are producer-only\n\nNo veto, no mutation of in-flight work. Two existing shapes cover everything\neither epic actually needs, and neither is a hook that can block something\nelse:\n\n- **Cheap wake, no payload.** `internal/domain/curator/wake.go`'s\n `WakeOnPrimaryInput` subscribes to the lossy `events.Bus` and nudges a\n runtime to check sooner; a dropped event only delays a check-in, never\n loses work, because the trigger decision is a deterministic state read at\n pass time, not the event payload. Model any \"check something sooner\"\n reaction on this, unchanged.\n- **Produce new work, real payload.** `daemon.go`'s existing `pollNATS` -\u003e\n `publishTask` path is already exactly a reaction that observes forge state\n and produces a `workintake.TaskEnvelope`. A webhook source is a second\n producer into the same path, not a new capability -- see question 4.\n\nA reaction that can veto or mutate an _in-flight_ task (block a PR from\nopening, alter a running stage) is a different, harder problem with its own\nordering and failure semantics, and belongs to `archie-core-h019` (adversarial\nself-review) if and when that epic needs one -- not here. Do not build a\ngeneral block-capable hook mechanism for either epic; nothing in their stated\nscope requires it, and it would duplicate whatever h019 eventually needs.\n\n## 4. No single generic intake interface; each domain keeps its own\n\nA `Source` interface generic enough to cover a forge issue, a chat message,\nand a cron tick would need an `any` payload -- the untyped-hook shape the\nplugin engine rule exists to prevent, just moved into a new package instead\nof the old one.\n\n**What's genuinely shared, and belongs in a small cross-cutting package**\n(imports nothing per the cross-cutting rule): the mechanics from\n`archie-core-t2db.5` that apply to any inbound HTTP source regardless of what\nconsumes it -- HMAC verification (already a standalone function,\n`internal/channels/webhook/webhook.go`'s `validateHMAC`), per-source rate\nlimiting, capture/audit, and the draft/pending_approval/armed gate. This\npackage owns none of forge, chat, or workintake's vocabulary.\n\n**What stays domain-owned, per existing convention:**\n\n- Forge-issue intake needs no new type. `internal/domain/workintake.TaskEnvelope`\n already is the typed contract -- `Subject()`, `IdempotencyKey()` (keyed on\n `owner/repo/number`, not on delivery source), `Encode()`. A webhook-sourced\n envelope for the same issue produces the _same_ idempotency key as a\n poll-discovered one, so `PublishUnique` dedups the two paths for free. This\n is `7d5u.5`'s answer, not a new mechanism: reuse `TaskEnvelope` and\n `publishTask`, don't invent a second envelope shape.\n- Chat-webhook intake stays exactly what it is:\n `internal/channels/webhook`'s `RouteConfig` routes into `gateway.Router`,\n documented as chat-scoped (\"External services push messages into\n archie-core's gateway for LLM processing\"). Do not generalise this package\n into a multi-domain receiver -- it already does one thing per \"a package\n owns its own format end to end.\"\n- A forge webhook receiver (`7d5u.4`) is new: it builds on the shared\n mechanics above, decodes a GitHub payload into the _same_\n `workintake.TaskEnvelope`, and calls the _same_ `publishTask` the poller\n calls. It does not get its own dispatch predicate -- `7d5u.4`'s bead\n already requires reusing the poller's label/assignee predicate rather than\n reimplementing it, and this decision extends that to the envelope and\n publish path too.\n\n## Packages this touches\n\n- New: a cross-cutting intake-mechanics package (HMAC, rate limit, capture,\n approval state machine) -- name and exact location decided in `t2db.1`/`t2db.5`.\n- `internal/infrastructure/eventbus/nats`: a second stream/consumer\n configuration for reactions, distinct retention policy from `ARCHIE_TASKS`.\n- `internal/domain/workintake`: unchanged in shape; `TaskEnvelope` is reused,\n not extended.\n- `internal/daemon`: gains a webhook-sourced call path into the existing\n `publishTask`, alongside the existing poll-sourced one.\n- `internal/channels/webhook`: unchanged. Not the forge receiver.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/execution-tree-state-machine/",
|
||
"title": "Execution tree and enforced state machine",
|
||
"section": "PRDs",
|
||
"body": "# Execution tree and enforced state machine\n\n**Status:** Approved\n**Authority:** `docs/architecture/agent-system.md` (canonical workflow model);\nsettles the \"enforced WorkflowExecution state machine\" item in\n`docs/architecture/migration-decisions.md` section 4.\n**Compounds with:** `docs/prds/task-run-detail.md` (attempt attribution),\n`docs/prds/state-store-contract.md` (wire contract and task-scoped grants).\n\n## Decision\n\nA WorkflowExecution becomes the root of a tree of StepExecutions. Every stage\nrun, and every agent call a stage makes, is its own durable record with a\nparent, an attempt, a status and an outcome. Both levels move only through a\ndeclared transition table, enforced by the State Store in the same transaction\nthat writes the audit row and the domain event.\n\n## Problem\n\n- A run is one `tasks` row. The only per-step record is the mutable `stage`\n column, overwritten as `workflow.Run` advances, plus observability events.\n A failed attempt cannot say which step failed or what its children did\n without replaying the event stream.\n- `Store.Transition` guards on the expected `from` status, but any `from` to\n any `to` is accepted. A `queued` to `merged` write succeeds.\n- Stop cancels one context in `daemon.runningTasks`. Nothing persisted says\n which work was in flight, so an agent call that outlives its stage (a\n reviewer, a container) is invisible to the store.\n- `workflow.Run` writes `tc.Store.Update` and discards the error, then emits\n `KindStageStart` separately. State and events can diverge.\n\n## Model\n\n```text\nWorkflowExecution (tasks row, id E)\n└── attempt N\n ├── StepExecution stage \"plan\" depth 0\n ├── StepExecution stage \"implement\" depth 0\n │ └── StepExecution agent \"implement\" depth 1\n └── StepExecution stage \"review\" depth 0\n ├── StepExecution agent \"review:1\" depth 1\n └── StepExecution agent \"review:2\" depth 1\n```\n\n- A StepExecution belongs to exactly one WorkflowExecution and one attempt.\n `parent_id` is null for a stage and names the enclosing StepExecution\n otherwise. Depth is derived from the parent, never supplied by the caller.\n- `kind` is `stage`, `agent` or `call`. A new kind needs a named consumer\n before it is added.\n- A `workflow.call` step (`docs/prds/event-automation.md`) is a `call`\n StepExecution in the caller's tree. The callee is its own\n WorkflowExecution, with its own tree, acting as its own identity, and the\n `call` step records the callee's execution ID. Trees never nest across\n executions, so each run's tree stays inside that run's authority.\n- A retry starts a new attempt. Earlier attempts' StepExecutions are never\n rewritten.\n\n## Transition tables\n\nWorkflowExecution statuses are the existing `taskstate` constants, persisted\nstrings unchanged. `taskstate` gains the one table every writer checks:\n\n| From | To |\n| --------------- | ---------------------------------------------------------- |\n| `queued` | `running`, `closed_wont_do` |\n| `running` | `queued` (crash recovery), `waiting_human`, `pr_open`, `completed`, `parked`, `closed_wont_do` |\n| `waiting_human` | `queued` (approve), `closed_wont_do` |\n| `parked` | `queued` (retry), `dead` (abandon), `closed_wont_do` |\n| `pr_open` | `merged`, `rejected`, `queued` (remediation), `closed_wont_do` |\n| terminal | none |\n\nStepExecution statuses:\n\n| From | To |\n| --------- | ------------------------------------------------------ |\n| `pending` | `running`, `cancelled` |\n| `running` | `succeeded`, `failed`, `cancelled`, `interrupted` |\n| terminal | none |\n\n`interrupted` means the process running the step died or shut down. It is\ndistinct from `cancelled`, which an operator or a parent asked for, and from\n`failed`, which the step reported.\n\nA StepExecution cannot reach `running` while its WorkflowExecution is not\n`running`, and cannot start under a terminal parent.\n\n## Enforcement\n\n- The table lives in `taskstate` (no dependencies), so the store, daemon and\n gateway share one copy.\n- Every store method that changes `tasks.status` checks the table and writes\n the audit and event rows, not only `Transition`. That includes the claim,\n `ParkTask`, `Requeue`, `RetryTask`, `BeginRemediation` and `RecoverStale`\n paths, several of which change status without an audit row.\n- The State Store rejects an illegal transition with a new sentinel,\n `ErrIllegalTransition`, carried across the wire through `mapError` like\n `ErrStaleTransition`. A stale `from` stays `ErrStaleTransition`.\n- Each transition is one transaction: guarded row update, audit row, domain\n event row. The events table is written by the store, not emitted beside it.\n `KindStageStart` and `KindStageFinish` are produced by the step transition\n and published to the bus after commit.\n- `workflow.Run` stops discarding persistence errors. A failed step write\n parks the execution; the step does not run unrecorded.\n\n## Cancellation\n\nStopping a run is one store call, `CancelExecution(E, reason)`. In one\ntransaction it moves every non-terminal StepExecution of the current attempt\nto `cancelled` and the WorkflowExecution to `closed_wont_do` or `parked`,\nwhichever the operator action names. The daemon then cancels the in-memory\ncontext. A worker whose step write returns `ErrStaleTransition` because its\nstep is already `cancelled` stops without writing further.\n\nCancelling a caller cancels each callee it is waiting on (`wait: true`),\nthrough `CancelExecution` on the callee under the callee's own lifecycle. A\ncallee started with `wait: false` runs on.\n\nThe store is the record of cancellation. The context cancel is the delivery\nmechanism, so a lost cancel still leaves the store correct and the worker's\nnext write fails.\n\n## Crash recovery\n\n`RecoverStale` moves every `running` StepExecution of an interrupted\nexecution to `interrupted` in the same transaction that requeues the\nexecution. The next attempt starts with fresh steps.\n\n## Wire contract\n\nNew State Store RPCs, each with a matching Go method on a narrow consumer\nfacade:\n\n- `StartStep(execution, attempt, parent, kind, name) -\u003e step id`\n- `FinishStep(step, from, to, detail, tokens_used)`\n- `CancelExecution(execution, reason, to)`\n- `ListSteps(execution, attempt)`\n\nThe run credential of `docs/prds/orgs-and-access.md` authorises `StartStep`\nand `FinishStep` on its own execution only, and the container records its own\nagent calls with it. `CancelExecution` and `ListSteps` are dashboard, API and\ndispatch actions, decided by the Authorizer: `update` and `read` on the run.\n\n## Organisations\n\n`docs/prds/orgs-and-access.md` is the boundary. A StepExecution belongs to its\nWorkflowExecution's org and workspace and carries their `org_id` and\n`workspace_id`, like every owned record. The State Store stamps both from the\ncredential on `StartStep` and ignores them in the request. `ListSteps` for an\nexecution in another org returns the store's ordinary not-found error.\n\n## Storage\n\nOne new table, `step_executions`: `id`, `org_id`, `workspace_id`,\n`execution_id`, `attempt`, `parent_id`, `depth`, `kind`, `name`,\n`called_execution_id`, `status`, `detail`, `tokens_used`, `started_at`,\n`finished_at`. Indexed on `(execution_id, attempt, id)`.\nDeleting an execution (archive, clearing terminal tasks) deletes its steps.\nExisting rows gain no backfilled steps. Their history stays in `transitions`\nand `events`.\n\n`tasks.stage` is written from the step transition while any reader still uses\nit, then dropped once the dashboard reads `ListSteps`.\n\n## Out of scope\n\n- Running sibling steps in parallel. The tree records concurrency; changing\n the stage runner is separate work.\n- Cost in currency. `tokens_used` per step is the only usage field.\n- Workflow definition versioning and the `tasks` to `workflow_executions`\n rename.\n- A harness contract for swappable coding agents.\n\n## Plan\n\n1. `taskstate` transition tables with table-driven tests covering every\n legal and a sample of illegal pairs.\n2. `ErrIllegalTransition` enforced in the Postgres store and mapped over the\n wire.\n3. `step_executions` migration, `StartStep`/`FinishStep`, grant rules;\n `workflow.Run` records stages; events written in the transition\n transaction.\n4. Agent-call children recorded from the agent runtime.\n5. `CancelExecution`; daemon stop and chat/dashboard actions route through it.\n6. `RecoverStale` interrupts running steps.\n7. Dashboard run detail reads `ListSteps`; `tasks.stage` dropped.\n\n## Verification\n\n- An illegal WorkflowExecution or StepExecution transition returns\n `ErrIllegalTransition` over gRPC, and `errors.Is` matches on the client.\n- Killing the daemon mid-stage, then restarting, leaves that stage\n `interrupted` and the execution `queued` with a new attempt.\n- Stopping a run during review leaves every review child `cancelled`, and a\n late `FinishStep` from the container returns `ErrStaleTransition`.\n- A run credential cannot `StartStep` on another execution, and a member of\n org A listing steps of an org B execution gets not-found.\n- Cancelling a caller cancels its `wait: true` callee and leaves a\n `wait: false` callee running.\n- For every step transition there is exactly one event row, written in the\n same transaction.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/extensions-center/",
|
||
"title": "Extensions centre",
|
||
"section": "PRDs",
|
||
"body": "# Extensions centre\n\n**Status:** Draft\n\n\u003e Superseded by `docs/prds/store.md`.\n\nEpic: `archie-core-1786637490708-35-38424ead` / GitHub `#56` (\"Replace Skills\nwith a bundled Extensions centre\").\n\nSkills, Workflows, and Tools/MCP already exist as separate, working\ninventories with their own APIs (`api_skills.go`, `api_workflows.go`,\nconfig-driven MCP servers) and their own family-owned controller pattern\n(`curator.Registry`, `health.Registry`, `image.Registry`,\n`workflow.Registry` all already prove this shape). `plugin.Plugin`\n(`internal/plugin/plugin.go`) is already metadata-only, per\n`docs/architecture/plugins-and-extensions.md`. None of that is rebuilt here.\nWhat's missing, confirmed by grep, is: any config schema for enabling or\ndisabling a capability, any \"pending restart\" state tracking, and any single\ninventory that reconciles configured/bundled/discovered/active state across\nfamilies. This document decides those three things and the UI surface that\npresents them. It extends `plugins-and-extensions.md` rather than replacing\nit — that doc's decisions (metadata-only `Plugin`, typed capability\nfamilies, no generic dispatch) are unchanged.\n\nOut of scope: any remote or arbitrary-URL plugin installation path (the\nepic's own acceptance criteria forbid this, and `plugin.LoadDir`\n(`plugin.go`) already only reads `~/.config/archie/plugins/*.go` — this\ndocument does not add a fetch-by-URL variant to it).\n\n## Problem\n\n`internal/config/config.go`'s `MCPServer` and `ToolsConfig` structs have no\n`enabled` field (confirmed by the investigation grep). `api_config.go`'s\n`UpdateConfig`/`handleConfigUpdate` seam persists changes but has no state\ndistinguishing \"saved, takes effect now\" from \"saved, needs restart\" — the\nonly existing reload path (`ReloadChannel`, `server.go`) is\nchannel-specific and explicitly does not cover plugins or MCP servers.\n`server.go` already wires five separate optional inventories\n(Workflows, Skills, Curators, Channels, Captures/Mappings/Bindings) as\nindependent fields with no unifying type — an operator has no single\nplace to see \"what's available, what's enabled, what's actually running,\nand what's degraded\" across all of them. `ui/src/` has `settings/`,\n`skills/`, and `workflows/` directories already; there is no\n`ui/src/extensions/`.\n\n## Design\n\n### Unified inventory type\n\n```go\npackage extensions // internal/domain/extensions — new, app-level aggregator\n\ntype State string\n\nconst (\n StateAvailable State = \"available\" // bundled, not configured\n StateEnabled State = \"enabled\" // configured on, not yet live\n StateActive State = \"active\" // configured on and running\n StateDegraded State = \"degraded\" // active but unhealthy\n StateSkipped State = \"skipped\" // optional, disabled by config\n StatePendingRestart State = \"pending_restart\" // config changed, not yet live\n)\n\ntype Entry struct {\n Family string // \"skill\" | \"workflow\" | \"tool\" | \"plugin\" | \"integration\"\n Name string\n Core bool // non-disableable\n State State\n Health string // reuses whatever the family's own registry already reports\n}\n\ntype Inventory interface {\n List(ctx context.Context) ([]Entry, error)\n}\n```\n\n**Decision.** `extensions.Inventory` is a read-only aggregator, not a new\ndomain that owns state. It queries each family's existing registry\n(`curator.Registry`, `health.Registry`, `workflow.Registry`, the MCP client\nset, `plugin.LoadDir`'s result) for live state, and compares that against\nthe config-desired state (below) to compute `StatePendingRestart` — the diff\nis computed on read, not persisted, so there is no new durable state to keep\nconsistent with the registries themselves. This follows\n`migration-decisions.md`'s rule that cross-cutting aggregation composes\nexisting domain-owned contracts rather than duplicating their state.\n\n**Call site.** New package `internal/domain/extensions/inventory.go`.\nComposed in `internal/app/archied` (wherever `server.go`'s other optional\nfields — Workflows, Skills, Curators — are wired) since it needs\nreferences to every family's registry, which only application-layer\ncomposition has.\n\n### Config-backed enablement\n\n**Decision.** Add `enabled bool` (default `true` for pre-existing configured\nentries, so no upgrade silently disables anything already configured) to\neach family's config element: `MCPServer.Enabled`, and equivalent fields on\nwhatever config types back skills/workflows/plugins if they don't already\nhave an on/off switch. Core capabilities (identity, workflow engine, State\nStore connectivity — named in `organisation.md`'s cross-cutting packages)\nhave no `enabled` field at all; a config value can't be added to something\nthat doesn't expose the toggle, which is the \"non-disableable\" property\nenforced structurally rather than by a runtime check that could be\nbypassed.\n\n**Call site.** `internal/config/config.go` — add the field per family\nstruct, not one generic `map[string]bool` (typed fields keep\n`docsgen`/`contracts.json` generation working, per `tools/docsgen`'s\n\"treat flags as executable truth\" rule — a generic map has no type to\ngenerate from). `config.example.toml` documents each new field with its\ndefault.\n\n**Reuse check.** The config `Holder` overlay system\n(`internal/infrastructure/configuration`) already handles safe saves; this\nadds fields to existing structs, it does not add a second config-write path.\n\n### Pending-restart and reconciliation\n\n**Decision.** No new \"pending restart\" flag is persisted. `Inventory.List`\ncomputes `StatePendingRestart` by comparing the config's `Enabled` value\nagainst whether the family's registry reports that entry as\nlive — if they disagree, the entry is pending restart. This is simpler than\ntracking a flag through a save-then-restart cycle and can never go stale,\nbecause it's recomputed on every read.\n\n**Call site.** `internal/domain/extensions/inventory.go`'s comparison logic.\n\n### API and UI surface\n\n**Decision.** New `api_extensions.go` alongside the existing `api_skills.go`\n/`api_workflows.go` pattern, exposing `Inventory.List` and a save endpoint\nthat writes through the existing `UpdateConfig` seam (`api_config.go`) —\nthis document does not add a second config-write endpoint. New\n`ui/src/extensions/` (colocated `.js`/`.css` per CLAUDE.md's frontend rule)\nas an index/dashboard that **links to**, not replaces, the existing\n`ui/src/skills/` and `ui/src/workflows/` detail views — Extensions centre is\nwhere an operator sees everything and toggles enablement; the family-specific\npages remain where they inspect a family's own detail.\n\n**Call site.** `internal/webui/api_extensions.go` (new, follows\n`api_skills.go`'s shape), `ui/src/extensions/` (new).\n\n## Call site inventory\n\n| concern | file | change |\n| ------------------------------------------------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------- |\n| Plugin metadata type | `internal/plugin/plugin.go` | none — already handles this |\n| Family controllers (curator/health/image/workflow registries) | `internal/domain/curator/registry.go`, `internal/domain/health/health.go`, etc. | none — pattern already proven, reused not rewritten |\n| Unified inventory | `internal/domain/extensions/inventory.go` | new |\n| Composition wiring | `internal/app/archied/server.go` (near existing optional fields) | new |\n| Config enablement fields | `internal/config/config.go` | new fields per family |\n| Config docs | `config.example.toml` | new field documentation |\n| API surface | `internal/webui/api_extensions.go` | new |\n| Existing skills/workflows APIs | `internal/webui/api_skills.go`, `api_workflows.go` | none — linked from, not replaced |\n| Config save path | `internal/webui/api_config.go` (`UpdateConfig`) | reused, not duplicated |\n| UI | `ui/src/extensions/` | new |\n| Existing UI detail views | `ui/src/skills/`, `ui/src/workflows/` | none — linked from Extensions centre |\n\n## Execution: multi-agent team breakdown\n\n| sub-feature | issue | implementer scope | suggested council lenses | why |\n| ------------------------------------ | ---------------------- | ---------------------------------------------------------------- | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |\n| Unified inventory type + composition | file as child of `#56` | `internal/domain/extensions/inventory.go`, wiring in `server.go` | `lens-boundary`, `lens-deletionist` | boundary: aggregator must not duplicate family-owned state; deletionist: must not become a new domain that shadows existing registries |\n| Config enablement fields | file as child of `#56` | `config.go` per-family `Enabled` fields, `config.example.toml` | `lens-contract` | config field is a wire contract other tooling (`docsgen`) generates from |\n| API + UI surface | file as child of `#56` | `api_extensions.go`, `ui/src/extensions/` | `lens-maintainer` | new user-facing state machine (available/enabled/active/degraded/skipped/pending-restart) must be legible to an operator with no prior context |\n\n## File and link the beads\n\n- `#56` itself has no filed children yet — file the three rows above as\n child issues linking to this doc's matching `##` heading.\n- Each issue's acceptance criteria should restate this doc's per-family\n `Enabled` default (`true` for existing config, no field at all for core\n capabilities) so \"non-disableable core\" isn't re-litigated per bead.\n- Add the suggested lenses to each issue.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/external-agent-harness/",
|
||
"title": "External agent harnesses",
|
||
"section": "PRDs",
|
||
"body": "# External agent harnesses\n\n**Status:** Approved\n**Authority:** `docs/architecture/agent-system.md` (agent execution as a data\nboundary); settles the harness non-goal deferred by\n`docs/prds/service-decomposition.md`.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (a harness\nrun is one `agent` StepExecution), `docs/prds/binding-secret-encryption.md`\n(the at-rest envelope reused for harness credentials).\n\n## Decision\n\nA workflow stage can run on an external coding-agent CLI instead of the\nbuilt-in agent loop: Claude Code, Codex, Pi, OMP, GitHub Copilot CLI, or any\nother CLI packaged as a Docker Sandbox Kit v3. Each authenticates with an API\nkey or the operator's subscription login, and can carry the operator's own\nskills, plugins, subagents and instructions.\n\nTwo boundaries hold regardless of what the harness does:\n\n- **The real credential never enters the container.** The container sees\n sentinel values; archie's egress proxy substitutes the real credential on\n outbound requests and performs OAuth refresh on the host.\n- **Archie's workflow guarantees are enforced outside the harness.** Git,\n protected paths, read-only stages, gates and budgets are checked by archie\n around the process and against the worktree afterwards.\n\n## Kits\n\nA harness is packaged as a Kit v3 image: an OCI image whose\n`vnd.docker.sandbox.kit.descriptor` manifest annotation declares what the\nharness needs. Archie reads descriptors with the published\n`github.com/docker/sandbox-kit-spec/v3/spec` package and does not maintain\nits own descriptor format. Published Kits run unmodified.\n\nArchie is a partial Kit runtime. It implements these capability types:\n\n| Type | Archie behaviour |\n| ------------------- | ----------------------------------------------------------------------------------------- |\n| `credential@1` | Proxy-managed only, described below. |\n| `network-policy@1` | Enforced by the egress proxy. |\n| `volume@1` | A Docker volume keyed by WorkflowExecution, deleted with it. |\n| `lifecycle@1` | Install and startup hooks run on container create. |\n| `agent-sessions@1` | The headless `prompt` and `resume` invocations for the stage. |\n| `agent-skills@1` | The operator's skills store (`skills_dir`), mounted read-only whatever the Kit asks. |\n| `agent-context@1` | The workload's profile file beside the workspace, indexing each Kit's guidance. |\n| `resources@1` | Container CPU and memory limits. |\n\nEvery other type fails closed: a required entry refuses the launch, an\noptional one is skipped and recorded on the step. `credential@1` with\n`passthrough: true` and any credential for a forge service are refused.\n\nThe operator's own setup (plugins, subagents, instructions, settings) is a\nmixin Kit, published with the harness Kit as a Kit set: the set merges their\nlayers into one image. It is versioned image content, rewritten on every\ncreate, never state a run can modify.\n\n## Contract\n\nA harness run is a second implementation of `agentexec.Runner`. It takes the\nsame `Request` and returns the same `Result`, so workflows, the daemon and\nthe State Store do not know which runner ran a stage.\n\n`agent-sessions@1` supplies the invocation. Archie supplies, per CLI, an\noutput adapter that translates the CLI's JSON output stream into\n`ToolCallReporter` calls, `Usage` and a session ID. A CLI with no adapter can\nrun stages with no capture tools, reporting no usage; its token budget falls\nback to wall clock.\n\nWorkflow definition validation rejects, at load time, a stage whose needs the\nselected harness cannot meet: capture tools without an adapter, or a gate\nretry without a `resume` verb.\n\n## Selection\n\nA harness is an agent profile. `docs/prds/event-automation.md` defines a\nprofile as the execution environment a workflow selects; a harness profile\nnames one Kit pinned by digest, a workload Kit or a published Kit set, in\nplace of a container image. Archie runs that Kit's image and composes no Kits\nitself; a bare mixin is refused. Everything else about profiles\nholds: a workflow names its profile, a stage may name a different one, and a\nprofile holds no secrets.\n\n`archie-agent` builds the stage's runner from the stage's profile: the\nai-sdk agent loop for an image profile, the harness runner for a Kit\nprofile. Image profiles, and every workflow already written against them,\nare unchanged. A Kit profile named by no workflow is a validation error.\n\nA stage naming a different profile than its workflow's is a separate run,\nnot a mid-container runner switch: a Kit task runs in one container\nprovisioned for one profile (Container layout), so a stage that needs a\ndifferent one is its own callee, started the way `workflow.call` already\nstarts one (`docs/prds/event-automation.md` \"Workflows call workflows\"),\nwith its own agent profile resolved off its own task row. This needs no\nmechanism beyond `workflow.call` plus per-task profile resolution\n(`daemon.pinTaskProfile`), both of which already treat every task's\nprofile independently of any caller's -- see\n`TestWorkflowCallerConformance` (staterpc) for the callee's independent\n`Workflow` field and `TestWorkflowCallGivesACalleeItsOwnProfile` (daemon)\nfor the Kit-vs-image runner selection that independence produces.\n\nProfiles are org resources in the State Store, so a Kit profile belongs to\none org, is granted to its workspaces like any profile, and applies without\na restart.\n\n## Organisations\n\n`docs/prds/orgs-and-access.md` is the boundary; a harness adds nothing that\ncrosses it.\n\n- A Kit profile, its credential bindings and its OAuth tokens belong to one\n org. No record is shared between orgs.\n- A harness run is a run like any other: it acts as the workflow's identity,\n under the run credential issued at dispatch.\n- The egress proxy injects a credential only when the run credential carries\n the secret the binding names. A Kit that declares a service the identity is\n not granted gets no credential: a required one refuses the launch, an\n optional one is skipped.\n- A Kit's network policy is what the Kit asks for; the identity's network\n grants are what the org allows. The proxy enforces the intersection.\n- Concurrency slots are counted per binding within its org.\n\n## Credentials\n\nA credential binding maps a Kit's `credential@1` `service` name to an org\nsecret, which the org grants to identities like any secret. The secret holds\none of:\n\n- **API key:** a value in the org's secret store.\n- **OAuth:** tokens captured by the setup terminal, stored in the State Store\n under the at-rest envelope of `binding-secret-encryption.md` with its own\n domain separator.\n\nThe egress proxy resolves the binding for each request against the run\ncredential. The container gets the Kit's sentinel in the named variable or\ncredential file. OAuth refresh happens at the proxy, which intercepts the\nKit's `tokenEndpoint`, so a refreshed token is written to the State Store and\nnever returned to the container.\n\nRuns sharing an OAuth binding share its subscription's rate limit.\n`max_concurrent` on the binding (default 1) caps its simultaneous runs. A\nstage waiting for a slot is `pending`, not `running`.\n\n## Setup terminal\n\nThe dashboard opens a terminal (xterm.js over a WebSocket to a PTY) into an\nephemeral container built from a Kit profile, on the same egress path. The\nmember runs the CLI's own login. The proxy captures the tokens at the token\nendpoint into the org secret the binding names. The container is removed when\nthe terminal closes. No worktree is mounted into it.\n\nOpening the terminal is the `update` action on the secret, decided by the\nAuthorizer like any other; the shipped roles give it to org owners and\nadmins. It is the only interactive path into a harness container.\n\n## Egress\n\nOnly Kit profiles run on the egress path. An image profile keeps the network\naccess its identity is granted, because workflows such as a network\ninvestigator need raw DNS, ICMP and arbitrary TCP that no HTTP proxy\ncarries.\n\n- **Sandbox network.** Each Kit run gets its own internal Docker bridge on\n which the host holds no address. Host services, including those listening\n on every interface, are unreachable from it by any address.\n- **Relay.** One relay container, `archie-agent relay`, sits on the sandbox\n networks under a fixed alias and on the host-reachable network. It\n forwards three fixed ports, to the proxy, NATS and the State Store, and can\n reach nothing else. It holds no secrets.\n- **Proxy.** The proxy runs inside `archied`, listening on the host gateway\n of the host-reachable network. It terminates TLS with a per-daemon CA\n installed into the container at create, identifies the container by a\n per-run proxy token, and applies that run's network policy and credential\n inject rules. Upstream dials refuse loopback, and refuse internal addresses\n unless the policy names the exact IP, so an allowed name cannot resolve\n back into the host.\n\n## Container layout\n\nA Kit task runs in one container, from the Kit composition's image, on the\ntask's sandbox network. The worker and every harness invocation share it;\nthey are separated by user, not by container, so a run's cost scales with\nthe run and not with how many agents it fans out to.\n\n- **Worker.** Archie mounts its own `archie-agent` binary read-only and runs\n it as the container's entrypoint, as root. It holds the task's NATS and\n State Store credentials in its own environment and in root-only files,\n runs the workflow, and serves `archie-agent mcp`. The Kit image's\n entrypoint is not PID 1.\n- **Harness.** Every invocation runs as the Kit's user (the image's `User`,\n default uid 1000) with an explicit environment: the Kit's variables, the\n proxy environment and the credential sentinels, and nothing inherited from\n the worker. That user cannot read the worker's environment or files, or\n trace it. Sibling invocations in one run share the user and the worktree.\n- **Hooks.** `lifecycle@1` install and startup hooks run as the user each\n declares, before the worker accepts work.\n- **Mounts.** The task worktree at the Kit's workspace directory, owned by the\n Kit's user, and the read-only skills store. Nothing else from the host.\n- **Network.** The relay forwards three fixed ports on the sandbox network:\n the egress proxy, NATS and the State Store. The worker reaches NATS and the\n State Store with its credentials; a harness can open those ports but holds\n no credential either accepts.\n- **Volumes.** `volume@1` paths persist for the life of the\n WorkflowExecution, which covers session resume across gate iterations and\n retries. They are not shared between executions.\n\n## Archie tools inside the harness\n\nCapture tools, which are how a stage returns structured results, are served\nby `archie-agent mcp`, a stdio MCP server registered through the Kit's MCP\nconfiguration or the adapter's flags. It serves the stage's `CaptureTools`\nand the central tools the agent profile's `AllowTools` permits, never the\n`workspace` toolset. The runner's `ToolLimits` bound its results; the\nharness's own built-in tools are outside archie's limits.\n\n## Enforcement around the harness\n\n| Property | Enforcement |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Model never runs git | No forge credential can be bound. Archie records `HEAD` and every ref before the run; any change fails the step (`failed`, policy class). The workflow commits the stage's changes. |\n| Protected paths | A diff touching a `Protection` path fails the step. |\n| Read-only stage | Any worktree diff fails the step. |\n| Gate | Archie runs the stage's gate after the harness exits. On failure it resumes the session with the gate output, up to the stage's iteration budget. |\n| Budget | Archie kills the process on wall-clock timeout, or when streamed usage passes the token budget. |\n| Cancellation | Cancelling the step kills the process group, then the container. |\n| Egress | Only the proxy is reachable; the Kit's network policy decides the rest. |\n\nHarness settings that deny git or protected paths are added where the CLI\nsupports them, to save wasted turns. They are never the enforcement.\n\n## Execution tree\n\nA harness run is one `agent` StepExecution. Subagents the harness spawns\ninternally are not recorded as children. Their usage is included in the\nstep's `tokens_used` when the adapter reports it.\n\n## Agents never hold a harness\n\nAn Agent reaches a harness only by invoking a Workflow whose stage names it.\nNo Agent tool, chat tool or capability exposes a harness, its setup terminal\nor its credential. The stage boundary is what carries the container, the git\nand path checks, the gate and the budget; a harness outside a stage has none\nof them.\n\n## Out of scope\n\n- Chat through a harness. The gateway chat loop stays on the built-in\n runtime.\n- `network-policy@2` HTTP method and path rules; a Kit requiring it is\n refused.\n- Recording the harness's internal subagents as StepExecutions.\n- Cost in currency.\n- Choosing a harness automatically per task.\n\n## Plan\n\n1. Descriptor resolution with the Kit spec package; capability support\n table and fail-closed refusal.\n2. Egress proxy: internal network, per-daemon CA, per-container token,\n `network-policy@1`.\n3. `credential@1`: API-key bindings, sentinel rendering, inject rules.\n4. Harness `Runner`: container create with worktree, volumes, lifecycle\n hooks and the mounted `archie-agent`; `agent-sessions@1` invocation;\n worktree checks for git, protected paths and read-only; budget and\n cancellation.\n5. Claude Code output adapter; `archie-agent mcp`; gate-and-resume loop.\n6. Kit profiles, runner selection from the stage's profile, and definition\n validation.\n7. OAuth bindings as org secrets: token-endpoint interception, encrypted\n storage, per-binding concurrency; dashboard setup terminal.\n8. Output adapters for Codex, Pi, OMP and GitHub Copilot CLI.\n\n## Verification\n\n- The same workflow definition completes on `builtin` and on the Claude Code\n Kit with the same `Result` shape and captures.\n- After `/login` in the setup terminal, the container's credential file holds\n only sentinels, the State Store holds the encrypted token, and a task run\n authenticates.\n- A real API key or OAuth token never appears in the container environment,\n filesystem, worktree, task log or event stream.\n- A request to a host outside the Kit's runtime allow list fails, including a\n direct connection that bypasses the proxy settings.\n- A harness instructed to `git commit` fails its step, and the task branch is\n unchanged.\n- A harness edit to a protected test file fails the step in a TDD fix stage.\n- A read-only feasibility stage that writes a file fails.\n- A file a run writes into a Kit-managed path such as the CLI's settings file\n is gone on the next execution.\n- A Kit requiring an unsupported capability is refused at launch with the\n capability named.\n- A stage with a failing gate resumes the same session with the gate output,\n then succeeds or exhausts its iterations.\n- Two tasks sharing an OAuth binding with `max_concurrent = 1` run one at a\n time; the second waits as `pending`.\n- Stopping a run kills the harness process and removes the container.\n- A harness invocation cannot read the worker's NATS or State Store\n credential from its environment, its files or `/proc`.\n- A run fanning out to many agent invocations uses one container.\n- A Kit profile in org A cannot be selected, and its binding cannot be\n resolved, by a workflow in org B.\n- A harness run whose identity is not granted a Kit's required credential is\n refused at launch; the proxy never injects a secret outside the run\n credential.\n- The firewall investigation example runs unchanged on an image profile with\n direct network access.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/identity-decision/",
|
||
"title": "Identity for users, agents and external agents — decision",
|
||
"section": "PRDs",
|
||
"body": "# Identity for users, agents and external agents — decision\n\n**Status:** Draft\n**Date:** 2026-09-22\n**Authority:** `docs/architecture/organisation.md` for placement;\n`docs/architecture/migration-decisions.md` §3 for the attribution semantics this\ndocument implements; `docs/prds/agent2agent-protocol.md` for the A2A surface that\nconsumes it.\n\n## Decision\n\nOne identity record covers every actor archie has: users, agents, external agents,\nservice accounts and the system. Authentication is delegated to the estate's OIDC\nprovider, and attribution stays archie's.\n\nAuthentication and authorisation are OSS archie integrates; a design that\nhand-rolls either fails this document.\n\n## The provider\n\n**Authelia is the only identity provider**, because the estate already runs it in\nfront of the dashboard and it issues credentials for both kinds of caller: humans\nuse the authorisation code flow, and machines use the `client_credentials` grant,\nwhich yields a token whose subject is the client's own identifier.\n\n**Each agent is its own provider client.** A shared client would make two agents\nindistinguishable in the audit trail and revocable only together, which is the\nproperty attribution needs and the reason per-client registration is a\nrequirement.\n\n**A person's client and an agent's client are registered separately**, because the\ntwo token shapes differ and a shared registration would collapse the distinction\nthe attribution rests on. A person's token names them in `sub`. A machine's token\ncarries no `sub` at all and names its client instead, so a machine identity is keyed\non its client registration and exists one per client. A token asserting neither is\nrefused.\n\n**A person's token requests the audience at the authorisation endpoint**, not only\nat the token endpoint. A provider that otherwise returns the audience empty\nproduces a valid token archie must refuse, and that refusal looks like a signing\nfault rather than a missing request parameter.\n\n**SPIFFE/SPIRE is not adopted.** It is adopted and maintained, and its attestation\nis strictly stronger than a secret: a workload proves what it is rather than what\nit holds. It is rejected here on operational fit, not on quality: it adds a\ncontrol plane archie does not have, and the property it buys — identity attested\nfrom the platform rather than from a presented secret — answers a threat archie\ndoes not face, because archie's agents run as containers the operator launches and\nalready receive their credentials from the daemon that owns them. It is the right\nanswer when agents run on infrastructure the operator does not control.\n\n## What archie owns\n\n- **The identity record.** `internal/domain/identity` survives as archie's own\n record of an actor and is not replaced by the provider's directory, because the\n record carries what the directory does not: the lifecycle, the bindings and the\n attribution below.\n- **The lifecycle.** Active, suspended and retired, with rename, suspend,\n reactivate and retire, remain archie's. Suspending an identity must stop it\n acting whether or not the provider still authenticates it.\n- **The subject binding.** An identity that proves itself is bound to the provider\n subject that asserts it. This is the same bridge `identity_aliases` already\n provides from a legacy name to an identity, so the second binding is the same\n mechanism rather than a new one.\n- **Attribution.** Who acted, and under whose authority.\n- **The audit trail of identity changes**, which already records an actor, a source\n and a request identifier for every mutation.\n\n## What archie never owns\n\nA credential, a password, a session, a token, a signing key, a second-factor\nenrolment, or a permission rule.\n\n## How an agent proves who it is\n\n**An agent authenticates by presenting a token the provider issued to its own\nclient**, and archie validates it against the provider's published signing keys and\ntakes its subject as the acting identity. The agent holds a client secret; archie\nholds only the public keys, so a compromised archie cannot mint an identity.\n\n**A WebMCP tool call is not a proof of agent identity and cannot be made one.** A\npage can invoke its own tools, so an agent's call and a page script's call arrive\nat archie identically. No field, header or convention distinguishes them, because\nthe difference does not exist on the wire.\n\n**A WebMCP tool call is therefore attributed to the identity the request\npresented, with its source recorded as a hint.** It may record that the request\ncame from the dashboard; it never records that an agent acted, and it never\nrecords that a human approved. A tool call that a page script can forge must not\nbe the thing an audit trail rests on.\n\n**Agent-initiated action therefore arrives on a surface that carries a credential**\n— the A2A surface, or an authenticated agent call — because that is the only place\nan agent's identity is verifiable. An approval an agent makes is recorded as an\nagent's action.\n\n## The two attributions\n\nEvery task action records two identities, and `taskactions` gains both:\n\n- **The acting identity**: the identity that performed the action, derived from\n the credential the request presented and never from a string the request\n supplied.\n- **The authorising principal**: the identity whose authority the action used.\n\n**The event kind describes the actor.** An action performed by an agent is\nrecorded as an agent's action even when a human's standing approval permitted it,\nso `KindHumanApproved` is recorded only when the acting identity is a human. An\nagent acting under a standing approval records itself as the actor and the human\nas the authorising principal.\n\n**An action with no authorising principal is recorded as unattributed rather than\nattributed to the actor's own authority.** An agent's own decision and an agent\nexecuting a human's delegation are different facts, and a record that collapsed\nthem would answer the audit question \"approved by whom\" with the wrong identity.\n\n**The review gate's approver is the authorising principal of the approval on that\ntask's action record.** A task parked for a person names that person, not the\nsurface they used.\n\n## External agents\n\n**External agents are first-class, not a special case.** A peer is an identity of\nkind `service_account`, bound to its own provider client, with the same lifecycle\nas any other identity.\n\nThe cost is a provider client per peer, a subject binding, and a client secret\nwhose rotation is the peer's responsibility — and retiring a peer is two\noperations rather than one, because its record and its provider client are retired\nseparately.\n\n**A peer's identity is attributed like any other.** The acting identity is the\npeer; the authorising principal is the identity that delegated to it, so an\nexternal agent acting for a user records both.\n\n## Authorisation is deferred\n\n**No authorisation engine is built, deployed or configured.** The credential's\nscopes are the only permission input, and they are verified by the same provider\nthat issued them, so the interim authorisation is OSS archie does not maintain.\n\n**Two things are not built now**: a policy language, and a decision point that\nanswers per-skill or per-task questions beyond a scope.\n\n**When a caller must be limited to a subset of what a scope permits** — A2A's\nper-skill model, where a peer may invoke some skills and not others — the decision\npoint is added behind a single archie interface that takes a subject, an action and\na resource. The engine behind it is an in-process library or a policy service, and\nthe choice waits for that requirement rather than being made now.\n\n## Verification\n\n- An agent obtains a token from the provider with its own client credentials and\n acts; its identity is recorded as the acting identity.\n- A call with no credential is refused, and a call with an expired or\n wrongly-signed token is refused.\n- A token issued to one agent is not accepted as another agent's identity.\n- A suspended identity cannot act, with a valid provider token.\n- A WebMCP tool call records the presented identity and its source, and never\n records an agent as the actor or a human as the approver.\n- An agent's approval records the agent as the acting identity and the human whose\n authority it used as the authorising principal, and does not record the human as\n the actor.\n- An approval with no authorising principal is recorded as unattributed.\n- No password, session, token, signing key or permission rule exists in archie's\n storage.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/image-capability-contract/",
|
||
"title": "Image capability contract and configuration",
|
||
"section": "PRDs",
|
||
"body": "# Image capability contract and configuration\n\n**Status:** Approved\n\nEpic: archie-core-1786748942120-1-6636e629 / GitHub #496 (\"Image generation\ncapability: hosted API baseline plus local GPU workflows\").\n\nThe contract and configuration (#497) shipped first and is settled — see\n\"Package: `internal/domain/image`\" below. This document now covers the full\nfeature set: hosted provider (#498), local GPU provider (#499), `/image`\nrouting and clarification (#500), delivery through channels and web (#501).\nDocumentation (#502) and the cross-cutting test matrix (#503) are execution\nwork against the design below, not open design questions, so they are not\ngiven their own sections — each preceding section states its own test\nobligations inline.\n\n## Problem\n\nArchie needs a provider-neutral boundary for image generation/editing so a\nhosted API (gpt-image-2-class) and a future local GPU backend can both\nimplement it without either one leaking into the domain contract. Nothing\nin the repo defines this (`grep -ri image` over `internal/domain` and\n`internal/config` returns nothing image-specific).\n\n## Design\n\nFollows the plugin engine family shape already established by\n`internal/domain/curator` and `internal/domain/memory`\n(`docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`): a\ntyped contract with real domain operations,\nan owning `Registry` (register/get/names, no start/stop lifecycle — see\nbelow), and narrow host access via a `Registrar`.\n\n### Package: `internal/domain/image`\n\n**Why no `Lifecycle` (`Start/Health/Stop`)**: curator and memory engines are\nlong-running — they hold a background cadence or a persistent connection\nworth health-checking. An image provider is call-scoped: it either serves a\n`Generate`/`Edit` request or it does not. Health here means \"is this\nprovider usable at call time\", which is exactly what `Generate`/`Edit`\nreturning a typed `ErrUnavailable` already communicates — a separate\n`Health()` poll would just duplicate that. `Registry` still owns\nregistration and lookup, so provider selection is testable with fakes per\nthe acceptance criteria, but there is no `Start`/`Stop` to orchestrate.\n\n```go\n// Capability declares what one provider supports, discoverable before a\n// request is built so a caller can reject an unsupported combination (e.g.\n// edit-with-mask on a generate-only provider) instead of sending it and\n// parsing a provider-specific error back out.\ntype Capability struct {\n Generate bool\n Edit bool // edit requires at least one input image\n Mask bool // edit supports a mask restricting the edited region\n MaxInputs int // 0 = Edit unsupported\n Sizes []string // provider-declared output sizes, e.g. \"1024x1024\"\n}\n\n// Provider is the typed contract every image backend implements — hosted\n// (gpt-image-2-class) or local (GPU recipe). No Lifecycle: a provider is\n// call-scoped, not long-running (see design doc rationale).\ntype Provider interface {\n Name() string\n // Class distinguishes hosted (spends money/quota per call) from local\n // (spends host GPU/time). Selection logic and config validation key off\n // this, never off Name, so a new provider cannot silently slip through\n // the hosted-cost gate under an unrecognized name.\n Class() ProviderClass\n Capability() Capability\n Generate(ctx context.Context, req GenerateRequest) (Result, error)\n Edit(ctx context.Context, req EditRequest) (Result, error)\n}\n\ntype ProviderClass string\n\nconst (\n ClassHosted ProviderClass = \"hosted\"\n ClassLocal ProviderClass = \"local\"\n)\n```\n\nRequest/result types (provider-neutral, no OpenAI types):\n\n```go\ntype GenerateRequest struct {\n Prompt string\n Size string // \"\" = provider default\n N int // number of images; 0 = 1\n Options map[string]string // provider-specific overflow (e.g. \"quality\"),\n // deliberately untyped so the contract does not\n // grow a field per provider\n}\n\ntype EditRequest struct {\n Prompt string\n Inputs []ImageData // 1..Capability.MaxInputs\n Mask *ImageData // nil unless Capability.Mask\n Size string\n N int\n Options map[string]string\n}\n\n// ImageData carries image bytes in-process, never a path or URL: the\n// domain layer must not assume a filesystem or a fetchable location (mirrors\n// why curator's Registrar never receives *daemon.Daemon — no ambient host\n// access smuggled in through a \"just a string\" field).\ntype ImageData struct {\n Bytes []byte\n MIMEType string\n}\n\ntype Result struct {\n Images []ImageData\n Provider string\n Model string\n CreatedAt time.Time\n}\n```\n\n### Error taxonomy\n\n```go\nvar (\n ErrUnavailable = errors.New(\"image provider unavailable\") // missing credential, backend down\n ErrUnsupported = errors.New(\"image capability unsupported by provider\") // e.g. Edit on generate-only\n ErrInputTooLarge = errors.New(\"image input exceeds configured limit\")\n ErrTimeout = errors.New(\"image request timed out\")\n ErrCanceled = errors.New(\"image request canceled\")\n)\n```\n\nEvery provider wraps these with `fmt.Errorf(\"%w: %s\", ErrX, detail)` so\ncallers can `errors.Is` regardless of backend. `ErrUnavailable` is the\nexplicit instruction from the epic: \"a missing credential or unavailable\nbackend must make the capability unavailable with a useful user-facing\nerror, not prevent Archie from starting or break text chat\" — Register\nnever rejects the daemon's boot, only marks that one provider absent from\n`Registry.Names()`.\n\n### Registry\n\n```go\ntype Registry struct { /* map[string]Provider, mutex, sorted Names() */ }\n\nfunc NewRegistry() *Registry\nfunc (r *Registry) Register(p Provider) error // ErrDuplicate on name clash\nfunc (r *Registry) Get(name string) (Provider, bool)\nfunc (r *Registry) Names() []string\nfunc (r *Registry) ByClass(class ProviderClass) []string\n```\n\nNo `Registrar`/host-injection type: unlike curator (needs `LLM`, `Skills`,\n`Conversations`) and memory (needs `Clock`, `Events`), an image provider's\ndependencies (HTTP client, API key, local backend handle) are entirely its\nown construction-time concern in `internal/infrastructure/image/*`, not\ndaemon services it needs injected. Introducing an empty `Registrar` now\nwould be exactly the \"field nobody sets\" pattern to avoid.\n\n### Configuration\n\nFollows `MinimaxConfig`'s hosted-cost-guard convention exactly: absent\nconfig or absent explicit `enabled = true` per provider means that\nprovider does not register, not \"registers but errors on first call\".\n\n```go\n// ImageConfig configures the image generation capability.\ntype ImageConfig struct {\n // Default selects which registered provider name /image uses when the\n // request/session has not already picked one. Empty means /image must\n // always ask (the epic's stated default UX) — an empty Default is valid\n // and is not a misconfiguration.\n Default string `toml:\"default\"`\n Hosted map[string]ImageHostedProvider `toml:\"hosted\"` // keyed by provider name\n Local map[string]ImageLocalProvider `toml:\"local\"` // keyed by recipe name\n}\n\n// ImageHostedProvider configures one hosted (paid) provider. Mirrors\n// MinimaxConfig: Enabled defaults false, spending real credits requires an\n// explicit opt-in, never an inferred one from the presence of a key alone.\ntype ImageHostedProvider struct {\n Enabled bool `toml:\"enabled\"`\n Class string `toml:\"class\"` // \"openai\" today; provider-neutral field name\n APIKeyEnv string `toml:\"api_key_env\"`\n APIKey secret.SecretRef `toml:\"api_key\" json:\"-\"`\n BaseURL string `toml:\"base_url\"`\n}\n\n// ImageLocalProvider configures one local GPU recipe. Enabled defaults\n// false too: a local backend can still saturate the host GPU/CPU, which is\n// its own kind of unwanted cost, so it gets the same explicit gate as a\n// hosted provider rather than an on-by-default WebFetch-style treatment.\ntype ImageLocalProvider struct {\n Enabled bool `toml:\"enabled\"`\n Backend string `toml:\"backend\"` // recipe identifier, defined by the local-backend bead\n}\n```\n\nValidation (`internal/infrastructure/configuration/validate.go`, alongside\n`validateMemory`):\n\n- `Default`, if set, must name a provider present (enabled) in `Hosted` or\n `Local` — rejects a config that silently falls back to \"always ask\" when\n the operator thought they'd pinned one.\n- A `Hosted` entry with `Enabled = true` must have `Class` non-empty and\n either `APIKeyEnv` or a resolvable `APIKey` — the same shape check\n `Provider` (chat) already gets, not a new invention.\n- No cross-field surprise: an _absent_ `[image]` section is valid and\n produces zero registered providers, matching the epic's non-goal\n (\"silently selecting a paid hosted provider\" must never happen by\n omission either).\n\n### Capability discovery\n\n`Registry.Names()` / `ByClass()` plus `Provider.Capability()` give a caller\n(the future `/image` routing bead) everything needed to ask \"what can I\noffer this user\" without a type switch on provider identity — this is the\nwhole point of the contract being provider-neutral.\n\n### Timeouts, size limits, cancellation\n\n- Every `Provider.Generate`/`Edit` call takes the caller's `ctx`; a\n provider that ignores cancellation is a provider bug, not a contract gap\n — mirrors every other Lifecycle-less call in the codebase (chat's\n `runtime.Chat`).\n- `ImageData` size limits are enforced by the _caller_ (the future routing\n bead) before constructing a request, using a configured max — not by this\n contract, which has no config surface for \"current caller's limit\". This\n package only defines `ErrInputTooLarge` so a provider that discovers an\n oversized input itself (rather than trusting the caller) has a typed way\n to say so.\n\n## Testing\n\n- `internal/domain/image/{contract,registry}_test.go` mirror memory's test\n suite shape: registration (duplicate, nil provider), `Names`/`ByClass`\n filtering, `Get` miss, table-driven `Capability` checks via a fake\n provider.\n- `internal/infrastructure/configuration/validate_test.go` gains cases for\n `ImageConfig`: absent section, valid hosted+local, missing `Class`,\n missing credential, `Default` naming an unregistered/disabled provider.\n- No provider implementation exists yet, so no live-call tests — those\n belong to the hosted/local provider beads.\n\n## Hosted provider (#498)\n\n`github.com/samcharles93/ai-sdk` (already a repo dependency) has its own\nprovider-agnostic image facility: `image.Provider` (`Name`,\n`GenerateImage(ctx, GenerateImageRequest) (GenerateImageResponse, error)`),\n`image.Client`, and `registry.Registry.RegisterImage`/`.Image(name)` — the\nsame shape chat model registration already uses. Three backends implement it\n(azure, togetherai, xai); OpenAI does not.\n\n`internal/infrastructure/image.\u003cname\u003eProvider` wraps an `ai-sdk/image.Provider`\nbehind this package's `Provider` interface, the same way `curatorLLMRunner`\n(`internal/app/archied/main.go`) adapts `runtime.Runtime` for chat rather\nthan the domain layer talking to ai-sdk directly.\n\n**Decision: generation goes through ai-sdk, edit does not.** ai-sdk's\n`image.GenerateImageRequest` is text-to-image only — no input-image or mask\nfields — so there is nothing to adapt `EditRequest`/`Capability.Edit` onto.\nBlocking the entire hosted provider on an upstream ai-sdk change is not\nthis epic's call to make on someone else's project's timeline. Instead:\n\n- `Generate` wraps `ai-sdk/image.Client.GenerateImage` against whichever\n backend `ImageHostedProvider.Class` names (azure/togetherai/xai).\n- `Edit` is a direct HTTP client hitting the OpenAI-compatible\n `/images/edits` multipart endpoint, built the same way `minimax.Client`\n (`internal/tools/minimax/minimax.go`) is a direct client for an API\n ai-sdk doesn't cover — not a second abstraction layer, one `http.Client`\n call with request/response mapping into this contract's `EditRequest`/\n `Result`.\n- A provider that only wraps ai-sdk (no edit client configured) declares\n `Capability{Generate: true, Edit: false}` truthfully rather than\n returning `ErrUnsupported` at call time for something `Capability`\n already said it couldn't do.\n- If ai-sdk gains edit support upstream later, the direct client is deleted\n and `Edit` moves onto it — recorded here so that's a known, not a\n surprise, follow-up.\n\n**Provider error mapping** (every ai-sdk/HTTP error translated at the\nadapter boundary, never leaked as-is):\n\n| ai-sdk / HTTP condition | contract error |\n| ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| `image.ErrNoProvider`, `image.ErrAuthFailed`, missing/invalid credential, backend unreachable | `ErrUnavailable` |\n| `image.ErrEditNotSupported`, request needs a `Capability` the provider didn't declare | `ErrUnsupported` |\n| input image over configured byte limit (checked before the call, per \"Timeouts, size limits, cancellation\" above) | `ErrInputTooLarge` |\n| `ctx.Err() == context.DeadlineExceeded` | `ErrTimeout` |\n| `ctx.Err() == context.Canceled` | `ErrCanceled` |\n| `image.ErrRateLimited`, `image.ErrContentFiltered`, any other 4xx/5xx | wrapped verbatim under `ErrUnavailable` with the provider's own message as detail — a rate limit or content filter is \"unavailable for this request\", not a new sentinel the caller needs to branch on differently |\n\n**Testing**: `httptest.Server` fixtures for the edit client (request\nmultipart shape, response decoding, 4xx/5xx mapping); a fake `ai-sdk/image.Provider`\nfor the generate path so the adapter's translation is tested without a live\nbackend, matching #498's \"no live API key needed for tests\" criterion.\n\n## Local GPU provider (#499)\n\n**Decision: ComfyUI is the baseline recipe**, not a name this doc picked\narbitrarily — it is the de facto scriptable HTTP API for Stable-Diffusion-class\nlocal generation, already shaped like the async submit/poll pattern\n`minimax.Client.GenerateAndWait` established in this codebase (submit a\nworkflow, poll `/history/{id}` until the output node has files, download).\nNo other local backend is referenced anywhere in this repo or its docs, so\nthis decision has no competing prior art to reconcile — it is a fresh\nchoice, recorded here so the implementing bead does not re-litigate it.\n\n```go\n// internal/infrastructure/image/comfyui\ntype Config struct {\n Enabled bool\n BaseURL string // e.g. http://127.0.0.1:8188\n Workflow string // path to a saved ComfyUI workflow JSON template\n // (prompt/size/seed substituted at call time)\n Timeout time.Duration\n PollInterval time.Duration\n MaxWait time.Duration\n}\n```\n\n- `Generate`/`Edit` load `Workflow`, substitute the templated fields (prompt,\n size, seed, and for `Edit` the input image as a base64 node input — ComfyUI\n supports img2img/inpaint workflows natively, so `Capability.Edit`/`Mask`\n can both be true for a correctly authored workflow, unlike the hosted\n provider), POST to `/prompt`, then poll `/history/{prompt_id}` the same\n shape MiniMax's poll loop already proves out in this codebase.\n- `Capability.Sizes` is empty (ComfyUI workflows declare their own\n resolution in the template, not a fixed enum) — the contract already\n allows an empty `Sizes` to mean \"provider does not constrain size\".\n- Backend-down (`BaseURL` unreachable) and workflow-missing are both\n `ErrUnavailable`, not a startup failure — same \"degrade the capability,\n never break text chat\" rule the contract already states.\n- Per #499's own acceptance criteria, at least one recipe must be exercised\n end-to-end on real hardware or a reproducible harness, and no model\n weights are ever committed or fetched by Archie itself — the operator\n points `Workflow`/`BaseURL` at their own running ComfyUI instance and\n already-downloaded checkpoint, exactly like `[providers.ollama]` points at\n an already-running Ollama rather than Archie managing the model.\n\n**Testing**: a fake HTTP server standing in for ComfyUI's `/prompt` and\n`/history` endpoints (submit, poll, malformed history, timeout, job\nnot-found); a real-hardware smoke test documented as a manual `task`\ntarget per #503, not part of `task check` — the same reasoning `task vuln`\nand `task deadcode` are excluded from the gate.\n\n## `/image` routing and clarification flow (#500)\n\n**Call site: `internal/gateway/gateway.go`'s `Router`**, following the\n`/model` shape exactly — `/model`, `/spawn`, `/approve` etc. are already\n\"commands handled without the LLM\" (`Router.handleModel`, dispatched via\n`isLocalCommand`/`parseCmd` against `messaging.LocalCommandSpecs()`).\n`/image` is a new entry in `internal/domain/messaging/commands.go`'s\n`localCommandSpecs`, and a new `Router.handleImage(ctx, arg)` beside\n`handleModel`, gated on a new `Router.Images *image.Registry` field (nil =\n\"/image not configured\", the same pattern `Models`/`Tasks`/`Controller`\nalready use — not a new gating convention).\n\n**Decision: mechanism selection is per-request, not session-persisted.**\nThe epic allows \"a remembered preference only where existing session/config\nconventions support it\" — and checking `Router`'s actual fields\n(`ModelPersist`, `Identity`, the rest) shows there is no generic per-session\nkey/value store the neutral command path can lean on; `/model`'s own\n`--session`/`--global` scoping lives entirely in\n`internal/channels/telegram/model.go`, a Telegram-specific enrichment, not\nsomething `Router` provides generically. Inventing a new generic\nsession-preference store just for `/image` would be exactly the kind of\nscaffolding CLAUDE.md's Architectural Simplicity rule warns against for one\ncaller. So: every `/image` invocation with no explicit mechanism produces\nthe clarification prompt; an explicit `--hosted`/`--local` flag (mirroring\n`parseModelRest`'s `--provider` flag shape) skips it for that one call.\nRevisit if a second command wants session-scoped preference too — that's\nthe point at which shared infra earns its cost.\n\n**Flow**, matching `handleModel`'s reply-string shape (no separate state\nmachine object, since the whole exchange is \"ask once, use the answer\ninline in the next message\" — the model-selector's own worked example):\n\n1. `/image \u003cprompt\u003e` with no `--hosted`/`--local` → reply lists the\n registered classes (from `Registry.ByClass`) and asks the user to repeat\n the command with `--hosted` or `--local`. No provider is called.\n2. `/image --hosted \u003cprompt\u003e` or `--local` → resolves to\n `Registry.ByClass(class)`, picks `ImageConfig.Default` if it names a\n provider of that class, else the sole registered provider of that class,\n else (more than one, no default) asks which named provider.\n3. Edit requests (`/image --edit \u003cprompt\u003e` with an attached image on the\n inbound message) validate against `Capability.Edit`/`MaxInputs`/`Mask`\n before calling — an edit request to a generate-only provider is a\n clarification reply (\"this provider doesn't support editing\"), not a\n provider call that returns `ErrUnsupported`.\n4. A successful call returns a `Result`; delivery is the next section's\n concern, not this one's — `handleImage` returns the same\n `(string, error)` shape `handleModel` does for the text portion, and a\n separate mechanism (below) carries the actual image bytes.\n\n**Testing**: channel-neutral state-machine tests for `handleImage` against\na fake `*image.Registry` (mirrors `#500`'s own acceptance criterion); Telegram\nadapter tests only for wiring, not for re-testing the state machine.\n\n## Delivery through channels and web (#501)\n\nThis is the smallest section because most of the delivery path **already\nexists and is already proven** — by `generate_video`\n(`internal/tools/minimax`), which ships. Do not rebuild it.\n\nThe existing pipeline: a result carrying media returns\n`tools.MultimodalResult{IsMultimodal: true, URLs: []tools.MediaRef{{Type: \"image\", ...}}}`\n→ `multimodalMediaRefs` (`internal/app/archied/telegram_setup.go`) decodes\nit → `TurnStream.Media(gateway.MediaEvent)` → each channel's own\n`Media`/`MediaAttachment` handling. `MediaRef.Type` already documents\n`\"image\"` as a valid value alongside `\"video\"`; nothing in the pipeline is\nvideo-specific.\n\n**Per-channel state, checked directly rather than assumed:**\n\n| channel | image delivery | evidence |\n| ----------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Telegram | **already works** | `internal/channels/telegram/media.go`'s `telegramMediaSender.dispatch` already branches on `att.Type` and calls `bot.SendPhoto` for `\"image\"` (`SendVideo` for `\"video\"` is the same function) |\n| Dashboard (webui) | **tracked gap, not new** | `internal/webui/api_chat.go`'s `chatStreamSink.Media` has no inline rendering path yet and degrades to a link/error string — already tracked as archie-core-1786748942243-6-f109697e. This epic does not need to fix it, but #501's acceptance criterion (\"unsupported paths report a clear limitation\") is already met by the existing degrade-to-text behaviour; closing the dashboard gap is that bead's job, not this one's |\n| Email | **no attachment sender exists** | `grep MediaAttachment internal/channels/email` returns nothing. Out of scope for this epic per #496's non-goals (no channel-abstraction rework) — email delivery of a generated image degrades the same way the dashboard does which is acceptable under #501's \"or unsupported paths report a clear limitation\" clause |\n| Webhook | not applicable | webhook is an inbound intake channel (`internal/channels/webhook`), not an outbound chat reply target — no `/image` command can originate there in the first place |\n\n**The only new work `internal/tools/minimax` didn't already need**: the\nautonomous-tool call site. `/image` (previous section) calls a provider\ndirectly and does not go through the model's tool-calling loop at all.\nSeparately, exposing `generate_image`/`edit_image` as an ordinary tool\n(so an agent task can generate a diagram or illustration mid-task, not only\na human typing `/image`) is a thin wrapper — `internal/tools/image/tool.go`,\nsame shape as `internal/tools/minimax/tool.go`, calling\n`image.Registry.Get(name).Generate/Edit` and returning the identical\n`MultimodalResult` shape. Wired in `bootstrap.go` next to\n`registerMinimaxTool`, gated the same way (registered only when a provider\nresolved a working credential/backend, never advertised broken). This\nreuses the entire delivery pipeline above with zero new plumbing — the tool\ncall site is what's new, not the delivery.\n\n**Testing**: `multimodalMediaRefs` already has coverage for the shape;\nadd a table case for `Type: \"image\"` if one doesn't already exist. The tool\nwrapper gets the same fake-provider test shape as the routing flow above.\nNo new delivery-path tests are needed — that pipeline's tests belong to\nwhichever change originally proved it out.\n\n## Call site inventory\n\nEvery file a full implementation touches, so no bead re-derives this by\ngrep:\n\n| concern | file | change |\n| -------------------------- | ------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- |\n| domain contract | `internal/domain/image/{contract,registry}.go` | done (#497) |\n| config schema + validation | `internal/infrastructure/configuration/{types,validate}.go` | done (#497); extended here only if ComfyUI's `Workflow` field needs a new validation rule (path exists, non-empty when `Local.Enabled`) |\n| hosted provider | `internal/infrastructure/image/openai/*.go` (or per `Class`) | new (#498) |\n| local provider | `internal/infrastructure/image/comfyui/*.go` | new (#499) |\n| `/image` command spec | `internal/domain/messaging/commands.go` | new entry in `localCommandSpecs` |\n| `/image` routing | `internal/gateway/gateway.go` (`Router.Images`, `handleImage`) | new (#500) |\n| autonomous tool | `internal/tools/image/tool.go` | new (#501) |\n| tool registration | `internal/app/archied/bootstrap.go` (`registerImageTool`, beside `registerMinimaxTool`) | new (#501) |\n| `Registry` wiring | `internal/app/archied/bootstrap.go` (construct `image.Registry`, register configured providers, hand to `Router.Images`) | new — this is the \"nothing to wire until a real provider exists\" step the original contract doc deferred |\n| Telegram delivery | `internal/channels/telegram/media.go` | none — already handles `\"image\"` |\n| Dashboard delivery | `internal/webui/api_chat.go` | none in this epic — tracked separately (archie-core-1786748942243-6-f109697e) |\n| docs | `docs/` user-facing guide (#502) | new, against this design |\n\n## Execution: multi-agent team breakdown\n\nEach row is independently implementable from its own section above plus the\ncall-site inventory — an implementer does not need this whole document\nre-explained, only its row. Lenses are picked from\n`.claude/workflows/council.js` (`boundary`, `contract`, `deletionist`,\n`operator`, `maintainer`) by what each slice actually risks getting wrong,\nnot all five by default.\n\n| sub-feature | issue | implementer scope (call-site rows) | suggested lenses | why |\n| -------------------------- | ----- | ------------------------------------------------------------------------------------------------------- | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Hosted provider | #498 | `internal/infrastructure/image/openai/*.go` (generate via ai-sdk, edit via direct HTTP client) | `lens-contract`, `lens-operator` | contract: the error-mapping table must actually produce the sentinels it claims, not leak ai-sdk/HTTP errors raw; operator: a live paid API's rate-limit/timeout/auth-failure paths are exactly the 3am-unattended-host case this lens exists for |\n| Local GPU provider | #499 | `internal/infrastructure/image/comfyui/*.go` (submit/poll against ComfyUI) | `lens-operator`, `lens-deletionist` | operator: backend-down, workflow-missing, and stuck-job failure modes on a host the operator's GPU actually runs; deletionist: a second, unevidenced \"recipe\" added later without real-hardware proof is exactly the speculative scaffolding this lens rejects |\n| `/image` routing | #500 | `internal/domain/messaging/commands.go`, `internal/gateway/gateway.go` (`Router.Images`, `handleImage`) | `lens-boundary`, `lens-maintainer` | boundary: a new `Router` field and its dependency on `image.Registry` must respect the same layering `Models`/`Tasks`/`Controller` already do; maintainer: this is the first local command with an explicit ask-once clarification exchange, not a single-shot reply — legibility for the next person who adds a command matters here |\n| Delivery + autonomous tool | #501 | `internal/tools/image/tool.go`, `internal/app/archied/bootstrap.go` (`registerImageTool`) | `lens-deletionist`, `lens-contract` | deletionist: the whole point of this section is \"reuse `generate_video`'s pipeline, do not rebuild it\" — this lens is the check that the implementation actually did that; contract: the new tool's `MultimodalResult`/`MediaRef{Type: \"image\"}` output must match the shape `multimodalMediaRefs` already decodes, not a near-miss |\n\nRun `/council --lenses \u003cpicked\u003e` once a row's implementation is gate-clean\n(`task check` passing), before opening its PR — same timing as any other\nopen-decision residue at the end of a session, not a standing per-commit\ngate.\n\n## File and link the beads\n\n`#498`–`#501` already exist (bd `archie-core-1786748942169-3-ae7cfaea`,\n`…-219-5-9844a72e`, `…-193-4-5072b564`, `…-243-6-f109697e`), filed under\nepic #496 before this document's decisions were made. Each one's\ndescription now links back to its matching section above instead of\nrestating it, and carries its row from the Execution table above so picking\nit up needs no round-trip back to this document.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/inline-review/",
|
||
"title": "Inline review -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Inline review -- decision\n\n**Status:** Finalised\n**Beads issue:** `archie-core-q9au`\n**Builds on:** `archie-core-h019` (adversarial self-review), `docs/architecture/adversarial-review.md`\n\n## Problem\n\nFindings reach the operator only as a flat list in the PR body\n(`renderPRReviewSection`, h019.6): ``- `file.go` (confirmed, error): \u003cdefect\u003e\n— \u003cscenario\u003e``. That is a report _about_ the code, not a review _of_ it. A real\nreviewer anchors each finding to the line it is about, so the author sees it in\ncontext — and, when the fix is mechanical, attaches the replacement so it can be\napplied in one click.\n\nBoth halves are missing:\n\n- **No line-anchored comment.** `forge.Forge.Comment` posts to the PR\n _conversation_ (`Issues.CreateComment`); nothing creates a review comment at\n `path` + `line`.\n- **No suggested fix.** `ReviewFinding` carries `File`/`Line`/`Defect`/\n `FailureScenario`/`Verdict`/`Level`/`Category` — no replacement code, so even\n an anchored comment could only describe the fix, not offer it.\n\n## Decisions\n\n### 1. A narrow, type-asserted write surface\n\nThe set of comments travels in one call, so a forge that can only carry them on\na submitted review needs one request rather than one per finding:\n\n```go\ntype ReviewCommentWriter interface {\n CreateReviewComments(ctx context.Context, owner, repo string, number int,\n comments []InlineReviewComment) error\n}\n```\n\nType-asserted like `PullRequestReader`/`PullRequestReviewReader`, so a forge\nthat cannot do it is skipped rather than faked. The stage reaches it through\n`workflow.Forger`, whose production implementation is `forgerpc.Client` — the\nworker holds no forge credentials, so the daemon's forgerpc server type-asserts\n`forge.ReviewCommentWriter` and answers an incapable forge with an error the\nbest-effort stage logs.\n\nThe comment's line is anchored to the pull request's **current head revision**,\nresolved inside each implementation rather than passed in by the caller: it is\nthe revision the comments are attached to, and the caller that produced the line\nnumbers holds no forge credentials to read it. The caller does pass the revision\nthose line numbers were **measured on** (`TaskContext.ReviewedHeadSHA`,\nrecorded when `StageOpenPR` opens the pull request), and the implementation\nposts only while the head it read still matches it — see §4.\n\nPer-forge reality, verified against the SDKs:\n\n- **GitHub** — direct: `PullRequests.CreateComment` with\n `{Body, Path, Line, Side: \"RIGHT\", CommitID}`.\n- **Gitea** — no direct inline-comment call. Inline comments ride on a submitted\n review (`CreatePullReview` with `Comments` and a `COMMENT` state). The Gitea\n implementation therefore submits a single `COMMENT`-state review carrying the\n findings, or the capability is simply absent (the type assertion skips it).\n Decided: implement Gitea as a `COMMENT`-state review; do not block the feature\n on it.\n\n### 2. `Suggestion`, and only on confirmed findings\n\n`ReviewFinding` gains an optional `Suggestion string` — the replacement text for\nthe anchored line, rendered as a fenced `suggestion` block (GitHub's one-click\n\"Commit suggestion\").\n\n**Only `confirmed` findings may carry one.** A wrong suggestion is a\none-click path to breaking code a reviewer then has to notice; `confirmed` is by\ndefinition the verdict the reviewer traced and can state the failure for. A\n`plausible` finding is a worry, and a worry gets a sentence, not a patch. This is\nthe h019.5 calibration rule reused, not a new one.\n\n`Suggestion` replaces the anchored line exactly, and is only rendered when\n`Verdict == confirmed`. Single-line only in this cut;\nmulti-line (GitHub's `StartLine`/`StartSide`) is a follow-up, because a\nmulti-line suggestion that is off by one line silently corrupts the file. The\nrule is enforced in `ReviewFinding.Validate` — an embedded newline is rejected\nthere, so the `record_finding` tool refuses it with feedback the model can act\non, rather than the renderer silently posting a fence that applies only its\nfirst line.\n\n### 3. Which findings are posted inline\n\nOnly findings that are **line-anchored** (`Line \u003e 0`). A whole-file finding\n(`Line == 0`, e.g. \"README.md is stale\") has no line to attach to and stays in\nthe PR-body list. The body section keeps listing everything, so it remains the\ncomplete record and the fallback.\n\nFindings that block never reach this path: a surviving blocking finding parks\nthe task before `StageOpenPR`, so there is no PR to comment on. Inline comments\nare the non-blocking, line-anchored subset — which is exactly the set that would\notherwise be a body-only list the author has to map back to lines by hand.\n\n### 4. A best-effort stage after `StageOpenPR`\n\nNew `StagePostReviewComments`, after `StageOpenPR`:\n\n1. Read the report stashed by `StageReview` (`tc.ReviewReport`, h019.6).\n2. Post each line-anchored finding as an inline comment, with a suggestion block\n when `Verdict == confirmed \u0026\u0026 Suggestion != \"\"`. The head SHA the comments are\n anchored to is resolved by the forge implementation, not here; there is no\n `PullRequestReader` on this side of the agent boundary. The stage passes\n `tc.ReviewedHeadSHA` alongside them.\n\nNothing in the run pushes between `StageReview` and this stage, so at that\nmoment the pull request's head _is_ the revision the review read. That was an\nassumption, and a collaborator can falsify it by pushing to the branch between\nthe review and the posting. A line number is not validated against a revision:\nboth forges anchor it to whatever now occupies that line, so a stale anchor\nmislabels rather than failing. The revision is therefore carried with the\ncomments and the set is refused when the head has moved off it\n(`reviewHeadDrift`) — a refusal lands in the failure path below, leaving the PR\nbody as the record. With no measured revision (a `Trees` that cannot report one)\nthe comments post unverified rather than being dropped.\n\n**Failures log and never fail the task.** The PR is already open; a comment that\nfailed to post must not park a task whose actual work succeeded, and the body\nsection already carries every finding. This mirrors `OpenPR`'s existing\nbest-effort `LinkBranch` call.\n\n### 5. No separate opt-in flag\n\nInline comments are posted whenever a review ran and produced line-anchored\nfindings — the same `repo.review_enabled` gate already covers it. A second flag\nis a knob to add if it proves noisy in practice; adding it speculatively is the\nsmallest-change principle's opposite.\n\n## Out of scope\n\n- Commenting on the blocking/parked case (no PR exists).\n- Resolving/dismissing threads, or reacting to replies (that is the\n `archie-core-8li9` remediation direction).\n- Multi-line suggestions.\n- Gitea's \"suggested change\" rendering: Gitea has no one-click apply, so the\n `Suggestion` there is informational text in the comment body.\n\n## Packages this touches\n\n- `internal/domain/workflow` — `ReviewFinding.Suggestion`; `StagePostReviewComments`;\n the `Forger` subset gains the writer (or `TaskContext` gains the writer seam).\n- `internal/forge` — `ReviewCommentWriter`, GitHub + Gitea implementations.\n- `internal/app/agentworker` — the reviewer prompt asks for a `Suggestion` on\n confirmed, mechanically-fixable findings; `record_finding` accepts it.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/mcp-tool-completion/",
|
||
"title": "MCP client and tool-guardrail completion",
|
||
"section": "PRDs",
|
||
"body": "# MCP client and tool-guardrail completion\n\n**Status:** Approved\n\nEpic: `archie-core-1786637493439-140-2cee2e9b` / GitHub `#161` (\"[EPIC] Tools:\ncentral registry, MCP client, guardrails\").\n\nThis document is scoped to five sub-features of that epic: MCP\nserver-initiated sampling, the per-server parallel-tool-calls flag, sandboxed\nMCP tool servers, a Firecracker sandbox backend, and sequential dispatch with\nper-result previews.\n\nOut of scope, and not to be touched: the central registry\n(`internal/tools/registry.go`), all three MCP transports (stdio/HTTP/SSE,\n`internal/tools/mcp/transport*.go`), reconnect-with-backoff\n(`transport.go`), guardrail loop detection\n(`internal/tools/guardrail.go`), idempotent/mutating classification\n(`internal/tools/classification.go`), concurrent tool dispatch (`#183`), and\nprogressive disclosure wiring (`internal/app/archied`). These are settled and\nproduction-wired.\n\n## Problem\n\n`bd search \"MCP client\"` shows five closed transport beads and one open:\nserver-initiated sampling (`#151`). `internal/tools/mcp/client.go`\nholds a single `callMu *sync.Mutex` that serialises every `tools/call`\nregardless of server — `ParallelToolCalls` does not exist on the `MCPServer`\nconfig struct (`internal/config/config.go`), confirming `#177` is a\nreal gap, not a stale bead. `#178` (wire MCP tool servers for the chat agent,\nsandboxed) and `#179` (Firecracker substrate) have no corresponding code —\nMCP servers run as whatever the config's `Command`/`Args` launches,\nunsandboxed. `#182` (sequential tool dispatch with per-result previews) is\nopen; concurrent dispatch (`#183`) is closed, so there are two dispatch paths\nand this PRD decides how they coexist.\n\n## Design\n\nSub-features are independent; there is no new shared type this document\nintroduces. Each section below stands alone.\n\n## MCP server-initiated sampling (`#151`)\n\n**Decision.** Implement as a callback on `mcp.Client`, not a second\ntransport. The MCP spec's `sampling/createMessage` request flows server→client\nover the same connection each transport already maintains\n(`transport.go`'s stdio/HTTP/SSE all multiplex request and response frames).\nAdd a `SamplingHandler func(context.Context, SamplingRequest) (SamplingResult, error)`\nfield to the client's config struct next to the existing transport fields,\ndefaulting to nil (server sampling requests get a JSON-RPC \"not supported\"\nerror when unset — never a silent drop).\n\n**Call site.** `internal/tools/mcp/client.go` — add the handler field and a\ndispatch branch in whatever function demuxes incoming server\nmessages by method name (the same switch that already routes\n`notifications/*`). Wire a concrete handler in\n`internal/app/archied/main.go` that routes the sampling request through the\nsame chat-model call path `chatGenerateOptions` already uses, so a\nserver-initiated sample gets the daemon's configured model, not a second\nprovider client.\n\n**Reuse check.** The chat model invocation path already exists\n(`chatGenerateOptions`, `main.go`); this reuses it rather than adding a\nparallel model-calling path for MCP sampling.\n\n**Failure mapping.**\n\n| condition | contract error |\n| ------------------------------- | ------------------------------------------------------------------------------------------ |\n| no `SamplingHandler` configured | JSON-RPC method-not-found, per spec |\n| handler returns error | JSON-RPC internal error, message from handler |\n| handler timeout | JSON-RPC timeout error; use the transport's existing request-timeout config, not a new one |\n\n**Testing.** Fakeable entirely — a stub `SamplingHandler` and a fake\ntransport that emits a `sampling/createMessage` frame. No real backend\nneeded; belongs in `task check`.\n\n## Per-server parallel-tool-calls flag (`#177`)\n\n**Decision.** `ParallelToolCalls bool` (default `false`) goes on `MCPServer`\nin `internal/config/config.go`, next to `Transport`/`Command`. The\nmutex in `client.go` becomes per-server: when `ParallelToolCalls` is\nfalse (default, and the only behaviour), keep the single `callMu`\nserialising that server's calls exactly as now; when true, drop the mutex for\nthat server's `Client` instance entirely and let the caller's own\nconcurrency (already proven by the closed concurrent-dispatch bead, `#183`)\nhandle it. This is additive — no server's behaviour changes unless an operator\nopts in.\n\n**Call site.** `internal/config/config.go` (struct field),\n`internal/tools/mcp/client.go` (make the mutex conditional on the\nfield), and wherever `MCPServer` configs are turned into `Client` instances\nat daemon startup (the constructor that reads `internal/config`'s MCP block).\n`config.example.toml`'s commented MCP block gets the new field documented\nwith its default.\n\n**Testing.** Unit test asserting two concurrent `Call`s to the same\n`Client` block on each other when the flag is false and don't when true —\nfakeable with a slow fake transport. `task check`.\n\n## Sandboxed MCP tool servers for the chat agent (`#178`) and Firecracker substrate (`#179`)\n\n**Decision.** These two beads are one design decision, not two: `#179` is an\nalternate execution backend for `#178`, not separate scope. Sandbox MCP\nservers that a chat-turn config marks as untrusted (a new\n`Sandboxed bool` field on `MCPServer`, default `false` — existing\nconfigured servers are unaffected) behind the same `ContainerPool` the\ndaemon already uses to run agent work\n(`internal/container/pool.go`, per CLAUDE.md's `WriteTaskJSON` trap and the\n`Acquire`/`Release` lifecycle `daemon.go`'s `acquireTaskContainer` already\nproves). Do not build a second sandbox mechanism: an MCP stdio server is a\nsubprocess exactly like an agent task container's entrypoint, so launching it\nvia `ContainerPool.Acquire` with stdio piped to the `mcp.Client` transport is\nthe smallest change, not new plumbing.\n\nFirecracker is **not** built now. `ContainerPool`'s existing Docker-backed\nisolation is the shipped backend; Firecracker becomes a second\nimplementation of the same acquire/release contract only if a concrete\nescape or resource-isolation gap is measured against Docker-backed sandboxing\nin production. Recording this as the deletion/promotion gate keeps `#179`\nhonestly `OPEN` instead of speculatively building an unused backend, per\nCLAUDE.md's Architectural Simplicity rule.\n\n**Call site.** `internal/config/config.go` (`MCPServer.Sandboxed`), the MCP\nclient constructor (route sandboxed servers' process launch through\n`container.Pool.Acquire` instead of `os/exec` directly), and\n`internal/container/pool.go` (confirm its `Acquire` contract accepts a\nstdio-piped, non-worktree workload — if it assumes a git worktree\nper CLAUDE.md's `WriteTaskJSON` note, add the narrower entrypoint variant\nthere rather than duplicating the pool).\n\n**Error/failure mapping.**\n\n| condition | contract error |\n| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |\n| pool exhausted (no container slot) | MCP client returns connection-refused equivalent; server marked unavailable in the registry until retried |\n| container fails health/start | same as `#151`'s handler-error path — surfaced through the client's existing reconnect/backoff (`transport.go`), not a new retry loop |\n\n**Testing.** `Sandboxed=false` path is the existing, already-tested\nunsandboxed flow — no change. `Sandboxed=true` needs a real\n`ContainerPool` (Docker) — manual smoke command, not `task check`; document\nthe exact command in the runbook this PRD's issue links to once implemented.\n\n## Sequential tool dispatch with per-result previews (`#182`)\n\n**Decision.** This coexists with concurrent dispatch (`#183`, closed) as a\nsecond, explicit mode selected by the caller — not a replacement. Concurrent\ndispatch is for independent tool calls in one turn; sequential-with-preview\nis for a turn where each result should be surfaced to the operator (or the\nnext tool call) before the next call fires — e.g. a mutating tool chain where\nshowing intermediate state matters. Add a `DispatchMode` enum\n(`Concurrent`/`Sequential`) to whatever type selects the closed\n`#183` concurrent path, defaulting to `Concurrent` (its only behaviour).\n\n**Call site.** The tool-dispatch entry point that `#183`'s closed\nimplementation lives in (grep the concurrent-dispatch symbol added for\n`#183` — same file gets the sequential branch, not a parallel file). Preview\nsurfacing reuses whatever channel-adapter interface already streams\nintermediate turn output (the same seam `internal/channels/telegram/` and\nother adapters use for streaming deltas) — do not invent a second preview\ntransport.\n\n**Testing.** Fakeable with a fake tool that returns staged results; assert\npreview callbacks fire in order before the next call starts. `task check`.\n\n## Call site inventory\n\n| concern | file | change |\n| --------------------------------------------------- | ---------------------------------------------------------------------------------- | -------------------------------------------- |\n| MCP client transports (stdio/HTTP/SSE) | `internal/tools/mcp/transport*.go` | none — already handles this |\n| MCP client call serialisation | `internal/tools/mcp/client.go` | change (`#177`) |\n| MCP server config | `internal/config/config.go` | new fields: `ParallelToolCalls`, `Sandboxed` |\n| Sampling dispatch | `internal/tools/mcp/client.go` | new (`#151`) |\n| Sampling handler wiring | `internal/app/archied/main.go` | new (`#151`), reuses `chatGenerateOptions` |\n| Sandboxed MCP launch | MCP client constructor + `internal/container/pool.go` | new (`#178`/`#179`) |\n| Concurrent tool dispatch | dispatch entry point (closed `#183`) | none — already handles this |\n| Sequential tool dispatch + previews | same dispatch entry point | new (`#182`) |\n| Guardrails / classification / registry / disclosure | `internal/tools/guardrail.go`, `classification.go`, `registry.go`, `disclosure.go` | none — already handles this |\n| `config.example.toml` MCP block | `config.example.toml` | doc update for new fields |\n\n## Execution: multi-agent team breakdown\n\n| sub-feature | issue | implementer scope | suggested council lenses | why |\n| ------------------------------------ | -------------- | ---------------------------------------------------------------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------- |\n| Server-initiated sampling | `#151` | `client.go` dispatch branch, `main.go` handler wiring | `lens-contract` | new JSON-RPC method surface crossing the MCP wire contract |\n| Parallel-tool-calls flag | `#177` | config field, conditional mutex, constructor wiring | `lens-boundary` | per-server behaviour change must not leak into other servers' serialisation |\n| Sandboxed MCP + Firecracker deferral | `#178`, `#179` | config field, constructor routing through `ContainerPool`, pool entrypoint variant | `lens-operator`, `lens-deletionist` | operator: sandbox-down failure path; deletionist: keeps `#179` from becoming unused scaffolding |\n| Sequential dispatch with previews | `#182` | `DispatchMode` enum, sequential branch, preview reuse of streaming seam | `lens-maintainer` | two coexisting dispatch modes need a reader to tell which applies without re-deriving both |\n\n## File and link the beads\n\n- `#151`, `#177`, `#178`, `#179`, `#182` each get their description updated to\n point at this doc's matching `##` heading instead of restating the design.\n- Close `#179` as a design decision (link here) rather than leaving it open\n as separate scope — it is not built now, and the doc records why.\n- Add the suggested lenses to each issue directly.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/memory-engine-unification/",
|
||
"title": "Memory engine unification: four scopes, CRUD, and one write path -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Memory engine unification: four scopes, CRUD, and one write path -- decision\n\n**Status:** Approved\n**Date:** 2026-09-16\n**Findings:** `memory-4` and its dependants in the adversarial sweep\n(`.local/issue-tracker`); the real defect is the split described below, for\nwhich `memory-4` is a symptom\n**Supersedes:** the draft `memory-turn-context.md` (its synchronous-read\nargument survives and is reused; its `Manager` method and consumer-owned port do\nnot)\n**Settles:** four open items in `docs/architecture/migration-decisions.md#5`\n(listed at the end)\n\n## Problem\n\nThere are **two live memory stores for one assistant**, split by which component\nwrites them, and the chat turn reads neither.\n\n| Family | Written by | Root | Format |\n| ----------------------------------------------------------- | ----------------------------------------------------------------------------- | ------------------------- | ----------------------- |\n| `internal/memory` (`Manager`, `builtin.Provider`) | the chat agent's `memory_edit` tool via `memorytoolprovider` (`bootstrap.go`) | `\u003cworkDir\u003e/memory` | `MEMORY.md` / `USER.md` |\n| `internal/domain/memory` + `internal/infrastructure/memory` | the session curator (`sessioncurator.go`) | `\u003cworkDir\u003e/memory-engine` | identity-keyed blocks |\n\nThey share no code, no format and no scope vocabulary. Neither reaches a prompt:\n`Manager.SystemPromptBlock` (`internal/memory/manager.go`) has no production\ncaller anywhere, so the agent can write memory through a tool and never read it\nback.\n\nThree further defects fall out of the same split:\n\n1. **The engine's contract is not CRUD.** `Store` is `Write`/`Query`/`List`/\n `Forget` — create, read, delete, and **no Update**. The retired file provider\n _does_ have `handleAdd`/`handleReplace`/`handleRemove`, so the family being\n deleted is operationally richer than its replacement.\n2. **There is no scope model.** The contract carries one opaque `Identity\nstring`, and the curator fills it with a **session id** (`sessioncurator.go`)\n — which is none of the four scopes `docs/architecture/agent-system.md`\n requires, and stops meaning anything when the session ends.\n3. **The chat turn has no user identity on half the channels.** Telegram and\n email populate `SenderID` with a person; the dashboard populates it with\n nothing, and a webhook populates it with nothing either — a route path names\n a source, not a person, so `webhook.go` leaves `SenderID` empty and carries\n the route in `Inbound.BudgetKey` for rate limiting. `sender_id` is persisted\n (`session_store_postgres.go`), so whatever a channel puts there reaches the\n consumers that read it.\n\n`agent-system.md` says the write-scope selection and cross-scope sharing\nrules \"require a focused memory design decision\". This is that decision.\n\n## Decision\n\n### 1. One engine, addressed by four typed scopes\n\n`internal/domain/memory` is the only memory engine. `Observation.Identity`,\n`Query.Identity`, `Record.Identity` and `List(identity)` are deleted, not\nextended: a field whose meaning depends on the caller cannot carry a scope.\n\n```go\ntype ScopeKind string\n\nconst (\n ScopeGlobal ScopeKind = \"global\" // shared across the application\n ScopeAgent ScopeKind = \"agent\" // private to one Agent\n ScopeUser ScopeKind = \"user\" // one user, across their Agents\n ScopeAgentUser ScopeKind = \"agent-user\" // one Agent's relationship with one user\n)\n\ntype Scope struct {\n Kind ScopeKind\n Agent AgentID // required for ScopeAgent, ScopeAgentUser\n User IdentityID // required for ScopeUser, ScopeAgentUser\n}\n\nfunc (s Scope) Validate() error // rejects a component present where it is not required\nfunc (s Scope) Key() string // canonical, collision-free storage key\n```\n\n`AgentID` and `IdentityID` are opaque. This package never resolves or\ncanonicalises them.\n\nA turn's resolved identity is one value:\n\n```go\ntype Subject struct {\n AgentID AgentID\n UserID IdentityID // \"\" when the channel carries no per-person identity\n}\n\nfunc (s Subject) Scopes() []Scope // readable, most specific first\nfunc (s Subject) WritableScopes() []Scope // never includes ScopeGlobal\n```\n\n**Access control is naming a scope, and nothing else.** The read path names\nexactly the scopes of the resolved `Subject`; the engine never unions beyond what\nit is handed and applies no policy of its own. An engine that enforced policy\nwould need to know about channels, sessions and bindings, and every backend would\nreimplement it. \"Agent-wide is private to its owning Agent by default\"\n(`agent-system.md`) falls out mechanically, because `ScopeAgent{a}` and\n`ScopeAgent{b}` are different storage keys.\n\n`ScopeGlobal` is never model-writable. Writing it is an operator action; nothing\nproduces one yet, but the contract models it because the read set includes it.\n\n### 2. CRUD, with revisions retained\n\n```go\ntype Store interface {\n Create(ctx context.Context, in NewRecord) (Record, error)\n Get(ctx context.Context, scope Scope, id RecordID) (Record, error) // ErrNotFound\n Query(ctx context.Context, q Query) ([]Record, error) // heads only\n List(ctx context.Context, scope Scope) ([]Record, error) // heads only\n Update(ctx context.Context, in RecordUpdate) (Record, error) // ErrNotFound, ErrStaleRevision\n Forget(ctx context.Context, scope Scope, id RecordID) error // idempotent\n Revisions(ctx context.Context, scope Scope, id RecordID) ([]Revision, error)\n}\n```\n\n`Get` and `Forget` take a scope. The current `Forget` scans every identity's\nstore to find a marker (`builtin_engine.go`) purely because the contract\ngave it nothing better.\n\n`Record` carries what `agent-system.md` requires be retained: `Scope`,\n`Kind`, `Content`, `Revision`, `Author`, `OriginUser`, `Source`, `CreatedAt`,\n`UpdatedAt`. `Metadata` is dropped — nothing sets it and the parser never\nrestores it, so it is a field that silently does not round-trip.\n\n`Update` supersedes rather than overwrites. The superseded state is appended to\nthe scope's `HISTORY.md` **before** the live block is replaced, so a crash\nbetween the two leaves an extra history entry and an unchanged record —\nrecoverable — never a record whose previous state was lost. `Update` carries no\n`Scope`: a record may never move between scopes, and the API makes that\nunrepresentable rather than validating it. `Expected` optionally names the\nrevision being replaced and returns `ErrStaleRevision` on mismatch, which matters\nbecause two processes (`archied` and `archie-gateway`) can hold the same scope's\nfile.\n\n`Forget` records a deleted revision before removing the live block, so a record's\nprovenance outlives its content.\n\n### 3. The caller resolves identity; the engine never does\n\nResolution happens in the gateway turn path, where the native sender id and the\nsession are both in hand, and it is supplied **per channel by the composition**:\n\n```go\n// TurnRunnerConfig\n// UserIdentity resolves the initiating user's identity for one inbound message.\n// false means the channel carries no per-person identity, and the turn gets\n// global and agent scopes only. Nil means the same.\nUserIdentity func(msg messaging.Message) (IdentityID, bool)\n```\n\nThis must be injectable rather than a blanket `channel + \":\" + SenderID`\nconcatenation, because a channel's sender id is not necessarily a person: a\nwebhook's route path names a source, and treating it as an identity would give\na URL one -- which is why `webhook.go` leaves `SenderID` empty and carries the\nroute in `Inbound.BudgetKey` instead. Telegram and email supply their sender\nid; the dashboard and webhook resolve nothing.\n\n**Fail closed.** When `UserID` is empty the turn proceeds with global and agent\nscopes only, and a write targeting `user` or `agent-user` fails with a clear\nerror. `agent-system.md` forbids showing one user's memory to another, so\n\"we do not know who this is\" must mean \"show none of it\", never \"fall back to a\nwider scope\".\n\n### 4. The chat read path\n\nSynchronous and total. It runs inside `prepareTurn` (`turn.go`) before the\nprompt is built, never returns an error to the caller, and degrades to an empty\nblock on any engine failure or provider panic. A fire-and-forget hook would race\nthe prompt build in the same turn, which is why this is not a lifecycle hook.\n\nThe read is a scope-only `Query` over the subject's scopes — **not** ranked\nsearch, which remains open in `migration-decisions.md#5`. A ranked read on the\nhot path with no ranking implementation would be a lie.\n\n`SystemPromptConfig` gains a `Memory` field, rendered into\n`templates/archie.md.tpl` between the `\u003ctools\u003e` and `\u003cenv\u003e` blocks:\n\n```\n\u003cmemory purpose=\"durable_context\" trust=\"data\"\u003e\n```\n\n`trust=\"data\"` and inclusion in the precedence clause follow the `\u003ctools\u003e`/`\u003cenv\u003e`\nprecedent exactly: memory content is model-written from untrusted input and must\nbe framed as data, never instructions. The rendered block carries each record's\nshort id, because the write path addresses records by id — one source of truth\nfor both directions.\n\nBoth a per-scope record limit and a byte cap on the rendered block are named\nconstants. The cap is about size, not latency; a local read needs no timeout.\n\n**Prerequisite:** `setupGateways` runs before `setupMemoryAll`\n(`main.go` vs `:428`), and `startGatewayRuntime` before `setupMemory`\n(`gateway.go` vs `:115`), so the engine is nil at every turn-runner\nconstruction site. The ordering is the root-cause fix and nothing in either\nmemory step reads what those later steps produce.\n\n### 5. The chat write path\n\nThe memory tool is built **per turn**, not registered once at boot, because it\nmust write into a scope derived from the turn's resolved subject:\n\n| action | engine call |\n| -------- | ---------------- |\n| `create` | `Create` |\n| `update` | `Update` (by id) |\n| `delete` | `Forget` (by id) |\n| `list` | `List` |\n\n**The model chooses the scope kind; it never chooses the ids.** Agent and user\nids come from the turn's resolution and there is no schema field to put anything\nelse in — even a model that emits another user's id has nowhere to write it. That\nis the root-cause form of `agent-system.md`, matching this system's other\nnon-waivable boundaries (the model never runs git; gates cannot be waived by the\nagent). `global` is not offered at all.\n\nThe tool takes a record **id** where the retired provider took a content\nsubstring (`handleReplace`/`handleRemove`). The substring addressing existed\nbecause a markdown file was hand-edited and the model had no stable handle on a\nblock; with ids rendered into the prompt it disappears, and one addressing scheme\nreplaces two.\n\n### 6. The curator's write\n\n`sessioncurator.reviewOne` stops writing `Identity: sessionID` and writes\n`ScopeAgentUser{AgentID: \u003csession's agent\u003e, UserID: \u003cparticipant\u003e}`. Relationship\nscope is the only defensible home: agent-wide is explicitly unsafe for user facts\nand would leak user A's extracted facts to user B through the same agent;\nuser-wide would pool facts across agents; and a session id cannot survive the\nsession ending, which is the whole requirement.\n\nThis needs data the tree does not persist: the `messages` table\ngains `sender_id`, and `curator.SessionSummary` / `ConversationMessage` gain the\nagent and participant. The participant is derived from the session's own\nuser-role messages: exactly one distinct non-empty sender writes; zero (a\ndashboard or webhook session) or several (a group chat) writes nothing and\nrecords why. Ambiguity fails closed rather than guessing.\n\nThe payoff is the point of the exercise: chat-written and curator-derived memory\nnow occupy the same scopes, so a future turn's prompt contains both, and a\ncurator's bad note becomes id-addressable and therefore correctable by the agent.\n\n### 7. The content scanner moves to the engine\n\n`internal/memory/scanner.go` and its installation at `bootstrap.go` are what\nmake `configuration.md`'s \"Memory safety scanner\" row describe a control that\nactually runs. Deleting the package would silently remove prompt-injection\nscanning on memory writes. The scanner moves into the engine and is applied in\n`Create` and `Update`, which is the single choke point every producer crosses —\nso the curator's model-extracted content is covered too, not just the chat tool's.\n\n### 8. `internal/memory` is deleted\n\nNot shimmed, not partially, not behind a flag: `manager.go`, `provider.go`,\n`lifecycle.go`, `sync.go`, `scanner.go` (moved), `schema.go`, `backup.go`,\n`config.go`, `builtin/provider.go`, and their tests. Its markdown store\n(`builtin/store.go`) is the one survivor, relocated to\n`internal/infrastructure/memory/builtin/` — `organisation.md`'s \"a package owns\nits on-disk format end-to-end\" makes infrastructure its correct home, and the\nengine already parses that format.\n\nAlso deleted: `internal/tools/provider/memory`'s manager wrapping (rewired to the\nengine), `config.Memory.{Provider,ProviderConfig}` and the\n`validateMemory` rejection that exists only to complain about `Provider`\n(`validate.go`), and `webui.MemoryStatus`/`MemoryProviderHandle` — never\nimplemented in production, so `/api/memory` reports empty forever unless it is\nrepointed at the engine registry.\n\nExisting data at `\u003cworkDir\u003e/memory` is left on disk, unreferenced. No migration.\n`\u003cworkDir\u003e/memory-engine` is likewise orphaned: its directories were keyed by\n`sha256(sessionID)` and no scope key can hash to one.\n\n## What we deliberately do NOT do\n\n- **No cross-scope sharing or grants in this wave.** Promoting a record between\n scopes is a future explicit command, and `Update` cannot change scope, so it\n cannot happen by accident.\n- **No canonical user identity.** `SenderID` is channel-native, so one person on\n Telegram and email is two identities. Unifying needs an Identity aggregate that\n does not exist (`internal/domain` has no identity package and `IdentityID`\n appears nowhere in Go). The contract does not change when it lands.\n- **No ranked retrieval.** `Query.Text` stays, unused by the prompt path.\n- **No data migration**, per the decision above.\n- **No prompt-memory timeout mechanism.** The read is bounded by construction.\n- **No `Metadata` field.** It is write-only; a caller needing it later\n needs one field, not a speculative contract.\n\n## Acceptance criteria\n\n1. A record created through the memory tool in turn 1 appears in turn 2's prompt,\n asserted on content the test planted, with its id.\n2. The same record is **absent** from a different user's prompt on the same\n channel and agent. Isolation asserted end to end, not at the engine boundary.\n3. An agent-scope record is visible to both of that agent's users; a global\n record is visible to a second agent; an agent-user record is not.\n4. A curator pass over a session writes a record a later turn reads, and which\n survives that session being deleted.\n5. `create` → prompt shows it; `update` → prompt shows new content and\n `Revisions` returns the prior content with author and time; `delete` → gone\n from prompt and `List`. Round-trip is asserted as `parse(render(r)) == r`,\n never against a fixture string.\n6. A panicking engine and a failing engine both complete the turn with no memory\n block.\n7. A channel with no resolvable user renders only global and agent scopes, and\n the tool refuses `user`/`agent-user` with a reason. A webhook route path never\n becomes a user identity.\n8. A write that would exceed the file cap fails loudly rather than silently\n dropping content.\n9. `internal/memory` is gone, `task check` is green, and no document describes a\n manager that no longer exists.\n\nEvery criterion asserts behaviour or planted content. None asserts on the text of\na hand-written file, and none pins that a method was called. In particular none\nasserts the composition calls setup in a given order: if the ordering were\nswapped back, criteria 1 and 4 would fail, and that is the assertion.\n\n## Sequencing\n\nEach step is independently verifiable and lands green on its own.\n\n1. **Persist the participant.** `sender_id` column on `messages`,\n insert/select/migration, `curator.SessionSummary.AgentID` and\n `ConversationMessage.SenderID`. Touches no memory contract and is shippable\n alone — nothing consumes it yet.\n2. **Scope, CRUD and revisions.** The `internal/domain/memory` contract, the\n builtin engine with `HISTORY.md`, the store relocation, the scanner move, and\n the curator rewritten onto agent-user scope (it is the engine's only live\n consumer, so it belongs in this slice). _Depends on 1 for the curator half._\n3. **Chat read path.** The `Memory` seam, the template block, subject resolution,\n and the composition-ordering fix. _Depends on 2._\n4. **Chat write path.** The tool adapter onto CRUD and the turn's subject on the\n context. _Depends on 3, because a write is only observable through the read._\n5. **Delete `internal/memory` and the dead config/dashboard surface**, and\n rewrite the \"Memory engine family\" section of\n `docs/architecture/plugins-and-extensions.md` to describe the\n engine family rather than a manager it deleted. _Depends on 3 and 4._\n\nSteps 3 and 4 are separable from each other; step 1 is separable from everything.\n\n## Settles these open items in `migration-decisions.md` §5\n\n| §5 open item | Settled by |\n| ---------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| authoritative memory record and revision model | §1, §2 — the live marker becomes JSON, since the ratified `\u003c!--mem:\u003culid\u003e--\u003e` cannot carry provenance; superseded states live in a per-scope `HISTORY.md` |\n| provider and infrastructure boundaries | §2 — contract in `internal/domain/memory`, builtin format in `internal/infrastructure/memory/builtin` |\n| retrieval and access enforcement | §1 — caller-named scope sets, no engine-side policy |\n| provenance representation | §2 |\n\n## Not determined\n\n- **Group chats produce no derived memory**, because the curator fails closed on\n mixed-sender sessions. Whether extraction should instead become\n per-participant is a product call.\n- **Two processes, one scope file.** After step 3 both `archied` and\n `archie-gateway` hold an engine over the same root, and the engine caches a\n store per scope and rewrites the whole file, so concurrent writers on one scope\n can lose a block. `Update.Expected` makes that detectable for one record; it\n does not make the file safe. A reload-before-mutate or advisory lock is the fix\n and is not designed here.\n- **Unbounded history.** `HISTORY.md` grows without limit; no compaction policy.\n- **Size bounds.** `defaultMaxFileBytes` is now per scope rather than per\n identity; whether that is right is unexamined.\n- **Whether the dashboard should carry a user identity.** It has one bearer\n token, not users. Until it has users, web turns are correctly limited to global\n and agent scope.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/messaging-service-boundary/",
|
||
"title": "Messaging Service boundary and transport design",
|
||
"section": "PRDs",
|
||
"body": "# Messaging Service boundary and transport design\n\n**Status:** Approved\n**Revision:** 1 \n**Date:** 2026-09-19 \n**Beads issue:** `archie-core-8cda.6.1` \n**Parent:** [`service-decomposition.md`](/dev/prds/service-decomposition/) \n**Prerequisites:** Gateway contract extraction (Phase 1), State Store extraction (Phase 2), UI extraction (Phase 3)\n\nThis document is the authoritative specification for Phase 4: Messaging Service extraction. It fixes the process boundary, configuration ownership, client contracts, security posture, readiness behaviour, channel invariants, and deletion criteria required by `archie-core-8cda.6` and its child tasks.\n\n## Decision\n\nThe Messaging Service (`cmd/archie-messaging`, composed via `internal/app/archiemessaging`) owns external channel connections (Telegram, email, webhook) and dispatches inbound messages and commands to the standalone Gateway Service over gRPC via `messaging.ChatContract`.\n\nIt is a peer service to the UI Service and the daemon:\n\n- It serves external chat participants across configured communication platforms.\n- It consumes Gateway's `messaging.ChatContract` exclusively for chat turn execution, streaming, session queries, cancellation, persona selection, and task control actions.\n- It holds no direct database connections, owns no LLM execution or model provider credentials, and has no dependency on the daemon runtime or the UI HTTP server.\n\n## Boundary and Ownership\n\n| Concern | Messaging Service owns | Messaging Service consumes | Messaging Service must not own or link |\n| ---------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------ |\n| **Channel transports** | Telegram bot long-polling, Email IMAP/SMTP loops, Inbound Webhook HTTP listener | Gateway `ChatContract` | Direct LLM providers, model runtimes, tokenizer libraries |\n| **Turn dispatch** | Inbound message packaging, typing indicators, live streaming updates | `ChatContract.Route`, `ChatContract.Stream` | `gateway.Router`, `runtime.Runtime`, agent prompt builders |\n| **Sessions \u0026 History** | Platform-to-session key mapping, conversation thread ID translation | `ChatContract.GetSession`, `RecentMessages`, `RecentTurns` | store implementations, `SessionStore` database handles |\n| **Control \u0026 Actions** | Platform command parsing (`/status`, `/tasks`, `/cancel`, `/stop`, `/persona`) | `ChatContract.Snapshot`, `ApplyTaskAction`, `Cancel`, `SetPersona` | `daemon.Daemon`, workflow execution engine, worktree operations |\n| **Task Approvals** | Channel interactive keyboards, approval callbacks | `ChatContract.ApplyTaskAction` | Direct task status mutations, forge comment posters |\n| **Configuration** | Platform bot tokens, allowed sender/user IDs, webhook listen ports | `[chat.*]` channel configs, Gateway target/token | `config.Holder`, forge tokens, repo configs, agent container specs |\n| **Readiness** | Messaging process liveness, channel connectivity, Gateway probe | Gateway `ChatContract.Snapshot` | Internal daemon subsystem probes |\n\n## Target Service Shape\n\n1. **Binary entry point**: `cmd/archie-messaging/main.go` parses CLI flags (`-config`, `-config-overlay`, `-gateway-target`, `-gateway-token`) and hands execution to `internal/app/archiemessaging.Run`.\n2. **Application composition**: `internal/app/archiemessaging/` wires:\n - Config loading scoped strictly to `[chat.*]` and `[services.gateway]`.\n - Secret resolution for channel credentials (e.g. Telegram bot token).\n - Gateway gRPC client (`internal/infrastructure/gatewayrpc.Client`).\n - Channel lifecycle managers (`internal/channels/status`).\n - Channel instances (`internal/channels/telegram`, `email`, `webhook`).\n3. **Turn and Stream Execution**:\n - Channels adapt inbound platform messages to `messaging.Inbound`.\n - Streaming channels (Telegram) consume `ChatContract.Stream(ctx, inbound)` which returns `\u003c-chan messaging.ChatEvent`, mapping `started`, `delta`, `tool`, `media`, `done`, and `error` events directly to live message frames.\n - Non-streaming channels (email, synchronous webhooks) consume `ChatContract.Route(ctx, inbound)`.\n\n## Load-Bearing Channel Invariants\n\nThe Messaging Service preserves existing invariants defined in `AGENTS.md` and channel documentation:\n\n1. **Telegram long-polling**: Must call `dropPendingUpdates(ctx, b)` before `b.Start(ctx)` and on `/restart`.\n2. **Telegram command menus**: Must register command menus simultaneously to `default`, `all_private_chats`, and `all_group_chats` via `commands.go` to prevent menu shadowing.\n3. **No UI detour**: Turns and task actions route directly to Gateway's `ChatContract`; no HTTP request may ever be sent to `archie-ui` or `internal/webui`.\n4. **Sender Confinement**: Telegram sender checks against `allowed_user_ids` must continue to fail closed before initiating any turn dispatch.\n\n## Configuration Ownership\n\n`cmd/archie-messaging` reads only:\n\n- `[services.gateway].target` and `[services.gateway].target_token` (or CLI flags).\n- `[chat.telegram]` (token, allowed user IDs).\n- `[chat.email]` (IMAP/SMTP host, ports, credentials).\n- `[chat.webhook]` (listen address, secret).\n- `[secrets]` (for resolving token secret references).\n\nIt does not read or decode `[forge]`, `[repos]`, `[models]`, `[providers]`, `[runners]`, `[containers]`, or `[nats]`.\n\n## Deletion Gate and Verification\n\nThe extraction is guarded by `cmd/archie-messaging/architecture_test.go`, verifying that `go list -deps ./cmd/archie-messaging` links zero banned runtime packages:\n\n- `modernc.org/sqlite`, `github.com/jackc/pgx`, `internal/infrastructure/postgres` and `internal/infrastructure/legacyread` (State Store)\n- `internal/daemon`, `internal/app/archied`, `internal/container`, `internal/worktree` (Daemon runtime)\n- `internal/domain/workflow`, `internal/agentexec`, `internal/taskrun` (Workflow engine)\n- `internal/forge`, `internal/forgerpc` (Forge client)\n- `internal/gateway` runtime (only `internal/domain/messaging` and `internal/infrastructure/gatewayrpc` are permitted)\n- `internal/webui` (Dashboard)\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/module-position/",
|
||
"title": "Module position -- design",
|
||
"section": "PRDs",
|
||
"body": "# Module position -- design\n\n**Status:** Draft\n**Date:** 2026-09-03\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n\n## Problem\n\n`eda-playbook-engine.md` names four logic positions a playbook action can\ntarget: Workflow, Module, Channel, Forge. Workflow (`t2db.9`/`.10`/`.11`)\nand the standalone lint tool (`t2db.12`) are shipped. Module -- the\nposition that makes an action \"not restricted to code-generation\" -- has\nnot started, because it needs a schema-generation mechanism the parent\ndoc deliberately deferred: \"This document does not attempt to design the\nfull generator.\" This is that design.\n\n## Do not invent a new mechanism -- generalise the one used three times\n\nYaegi-interpreted extension code with a fixed, resolved-by-name exported\nfunction already exists in this codebase for three surfaces, and all three\nfollow the identical shape:\n\n| Surface | Fixed signature | Export path | Symbol table |\n| ----------------------------------------- | --------------------------------------- | -------------- | --------------------- |\n| Custom gate (`internal/gate/gateeval`) | `func(gate.GateContext) []gate.Finding` | `\"gate.Check\"` | `gateextract.Symbols` |\n| Workflow stage plugin (`skillbuild`) | stage-specific | skill-declared | `wfextract.Symbols` |\n| Skill plugin (`internal/skill/plugin.go`) | `func(string) string` | `\"main.Run\"` | none (stdlib only) |\n\nEach is: read a `.go` file -\u003e `yaegiutil.New` with that surface's symbol\ntable -\u003e `yaegiutil.Resolve[T](i, src, exportPath)` -\u003e call the resolved\nfunction wrapped in `yaegiutil.Safe` for panic recovery. **Module is a\nfourth instance of this same shape**, parameterised by action kind instead\nof hardcoded to one fixed signature. No new interpreter construction, no\nnew panic-recovery mechanism, no new symbol-table generation tool --\n`go generate ... yaegi extract` is already how `wfextract`/`gateextract`\nare produced that way (see their `//go:generate` directives), and Module\nschema generation reuses that exact command.\n\n## No generic `Module` interface -- one typed contract per action kind\n\nThe plugin engine rule's invariant 1 requires \"real capability-specific\noperations,\" and invariant 5 forbids adding capability methods to a\ngeneric interface. A single `Module` interface with `Run(args any) (any,\nerror)` would be exactly the untyped hook the rule (and\n`event-sources-and-reactions.md`'s rejection of a generic `Source`\ninterface) exists to prevent -- restating the mistake the whole EDA\ndocument was written to move away from (`wfextract`'s hardcoded\n`reflect.ValueOf(workflow.Implement)` table is what a closed, un-typed-per-\ncase system already looks like in this repo).\n\nInstead, **each action kind is its own tiny package with its own\ngenerated contract**, exactly like `gate.GateContext`/`gate.Finding` are\none hand-written Go file that `gateextract` turns into a Yaegi symbol\ntable:\n\n```go\n// internal/domain/eda/module/notify/types.go (hand-written, the schema source)\npackage notify\n\n// SchemaVersion is independent of a playbook's own workflow_version, per\n// eda-playbook-engine.md's schema-versioning point.\nconst SchemaVersion = 1\n\ntype Args struct {\n Channel string\n Message string\n}\n\ntype Result struct {\n Delivered bool\n}\n\n//go:generate go run github.com/traefik/yaegi/cmd/yaegi extract -name notifyextract github.com/samcharles93/archie-core/internal/domain/eda/module/notify\n```\n\nA Yaegi module implementing this kind exports a fixed function name\nagainst these generated types, resolved the same way `gate.Check` is:\n\n```go\n// operator's notify.go, interpreted at load\npackage notify\n\nfunc Run(a notify.Args) notify.Result { ... }\n```\n\n`internal/domain/eda/module/notify` is the schema; `notifyextract.Symbols`\nis its generated Yaegi bridge; the operator's file is the implementation.\nThis is the same three-way split `internal/gate` / `gateextract` /\n`.archie/gate.go` already has.\n\n## The registry: `ModuleRegistry`, keyed by kind, not a service locator\n\nPer invariant 2 (owning family manager) and invariant 4 (narrow host\naccess): a `ModuleRegistry` in `internal/domain/eda/module` maps a kind\nname (`\"notify\"`, `\"image-gen\"`) to a resolved, type-erased invoker --\nthe same `map[string]reflect.Value`-shaped erasure `wfextract.Symbols`\nalready uses to bridge Yaegi and Go, not a new pattern and not exposed as\nthe plugin's contract with the daemon (the plugin still writes and is\nchecked against a fully typed `func(Args) Result`; the registry's\ninternal storage is the only place erasure appears, exactly as it already\ndoes inside `wfextract`).\n\n```go\ntype ModuleRegistry struct {\n // kind -\u003e (validate args against generated schema, invoke, marshal result)\n}\n\nfunc (r *ModuleRegistry) Register(kind string, dir string) error // discovers \u003ckind\u003e.go, resolves against that kind's generated symbols\nfunc (r *ModuleRegistry) Invoke(ctx context.Context, kind string, rawArgs map[string]any) (map[string]any, error)\n```\n\n`Invoke` decodes `rawArgs` into that kind's generated `Args` struct before\ncalling the interpreted function -- a shape mismatch is a reported load/\ndispatch failure per the design doc's \"schema defines the accepted\nmessage\" rule, not a silent zero-value fill. This mirrors `plugin.Registry`\n(`internal/plugin/plugin.go`)'s `Register`/dispatch split, scoped to one\ncapability family instead of generic daemon plugins.\n\nLifecycle (invariant 3): stateless invocation is the default -- explicit\nno-op start/health/stop, matching \"Stateless providers may make these\noperations explicit no-ops.\" A future Module kind that owns a resource\n(e.g. a persistent connection) can add optional lifecycle methods later,\nthe same way `internal/memory.MemoryProvider`'s optional capability\ncontracts (system-prompt contributions, prefetch, shutdown) layer onto a\nminimal required set without forcing every provider to implement them.\n\n## Trust boundary (closes `eda-playbook-engine.md` open question 3)\n\nInvariant 6 draws the line: operator-trusted in-process code vs.\nrepository-supplied code that must run in a container. Module code is\n**operator-installed, in-process, daemon-privileged** -- the same trust\ntier as `PluginDir` (daemon plugins) and `SecretEngineDir` (secret engine\nplugins) already occupy, both Yaegi-interpreted and already running\nin-process. It is **not** repository-supplied code from a task\n(that stays containerized, unchanged). A Module directory is therefore a\nnew config field the operator points at their own trusted directory,\nmirroring `PluginDir`'s shape exactly -- not something a forge webhook or\nan untrusted playbook source can populate.\n\n## Playbook YAML shape for a Module action\n\n```yaml\nactions:\n - position: module\n kind: notify\n args:\n channel: telegram\n message: \"build finished\"\n```\n\n`args` is validated against `notify.Args` (via the registry's decode\nstep) before invocation -- an unknown or missing required field is the\nsame \"dropped and reported to the caller\" failure the routing layer\nalready applies to unrecognised kinds/labels, not a new error philosophy.\n\n## Interaction with the unresolved execution-time gaps\n\n`eda-playbook-engine.md`'s \"Execution-time gaps\" section already gates\nthis: mid-run failure semantics and idempotency keying \"must be resolved\nbefore Channel/Forge actions or multi-action playbooks ship.\" Module is\nwhere those gaps stop being theoretical -- a `notify` action has a real\nside effect. Concretely:\n\n- A **first Module kind should be side-effect-free** (e.g. a `log`/`echo`\n kind that only writes to the daemon's own log) so the schema-gen -\u003e\n registry -\u003e playbook-dispatch mechanism can be proven end to end without\n yet needing an idempotency answer.\n- **Any Module kind with a real external side effect (`notify`,\n `image-gen`, forge writes) is blocked on resolving those two gaps\n first** -- this document does not relitigate or resolve them, it\n inherits the existing gate.\n\n## Recommended first slice\n\n1. `internal/domain/eda/module` package: `ModuleRegistry`, the\n discover/resolve/invoke mechanics, generalising `gateeval`'s pattern.\n2. One proof-of-concept kind, side-effect-free (`log`), with its schema\n source + generated extract file, to prove schema -\u003e `go:generate` -\u003e\n Yaegi load -\u003e registry -\u003e playbook dispatch end to end.\n3. Config field for the Module directory (`ModuleDir`, mirroring\n `PluginDir`), wired at daemon startup alongside the existing\n `PlaybookDirs` loading.\n4. Do **not** implement `notify` or any side-effecting kind in this slice\n -- that is explicitly gated on the execution-time gaps above.\n\n**Resolved** (`archie-core-t2db.13`). The schema-gen -\u003e\ngo:generate -\u003e Yaegi load -\u003e registry mechanism is proven end to end with\n`internal/domain/eda/module/log` (hand-written schema + `logextract`\ngenerated symbols) and `ModuleRegistry` (Register/Invoke, strict arg\ndecode, panic recovery via yaegiutil -- no new interpreter-construction\nor recovery mechanism). `ModuleDir` config field mirrors `PluginDir`;\nwired in bootstrap.go with log-and-abort on load failure.\n\nPlaybook YAML action dispatch is **deliberately out of this slice**: a\nminimal 'run this one module action, no chaining' dispatch still requires\nthe parent doc's data-flow mechanism (now resolved: CEL expressions\nover an evaluation context -- see eda-playbook-engine.md's resolved open\nquestion 1) and what the result feeds. Dispatch is a follow-up ticket\nthat lands once the coordinator exists, and it will consume this\nregistry as-is.\n\n## Open questions\n\n1. **Discovery convention.** Does a Module kind's implementation file need\n to declare its kind name explicitly (frontmatter-style, like a skill's\n `metadata.archie.workflow`) or is the kind inferred from directory/file\n naming (`module/notify/impl.go` implies kind `notify`)? Follow the\n skill catalogue's existing declared-metadata convention unless there's a\n reason not to -- not decided here.\n2. **Where schema source files live.** This doc assumes\n `internal/domain/eda/module/\u003ckind\u003e/types.go` per kind, but the exact\n package layout (one package per kind vs. one package with per-kind\n subtypes) should be confirmed against `organisation.md`'s file-layout\n rule before the first slice starts.\n3. **Result surfacing.** A Module's `Result` needs to reach later actions\n in the same playbook once data-flow is designed -- resolved in the\n parent doc as CEL expressions over `actions.\u003cid\u003e.result.\u003cfield\u003e` (see\n eda-playbook-engine.md's resolved open question 1); the `map[string]any`\n result shape above is what the evaluation context exposes, so it stays\n compatible by construction.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/multi-action-playbooks/",
|
||
"title": "Multi-action playbooks and typed action results",
|
||
"section": "PRDs",
|
||
"body": "# Multi-action playbooks and typed action results\n\n**Status:** Approved\n**Date:** 2026-09-22\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-nx30` (unblocks `archie-core-t2db.27` and `archie-core-t2db.31`)\n\n## Why this exists\n\n`eda-playbook-engine.md`'s hard boundary rejects any playbook that is not\nexactly one `workflow` action, so the parent document's data-flow, Module and\nChannel/Forge decisions have no loadable shape to attach to. The boundary's own\nstated condition is now met: both execution-time gaps are resolved\n(`eda-playbook-engine.md`, \"Execution-time gaps\"; the `playbook_dispatches`\nledger). This document designs the relaxation.\n\n## A playbook is one of two shapes\n\n- A **workflow playbook** is exactly one `workflow` action. Unchanged: it names\n a workflow definition and is routed by the daemon's definition pin.\n- An **action playbook** is one or more `module` actions in order. Each action\n has an optional `id`, a `kind` naming a registered Module kind, `args`, and an\n optional `when`.\n\nOne playbook does not mix the two shapes.\n\n```yaml\ntrigger: { kind: bug }\nactions:\n - id: build\n position: module\n kind: log\n args: { message: '\"build started\"' }\n - id: done\n position: module\n kind: log\n args: { message: '\"done: \" + string(actions.build.result.written)' }\n when: actions.build.result.written == true\n```\n\nThe first Module kind is the side-effect-free `log` kind\n(`module-position.md`, \"Recommended first slice\"). A Module kind with an\nexternal side effect, and the Channel and Forge positions, are out of scope.\n\n## A result type is the kind's generated Result struct\n\nEach Module kind owns a hand-written `Args` and `Result` struct\n(`internal/domain/eda/module/log` is the worked example); `ModuleRegistry.Invoke`\nreturns the result as a flat `map[string]any`. `ModuleRegistry.DecodeResult`\n(`internal/domain/eda/module`) is the single site that converts that map into\nthe kind's `Result` struct before a later expression reads it, so the value the\nchecker validates and the value the run reads have one definition.\n\n`actions.\u003cid\u003e` is typed by the kind of the action that declares `\u003cid\u003e`:\n`{ result: \u003cKindResult\u003e }`. `event` stays `dyn` (J4,\n`playbook-expression-syntax.md`).\n\n## Reject-at-load, preserved\n\nRelaxing the boundary must not make a bad reference resolvable. The load fails,\nnaming the playbook and the offending action, for:\n\n- an `id` missing where a later action references it, duplicated, or outside\n its shape's grammar: a workflow action id keeps the shared\n stable-identifier grammar (J1, already enforced), while a module action id\n must be a lowercase CEL field name writable as `actions.\u003cid\u003e` (so a CEL\n keyword such as `in`, `true`, `false`, or `null` is rejected at load);\n- an `actions.\u003cid\u003e` read of an id no earlier action declares;\n- an `actions` read that is not statically resolvable to an id (any map index,\n literal or dynamic, `in`, a comprehension): the typed `actions` root is an\n object type, so only field selection `actions.\u003cid\u003e` resolves and the\n `actions[\"id\"]` spelling is not accepted;\n- a `result` field the referenced kind's `Result` does not define;\n- a reference to an earlier action that produces no result;\n- an unknown `position`, an unknown `kind`, or a `module` action with no `kind`;\n- a field belonging to the other shape (`kind` on a `workflow` action, or\n `workflow` on a `module` action);\n- an arg KEY the kind's `Args` schema does not define;\n- an arg value whose checked CEL type cannot fill its `Args` field (`'123'`\n for a string field). A `dyn` value, such as an `event` read, passes: its\n type is known only per event, and the module decode refuses a bad one at\n run.\n\nDeclaring the kind's `Result` schema to CEL at load makes a result-field typo a\nload failure rather than a dispatch failure. Matching arg keys against the\nkind's `Args` schema makes an arg-key typo or a wrongly typed value a load\nfailure.\n\n## The environment is per playbook\n\nResult typing depends on the ids a playbook declares and the kind each id runs,\nso the CEL environment is built from one playbook's action list rather than once\nper `Store`, and `expr.NewEnv` takes that list: each declared id becomes a field\nof the `actions` object typed by its kind's `Result`.\n\n## Running an action playbook is out of scope\n\nThis document makes an action playbook load and its references type-check.\nRunning one is designed in `docs/prds/action-playbook-run.md`.\n\n## Decisions\n\n**D1 — dispatch before `t2db.31`.** Superseded by\n`docs/prds/action-playbook-run.md`. Was: an action playbook loads and\nvalidates but is not routed; the definition-pin dispatch reports no match for it,\nand the daemon logs a visible warning naming it.\n\n**D2 — mixing positions.** Settled: one shape per playbook. A `workflow` action\nyields a task, not a result, so it cannot feed `actions.\u003cid\u003e.result`.\n\n**D3 — the CEL typing mechanism.** Settled: a per-playbook environment whose\n`actions` root is an object type with one field per declared id, each field typed\nby the kind's `Result` struct. `internal/domain/eda/expr` implements it with the\ngeneric `actionResultProvider`.\n\n**D4 — `args` against the kind's `Args` schema.** Settled: keys and statically\ntyped values are checked at load; `dyn` values are checked by the module decode.\n\n**D5 — where the result map becomes the `Result` struct.** Settled:\n`ModuleRegistry` stays schema-agnostic. `ModuleRegistry.DecodeResult`\n(`internal/domain/eda/module`) is the conversion site.\n\n## Verification\n\n- A two-action `module` playbook with a valid `actions.\u003cid\u003e.result.\u003cfield\u003e` read\n loads; the same read of a field the kind's `Result` does not define fails the\n load naming the field; a read of an id no earlier action declares fails; a\n dynamically indexed `actions` read fails.\n- An arg KEY the kind's `Args` schema does not define fails the load, and so\n does `message: '123'` for the `log` kind. `message: event.title` loads.\n- The one-action `workflow` playbook shape, its dispatch, and its existing\n load-time checks are unchanged.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/news-and-releases/",
|
||
"title": "Public news and per-version release notes",
|
||
"section": "PRDs",
|
||
"body": "# Public news and per-version release notes\n\n**Status:** Approved\n**Date:** 2026-09-21\n**Beads issue:** `archie-core-2ncm.2`\n**Superseded in part:** `docs/prds/release-stream-and-commit-rules.md` defines the\nsingle release stream. `CHANGELOG.md` carries every release and one `vX.Y.Z` tag\nnames each release.\n\nThe Astro website renders public news and per-version release notes from the\nartifacts generated here. Both derive from `CHANGELOG.md`, so publishing a\nrelease adds no writing.\n\n## Source of truth\n\nRelease notes come from the frozen sections in `CHANGELOG.md`:\n`tools/release.sh --prepare` writes a section, and `--tag` refuses to tag a\nversion without it. The generated artifact has one entry per release version.\n\nTags and raw commits are not the source. The changelog is the edited release\nrecord, and its `## [\u003cversion\u003e] - \u003cdate\u003e` headings provide the version and date.\n\n## Release data\n\nThe generator skips `## [Unreleased]`. Any other `## [` heading that is not\n`[version] - date` stops generation. A relative Markdown link in a changelog\nbody also stops generation because the Astro site must be able to render the\ncontent without depending on repository-relative paths.\n\n`archied` and `archie-agent` version independently. Each\n`docs/news/releases.json` entry carries the tag prefix (`archied`, `archie`),\nproduct name (`archied`, `archie-agent`), version, date, and changelog body.\n`docs/news/redirects.json` maps retired per-component news URLs to each unified\nrelease route. The Astro site must apply this table so previously published\nrelease URLs continue to resolve.\n\nBoth JSON files are committed generated artifacts. `task news` regenerates\nthem, `task news:check` detects drift, and `tools/release.sh --tag` regenerates\nand stages them with the release commit. The Astro site renders from these\nartifacts rather than re-parsing `CHANGELOG.md`.\n\n## Site ownership\n\nThe Astro website is maintained outside this repository and serves the\ndocumentation at `offloaded.dev/docs/`. This repository generates and validates\nthe news and documentation data artifacts; it does not build, preview, or\ndeploy the Astro site.\n\n## Not in scope\n\n- Generated summaries, categories, or section headings; the changelog has no\n structured summary field to derive one from.\n- Per-release download or install links; where to get the artifacts is\n documented once, not restated for every release.\n- Hand-written news items.\n- Changes to `infra/ansible` or to the external Astro website repository.\n\n## Verification\n\n- `go test ./tools/newsgen/...` covers parsing, ordering, artifact generation,\n heading and link refusals, and check-mode drift.\n- `task news:check` passes inside `task check`.\n- `task docs:artifact:check` verifies the generated documentation data against\n its Markdown sources.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/operator-pr-review/",
|
||
"title": "Operator-triggered PR review",
|
||
"section": "PRDs",
|
||
"body": "# Operator-triggered PR review\n\n**Status:** Draft\n**Beads issue:** `archie-core-h019.8`\n\n\u003e Superseded by `docs/prds/pr-review-agent.md`.\n\nThis PRD reuses the adversarial-self-review contract\n(`docs/prds/adversarial-self-review.md`) and the `workflow.Reviewer` /\n`workflow.ReviewReport` types from `internal/domain/workflow`. It decides\nonly what is new: how an operator asks for a review of an _existing_ PR, and\nhow that PR's content reaches the reviewer.\n\n## Problem\n\nThe adversarial reviewer (`StageReview`) runs automatically before `StageOpenPR`\nand reviews the task's own worktree HEAD. There is no way to review a PR that\nalready exists: one archie opened, a human opened, or one whose findings the\noperator wants re-checked after a push.\n\n## Shape\n\nOne channel-neutral command — `review_pr` — accepts a repository and PR number\nand returns the structured `ReviewReport`. The Web UI chat and Telegram invoke\nthe same command; neither duplicates review behaviour. The reviewer model is\n`models.reviewer` (falling back to `models.builder`), already the rule the\nworkflow stage uses, so an operator configures it independently of\nplanner/builder.\n\n## Decisions\n\n1. **Forge read.** `GetPullRequest(owner, repo, number)` returns a\n forge-neutral `PullRequest{Number, Title, Body, HeadRef, BaseRef, HeadSHA,\nBaseSHA, State}`. Declared as a narrow `PullRequestReader` interface,\n implemented by GitHub and Gitea; the noop forge does not implement it, so\n the composition root type-asserts and refuses review with \"forge does not support PR\n review\". Adding to the fat `Forge` interface would churn every test fake for\n a capability the noop forge cannot honestly provide.\n\n2. **Materialisation.** `worktree.Manager.CheckoutPR(owner, repo, headRef,\nbaseRef)` clones the repository fully and checks out `origin/\u003cheadRef\u003e`.\n `Diff(dir, baseRef)` and `Snapshot(dir, destDir)` then work unchanged:\n `Diff` resolves `origin/\u003cbaseRef\u003e` via the existing `resolveBase`, and\n `Snapshot` strips `.git` exactly as the workflow stage does. The head must\n already be pushed to the same repository — archie's own PRs always are; a\n cross-repo or deleted head is refused with that reason, not half-reviewed.\n\n3. **Reviewer reuse.** `agentworker.NewOperatorReviewer(models, providers)`\n builds the same `subagentReviewer` the stage uses, from daemon-level config\n rather than a `taskrun.Request`. It drops only the `Repo.ReviewEnabled`\n gate (the operator asked explicitly) and the task-run scaffolding.\n\n4. **Command.** A gateway tool `review_pr` (toolset `tasks`) bound to the\n channel's identity at construction — the model never names an identity,\n matching `task_spawn`/`task_action`. The Gateway process serves it: it\n composes the worktree manager it needs to materialise the PR head, and the\n composition root supplies `ChatPRReviewer`, a narrow gateway interface\n whose adapter maps `workflow.ReviewReport` into a gateway-owned result\n shape (gateway states what it needs, the composition root adapts — the\n `ChatTaskActor` pattern).\n\n5. **Authorisation.** Identity-scoped repository allow-list, the\n `task_spawn` rule: a chat identity may only review a repo in its\n `TaskProfile.Repos`. `identity == nil` denotes an authenticated dashboard\n operator who may review any configured repository.\n\n6. **Determinism.** One review per (owner, repo, number) at a time: the\n Gateway holds an in-flight set; a concurrent request for the same PR returns\n \"review already in progress\" rather than launching a second reviewer and\n racing on the snapshot.\n\n## Files\n\n- `internal/forge/forge.go`, `github.go`, `gitea.go` — `PullRequest`,\n `PullRequestReader`, `GetPullRequest`.\n- `internal/worktree/worktree.go` — `CheckoutPR`.\n- `internal/app/agentworker/review.go` — `NewOperatorReviewer`.\n- `internal/gateway/review_tools.go` — `ChatPRReviewer`, result shape, `review_pr`.\n- `internal/gateway/turn.go`, `gateway.go` — tool wiring into the turn.\n- `internal/app/archied/pr_review.go` — the `ChatPRReviewer` adapter.\n- `internal/app/archied/{bootstrap.go,gateway.go,main.go}` — composition.\n\n## Not in scope\n\n- Surfacing findings on the PR body or the dashboard (h019.6).\n- Reviewer model per-invocation override: configuration via `models.reviewer`\n is the selection surface; a per-call override is a later knob.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/orgs-and-access/",
|
||
"title": "Orgs, workspaces and access control",
|
||
"section": "PRDs",
|
||
"body": "# Orgs, workspaces and access control\n\n**Status:** Approved\n**Date:** 2026-09-24\n**Parent:** `docs/prds/event-automation.md`, epic `archie-core-t2db`\n\n## Why this exists\n\nArchie has no tenant boundary and no access control: one operator, one shared\ntoken, and no org on any record. The event automation design needs orgs to own\nsources, bindings, workflows and agents, and an agent's reach must be decided\nby the policies over it, never by the event that started it.\n\n## Concepts\n\n- **Org.** The tenant boundary. An instance admin creates an org and adds its\n first owner. An owner or admin adds further members by the sign-in\n provider's subject or email.\n- **Workspace.** A division of an org, named freely (`networking-prd`,\n `service-desk-dev`), with an optional `environment` attribute (any string)\n that policies can test. Sources, event types, mappings, bindings and runs\n belong to a workspace.\n- **Identity.** Every principal is an identity (`internal/domain/identity`):\n - a person is an identity of kind `user`, bound to the sign-in provider\n through the existing subject binding;\n - an agent is an identity of kind `bot` or `service_account`, assigned to\n one org and granted access, secrets and permissions the way a user is.\n\n Signing in and verifying credentials stay separate from identity; identity\n is who acts. Memberships, roles, grants and audit records all reference the\n identity ID.\n\n- **Instance admin.** Operates the installation: orgs, the sign-in provider,\n instance policies. Instance admins see an org's resources only as members\n of it.\n\n## What belongs where\n\n| Level | Owns |\n| --------- | ---------------------------------------------------------------------------------------------------------------- |\n| Instance | the shipped workflow catalogue, instance policies |\n| Org | members, identities and their grants, secrets, agent profiles, workspaces, org workflows and their enabled state |\n| Workspace | sources, event types, mappings, bindings, runs, captures, run events and logs, chat sessions, memory |\n\nEvery owned record carries its `org_id`, and workspace records also carry their\n`workspace_id`. An identity, secret or profile is used in a workspace only\nafter the org grants it to that workspace.\n\nA secret is granted to an identity. An agent profile holds the container\nimage, tools and network access, and no secrets; this supersedes the secret\nreferences in `event-automation.md`'s agent profiles.\n\nA workflow names the identity it runs as. A workflow called by another runs\nas its own identity, and the caller's identity must be permitted `run` on it.\n\n## Access decisions\n\n### Where it lives\n\n- `internal/domain/access` owns the policy schema, the action list, how\n principals and resources are turned into policy entities, and the\n `Authorizer` contract.\n- The Cedar implementation, using `github.com/cedar-policy/cedar-go`, lives in\n `internal/infrastructure/access`; `internal/app` wires it.\n- Exactly two places call the `Authorizer`: the dashboard and API request\n path, and dispatch. Every other service verifies a credential and never\n evaluates policy.\n\n### The model\n\n- **Principal:** an identity, with its memberships, roles and grants.\n- **Resource:** the record acted on, with its workspace and org as parents,\n its type, owner and state.\n- **Action:** one of `read`, `create`, `update`, `delete`, `approve`, `run`,\n `read_logs`, `read_secret`, `manage_members`, `manage_identities`,\n `manage_policies`.\n- **Context:** the event's signature result, the address it came from, the\n time, and the run and step when the principal is an agent.\n\n### The policy chain\n\n```\ninstance (+ network rules) ─▶ org ─▶ workspace ─▶ identity + workflow ─▶ object\n```\n\n- **Instance** policies hold across every org. They include network rules:\n which addresses may deliver events to any source on the instance.\n- **Object** policies attach to one object (an event source, a secret, an\n identity, a binding, a workflow): \"only `soc-responder` may read this\n secret\", \"this source accepts events only from 10.20.0.0/16\".\n- A request is allowed only if every level with policies for it permits it. A\n lower level narrows and never widens; a forbid at any level wins; with no\n permit the request is denied.\n- The shipped role policies mean the org level always has policies, so an\n object or workspace policy alone can never grant access.\n- Archie ships an instance policy, which no org can remove, forbidding any\n principal to act on a resource outside the orgs it belongs to.\n\n### Storing and changing policies\n\n- Policies are stored in the State Store per level, versioned, and every\n change is an audit event naming who made it.\n- A policy is checked against the current schema when it is saved, and every\n stored policy is checked again when archie starts. An invalid org, workspace\n or object policy makes its level deny everything and is reported as a health\n issue naming the policy and the error. An invalid instance policy stops\n archie serving until it is fixed, with the same report.\n- A policy change that would leave no owner able to `manage_policies` is\n refused.\n\n### Roles\n\nArchie ships these org roles as policies. A member may hold a different role in\neach workspace.\n\n| Role | Can |\n| --------- | ----------------------------------------------------------------------------------- |\n| Owner | everything in the org, including members, identities, policies and deleting the org |\n| Admin | everything except deleting the org and changing owners |\n| Developer | create and edit workspace resources and org workflows; run workflows; read logs |\n| Viewer | read everything except secret values |\n\n### Approvals\n\nWhat needs approval, and who may give it, is org and workspace policy. Archie\nships one default: arming a binding needs a member other than the one who made\nthe change, holding the admin or owner role. No policy can let a change approve\nitself.\n\n### Denials\n\n- A lookup is filtered by org before any policy check, so a record in another\n org is never loaded and is reported as not found, with the same error the\n store returns for a record that does not exist.\n- A request denied inside the caller's org is reported as forbidden, without\n the reason.\n- Every denial is recorded with the principal, action, resource, the level\n that decided, and the deciding policies. Admins can see this record for\n their org; instance admins for instance policies.\n\n### Recovering from a locked-out org\n\n`archied access reset --org \u003corg\u003e` restores the shipped role policies for one\norg and removes its other org-level policies; `--instance` does the same for\ninstance policies. It runs only on the State Store host, never over the\nnetwork, and is recorded as an audit event.\n\n## Credentials\n\n### People and services\n\n- The State Store accepts two kinds of credential: the instance service\n credential, held by the daemon and Gateway for dispatch and background work\n across orgs, with every use attributed; and a principal credential issued to\n one identity, from which the State Store derives the org and workspace scope\n itself. No request field sets scope.\n- A person signs in through the provider or uses a personal API token issued\n to their identity. The shared token of a single-operator install\n authenticates as the owner of the `default` org.\n- If the sign-in provider is unreachable, sign-in fails with a health issue\n distinct from a rejected credential, `archied access reset` still works, and\n dispatch and running agents are unaffected.\n\n### Agents: decided once, carried as a run credential\n\n1. When a binding dispatches, the orchestrator evaluates the chain for the\n workflow's identity: the identity may `run` this workflow in this\n workspace, and the workflow and binding are enabled.\n2. It issues one run credential holding the org, workspace, run, workflow,\n identity, and the capabilities and secrets the identity is granted,\n narrowed by the workflow's policy. This single credential replaces the\n separate State Store and worktree grants.\n3. The State Store, forge RPC, worktree RPC and secret retrieval each verify\n every call against the credential and refuse anything outside it, recording\n the refusal on the run. None of them evaluates policy. The State Store's\n task-scoped calls stay limited to updating, transitioning and adding events\n to the credential's own run.\n4. The credential lives in the State Store, stored as a digest, so it survives\n a restart. It expires at the workflow's timeout plus a margin, is revoked\n when the run ends, and credentials of runs that are no longer running are\n revoked when archie starts. A run whose credential is refused as unknown is\n parked for retry, not failed.\n5. Starting an identity is recorded as an audit event: identity, run,\n workflow, the binding or member that started it, and the event behind it.\n The State Store stamps the actor, principal, org and workspace on every\n event from the credential and ignores those fields in the request.\n\nForge RPC subjects are keyed by the identity's immutable ID, so identities with\nthe same name in two orgs never share a responder.\n\n## Events and task identity\n\n- The capture receiver is the only place network rules are evaluated. A\n refused event is not stored; it is counted on its source with the address it\n came from (the most recent addresses only). A source that refuses more\n events than it accepts over an hour is reported as a health issue.\n- The task idempotency key is `org/identity/owner/repo/number`, produced in one\n place. The poller and the webhook receiver both resolve the org and identity\n before publishing, so the same issue delivered both ways still gives one\n key. The review-comment key follows the same rule. Task uniqueness in the\n store uses the same fields.\n\n## Upgrading existing installs\n\n- An install with no sign-in provider has one org, `default`, with one\n workspace, `default`, and the local operator as owner.\n- The upgrade runs in the State Store first, then in the event store, each\n resumable from a recorded version, after a backup of the database. Every\n existing record gets the `default` org and workspace, and every existing\n identity belongs to the `default` org.\n- The State Store refuses scoped calls until its upgrade has finished, rather\n than serving records with no org.\n- People's identities and memberships live in the State Store, not the event\n store.\n\n## Audit retention\n\nAudit and denial records are kept for a configured number of days. Repeated\nidentical denials within a minute are stored once with a count. Their size is\nshown on the health surface.\n\n## Delivery order\n\n1. Orgs, workspaces, people as identities, memberships, identities assigned to\n orgs; `org_id` and `workspace_id` on every owned record; the resumable\n `default` upgrade; the new idempotency key.\n2. `internal/domain/access` with the chain, shipped roles, the cross-org\n forbid and the startup re-check; applied to the dashboard, API and\n dispatch; denial records; `archied access reset`.\n3. Instance and principal credentials with server-derived scope; personal API\n tokens.\n4. Identity grants; the run credential replacing both existing grants; audit\n on every start.\n5. Approval policy with the shipped default.\n6. Instance network rules at the capture receiver, and object policies.\n7. Org, workspace and object policy editing.\n\n## Out of scope\n\n- Self-service org sign-up, billing and quotas.\n- Groups or teams as principals.\n- Agents acting with the permissions of the person who triggered an event.\n\n## Verification\n\n- A member of org A reading an org B record by ID gets the store's ordinary\n not-found error, and org B records never appear in org A's lists.\n- A workspace policy permitting what its org forbids has no effect.\n- A developer can create a binding and cannot approve it; an admin can.\n- A policy naming an unknown attribute is refused when saved; a stored policy\n that became invalid after an upgrade denies its level and shows as a health\n issue.\n- A policy change leaving no owner with `manage_policies` is refused, and\n `archied access reset --org` restores access for a locked-out org.\n- An identity not granted `secret:firewall-api` cannot retrieve it, and the\n refusal is on its run.\n- A run keeps working across a State Store restart, and a crashed run's\n credential is revoked at the next start.\n- Every run shows the identity it started and what started it.\n- An instance network rule refuses an event from outside its range for every\n org, and the refusal is counted on the source.\n- An object policy on a secret narrows which identities can read it.\n- A policy forbidding `run` where `environment` is `production` applies to\n every such workspace.\n- The same issue delivered by webhook and by poll produces one task.\n- A single-operator install upgrades with every record in `default` and works\n unchanged; an interrupted upgrade resumes.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/payload-field-mapping/",
|
||
"title": "Payload field mapping -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Payload field mapping -- decision\n\n**Status:** Approved\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-t2db.3`, depends on `t2db.1`, blocks `t2db.4`\n\nAnswers what a \"mapping\" is, where it lives, and what \"resolve against a\nreal event\" means precisely enough to implement. `t2db.4` (binding: matcher\n\n- mapping + workflow) is out of scope -- this bead ships the mapping\n concept and its preview only, as a standalone entity `t2db.4` will later\n reference by ID.\n\n## Decision: a new `internal/domain/mapping` package, SQLite storage\n\n\u003e Storage superseded: every store now lives in PostgreSQL behind\n\u003e `internal/domain/storecontract`, implemented in `internal/infrastructure/postgres`\n\u003e (`docs/prds/state-store-contract.md`). SQLite names below describe the\n\u003e original design.\n\n`internal/domain/mapping` owns the mapping vocabulary and the pure\nresolve/preview rule; `internal/store` persists it (mirrors\n`captures.go`'s shape exactly -- inline schema const, narrow store\ninterface, `*store.Store` implementation); `internal/webui/api_mapping.go`\nis the transport. This follows `docs/prds/event-capture-storage.md`'s own\nplacement precedent: `internal/store` is where this daemon's persistence\nalready lives, not `internal/infrastructure` (that layer holds\ncross-domain services -- config, eventbus, modelcatalog -- not this\ndaemon's primary store).\n\n### Why a domain package and not just a store table\n\nResolving a path against a payload and deciding \"missing vs null vs\ntype-changed\" is a rule, not a query -- CLAUDE.md's domain layer owns\n\"language, state, rules, operations\" precisely so this logic is testable\nwithout a database and reusable by `t2db.4`'s runtime trigger path without\neither package importing the other's transport concerns.\n\n## Mapping shape\n\n```go\ntype Mapping struct {\n\tID int64\n\tName string\n\tSourceHint string // free-text label (e.g. \"sentry\"), organisational only -- not enforced against Capture.Source\n\tFields []Field\n\tCreatedAt time.Time\n\tUpdatedAt time.Time\n}\n\ntype Field struct {\n\tName string // the name the workflow receives\n\tPath string // dot/bracket path into the payload, see below\n\tType FieldType // expected type, checked at resolve time\n\tRequired bool // Required=false + missing is not a failure; Required=false + wrong type still is\n}\n\ntype FieldType string // \"string\" | \"number\" | \"bool\" | \"object\" | \"array\" | \"any\"\n```\n\n`SourceHint` is advisory only: an operator authors a mapping by clicking\nthrough one example capture, but the mapping is not pinned to that\ncapture's exact source string, because `t2db.4`'s matcher (not this bead)\ndecides which events a binding applies to. Pinning here would duplicate\nthat decision in the wrong layer.\n\n## Path syntax\n\nDot-separated keys with `[N]` for array indices: `data.items[0].id`. No\nwildcards, no filters, no external library -- the grammar is exactly what a\nclick-through UI needs to express (this key, then this key, then this\nindex), and every path in a mapping was generated by clicking a value in a\nreal payload, never hand-written against a spec. A hand-rolled parser\n(~30 lines: split on `.`, strip `[N]` suffixes) is less risk than a new\ndependency for a grammar this small.\n\n## Resolve semantics (the preview AND the future runtime path)\n\n```go\nfunc Resolve(fields []Field, payload []byte) (values map[string]any, failures []Failure)\n\ntype Failure struct {\n\tFieldName string\n\tPath string\n\tReason string // \"missing\" | \"null\" | \"type_mismatch\"\n}\n```\n\nDecode with `json.Decoder.UseNumber()` (matches `webhookguard.RedactPayload`'s\nexisting convention) so int/float distinction survives for type checking.\nWalk each field's path independently -- one field's failure does not stop\nresolution of the others, because the whole point of preview is showing\nthe operator every problem at once, not the first one.\n\n- Path segment doesn't exist, or indexes past an array's length -\u003e `missing`\n- Path resolves to JSON `null` -\u003e `null`\n- Path resolves to a value whose JSON type doesn't match `Field.Type`\n (`\"any\"` always matches) -\u003e `type_mismatch`\n- `Required: false` suppresses `missing` only, never `null` or\n `type_mismatch` -- a field that's present but the wrong shape is always\n worth surfacing, since silently coercing it is exactly the \"agent\n confabulates around a blank field\" failure this bead exists to prevent.\n\n**This is the same function both consumers use.** `t2db.3`'s preview calls\nit against an operator-picked capture and renders every result, pass or\nfail, in the UI -- that satisfies this bead's \"fail loudly\" acceptance\ncriterion, because at this stage the only runtime consumer of a mapping\n_is_ the human previewing it. `t2db.4` will call the identical function\nwhen a live event triggers a binding and turn any non-empty `failures` into\na visible operator-facing failure (park the task / surface in the\ninspector) instead of starting a workflow with blank fields -- that wiring\nis `t2db.4`'s job, not duplicated here.\n\n## Storage\n\nThe `mappings` table: `id, name,\nsource_hint, fields_json, created_at, updated_at`. `fields_json` is the\n`[]Field` slice JSON-encoded into one column -- fields have no independent\nquery need (nothing filters \"mappings with a field named X\"), so a\njoin table would be structure with no consumer, the same failure mode\n`organisation.md` bans for empty layering.\n\n`MappingStore` interface (narrow, mirrors `CaptureStore`):\n`InsertMapping`, `ListMappings`, `GetMapping`, `UpdateMapping`,\n`DeleteMapping`.\n\n## HTTP surface\n\nAll under `/api/mappings`, token-gated like every other `/api/*` route (no\nbypass -- unlike capture, nothing here accepts unauthenticated input):\n\n- `GET /api/mappings` / `POST /api/mappings` -- list / create\n- `GET|PATCH|DELETE /api/mappings/{id}`\n- `POST /api/mappings/preview` -- body `{fields: []Field, capture_id:\nint64}`, calls `mapping.Resolve` against that capture's stored (already\n redacted) `Body` and returns `{values, failures}`. Deliberately takes\n loose `fields` rather than a saved mapping ID, so an operator can iterate\n before saving anything -- \"never ship a mapping the operator has not seen\n resolve against real data\" requires preview to work pre-save.\n\nPreview resolves against `CapturedEvent.Body`, which is already\nredaction-passed (`webhookguard.RedactPayload`, per\n`event-capture-storage.md`) -- a mapped field pointing at a\nsensitive-keyed path previews as the literal string `\"[redacted]\"`, not the\nreal value. This is accepted, not fixed here: it is the stored ground\ntruth, the same payload `t2db.4`'s runtime path will resolve against, and a\npreview mismatch would be worse than an honest one.\n\n## Frontend\n\nNew `ui/src/mappings/` feature folder, same three-file split as\n`ui/src/captures/`: `mappings.js` (page: list, fetch, create/edit flow),\n`mapping-editor.js` (pure, unit-testable: renders a captured payload as a\nclickable tree -- clicking a leaf appends a field row with `Path`\npre-filled -- plus the field table and preview-result rendering),\n`mappings.css`. Creating a mapping starts from an existing captured event\n(reuses `api.captures`), so the editor always has a real payload to click\nthrough and preview against, never a blank form.\n\n## What this does not decide\n\nThe binding entity, its matcher, its draft/pending_approval/armed state\nmachine, and turning a runtime `Resolve` failure into a parked task are\n`t2db.4`. Per-source HMAC secret registration is `t2db.5`. This bead ships\nthe mapping concept, its storage, and its preview only.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/playbook-expression-syntax/",
|
||
"title": "Playbook data-flow and condition syntax",
|
||
"section": "PRDs",
|
||
"body": "# Playbook data-flow and condition syntax\n\n**Status:** Approved\n**Date:** 2026-09-03\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-t2db.14` (expression environment).\nUnbuilt decisions: `archie-core-t2db.25` (J1 action ids),\n`archie-core-t2db.26` (J2 args as CEL),\n`archie-core-t2db.27` (Result schemas declared to CEL).\n\n**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression\nmechanism covers both an action's `when` condition and its `args` values.\n\n## The requirement, stated directly\n\nA playbook action is gated and parametrised by runtime data: (a) a condition\ndeciding whether the action runs; (b) a later action's `args` values referencing\nan earlier action's `Result` and the triggering event's fields. Both are **CEL\nexpressions**, evaluated against one evaluation context. There is no separate\ninterpolation syntax: CEL reads nested maps and fields natively, so `args`\nvalues that need data are written as CEL, and only plain literal args remain\nplain YAML scalars.\n\nBoth halves are read-only: expressions may read context values and compute a\nresult from them, never mutate, never call host functions, never perform I/O.\nCEL enforces this structurally (no side effects, linear evaluation when macros\nare bounded, cost-limited).\n\n```yaml\nactions:\n - id: build\n position: module\n kind: log\n args:\n message: '\"build finished\"' # literal\n when: 'event.label == \"bugfix\"' # gate on event field\n - position: module\n kind: log\n args:\n message: '\"done: \" + string(actions.build.result.written)'\n when: \"actions.build.result.written == true\"\n```\n\n## Trust boundary of the expression\n\nA playbook YAML is **operator-installed, in-process, daemon-privileged** -- the\nsame tier as `ModuleDir`/`PluginDir`/`SecretEngineDir` (module-position.md's\ntrust-boundary section): loaded from a configured directory at startup, never\nfrom a webhook body or task worktree. The schema-by-example flow uses live\nevents to _design against_; the saved playbook is operator-authored file content\nin an operator-configured location.\n\nTwo consequences:\n\n1. The expression string is **trusted**. Operator-authored config, not attacker\n input; no sandboxing against a hostile expression string is required. If a\n future playbook source is less trusted (auto-generated bindings from\n unauthenticated captures), that is the playbook-file trust tier moving -- the\n loader's acceptance check is the right place, not the evaluator.\n2. The **data the expression reads is NOT trusted** (decoded webhook body, prior\n results through the same pipeline). Robustness against hostile data is\n required: no panic on wrong-typed/missing fields, no unbounded cost on\n deep/cyclic maps, bounded evaluation time. CEL provides this structurally\n (cost limits, linear non-Turing-complete evaluation, error values instead of\n panics -- verified in the vetting record below).\n\n## The gate precedent is deliberately NOT copied\n\n`internal/gate/gateeval` evaluates arbitrary interpreted Go (`.archie/gate.go`)\nas the repo's conditional-logic precedent. That is the wrong precedent for\nplaybook conditions because this document's premise is: **YAML is pure\norchestration data; Yaegi is where imperative logic lives** (Problem section;\nmodule-position.md's \"no generic hook\" section). Yaegi snippets in YAML would\nmove imperative logic into the data layer -- the inversion this design exists to\nprevent -- and make playbooks un-lintable.\n\nThe line: conditions are **declarative predicates over runtime data, not\nprograms**. CEL is non-Turing-complete and side-effect-free, so it cannot become\na program; a condition that needs real logic gets a Module kind (Yaegi),\nreferenced by the action, not a stronger expression body.\n\n## Plugin-engine-rule interaction\n\nThe plugin engine rule forbids a generic `Module` interface with an `any`\npayload; it exists to keep static implementation contracts typed. Evaluating CEL\nagainst runtime `map[string]any` is **not the same problem**: the rule concerns\n_implementation contracts_ (static operations a capability family exposes), not\n_runtime data values_ (legitimately dynamic -- an event payload's shape is\nunknown until it arrives). The Module contract stays fully typed (generated\n`Args`/`Result` schemas); CEL is a read-only data reader under the playbook\nengine family, not a new capability contract. Stated once.\n\n## Minimum viable evaluation context (what expressions see)\n\nOne flat context, namespaced from the playbook run, built at dispatch time:\n\n| Name | Type | Source |\n| --------- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `event` | `map(string, dyn)` | The triggering event's decoded payload (webhook body / forge issue / schedule tick). Field access via `event.\u003cfield\u003e`, map access via `event[\"field\"]`, presence via `has(event.\u003cfield\u003e)`. |\n| `actions` | per-playbook object type | Previous actions' results, one field per declared action `id` (`actions.\u003cid\u003e`), each field typed `{ result: \u003cKindResult\u003e }` so a later action reads an earlier one's `Result` struct (`actions.\u003cid\u003e.result.\u003cfield\u003e`) with field-level checking. The current action and later actions are not present. |\n\nExpressions may also read literal-only state (numbers, strings, booleans)\ndirectly. CEL's `has()` macro covers the missing-key case (`has(event.labels)`\n-\u003e bool), which replaces any bespoke `has`/ `contains` operator.\n\n`actions.\u003cid\u003e.result` is the kind's hand-written `Result` struct:\n`ModuleRegistry.DecodeResult` converts `Module.Invoke`'s flat `map[string]any`\n(`archie-core-t2db.13`) back into that struct before a later expression reads\nit, so the checker and the run read the same typed value.\n\nJudgement call J1: `action` id is a required field on every action that later\nactions reference; ids are validated at load (unique, stable-identifier-shaped),\nand a `when`/`args` expression referencing an unknown `actions.\u003cid\u003e` is a **load\nfailure**, not a runtime miss -- the same reject-at-load rule as collisions (see\nbelow).\n\n## Load-time validation (consistent with the collision rule)\n\nThe collision-handling rule is: schema defines the accepted message; anything\noutside it is rejected at load, logged plainly, reported to the caller -- never\ndeferred to runtime. Expressions are validated the same way, at playbook load:\n\n- **Syntax**: every `when` and every CEL-typed `args` value is parsed and\n checked by CEL's checker when the playbook is loaded. A syntax or type error\n is a reported load failure (the playbook is dropped and the error goes to the\n caller), not a runtime surprise.\n- **Context conformance**: with the context declared to CEL (`event` as\n `map(string, dyn)`, `actions` as a per-playbook object type with one typed\n field per declared id), unknown top-level identifiers are rejected at compile\n time. Field-level typing inside `dyn` values is deferred to runtime by CEL's\n design (verified: `data.items.lenght` with `data` dyn compiles clean and\n errors at eval) -- so the loader declares as much type as the schema provides\n (the action kinds' generated `Result` structs become CEL type declarations at\n load, giving field-level rejection for result reads), and `event` stays `dyn`\n because its shape is unknown by design (schema-by-example).\n- **Names**: unknown `actions.\u003cid\u003e` references and unknown roots are\n compile-time/lint-visible errors (J1).\n\nThis means the lint tool and the daemon share one checker -- same single-source\nargument as the playbook loaders.\n\n## Candidate comparison\n\n| Candidate | Maintenance | Safety surface | Cost (measured) | Verdict |\n| ------------------------ | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------ |\n| **CEL (cel.dev/cel-go)** | Actively maintained (google/cel, monthly releases) | Non-Turing-complete, linear eval, `CostLimit`/`CostTracking`/`ParserRecursionLimit`, no side effects, error values not panics | **+13.0MB** (2.34MB -\u003e 15.25MB) | **Chosen** |\n| `expr-lang/expr` v1.17.8 | Actively maintained | Side-effect-free, `DisableAllBuiltins`, depth limits (CVE fixed w/ tests) | +3.9MB (2.34MB -\u003e 6.28MB) | Not chosen: two safety CVEs this cycle; smaller ecosystem; no protobuf/type-model fit with future schema-gen |\n| `gval` v1.2.3 | Maintained, slower | No builtin-restriction surface; extras (ternary/`??`) not wanted | +3.0MB (2 transitive deps) | Not chosen |\n| `govaluate` | **ARCHIVED** (author archived 2024) | n/a | n/a | Not chosen: dead |\n| `starlark-go` | Maintained (Bazel) | Turing-complete with step limits | large (full language) | Not chosen: whole-language surface |\n\n## Vetting record\n\nSource reviewed at HEAD:\n\n- **Repo hygiene**: clean. No binaries/media in tree (the only non-source\n artifacts are expected generated protobuf files); 16-line `.gitignore`.\n- **Hard-boundary correctness**: the place correctness is hard for an expression\n evaluator is bounding evaluation and surviving hostile input. Dedicated\n regression tests exist and pass at HEAD: `TestCostLimit`, `TestCostTracking*`,\n `TestParserRecursionLimit`, and a large conformance suite. Verified by\n execution: evaluation against a 2000-deep nested map under `CostLimit`\n completes in ~10 microseconds and returns an error value for a missing key --\n no panic, no unbounded walk.\n- **Dependency footprint**: antlr (parser), protobuf + genproto (CEL's type\n model), `cel.dev/expr` (CEL spec types), yaml.v3. All mainstream, canonical,\n vendored widely by Google-adjacent projects. No runtime-impure third-party\n code; the REPL-only deps (readline, tview, tcell) are not in the library\n import graph.\n- **Security record**: no advisories on record for `cel.dev/cel-go` (GitHub\n Advisory Database).\n- **Module path**: migrated to `cel.dev/cel-go` (v0.32.0 is current); the old\n `cel.dev/cel-go/cel` path is the pre-migration identity. Pin `cel.dev/cel-go`.\n- **Binary size cost**: +13.0MB is the largest of the candidates (measured:\n baseline 2.34MB -\u003e 15.25MB with CEL). Accepted because: the daemon already\n embeds a large Yaegi runtime; CEL's type model is the load-bearing win\n (compile-time rejection of result-path typos, matching the reject-at-load\n rule); and the alternative bespoke grammar would re-implement CEL's checker to\n get the same guarantee. Flagged for sign-off: if 13MB is unacceptable for a\n specific deployment target, the fallback is a bespoke grammar with a\n re-implemented subset of this checker -- a strictly worse trade.\n\n## Why bespoke was not sufficient\n\nA bespoke grammar for the shipped cases (path read + comparison + bool\ncomposition) is small, but the load-time-validation requirement makes the\ncomparison: the reject-at-load rule requires _field-level_ static checking of\nresult paths against generated `Result` schemas, which is CEL's type-checker's\njob. Re-implementing that on a hand-rolled grammar means re-implementing a type\nsystem -- more code than the grammar itself, with worse diagnostics, no\nconformance suite, and no community surface for future expression needs (even\ndeclarative ones). The bespoke option only wins on binary size, and the size\ncost of CEL is measured and bounded.\n\n## Judgement calls (flagged for sign-off)\n\n- **J1 -- `id` required on referenced actions, unknown id = load failure.**\n Matches the reject-at-load rule. Alternative considered: an alternative is\n positional `actions[0]` access, which breaks on reorder; named is the\n operator-friendlier shape.\n- **J2 -- every `args` value is evaluated as a CEL expression.** No split\n between \"literal args\" and \"expression args\": a plain string value is a CEL\n string literal, a number is a CEL number literal, and any value that reads\n context data is written in CEL. One mechanism, no interpolation marker, no\n second syntax to learn. Alternative considered: cEL string literals must be\n quoted inside YAML (e.g. `message: '\"build finished\"'`), which is slightly\n noisy for mostly-literal args; accepted for uniformity -- the linter catches\n quoting mistakes at load.\n- **J3 -- condition failure semantics.** A condition that errors at runtime\n (missing/wrong-typed field) evaluates to **false**: the action is skipped,\n logged, run continues. This is the simplest candidate consistent with gap 1's\n bias against clever recovery; gap 1 (`archie-core-t2db.18`) commits\n stop-on-first-failure, so per-action control is not coming and J3 stands.\n Alternative considered: cEL separates false from error; we treat evaluation\n error as false + log.\n- **J4 -- `event` stays `dyn` (schema-by-example).** Load-time we cannot know\n event field types; declaring them typed would reject valid plays. Result reads\n get field-level checking via generated schema declarations. Alternative\n considered: when schema-by-example ships, tighten event field declarations\n from captured samples.\n- **J5 -- cost limit default.** A `CostLimit` at load time (e.g. 100,000) is\n applied to every expression; the linter and daemon use the same value so they\n agree. Alternative considered: exact number is a constant to tune at first\n use.\n\n## Interaction with the execution-time gaps\n\nThis resolution does not resolve the execution-time gaps (both are resolved\nseparately: gap 1 by `archie-core-t2db.18`, gap 2 by the gap-2 decision).\nConditions make gap 1 concrete (condition-failure vs. action-failure semantics,\nJ3) and CEL's read-only result access makes gap 2's keying derivation readable\n(an idempotency key expression can read event fields) -- both flagged as the\ngaps' first real exercise. Gap 2's exercise is subsequently solved by that\ndecision: the answer is a fixed structural derivation, and idempotency keys are\ndeliberately NOT CEL expressions (see the \"No CEL in the key\" passage).\n\n## Implementation placement (for the follow-up slice)\n\n- New `internal/domain/eda/expr` package: builds the CEL environment (context\n declarations, generated `Result` schema types as CEL type declarations, cost\n limit), compiles/validates playbook expressions at load, evaluates them at\n dispatch. Domain layer; no infrastructure imports; owned by the playbook\n engine family. One new dependency: `cel.dev/cel-go` (pinned), `go.mod`\n otherwise unchanged.\n- Consumption: the playbook event coordinator (open question 4's loader)\n evaluates `when` before dispatch and evaluates `args` values before\n `Module.Invoke`; the lint tool (`archie-core-t2db.12`) shares the same\n compile/validate path for author-time diagnostics.\n- Schema integration: action kinds' generated `Result`/`Args` structs\n (`archie-core-t2db.13`'s `logextract` pattern) are declared to CEL at load so\n `actions.\u003cid\u003e.result.\u003cfield\u003e` type-checks; `args` expressions type-check\n against the target kind's `Args` schema before the playbook is accepted.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/pr-review-agent/",
|
||
"title": "PR review agent",
|
||
"section": "PRDs",
|
||
"body": "# PR review agent\n\n**Status:** Approved\n**Authority:** `docs/architecture/agent-system.md`; supersedes the reviewer\nexecutor and blocking rule in `docs/architecture/adversarial-review.md`, and\nthe designs in `docs/prds/operator-pr-review.md` and\n`docs/prds/review-calibration.md`.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (every agent\ncall is a StepExecution), `docs/prds/external-agent-harness.md` (any agent\nstep may run on a harness), `docs/prds/inline-review.md` (posting),\n`docs/prds/pr-review-remediation.md` (consumes the findings).\n\n## Decision\n\nArchie ships a pull request reviewer as its first end-to-end product. It\nreviews any pull request on a watched repository, a PR an operator names, and\narchie's own PRs before they open.\n\nThe reviewer is a fixed pipeline, not a single agent. The review strategy is\ngenerated per PR: three lens agents propose review dimensions from the PR's\ncontent, one reviewer definition runs once per dimension, and later agents\nverify, challenge and combine the findings. Everything that can be computed\nis code: diff parsing, clustering, blast radius, evidence extraction,\nscoring, deduplication, line mapping and the review event.\n\nThe pipeline is **recall-first**. It posts every evidenced finding above the\nconfidence threshold, up to the comment cap. A precision gate exists as an\noperator dial and is off by default.\n\nAcceptance is measured, not asserted: see Verification.\n\n## Pipeline\n\nEach numbered phase is a stage of the `pr-review` workflow. Each agent call is\nan `agent` StepExecution; fan-out phases record one child per call.\n\n1. **Intake.** Code reads the PR metadata and diff statistics: type,\n complexity, languages, areas touched, risk signals. Depth comes from lines\n changed: under 100 is `quick` (at most 3 dimensions), under 500 is\n `standard` (6), otherwise `deep` (12). An operator may set depth\n explicitly.\n2. **Anatomy.** Code parses the diff into files and hunks, clusters related\n files, and computes the blast radius: unchanged files that import or are\n imported by changed files. One agent writes the PR narrative, risk\n surfaces, changes that do not fit the narrative, and gaps between the\n description and the diff.\n3. **Dimension selection.** Three lens agents run in parallel over the intake\n (including its machine-written score), anatomy and diff, each able to read\n the repository. A PR that intake scores as likely machine-written adds a\n hallucination-check dimension: claims, citations and behaviour the diff\n does not actually implement.\n - **behaviour:** where the old and new code diverge for any input; API\n contracts, concurrency and state, security, error handling, data flow;\n - **mechanics:** whether the code runs at the language and framework\n level; types, signatures and every caller, decorators and middleware,\n framework contracts, imports;\n - **fit:** patterns, complexity, abstraction, test adequacy, documentation\n that no longer matches, dependencies, migration completeness.\n\n Each lens returns review dimensions: a name, a reviewer prompt written for\n this PR, target files, context files and a priority. Code merges them,\n drops duplicates and keeps the highest-priority dimensions up to the\n depth's cap.\n4. **Review.** One reviewer definition runs once per dimension, in parallel,\n at most 8 at a time. Each reads its target and context files and may\n follow references. Each finding carries file, line range, severity\n (`critical`, `important`, `suggestion`, `nitpick`), title, body, an\n optional suggested fix, quoted evidence, confidence and tags. A reviewer\n that exhausts its turn cap or deadline before its terminal tool call\n records a distinct StepExecution status from a reviewer that completed\n and found nothing; synthesis and the posted review treat the two\n differently (the former is unreviewed, the latter is reviewed-clean).\n5. **Verification layer**, over the review findings:\n - code extracts an evidence package per finding: the code at the cited\n lines and snippets from its callers;\n - an evidence verifier checks high-priority findings against their\n packages;\n - an adversary confirms or challenges each finding, looks for issues every\n reviewer missed, and is more sceptical when intake scores the PR as\n likely machine-written;\n - a compound agent looks, within each cluster of confirmed findings, for\n defects that only exist in combination.\n6. **Coverage and consistency**, in parallel:\n - a coverage gate checks every cluster and high-exposure blast-radius file\n was reviewed, and runs gap reviewers for what was not, at most 2 rounds;\n - consistency verification lists every cross-location obligation the\n changed code creates (what must be true elsewhere for this line to be\n correct), then verifies each by reading both ends. A broken obligation\n becomes a finding.\n7. **Synthesis**, code only:\n - score = severity weight (1.0, 0.7, 0.3, 0.1) × confidence × multipliers:\n compound 1.5, adversary-confirmed 1.3, adversary-challenged 0.5, likely\n machine-written PR 1.2, blast radius over 10 files 1.2;\n - drop findings under the severity's confidence floor: critical 0.2,\n important 0.3, suggestion 0.4, nitpick 0.4;\n - merge exact duplicates (file, overlapping lines, category) in code, and\n near-duplicates through a small classification call that keeps both\n when unsure;\n - map lines to the diff's coordinates, sort by score, cap at 25 inline\n comments.\n8. **Merge gate.** One classification call per finding decides blocking or\n advisory: blocking only for broken builds, security, data loss, contract\n breaks and regressions. A failed call means advisory.\n9. **Output.** One call per comment tightens its wording, running in a\n bounded pool at most 8 at a time (matching phase 4), keeping the original\n on failure. The review is posted as one forge review with inline\n comments. Any blocking finding requests changes; otherwise the review is a\n comment. Archie never posts an approval.\n\n## Precision dial\n\nWith `review.precision_gate = true`, a post-worthiness pass runs after phase 4\nand before the verification layer. It keeps every concrete, evidenced defect\nand drops nitpicks, style and unverifiable claims, keeping a finding when\nunsure. It trades recall for precision and cuts verification cost by\nfiltering early.\n\n## Operator approval\n\nWith `review.approve_before_post = true`, synthesis ends in\n`waiting_human`. The operator chooses which findings to post, rejects the\nreview, or asks for a re-review with instructions. A re-review reruns phases\n3 to 8 with the instructions added to the lens and reviewer prompts, at most\n2 times. A review with no findings posts nothing and does not wait.\n\n## Isolation\n\nEvery agent in the pipeline reads a `.git`-free snapshot of the PR head, with\nread-only tools rooted at it, and never the implementer's worktree or\ntranscript. No pipeline agent holds a forge credential; posting is code.\n\n## Budget\n\nA run has a cost cap and a wall-clock cap, split into per-phase shares. A\nphase whose share is spent is skipped and recorded on its StepExecution. The\nposted review names every skipped phase, so an exhausted run never reads as a\nclean one. A run whose review stage fails posts nothing and\nfails.\n\n## Triggers\n\n- **Watched repositories:** a PR opened or updated on a repository with\n `review.enabled` starts a run. An update cancels the in-flight run for the\n same PR.\n- **Operator:** a PR URL from the dashboard or chat starts a run on any\n repository the identity can read.\n- **Archie's own PRs:** the implement workflow runs the pipeline before\n opening its PR. A blocking finding the adversary did not challenge parks\n the task with the findings. A pipeline that fails parks the task. Advisory\n findings are posted after the PR opens.\n\n## Organisations\n\nThe reviewer is a workflow, so `docs/prds/orgs-and-access.md` applies to it\nunchanged: it runs as the workflow's identity under the run credential, reads\nonly repositories that identity may read, and posts through the forge RPC\nunder that credential. A watched repository belongs to a workspace, and its\nPR events start reviews through that workspace's bindings. Findings,\nStepExecutions and the posted review carry the run's org and workspace.\n\n## Models\n\nEach agent phase names a model role. Lens, reviewer, verification,\nadversary, compound and consistency agents use the review role. Intake,\ncoverage, dedup, merge gate and polish calls use the classification role.\nEither role may run on the built-in runner or a harness.\n\n## Out of scope\n\n- SARIF and standalone report formats. The findings are StepExecution outputs\n and the forge review.\n- Reviewing a diff without a forge PR.\n- Self-consistency reruns.\n\n## Plan\n\n1. Deterministic core: diff parsing, clustering, blast radius, evidence\n extraction, scoring, dedup, line mapping, review event. Table-driven tests.\n2. The benchmark harness (Verification), run against the deterministic core\n with fixture findings.\n3. `pr-review` workflow: intake, anatomy, lenses, reviewers, synthesis,\n output, with the execution tree recording each call.\n4. Verification layer, coverage loop and consistency verification.\n5. Merge gate, polish, budget shares and skipped-phase reporting.\n6. Benchmark acceptance run.\n7. Triggers: operator, watched repositories, archie's own PRs replacing the\n single-reviewer stage.\n8. Precision dial and operator approval.\n\n## Verification\n\n- **Benchmark acceptance.** On the 38 runnable PRs of the Martian\n Code-Review-Bench offline set, one blind run per PR at `deep` with the\n review and classification roles on GLM-5.2, and an independent judge\n (`anthropic/claude-sonnet-4.6`) matching posted comments to golden\n comments: golden micro-recall is at least 0.70. Golden-only precision is\n reported alongside it. The run is reproducible from a script in the\n repository.\n- Scoring, thresholds, dedup and line mapping give identical output for\n identical findings.\n- A run whose phase share is exhausted posts a review naming the skipped\n phase.\n- An update to a PR under review cancels the running review and its child\n steps.\n- No pipeline agent can write to the snapshot, and none can reach a forge\n credential.\n- Archie's own PR with an unchallenged blocking finding is parked, not\n opened.\n- Archie never submits an approving review.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/pr-review-remediation/",
|
||
"title": "PR review comment remediation -- decision",
|
||
"section": "PRDs",
|
||
"body": "# PR review comment remediation -- decision\n\n**Status:** Approved\n**Date:** 2026-09-10, amended and ratified 2026-09-19\n**Beads issue:** (filed as `archie-core-…`, parent: `archie-core-7d5u` event sources and typed reactions)\n\nAnswers: how archie reacts to review comments on a PR it opened, from a human\nor from another review bot, and remediates them. Built on two settled decisions — the adversarial review\nepic's isolation contract (`docs/prds/adversarial-self-review.md`) and the\nevent-sources decision (`docs/prds/event-sources-and-reactions.md`) — plus the\nwebhook intake security contract (`docs/prds/webhook-intake-security.md`).\nNothing here re-decides those; it names the one concrete reaction each one's\nmachinery was built to carry.\n\n## Problem\n\nArchie opens a PR (`pr_open`) and then never learns what a human said about\nit. `forge.Forge` has no method to list PR reviews or review comments; the\ndaemon poll loop finds only new issues; the forge webhook receiver\n(`internal/forge/webhook/receiver.go`) decodes only `issues` events and\nexplicitly ignores `pull_request` events. A \"requested changes\" review, or an\ninline comment on a line, is invisible work.\n\n## Decisions\n\n### 1. Read surface: a narrow `PullRequestReviewReader`, type-asserted\n\nA new forge interface, implemented by GitHub and Gitea, not folded into the\nfat `Forge` interface (the h019.8 `PullRequestReader` precedent — the noop\nforge cannot honestly provide it):\n\n```go\ntype ReviewComment struct { ID int64; Author, Body, Path string; Line int; InReplyTo int64; CreatedAt time.Time }\ntype Review struct { ID int64; Author, State string /* requested_changes | approved | commented | dismissed */; SubmittedAt time.Time }\n\ntype PullRequestReviewReader interface {\n ListReviews(ctx, owner, repo string, number, sinceID int64) ([]Review, error)\n ListReviewComments(ctx, owner, repo string, number, sinceID int64) ([]ReviewComment, error)\n // ReplyToReview posts a reply to a review or comment thread.\n ReplyToReview(ctx, owner, repo string, number, inReplyTo int64, body string) error\n}\n```\n\nGitHub: `pulls/{n}/reviews` + `pulls/{n}/comments`; reply via\n`pulls/{n}/comments/{id}/replies`. Gitea: `pulls/{n}/reviews` +\n`pulls/{n}/comments`; reply via `pulls/{n}/comments/{id}`. `sinceID` is the\ncursor for both poll and webhook dedup.\n\n### 2. Event source: webhook first, poll as the universal backstop\n\n- **Webhook (GitHub, low latency):** extend `receiver.go` to also decode\n `pull_request_review` and `pull_request_review_comment` events, behind the\n existing HMAC verification, producing the same reaction as the poller. The\n receiver stays a forge-intake surface — not the generic chat webhook, not a\n new generic receiver (7d5u decision 4).\n- **Poll (both forges, no webhook dependency):** a periodic scan of `pr_open`\n tasks calls `ListReviews`/`ListReviewComments` with a persisted `sinceID`\n cursor per task. This is the Gitea path and the deployment-without-webhook\n path; it must stay first-class, not a legacy fallback (the repo's own\n webhook-intake rule). It is **one shared scan over all `pr_open` tasks**,\n not a scan inside each identity's poll loop, and it calls the forge client\n the scanned task's own identity owns. This is `reconcilePRs`'s shape\n exactly (`internal/daemon/daemon.go`: `Store.OpenPRs` then `forgeFor(\u0026t)`).\n A per-identity scan would call one identity's forge client against another\n identity's PR.\n\nBoth paths emit the **same** typed reaction, so `PublishUnique` dedups them\nfor free — the `TaskEnvelope.IdempotencyKey` pattern (7d5u decision 4) applied\nto reviews.\n\n### 3. Reaction contract: producer-only, resolves to an existing task\n\nA new `workintake.ReviewCommentEnvelope{Owner, Repo, PRNumber, Kind, CommentID,\nReviewID, Author, State, Body, Path, Line}`. Idempotency key is\n`owner/repo/pr-number/kind/id` — source-independent, so a webhook delivery\nand a poll of the same comment collapse.\n\n`Kind` (`review` or `comment`) is part of the key because a review ID and a\nreview-comment ID come from independent forge sequences. Keying on the bare id\nlets a review and an inline comment on the same PR collide, silently dropping\none of the two. This is a producer-only reaction\n(7d5u decision 3): it never vetoes or mutates in-flight work; the target task\nis idle (`pr_open`).\n\nThe daemon resolves `PRNumber` → the task that owns it. An unmatched PR (not\nan archie task) is dropped with a counter — a public webhook must not be able\nto start an agent against an arbitrary repository. A PR whose task is merged,\nclosed or archived is dropped through the same counter: the reaction is only\never work against a PR archie itself opened _and still owns_.\n\nThat lookup does not exist. `pr_number` is written by\n`internal/infrastructure/postgres/store.go` and never read in a `WHERE` clause, so this needs a\n`TaskStore` method keyed on `(owner, repo, pr_number)`, a matching\n`StateStoreService` RPC (a Go interface method on a store facade without one\nis forbidden, `CLAUDE.md`), and an index on those three columns. It is the\nauthorisation boundary, so it is sequenced as its own step below rather than\ncarried along with the envelope.\n\n**Identity is derived from the resolved task, never carried in the reaction.**\nThe task row already stores `identity` (`internal/infrastructure/postgres/store.go`), and it is\nthe resolved task's identity that selects the forge client, worktree manager,\nrepo config and `bot_user` for the remediation run and the `ReplyToReview`\nreply, through the existing `forgeFor`/`treesFor`/`repoFor` helpers. That is\nwhat decision 5's \"the identity that owns the task\" means, and it is the only\nplace identity enters this path.\n\nIdentity is deliberately **not** added to the envelope, the idempotency key or\nthe index:\n\n- A PR number is unique within an `owner/repo`, and the tasks table is keyed\n `UNIQUE(owner, repo, issue_number)` with no host or identity column, so\n archie already treats `owner/repo` as globally unique across identities. A\n `(owner, repo, pr_number)` lookup cannot return a task other than the one\n that owns that PR; there is no cross-identity resolution to guard against.\n- The webhook producer has no identity to carry: webhook intake refuses to\n start when `[[identities]]` is configured (`setupForgeWebhook`,\n `internal/app/archied/bootstrap.go`).\n- Keying on a field only one of the two producers can populate is precisely\n the delivery-source-specific key `CLAUDE.md` forbids, and it would break the\n webhook/poll dedup the key exists for. An index column that never appears in\n the `WHERE` clause is dead weight.\n\n### 4. Remediation: a `remediate` workflow on the existing branch\n\nA new workflow that runs when a reaction resolves to an archie-owned PR. It is\nnot a new branch/PR — it reuses the task's prepared worktree, refreshes it\nonto the PR branch (the `worktree.Manager.refresh` idempotency), and:\n\n1. Runs a builder agent whose mission is the review's actionable comments\n (each with body, path, line): \"Address these review comments with the\n smallest change; run the gate.\" One agent run per review, per decision 5. The toolset is the normal builder toolset — edit + gate + (never\n git; the orchestrator commits/pushes) — not the reviewer's read-only set.\n2. Runs the repo gate (`[[repos.gate]]`) and `diff_cap_lines` — the same\n enforcement as any other work (webhook-intake-security point 4; no\n relaxation for review-triggered edits).\n3. Commits and pushes to the **same** PR branch.\n4. Replies to the review/comment via `ReplyToReview`, summarising what\n changed.\n\nThe comment is injected through the task record (e.g. `Task.Notes` or a\ndedicated review field), not through a prompt-only path, so the reaction is\nrecoverable and auditable.\n\n### 5. Routing and lifecycle\n\nA **review** is the unit of remediation, never an individual comment.\n\n- `requested_changes` review → one remediation run addressing every actionable\n comment in that review.\n- `commented` review with actionable inline comments → one remediation run\n addressing every actionable comment in that review.\n- `approved` review → no remediation. Record the review state on the task;\n do **not** auto-merge — merging stays an operator action (the dashboard/chat\n `/approve` precedent). Auto-merge is a separate decision if ever wanted.\n\nOne run per review, strictly. A review carrying eight inline comments arrives\nas one `pull_request_review` event plus eight `pull_request_review_comment`\nevents; remediating each separately means eight builder runs, eight gate runs\nand eight pushes to answer one review. The comment events are collected\nagainst their parent `ReviewID` and remediated together. A standalone comment\nwith no parent review is its own unit of one.\n\nRemediation runs for a single task are **serialised**, whatever\n`[[repos]].allow_concurrent` says. Two runs against one PR branch would race\non the shared worktree and on the push; `allow_concurrent` is about unrelated\ntasks in a repo, not about one task reacting to itself.\n\n#### Whose comments count\n\nOnly archie's **own** comments are excluded, matched on the forge identity\nthat owns the task (`bot_user`, or the identity's `bot_user` under\n`[[identities]]`), so its `ReplyToReview` replies do not retrigger it.\n\nEvery other author is actionable, including other bots. The external review\nbot on this repository is the main source of findings on an archie-opened PR,\nand discarding comments because their author is a bot would discard the review\nthis feature exists to act on. There is no \"is a bot\" test anywhere in this\npath.\n\n(The original draft justified a broader exclusion by citing a `RepliesAfter`\n`exclude` precedent. No such mechanism exists in the tree; the citation is\nwithdrawn.)\n\n#### Bounding the exchange\n\nBecause bot comments are actionable, the exchange can run away without a human\never touching it: archie pushes a remediation, the review bot re-reviews the\nnew commit, comments, archie remediates again. Nothing in the reaction path\nterminates that on its own.\n\nA task therefore carries a remediation round count, bounded by the repo's\nexisting `max_retries`. On the cap, archie stops remediating, posts one\ncomment saying it has stopped and why, and parks the task for an operator.\nThis is a hard stop, not a backoff: the failure it guards against is\nunbounded spend, so slowing it down does not address it.\n\n### 6. Security posture\n\nThe webhook reaction is not a public no-code binding, so it does not pass\nthrough the `draft → pending_approval → armed` gate (webhook-intake-security\npoint 2) — but every other constraint holds: HMAC (already in the receiver),\nper-source rate limiting (add when the receiver gains a second event family),\nand blast radius bounded by the existing gate/sandbox/worktree enforcement,\n**plus** the new archie-owned-task check (decision 3), which is the actual\nauthorisation boundary: a review event only ever causes work against a PR\narchie itself opened and still owns.\n\nAccepting other bots' comments (decision 5) does not widen that boundary: an\nauthor is never a reason to act, only the owned-task check is. It does widen\nthe _spend_ surface, which the round cap bounds.\n\n### 7. Sequencing\n\n1. **The reaction stream.** `ARCHIE_REACTIONS`, under the fan-out\n `jetstream.LimitsPolicy` the parent decision requires\n (`event-sources-and-reactions.md` §2). `Config.Retention` is\n parameterised for exactly this (`archie-core-7d5u.2`), but no reaction\n stream exists and `ARCHIE_TASKS` carries `archie.task.\u003e` only, so every\n publish to `archie.reaction.*` fails against no matching\n stream. This step is first because nothing after it can be exercised\n without it.\n2. `PullRequestReviewReader` + GitHub/Gitea implementations + `ReplyToReview`.\n _(Landed.)_\n3. `ReviewCommentEnvelope` + the kinded idempotency key.\n4. **The owned-task guard**: `(owner, repo, pr_number)` lookup, its\n `StateStoreService` RPC, its index, and the drop-with-counter path for an\n unmatched, merged, closed or archived PR. Nothing may consume a reaction\n before this exists.\n5. `remediate` workflow: worktree refresh, one run per review, builder, gate,\n push, reply, round cap.\n6. Webhook receiver extension (`pull_request_review` / `_comment` events).\n7. Poll backstop with the persisted `sinceID` cursor.\n8. Tests + `task check`.\n\nSteps 6 and 7 are the producers and come last on purpose. Building a producer\nbefore its consumer and its authorisation guard yields a path that publishes\ninto nothing, and becomes an unguarded intake the moment a stream appears.\n\n## Out of scope\n\n- Auto-merge on approval (separate decision).\n- Reacting to comments on PRs archie did not open.\n- The calibration epic's confirm-before-surfacing (h019.5) — orthogonal.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/progressive-tool-disclosure/",
|
||
"title": "Progressive tool disclosure (bridge tools) — wiring",
|
||
"section": "PRDs",
|
||
"body": "# Progressive tool disclosure (bridge tools) — wiring\n\n**Status:** Finalised\n**Date:** 2026-08-15\n**Compound:** the \"keep the LLM tool schema small, give full access on demand\" work (issue #167)\n\n## Decision\n\nThe bridge-tool pattern is **implemented and tested** in\n`internal/tools/disclosure.go` (`tool_search`, `tool_describe`, `tool_call`\n\n- `ContextPressureGate` + `DisclosureMode`, section 17.14/17.15) — but it is\n **never wired into the daemon**. `BridgeTools` and `ContextPressureGate` are\n referenced only in `disclosure_test.go`. This task is therefore a **wiring**\n task, not a feature build.\n\nWire the existing bridge into the per-chat-turn toolset:\n\n1. **Compose a per-turn registry** seeded from the base registry plus the\n per-turn extra tools, register the bridge tools into it, and hand it to both\n the bridge handlers and the `ContextPressureGate`.\n2. **Run the gate in `chatGenerateOptions`** (`internal/app/archied/main.go`)\n so it decides full-vs-bridge disclosure per turn, and build the\n `core.ToolSet` from the gate's filtered output.\n3. **Thread the model's context window** into the preparation seam so the gate\n has a real `ContextWindowSize` (it disables itself at 0, falling back to\n full disclosure).\n\n## Why the naive \"register extras into the base registry\" is wrong\n\n`BuildToolSetFrom` (`internal/agentexec/toolset.go`) documents the invariant:\n\n\u003e the task tools carry the calling gateway's identity, so there is one set of\n\u003e them per gateway and they **cannot live in the process-wide registry**.\n\nThis is confirmed by wiring: gateways are per-channel\n(`telegram_setup.go` build a `TurnRunner` per channel), and the extra\ntools (`task_*`, `session_*`, `dashboard_*`) close over that channel's identity\nvia `TurnRunnerConfig` (`turn.go`). Registering them into `b.toolReg` would\nlet channel A's session/identity-bound tools be invoked by channel B — a real\ncross-channel privilege bug. The bridge handlers iterate `reg.All()`\n(`disclosure.go`), so the extras must be discoverable; the fix is a\n**per-turn composed registry**, not a mutation of the process-wide one.\n\n## Shape of the change\n\n### 1. Per-turn composed registry + gate inside `chatGenerateOptions`\n\nReplace the current base-set + extra-set merge (`main.go`) with a single\ncomposed-registry pass:\n\n```go\n// new helper, in internal/tools/disclosure.go or the archied wiring package\nfunc ComposeForTurn(reg *tools.Registry, extra []tools.ToolEntry) (*tools.Registry, error) {\n composed := tools.NewRegistry()\n for _, e := range reg.All() {\n if err := composed.Register(e); err != nil { return nil, err }\n }\n for _, e := range extra {\n if err := composed.Register(e); err != nil { return nil, err }\n }\n if _, err := composed.RegisterBatch(tools.BridgeTools(composed)); err != nil {\n return nil, err\n }\n return composed, nil\n}\n```\n\n`chatGenerateOptions` then:\n\n```go\ncomposed, err := ComposeForTurn(registry, extra)\n// ...\ngate := tools.NewContextPressureGate(contextWindow)\ngate.SetAlwaysVisible(...) // e.g. session/session_resume, task/task_list\ndisclosed := gate.FilterTools(composed)\ntoolSet, err := agentexec.BuildToolSetFrom(disclosed, toolOpts)\n```\n\n`FilterTools` (`disclosure.go`) already returns `all` in full mode and\n`bridge + always-visible` in bridge mode, and `BuildToolSetFrom` re-applies\navailability/approval/schema rules, so the two layers compose cleanly.\n\n### 2. Thread context window through the preparation seam\n\n`ContextPressureGate` disables at `ContextWindowSize == 0`. The model context\nwindow is resolved in `TurnRunner.prepareTurn` (`turn.go`) but is not\npassed to `TurnModel.Prepare`. Extend the seam:\n\n```go\ntype TurnPrepareContext struct {\n Model string\n Extra []tools.ToolEntry\n ContextWindow int // model's context window in tokens; 0 = gate disabled\n}\n```\n\n`TurnModel.Prepare(ctx, context TurnPrepareContext)`; `prepareTurn` fills\n`ContextWindow` from `modelDetails.ContextWindow`. This touches one interface\n(`turn.go`), one production impl (`chat_turn_model.go`), and the test\nmock (`turn_test.go`).\n\n### 3. Bridge-handler parity note\n\n`bridgeCallHandler` calls `e.Handler` directly (`disclosure.go`), bypassing\n`toolExecute` — so a `tool_call` invocation does **not** apply `CapPayload` /\n`MaxResultSizeChars` spilling or `OnToolCall` reporting, and it pre-refuses\n`RequiresApproval` tools (`disclosure.go`). This is acceptable for the\nfirst wiring (the bridge is read-discover/describe plus a guarded call), but a\nlater hardening should route the invoked tool through the same\ncap/approval path the model's direct call uses. Recorded as a follow-up, not a\nblocker for this task.\n\n## Files touched\n\n- `internal/app/archied/main.go` — `chatGenerateOptions` composes + gates.\n- `internal/app/archied/chat_turn_model.go` — `Prepare` accepts `TurnPrepareContext`.\n- `internal/gateway/turn.go` — `TurnModel.Prepare` signature; `prepareTurn` fills context window.\n- `internal/gateway/turn_test.go` — update the `turnTestModel.Prepare` mock.\n- `internal/tools/disclosure.go` — add `ComposeForTurn` helper (or place in archied).\n\n## Out of scope / follow-ups\n\n- Delegate `tool_call` through `toolExecute` for cap/approval/observability parity.\n- Cover the compose-and-gate path with tests (compressed footprint under a\n small window; always-visible minus bridge margin).\n- Reconcile any `ContextPressureGate` mode with the compression budget already\n computed in `prepareTurn` (`turn.go`).\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/public-and-dev-docs/",
|
||
"title": "Public and development documentation -- design",
|
||
"section": "PRDs",
|
||
"body": "# Public and development documentation -- design\n\n**Status:** Draft\n**Date:** 2026-09-22\n\n## Problem\n\nOne published set carries both kinds of document. A reader who wants to know\nwhat Archie is and how to point an event at it lands in a sidebar that also\nholds the repository organisation rules, the migration decision register, the\npackage review procedure and five PRDs. The PRDs are there for a reason that\nhas nothing to do with the reader: settled architecture cites them, and the\ngenerator refuses a link whose target it holds back, so publishing them was the\nonly way to keep those citations alive.\n\nMixing them is not an exposure problem. Every one of these documents can be\npublic. It is a navigation problem: a customer reading the site meets\ncontributor material with no signal that it is not for them.\n\n## Decision\n\nOne source tree, two published sets, two artifacts, two URL bases.\n\n| Set | Holds | Served under | Artifact |\n| ----------- | ------------------------ | ------------ | ----------------------------------- |\n| Public | `index.md`, `guides/` | `/docs/` | `docs/data/generated/docs.json` |\n| Development | `architecture/`, `prds/` | `/dev/` | `docs/data/generated/dev-docs.json` |\n\n`archive/`, `inspiration/`, `news/` and `github-token.md` stay out of both, for\nthe reasons the generator already records.\n\nThe two artifacts have the same shape. A renderer reads a second file and\nserves it under a second base; it needs no new field and no set discriminator,\nbecause the file a page arrived in is the set it belongs to.\n\n## Consequences the design depends on\n\n**Every PRD publishes.** Selective PRD publication existed to keep citations\nfrom settled architecture working. With both cited and citing documents in the\ndevelopment set, that pressure is gone, and the rule that produced it goes with\nit: `prds/` is a development tree, not a list of exceptions.\n\n**Cross-set links resolve.** A link from a public page to an architecture page\nrewrites to the development set's absolute path, and the reverse works the same\nway. Both sets are generated in one pass over one url map, so a link and its\ntarget cannot disagree about where the target lives. The generator's refusal\nstill fires, now only for a target genuinely in neither set.\n\n**Assets keep the public base.** `docs/public/` is one file set, referenced\nfrom either set, and serving one file under two paths would invent a second\nlocation for it.\n\n**The sidebar stays derived from the directory layout**, per set. A page added\nunder a directory appears without a second edit, which is the property the\ncurrent generator has and the reason its tree mirrors the tree on disk.\n\n## The public set is thin, and that is the honest state\n\nTwo pages: the landing page and the playbook guide. Nothing is lost by the\nsplit, because a reader was never served by the seventeen architecture pages\nnext to them. The pages a public site needs (installation, configuration\nreference, the event sources, the actions a playbook can run) are written into\n`guides/` and a `reference/` tree as they are written, and each lands in the\npublic set with no generator change.\n\n## Verification\n\n- `task docs:artifact` writes both files; `task docs:artifact:check` fails on a\n stale copy of either, without touching the tree.\n- A link from the public set to a development page, and a link the other way,\n each resolve to the other set's base path in the generated body.\n- A link to a page in neither set fails the generator, naming the file, the\n line and the target.\n- `docsgen` accounts for both artifacts, so neither tool reports the other's\n file as unowned.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/release-stream-and-commit-rules/",
|
||
"title": "One release stream, one page per version, and commit rules — decision",
|
||
"section": "PRDs",
|
||
"body": "# One release stream, one page per version, and commit rules — decision\n\n**Status:** Draft\n**Date:** 2026-09-22\n**Authority:** `RELEASING.md` for the process this changes, `AGENTS.md` for the\ncommit rules it enforces, and `tools/newsgen` for the artifact shape.\n\n**Supersedes:** `docs/prds/news-and-releases.md`, sections \"Source of truth\" and\n\"Two components\" — one changelog replaces two, and one page per version replaces\none page per component. Its remaining decisions stand.\n\n## Decision\n\n**A release is one version covering whatever changed.** One changelog holds every\nrelease section, one generated page holds every version's notes, and an operator\ncutting a release names one version.\n\n## One changelog\n\nEvery release section lives in `CHANGELOG.md`, newest first. `CHANGELOG.archied.md`\nand `CHANGELOG.archie.md` are deleted, and `CHANGELOG.md` stops being a pointer to\nthem. The changelog is the only source of release prose, and tags are not a source:\nCI and deployment check out shallow, so tags are absent from a site build while the\nchangelog is always present.\n\n### The merged history\n\nTwo version streams become one file, so a version that both components released\ncarries one heading and one date: **the newest date among that version's sections**,\nbecause a version's date is when the last part of it shipped. Where a component's own\ndate differs from it, that component's section label carries its own date, so the\nper-component fact survives the merge rather than being collapsed into the heading.\n\nThe merged history holds 65 versions: 33 that both components released, 31 only\n`archied`, and one only `archie`, from 64 and 34 sections respectively, so 98\npublished pages become 65. `## [Unreleased]` is not a release -- it carries no\nversion and gets no page -- but it stays at the top of the file, because it is\nthe one place unfinished work is recorded. Counting `##` headings rather than\nreleases overcounts by one.\n\nThis ambiguity is historical only. One version stream with one tag gives a version\nexactly one date, so the rule represents the merged past rather than a design that has\nto hold.\n\nThe merge is verified by content, not by the generator succeeding: the section count\nper source, the merged version set equal to the union of both sources with nothing\nextra, and every section title still present. A generator that agrees with the file it\njust wrote is evidence of nothing.\n\n## One version stream\n\n**`GATEWAY` and `RUNTIME` stop being release inputs.** `release:prepare` and\n`release:tag` take one version, and `tools/release.sh` loses the two flags.\n\nThey may stay as build-time stamps, because a binary must still report what it\ncontains. A release that must name three versions forces an operator to pass skip\nvalues for components that did not change, and that ceremony is what this decision\nremoves — along with the drift it allows, where two components' tags fall a release\napart.\n\n**Per-component detail stays, as sections inside the release.** A reader still needs\nto know whether `archied` or the agent changed; what goes away is the separate\nnumbers and the separate pages, not the information.\n\n**History is not re-tagged.** Existing tags keep their names, because tags are not a\nsource and nothing reads them to build a page. A future reader must not \"fix\" the\nunified changelog by rewriting tags to match it.\n\n## One release page per version\n\n`docs/news/\u003cversion\u003e.md` replaces `docs/news/\u003ccomponent\u003e/\u003cversion\u003e.md`, carrying the\nrelease's per-component sections inside it.\n\n**Every release URL already published keeps working.** `/news/archied/\u003cversion\u003e` and\n`/news/archie/\u003cversion\u003e` route to the unified page for that version. The generator\nemits the redirect table beside the pages so it cannot drift from them; if the\nrenderer needs the table in its own repository, that is a change in that repository\nrather than a hand-maintained list here.\n\n## The reviewer runs between prepare and tag\n\nRelease notes are prose, and the review agent exists to review prose. **`release:tag`\nrefuses to tag when the review has not been run for that version, or when it reports\nat the level.**\n\nThe invocation is verified rather than assumed. `tl rules --lint` reports `corpus ok`\nand exits 0. `tl review \u003cpath\u003e --deadline \u003cms\u003e --max-turns \u003cn\u003e` reports\n`0 findings (0 high, 0 medium) · 1/1 files reviewed · clean` and exits 0 on\n`docs/news/archied/1.36.0.md`.\n\n**`--fail-on` must be passed explicitly.** It defaults to `never`, so an invocation\nwithout it cannot refuse anything and the step would be decorative. `--fail-on high`\nis the level: a medium finding in a release note is worth acting on, and it should not\nstop a release on its own.\n\n**The rule is stated once.** It belongs in the reviewer's own rule set, which\n`tl rules --from-doc` can draft from this document; `RELEASING.md` points at that\nrule beside the step it governs rather than restating it, because two statements of\none rule drift.\n\n## Commit rules and their enforcement\n\n- **Conventional commits**, with the scope taken from the package map.\n- **One logical change per commit.** A commit that does two things is two commits.\n- **The subject is imperative and short**; the body says why the change is right\n rather than restating the diff.\n- **A commit depends on the gate's exit status.** The gate and the commit run as one\n conditional chain, never as separate statements on one command line, where nothing\n reads the gate's status but the operator who typed it.\n- **Enforcement is a `commit-msg` hook** installed by one documented target, plus a\n check in CI over the pushed range, so the rule holds for a person who never\n installed the hook.\n- **Scopes are derived, not invented**, and an unknown scope fails loudly rather than\n being accepted.\n\n## The release pipeline\n\n**A Woodpecker workflow performs the release**: gate, generate the release page, run\nthe reviewer over it, commit the changelog and the generated pages, tag, push,\npublish images, and trigger the site's artifact sync. No step in that list is an\noperator's.\n\n**It triggers on a push to main**, not on a manual run. A pipeline a human must start\nis a pipeline that can be forgotten, which is the property this change removes. The\ntrigger is also not dependent on repairing manual triggering: in this estate a manual\nrun fails with zero steps while the same configuration runs on a push, and the\nestate's own Woodpecker configurations trigger on push with cron, manual or\npull-request events _added_ to it rather than instead of it.\n\n**Prerequisite: Woodpecker must watch this repository.** Adding the workflow to the\nrepository and enabling the repository in Woodpecker are two steps, and the second is\na configuration action rather than a file. A release pipeline that is correct and\nnever runs is invisible until someone asks why no release happened, so the repository\nbeing enabled is a prerequisite of this design rather than an assumption behind it.\n\n**One tag, and the duplicate-build problem shrinks as a consequence.** Concurrency\nkeyed on a commit identifier treats an annotated tag as a ref distinct from its\ncommit, which starts three builds for one release. One version and one tag removes\ntwo of those refs. Nothing in the pipeline may depend on a tag object's\nidentity, and this is a consequence of one version stream rather than a separate fix.\n\n### The version is computed, not typed\n\nThe version is derived from the conventional commits since the last release, which is\nwhat the commit rules above exist for.\n\n- `feat` bumps the minor version; `fix`, `perf` and `refactor` bump the patch.\n- **A breaking change bumps the major version.** Its form is `!` after the type or\n scope, as in `feat(api)!:`, and a `BREAKING CHANGE:` trailer — either marks it.\n- `chore`, `docs`, `test`, `style` and `build` bump nothing alone.\n- **The scope decides whether a commit is the product's.** A commit whose scope is\n the repository's own machinery -- `release`, `ci`, `docs`, `build`, `test` -- does\n not bump the version even when its type is `feat` or `fix`, because it changes how\n the project is delivered rather than what it does. Without this filter the first\n improvement anyone makes to the release process inflates the product's version.\n- **No release is a legitimate outcome.** A commit set that justifies no version\n produces no version, and the pipeline reports that rather than inventing one.\n\n### `tools/release.sh` is designed for the pipeline\n\n- **It never prompts.** Every input arrives as an argument or in the environment, so\n it runs with no terminal.\n- **Its version is a result as well as an argument.** It may be given one or asked to\n derive one, and it reports the version it used in a form a pipeline can read.\n- **\"Nothing to release\" has its own exit code**, distinct from failure, so the\n pipeline finishes green without tagging.\n- **It commits and tags nothing.** The pipeline owns the commit, the tag and the push;\n the script writes the notes and reports what it did. A script that pushes cannot be\n run twice safely, and this one runs on every push to main.\n\n### Which CI is authoritative\n\nThree surfaces exist: GitHub Actions, the Gitea workflow files, and Woodpecker.\n\n- **Woodpecker owns releases**, which is the production path this repository's deploy\n workflow already names.\n- **GitHub Actions keeps the checks a pull request needs.**\n- **The Gitea workflow files are retired in favour of Woodpecker** once Woodpecker\n runs the same checks, and are not edited in the meantime: two copies of a pipeline\n diverge.\n- No check lives in two places with neither authoritative. Where a check must run in\n both, the second runs the same file rather than a copy of it.\n\n## Verification\n\n- One changelog file exists, the two component changelogs do not, and the generated\n pages match it.\n- A release prepared with `VERSION` alone writes its notes and tags, with no\n `GATEWAY` or `RUNTIME` input.\n- Every release URL published before this change resolves.\n- `release:tag` refuses after a review that reported at the level, and refuses when\n no review ran.\n- The `commit-msg` hook and the CI range check each refuse a non-conforming subject\n and an unknown scope, with the hook absent and present.\n- A push to main carrying releasable commits produces a version, a release page, a\n tag and a published image with no operator step.\n- A push whose commits justify no release ends green and tags nothing.\n- `tools/release.sh` runs to completion with stdin closed, and exits with its\n distinct code when there is nothing to release.\n- The release workflow reads no tag object's identity, so one release starts one\n build.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/review-calibration/",
|
||
"title": "Review calibration -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Review calibration -- decision\n\n**Status:** Draft\n**Beads issue:** `archie-core-h019.5` (parent `archie-core-h019`)\n\n\u003e Superseded by `docs/prds/pr-review-agent.md`.\n\nDecides the half of calibration `docs/prds/adversarial-self-review.md` left\nopen: \"the adversarial-confirmation mechanism that keeps a finding from\nsurfacing on a single reviewer's say-so.\"\n\n## The problem, precisely\n\nThe shipped contract (`internal/domain/workflow/review_findings.go`, h019.2)\nalready makes only a **confirmed** error-level finding block\n(`Blocking() == Verdict == confirmed \u0026\u0026 Level == error`), and a zero-finding\nreview is already distinguishable from one that never ran (`ReviewStatus`).\nWhat is missing is the other half: a reviewer can report _nothing_\nand say only \"I found nothing\" in free text. The operator cannot tell \"I\nchecked the five things that matter and they are clean\" from \"I did not look.\"\nThat is the failure the epic names -- a findings surface that consumes attention\nwithout earning trust.\n\n## Decisions\n\n### 1. \"Checked and cleared\" becomes structured, not prose\n\n`ReviewReport` gains `Checked []ReviewCheck`:\n\n```go\ntype ReviewCheck struct {\n Property string // what was verified clean, e.g. \"nil-safety of Foo callers\"\n Evidence string // how it was verified, e.g. \"read all 4 callers; each nil-checks\"\n}\n```\n\n`Evidence` is required, for the same reason `ReviewFinding.FailureScenario` is:\na check without evidence is an assertion, and the whole point is to separate a\nverified property from an unexamined one. A zero-finding review is credible\nonly when it lists what it checked.\n\n### 2. Report checks through a typed tool, like findings\n\nThe reviewer calls a new capture tool `record_checked` (mirroring\n`record_finding`) once per property it verified clean. The same environmental\nargument as findings applies: the reviewer cannot end its turn having silently\nskipped the checklist. `ReviewReport.Summary` stays as the human overview.\n\n### 3. Style nits are out of contract\n\nA finding must describe wrong behaviour or a crash. Style, formatting, naming,\nand preference nits are explicitly out of contract -- they are the padding that\ntrains the operator to skim. Stated in the reviewer prompt and in the\n`ReviewFinding` type's own documentation. Not machine-enforced: deciding \"is\nthis a style nit\" from free text is unreliable, and a wrong rejection is worse\nthan a tolerated one. The category enum already has no style bucket; the rule\ncloses the gap that would otherwise let one hide under `other`.\n\n### 4. Surfacing\n\nThe PR-body findings section lists the cleared properties alongside findings,\nso \"ran and found nothing\" arrives with its evidence. (Park reasons keep\nrendering only blocking findings -- what a human needs to act on, not the full\nchecklist.)\n\n## What this does not decide\n\n- Adding a second, independent reviewer pass. The verdict distinction plus the\n recorded checks is the confirmation mechanism; a second model pass is a\n separate, heavier decision and nothing in `h019.5` requires it.\n- How many findings can survive before `MaxSteps` is exhausted (a budget\n question, already satisfied by the per-stage budget).\n\n## Packages this touches\n\n- `internal/domain/workflow` -- `ReviewCheck`, `ReviewReport.Checked`, validation.\n- `internal/app/agentworker` -- `record_checked` tool, prompt rules.\n- `internal/domain/workflow/review.go` -- render cleared properties on the PR body.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/runtime-control-plane/",
|
||
"title": "Runtime control plane",
|
||
"section": "PRDs",
|
||
"body": "# Runtime control plane\n\n**Status:** Approved \n**Date:** 2026-09-20 \n**Tracking:** `archie-core-j28m`, `archie-core-1786637498420-327-bacadaae`\n\n## Outcome\n\nArchie can be managed from the Web UI or a messaging channel. Both use the same\nAPI.\n\n## Model\n\nEach feature owns its settings and commands. The database stores them. There is\nno global configuration document.\n\nThis includes:\n\n- workflows;\n- identities and bindings;\n- providers and models;\n- repository policies;\n- channels and schedules;\n- capture mappings and bindings;\n- tool settings;\n- prompts and personas;\n- plugin settings.\n\nPlugin code remains installed code. Only its settings are stored in the\ndatabase.\n\nEach feature defines its data, validation, permissions, editor fields, and how\nchanges take effect. Each feature also defines its allowed actions. For example,\nidentities can be created, suspended, and retired.\n\nRepository and identity settings remain separate. They are not stored as large\nrecords that unrelated features must share.\n\nThe State Store process hosts the control-plane API on its existing gRPC server.\nNo new process is added. Feature code validates changes. Storage code only\nwrites versions and audit records.\n\nEvery write records who made it, where it came from, a request ID, and the\nversion being edited. The API layer sets the actor; the request body cannot.\nThe server validates the change, then writes it with its audit record. It rejects\nedits to an old version.\n\n## API\n\nThe API has four operations:\n\n1. **Catalogue** tells clients what can be viewed or changed and how to edit it.\n2. **Query** reads data and versions without exposing secrets.\n3. **Command** asks the owning feature to make a change.\n4. **Watch** streams changes and errors to clients.\n\nComplex features can use custom screens. They still use the same commands.\n\nA dashboard session or channel sender must map to an Archie identity before it\ncan run a command. Messaging syntax maps to the same commands as the Web UI.\n\n`archie-ui` receives only an API client. It does not receive a database handle,\ndaemon pointer, global configuration object, or secret value.\n\nThe database stores the requested settings. Each process reports the version it\nis using or why it failed. The UI shows both.\n\nSome changes require a restart. The UI shows them as pending. When a live change\nreplaces a running component, Archie starts and checks the new one before\nswitching. If the check fails, the old one keeps running.\n\n## Bootstrap, migration, and recovery\n\nBootstrap contains only the database path, listen addresses, credentials, TLS,\nand encryption-key references. State Store and control-plane RPCs use the same\nendpoint.\n\nWhen upgraded, existing instances are migrated to the database. Existing\ndatabase values are not overwritten. After migration, settings in TOML are\nignored and cannot block State Store startup.\n\nThe Web UI remains available when `archied` or Gateway is down. It shows failed\nchanges and can correct or roll them back.\n\nOffline commands can back up, restore, validate, and roll back the database.\nRecovery does not require `archied` or the Web UI. The database stores secret\nreferences, not secret values.\n\n## Workflows\n\nUsers create and edit workflows in the Web UI. Workflows are stored as versioned\nYAML. Built-in workflows can be overridden and restored.\n\nA workflow is a list of steps. Each step has a known type and its own settings.\nPlugins can add step types. Each run keeps the workflow version it started with.\n\nYaegi, `.archie/stages/*.go`, and `.archie/gate.go` will be removed. Their useful\nbehaviour must first be available as normal workflow steps.\n\n## First implementation\n\nThe first implementation makes task limits editable: maximum model steps,\nmaximum runtime, and maximum consecutive gate failures.\n\nIt includes:\n\n- validated task-limit fields;\n- version history and audit records;\n- the four API operations;\n- authenticated Web UI and messaging access;\n- live updates and apply status;\n- a Pinia-backed editor in the Web UI;\n- a messaging command using the same API;\n- migration of existing `[budgets]` values;\n- an end-to-end restart test.\n\nRunning tasks keep the limits they started with. New tasks use the latest applied\nversion. This replaces the old settings overlay instead of extending it.\n\n## Not included in the first implementation\n\n- Moving bootstrap or secret values into normal settings.\n- A generic JSON editor.\n- Rebuilding every settings screen at once.\n- Changing the limits of a running task.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/self-update-transaction/",
|
||
"title": "Self-update: artifact install, topology migration, journaled transaction",
|
||
"section": "PRDs",
|
||
"body": "# Self-update: artifact install, topology migration, journaled transaction\n\n**Status:** Draft\n\nEpic: `archie-core-hj1t` / GitHub `#804` (\"[EPIC] Self-update: artifact\ninstall, topology migration, journaled transaction\").\n\nThe update _contract_ (`internal/releaseupdate`: `Catalog.Check`,\n`Installer.Install`, plus defer/watchdog/verify) is settled and unchanged by\nthis document — it stays a thin Go interface shelling out to\n`CommandCatalog`/`CommandInstaller`. This document decides what the reference\nadapter scripts (`scripts/archie-update-{check,install,watchdog}`) and\n`install.sh` do behind that contract, and settles the one thing the epic's\nown description got wrong: the dist-zip **is** already attached to GitHub\nReleases (`.github/workflows/deploy.yml`, `gh release upload`), not\nActions-artifact-only. Artifact install is therefore a smaller change than\nthe epic assumed — pointing the installer at an existing asset, not standing\nup new CI publishing. Topology migration and the journaled transaction are\ngenuinely greenfield: no topology-detection or transaction-journal code\nexists anywhere in the tree.\n\nOut of scope: rewriting the `internal/releaseupdate` Go interfaces\nthemselves, and any UI for triggering updates (channel dispatch, e.g.\nTelegram `/update`, is unaffected — this only changes what happens once an\ninstall is triggered).\n\n## Problem\n\n`scripts/archie-update-install:139-160` clones and compiles from source on\nevery update — the only path that exists. `deploy.yml` already publishes a\nper-platform archive plus (implicitly) checksums as a release asset, so the\nscript not using it is a missed-reuse gap, not a missing-CI gap.\n`install.sh:414-445` writes only `archied.service` while every shipped\ndeployment profile (`deployments/*.toml`) declares `[services.state]` and\n`[services.gateway]` sections needing their own units — a fresh install\nalready reproduces the broken single-unit shape `deployment_contract_test.go`\ndoesn't even catch, because that test only checks profile\nsyntax, not installed topology. No code anywhere enumerates which systemd\nunits exist on a host or compares that against what a release requires — the\ndaemon fails at `bootstrap.go`'s `[services.state].target` check with no\nprior diagnosis. The self-refresh trap is confirmed at\n`scripts/archie-update-install:195-199`: new adapter copies land on disk but\nbash keeps executing the in-memory copy, so a pre-flight check added to the\nscript only protects the _next_ update.\n\n## Design\n\n### Artifact install (`#804` item 1)\n\n**Decision.** `CommandInstaller`'s default script downloads the per-platform\narchive plus `SHA256SUMS` from the GitHub Release that `Catalog.Check`\nresolved (the same release `deploy.yml` already publishes to), verifies the\nchecksum, and stages it — it does not clone/compile. Source build\n(`scripts/archie-update-install`'s current clone+compile body) is kept as an\nopt-in path selected by an explicit `--from-source` flag or\n`ARCHIE_UPDATE_SOURCE_BUILD=1` env var, never the default, since a compile-on\nevery-host update is strictly slower and has a larger failure surface than\ndownloading a signed asset that CI already produced.\n\n**Call site.** `scripts/archie-update-install` — replace the clone+compile\nbody with a download-and-verify step; move the existing\nclone+compile body behind the opt-in flag rather than deleting it.\n`internal/releaseupdate/command.go` (`CommandInstaller`) is unchanged —\nit already just shells out.\n\n**Reuse check.** `deploy.yml` already builds and publishes the\narchive and uploads it as both an Actions artifact and a release asset; no\nnew CI work is needed, only pointing the installer at what exists.\n\n**Testing.** Checksum-verification logic is a shell function, testable with\na fixture archive + `SHA256SUMS` pair — add to whatever test harness already\ncovers `scripts/`. Full download-from-real-release is a manual smoke command\n(document the exact `archie-update-check`/`--dry-run` invocation once built),\nnot `task check`.\n\n### Topology migration (`#804` item 2)\n\n**Decision.** Add a topology _observer_ to the daemon, not the shell\nadapter — CLAUDE.md's own \"known trap\" note says the shell copy is defence\nin depth only, and the daemon is what \"knows the deployment and holds the\napproved snapshot.\" The observer inspects which systemd user units exist\n(`systemctl --user list-unit-files 'archie*.service'` or equivalent) and\nwhich config sections are present (`[services.state]`, `[services.gateway]`,\nUI service config), producing an `ObservedTopology` value with one field per\nunit: present/absent, and the config sections backing it. Each release's\n`release.json` (already produced by `deploy.yml` per the artifact-install\nwork above) gains a `required_topology` field naming the units and config\nsections that release's binaries need. `Catalog.Check` compares observed\nagainst required; when the release needs a unit the host doesn't have, the\nresult is not an error — it's a `MigrationPlan` value naming exactly which\nunits and config blocks must be created and in what order (state store →\ngateway → UI → daemon, per CLAUDE.md's process boundaries), which the\ninstaller must apply before installing binaries, or refuse with that plan\nprinted to the operator.\n\n**Call site.** New topology-observation code lives in\n`internal/app/archied/` next to `bootstrap.go` (which already owns the\n`[services.state]`/`[services.gateway]` checks this reuses), not in\n`internal/releaseupdate` — the update package consumes the daemon's\nobservation, it doesn't reimplement config parsing. `install.sh:414-445`\nreads the same `required_topology` declaration (from whichever profile the\noperator selected) so first-install and self-update stop diverging — this is\nthe fix for the \"three docs describe four units, install.sh writes one\"\ngap: one topology source of truth, consumed by both entry points.\n`deployment_contract_test.go` gains an assertion that each shipped profile's\ndeclared topology matches the units `install.sh` would create for it.\n\n**Failure mapping.**\n\n| condition | contract error |\n| -------------------------------------------------------------------------------- | ------------------------------------------------------------------------ |\n| observed topology cannot be determined (systemctl unavailable, non-systemd host) | refuse with \"topology unknown\" plan naming the manual steps, never guess |\n| required units missing, migration plan derivable | apply plan before binary install, in dependency order |\n| required units missing, no derivable plan (e.g. conflicting existing unit) | refuse, print the plan, do not proceed |\n\n**Testing.** `ObservedTopology`/`MigrationPlan` comparison logic is pure —\ntable-driven unit tests belong in `task check`. Actual `systemctl` inspection\nneeds a real systemd host; manual smoke test only.\n\n### Journaled transaction (`#804` item 3)\n\n**Decision.** Replace `archie-update-install`'s copy-and-trap body with: (1)\npre-flight (the existing check, kept), (2) write an intent journal\nto a path outside every directory the install touches — e.g.\n`${XDG_STATE_HOME:-~/.local/state}/archie/update.journal` — recording the\ncandidate version, the topology plan (if any), and each binary's\ncurrent-and-candidate path before touching anything; (3) stage each new\nbinary under a `.new` suffix in the same directory as the live binary, then\natomically rename over the live path (`rename(2)` is atomic on the same\nfilesystem — this replaces whatever non-atomic copy the current script uses)\nrather than overwriting in place; (4) run each replaced binary's existing\nhealth/ready check (the same checks `internal/domain/health` already exposes\nper the health registry pattern, not a new one) against the _candidate_, not\na hardcoded default port/binary; (5) hold the process open for a bounded\nobservation window (e.g. 120s) watching for crash-loop before declaring\nsuccess; (6) on any failure at any stage, roll back using the journal:\nrestore prior binaries via the same atomic rename, restore prior unit files\nand config if the topology-migration step touched them, and delete the\njournal only on confirmed rollback completion; (7) on daemon boot, check for\na journal file and resume rollback if one exists uncommitted (crash-during\n-update recovery). A single-writer lock (`flock` on a lock file outside the\nreplaced paths, e.g. next to the journal) prevents two concurrent updates.\n\n**Call site.** `scripts/archie-update-install` (journal write, stage-then\n-rename, lock acquisition), `internal/releaseupdate` package (health-check\ninvocation reusing whatever the health registry already exposes — cite the\nconcrete symbol once located, do not add a second health-check mechanism),\nand `internal/app/archied` startup (resume-on-boot journal check, next to\nexisting bootstrap sequencing).\n\n**Failure mapping.**\n\n| condition | contract error |\n| ---------------------------------------------- | ------------------------------------------------------------------------------------ |\n| lock held by another update | refuse immediately, name the holder's PID/journal |\n| health check fails post-switch | automatic rollback within the observation window, journal records the failure reason |\n| crash mid-transaction, journal present at boot | daemon completes rollback before normal startup, logs it |\n\n**Testing.** Stage-then-rename and rollback logic against a fixture\ndirectory tree is fully fakeable — `task check`. Real systemd unit\nstart/stop as part of rollback needs a real host; manual smoke command.\n\n### Channel, pinning, automation (`#804` item 4)\n\n**Decision.** Add `channel = \"stable\" | \"next\" | \"\u003cexact-version-pin\u003e\"`\nto whatever config section already governs update behaviour (next to\n`ARCHIE_UPDATE_CHANNEL`'s existing chat-channel field in\n`internal/releaseupdate/command.go` — name this new field distinctly,\ne.g. `ReleaseChannel`, to avoid colliding with the existing chat-notification\nchannel of the same word). `Catalog.Check` filters candidate releases by\nthis field before returning the newest match. Runtime image pinning: every\nshipped profile already pins by digest\n(`deployments/*.toml` confirmed), so the remaining gap is\n`scripts/archie-update-install`'s fallback default — it must read the\ndigest from the active profile's config rather than falling back to\n`:latest` when none is configured; refuse the update (don't silently pull\n`:latest`) if no digest is resolvable and the profile requires a pinned\nimage. Opt-in auto-update refuses to start when the daemon reports work in\nflight (reuse whatever \"tasks running\" signal the daemon already\nexposes for graceful shutdown — do not add a second in-flight tracker).\n\n**Call site.** `internal/config/config.go` (new `ReleaseChannel` field),\n`internal/releaseupdate` (`Catalog.Check` filtering),\n`scripts/archie-update-install` (digest resolution, remove `:latest`\nfallback), daemon shutdown/drain signal (reuse for the auto-update\nin-flight check).\n\n**Testing.** Channel filtering is pure — `task check`. Digest-refusal path\nis a shell-script unit test against a fixture profile.\n\n## Call site inventory\n\n| concern | file | change |\n| ------------------------------------------------ | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------ |\n| Update contract (Catalogue/Installer interfaces) | `internal/releaseupdate/service.go` | none — already handles this |\n| Command adapters | `internal/releaseupdate/command.go` | new `ReleaseChannel` field distinct from existing chat-channel field |\n| Install script body | `scripts/archie-update-install:139-160,170,195-199` | replace clone+compile default with download+verify; add journal, lock, stage-then-rename, digest check |\n| CI release publishing | `.github/workflows/deploy.yml` | none — already handles this |\n| Deployment profiles | `deployments/*.toml` | none — already digest-pinned |\n| First-install unit creation | `install.sh:414-445` | change — consume shared topology declaration |\n| Topology assertions | `deployment_contract_test.go` | new assertion: profile topology matches install.sh output |\n| Topology observation | `internal/app/archied/bootstrap.go` (new sibling code) | new |\n| Resume-on-boot rollback | `internal/app/archied` startup | new |\n| Health check reuse for candidate verification | `internal/domain/health` | reuse — confirm exact symbol before implementing |\n\n## Execution: multi-agent team breakdown\n\n| sub-feature | issue | implementer scope | suggested council lenses | why |\n| -------------------------- | ------------------------ | ------------------------------------------------------------------------------------ | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |\n| Artifact install | item 1 (file if unfiled) | `archie-update-install` download+verify, opt-in source-build flag | `lens-deletionist` | must not leave the clone+compile path as silent default cruft |\n| Topology migration | item 2 (file if unfiled) | daemon-side observer, `release.json` schema, `install.sh` consumption, contract test | `lens-boundary`, `lens-operator` | boundary: observation belongs in daemon not shell; operator: refusal-with-plan is the whole safety property |\n| Journaled transaction | item 3 (file if unfiled) | journal format, stage-then-rename, lock, resume-on-boot | `lens-operator`, `lens-maintainer` | operator: crash-mid-update is the core failure mode; maintainer: someone must be able to read a journal file during an incident |\n| Channel/pinning/automation | item 4 (file if unfiled) | config field, catalogue filtering, digest refusal, in-flight guard | `lens-contract` | digest-refusal changes what a config value is allowed to resolve to |\n\n## File and link the beads\n\n- The epic's four numbered work items are not yet individual child beads —\n file one bead per item above, each linking to its matching `##` heading\n here instead of restating the design, with acceptance criteria drawn from\n this doc's failure-mapping tables.\n- Add the suggested lenses to each new issue.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/service-decomposition/",
|
||
"title": "Service decomposition -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Service decomposition -- decision\n\n**Status:** Draft\n**Date:** 2026-09-05\n**Beads epic:** archie-core-8cda\n**Prior art:** `docs/inspiration/2026-09-05-awx-service-decomposition-and-ansible-task-model.md`,\n`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`,\n`docs/inspiration/service-discovery-research-2026-09-05.md`,\n`/work/artefacts/archie-core-current-state-archify.html`,\n`/work/artefacts/archie-core-ideal-state-archify.html`\n\nThe Phase 3 UI boundary and configuration decision is ratified in\n[`ui-service-boundary.md`](/dev/prds/ui-service-boundary/). That focused document is\nthe implementation authority for `archie-core-8cda.5.2`; this parent remains\nthe broader service-distribution proposal.\n\n## Decision\n\narchie-core moves from one Go binary (`archied`) hosting every domain\nin-process to a set of independently deployable services, each exposing\nexactly one typed contract. Consumers depend on the contract; they never\nreach into another service's internal types, session state, or config.\ngRPC is the synchronous transport; NATS remains for asynchronous fan-out\n(status, telemetry) rather than being replaced.\n\nThis is a **distribution decision**, not a rewrite of Archie's domain\nmodel. The six-domain split already approved in\n`docs/architecture/index.md` and `docs/architecture/organisation.md`\n(Identity, Agent, Messaging, Workflow, Work Intake, Plugin) is unchanged;\nthis PRD is about how the _processes_ running that domain logic get\npackaged, contracted, and discovered -- not about redrawing domain\nboundaries.\n\n### Target services\n\n| Service | Contract it owns | Existing code it replaces/wraps |\n| --------------------- | -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |\n| UI Service | thin client only; talks to Gateway and State Store contracts | `internal/webui` (SPA hosting stays; API surface becomes typed contract clients, not a peer with direct access) |\n| Gateway Service | routing / turn / session contract | `internal/gateway` |\n| Messaging Service | channel-plugin contract (Telegram, email, Discord, Teams, ...) | `internal/channels` |\n| Curator Service | curator contract; **optional**, installs as a Gateway plugin | `internal/domain/curator` |\n| Scheduler Service | cron/ticker contract | `internal/domain/scheduling` |\n| Execution Environment | execution-dispatch contract | `internal/domain/workflow`, task dispatch paths |\n| Runner | isolated worktree/container execution | `internal/container`, `internal/agentexec`, `internal/worktree` |\n| State Store | state-access contract | `internal/infrastructure/postgres` |\n\n### Discovery\n\nPer `docs/inspiration/service-discovery-research-2026-09-05.md`:\n\n- **Primary:** Kubernetes-native discovery (Service + headless-Service/DNS\n - EndpointSlice, served by CoreDNS), delivered as a Helm chart + Operator,\n on k3s for single-host installs and a real cluster for multi-host or a\n hosted offering.\n- **Fallback:** NATS-based discovery (micro `$SRV` announce + JetStream KV\n registry) for no-K8s single-host installs, with a custom gRPC resolver\n reading `host:port` from the KV bucket.\n- Both implementations sit behind one internal interface:\n\n ```go\n type ServiceRegistry interface {\n Resolve(service string) []Endpoint\n Watch(service string) \u003c-chan Event // join/leave\n }\n ```\n\n- An optional service (Curator; others later) that is not installed\n must resolve to a distinct `NotInstalled` outcome, never a `Down`/\n `Unhealthy` one. Under K8s this falls out of DNS returning no record;\n under the NATS fallback it requires an explicit \"installed\" marker in the\n KV registry, not just absence-implies-down.\n\n### Distribution model this supports\n\nBoth self-hosted (customer runs the Helm chart or the NATS-fallback\nbinaries themselves) and a managed/hosted offering (we run the same Helm\nchart) are in scope -- this is the RHEL / Ansible Automation Platform\nmodel: one architecture, sold both ways. This PRD does not decide the\nbusiness model; it decides that the architecture must not foreclose\neither shape.\n\n## Why this shape\n\n`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`\nestablished that Archie's existing domain split already does what AWX's\nservice-decomposition post argues for -- boundaries exist. What didn't\nexist was deployability: `internal/webui` was not a passive frontend. The\nre-baselined coupling inventory is below, and the ratified boundary is\n[`ui-service-boundary.md`](/dev/prds/ui-service-boundary/).\n\n- Telegram task actions now call Gateway's `ChatContract` directly through\n `chatTaskActorAdapter`; the old Telegram-to-webui HTTP detour is historical.\n- `internal/webui/api_chat.go` now holds `gateway.ChatContract`; direct\n `SessionStore`, `Router`, and `TurnHistory` access is historical.\n- The third coupling is closed. UI extraction shipped (`archie-core-8cda.5`,\n closed 2026-09-11): `cmd/archie-ui` serves the dashboard against remote\n Gateway and State Store contracts, holds no `config.Holder`, and relinks no\n daemon runtime (`cmd/archie-ui/architecture_test.go` enforces that on every\n `task check`). The `b.web.Cfg = b.d.Cfg` alias left in `buildDaemon` is now\n inside the daemon's own process -- it renders the published `ConfigView`\n snapshot and holds the readiness probes -- so it no longer crosses a service\n boundary. That cleanup is `archie-core-ml30` (P2); the UI boundary is\n ratified in [`ui-service-boundary.md`](/dev/prds/ui-service-boundary/).\n\n## Non-goals\n\n- Redesigning the domain model (Identity/Agent/Messaging/Workflow/Work\n Intake/Plugin boundaries are settled elsewhere and unchanged).\n- Replacing NATS. It remains the async transport; this PRD only adds gRPC\n for synchronous request/response and (optionally) K8s-native discovery.\n- Deciding the business model (self-hosted vs. managed vs. both). Kept\n open deliberately; the architecture must serve either.\n- Config relocation (`config.toml` -\u003e ConfigMaps/Secrets) is real work\n implied by a K8s-native deployment but is its own decision, not decided\n here. Flagged in the discovery research as a phased concern.\n- Any concrete adoption of Pi (or another external agent harness) as an\n execution runner. Raised in conversation alongside this decomposition,\n but it is a separate decision (agent execution as a strict data\n boundary, `docs/architecture/agent-system.md`) and does not gate or get gated by this PRD.\n\n## Open questions\n\nAll ten are resolved; each carries its resolution inline. Three phases have\nsince shipped against them (Gateway, State Store, UI), so the framing here is\nhistorical record rather than an outstanding gate.\n\n1. **Extraction order.** Which service gets pulled out of the monolith\n first? The webui entanglement findings above suggest Gateway Service's\n `ChatContract` should exist before UI Service is extracted, since UI is\n the most entangled consumer. Not yet sequenced.\n\n **RESOLVED.** Gateway Service first, then State Store,\n then UI, then Messaging, then Execution/Runner/Scheduler, then Curator\n (optional) — preceded by an in-process \"contract seams\" (modular\n monolith) phase. Confirms the hypothesis and strengthens it: the\n Gateway is already store-decoupled (it uses narrow contract adapters,\n not `internal/store`) and already owns an isolated session SQLite, so it\n is both the stickiest capability _and_ the one with a ready seam and\n isolated data — extractable alone, first. Full phase list and reasoning:\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#1`.\n\n2. **Contract definition mechanism.** Protobuf/gRPC service definitions\n need a home (a new `proto/` or `contracts/` tree, versioning scheme,\n codegen wiring into `Taskfile.yml`). Not yet designed.\n\n **RESOLVED.** buf v2 driving `protoc-gen-go` +\n `protoc-gen-go-grpc` (grpc-go, not connect-go — connect-go's net/http\n client breaks the discovery research's gRPC resolver plumbing for K8s\n DNS and NATS-KV). One committed `proto/\u003cservice\u003e/v1/` tree, generated\n Go committed to `internal/contracts/\u003cservice\u003e/v1/`, directory\n versioning (breaking change -\u003e new `vN`, enforced by `buf breaking` in\n the gate), and `task proto:generate`/`proto:lint`/`proto:check` wired\n into `task check` and mirrored into `.github/workflows/deploy.yml`\n (CI does not run `task check`). Full toolchain comparison, layout\n rationale, and scratch-verified `buf`/`buf breaking`/`connect-go`\n results:\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#2`.\n\n3. **In-process vs. out-of-process during migration.** Whether services\n can run multiplexed inside one binary behind the same `ServiceRegistry`\n interface during a transition period (so extraction is incremental and\n testable) or whether each service must be a separate binary from day\n one. Strongly recommend the former for a solo maintainer, but not\n decided here.\n\n **RESOLVED.** Multiplexed in-process is the migration\n default -- this is `docs/architecture/organisation.md`'s standing\n \"Process boundaries\" rule, not a new lean. Mechanism: one wire-safe Go\n contract interface per service; a local adapter and a generated gRPC\n client adapter, both satisfying the same interface via a compile-time\n assertion (the `var _ workflow.Store = (*Client)(nil)` idiom;\n `store.WorkflowStore` is superseded by the consumer-owned\n `workflow.Store`); composition (not `ServiceRegistry`, which stays\n network-only) picks the adapter by the presence of a\n `[services.\u003cname\u003e].target` address -- empty = local (in-process) adapter,\n set = dial the remote gRPC client. This replaced the earlier\n `mode = \"inproc\" | \"remote\"` enum, which was dropped (see the gateway's\n `services.gateway.target`). The adapter is chosen by presence of the\n target; Phase 2 then deleted every local fallback, so remote is now the\n only mode and an empty `[services.state].target` is a startup error\n (docs/prds/state-store-contract.md §12 step 7). A\n per-contract conformance suite runs against both adapters over\n `bufconn`; the flip to a real process is a same-commit deletion of the\n in-process path, no dual-live window. Consequence: Gateway's Phase 0\n (`ChatContract` seam) ships in-process first, with the network protocol\n staying out of the production path until Phase 1 -- no amendment to Q1's\n phase list needed. Full mechanism and evidence:\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#3`\n (the State Store's concrete ratified config shape is\n `docs/prds/state-store-contract.md` §10).\n\n4. **State Store's contract shape.** Whether every service gets its own\n storage (true microservice isolation) or several services share the\n State Store service's contract (less isolation, less migration risk).\n Not yet decided.\n\n **RESOLVED.** One State Store service owns the task data\n (now PostgreSQL, see `docs/prds/state-store-contract.md`) behind narrow typed contracts (`TaskStore`,\n `CaptureStore`, `MappingStore`, `BindingStore`, `WorkflowStore`) over\n gRPC. _(Ratified 2026-09-06: the workflow contract is now the\n domain-owned `workflow.Store`, which supersedes\n `internal/store`'s `WorkflowStore`. See note below and\n `docs/prds/state-store-contract.md`.)_\n No physical database-per-service: the solo-maintainer / team-\n topology rule of thumb plus the low value of per-service autonomy for\n one person rule it out, and one store means no distributed-transactions\n problem.\n Concrete migration path (generalise `storerpc`, swap consumers one\n contract at a time, stand up the store service, split the file only on\n evidence):\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#4`.\n\n **RATIFIED (2026-09-06, beads archie-core-8cda.4.1).** The authoritative\n contract boundary \u0026 transport design for this is fixed in\n `docs/prds/state-store-contract.md` (rev. 2c): one `StateStore` gRPC service,\n 42 RPCs; **ownership split** — consumer-owned `workflow.Store` +\n `workflow.Task`/`Status`/`Source` in the workflow domain (dependency rules\n #2/#7), producer-owned daemon/webui store surfaces in `internal/domain/storecontract`;\n the `Task`/`Status`/`Source` + interface relocation is a Phase 2\n prerequisite pulled from migration-decisions §4 (`store.Task →\nWorkflowExecution`); `RecordDispatch` drops `*sql.Tx`; `Close` off-wire;\n structured gRPC error codes with sanitised details; explicit agent State\n Store gRPC handoff (`STATE_STORE_URL`/`STATE_STORE_TOKEN`); loopback+bearer\n transport-security boundary; presence-based `[services.state].target`\n defaulting local; migration order `workflow.Store -\u003e archie-state-store -\u003e\nCapture -\u003e Mapping -\u003e Binding -\u003e TaskStore`. `.4.2` and `.4.3` implement\n against that doc.\n\n5. **Helm chart / Operator ownership.** Net-new work with no existing\n analogue in this repository (`deployments/` holds TOML profiles\n and a `docker-compose.yml`, not Kubernetes manifests). Scoping not yet\n started.\n\n **RESOLVED.** Three charts: an `archie-library` chart\n (shared Deployment/Service templates, named ports), an `archie` chart\n for the 7 core services, and an `archie-curator` chart + Curator\n Operator (CRD, controller, RBAC). The Curator reconciler is a\n level-triggered loop that fetches the CR, builds Deployment + Service,\n sets owner refs, applies idempotently (`CreateOrUpdate`), watches via\n `Owns()`, and reports a `Ready` status condition. Smallest real\n milestone (M1): install a Curator CR → Operator creates Deployment +\n Service → CoreDNS serves `curator` → Gateway's resolver flips it from\n `NotInstalled` to Installed with no redeploy; uninstall flips it back.\n Full scoping and first bead-issue list:\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#5`.\n\n## Completion criteria\n\nNot applicable yet -- this PRD records the target shape and the discovery\ndecision. It does not authorise implementation. Before work begins: file\na beads epic, resolve the open questions above (at minimum #1-#3), and\nget explicit go-ahead given the scale of this change relative to\n`AGENTS.md`'s \"smallest workable change\" default for this solo project.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/service-registry/",
|
||
"title": "Service registry — one declaration per service",
|
||
"section": "PRDs",
|
||
"body": "# Service registry — one declaration per service\n\n**Status:** Draft\n**Supersedes:** `state-store-contract.md` §10's `State ServiceConnection`\nstruct-field instruction.\n\n**Base:** the `listen` field, `internal/app/archied/listen.go`, and the listen defaults cited\nbelow exist on `feat/attempt-attribution`, not on `main`. This design therefore lands on top of\nthat branch: `RegisterService`'s `listen` argument has nothing to carry without it.\n\n## Problem\n\nA service is already one Go type, `config.ServiceConnection`. What is duplicated is its\n**name**, in three layers:\n\n| Layer | Location | What is hard-coded |\n| ----------- | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |\n| Declaration | `internal/config/services.go` | `Gateway` and `State` as struct fields |\n| Defaulting | `internal/infrastructure/configuration/defaults.go` | three `if cfg.Services.X.Y == \"\"` branches |\n| Consumption | `internal/app/archied/{bootstrap.go,state_store.go,gateway.go}`, `internal/app/archieui/config.go` | `cfg.Services.State` / `cfg.Services.Gateway` field access |\n\nAdding a service means editing all three. The drift this permits is already visible:\n`stateStoreResolvedToken` (`state_store_client.go`) and `gatewayResolvedToken`\n(`chat_service.go`) are byte-identical apart from the env var name, and\n`gateway.Target` is defaulted while `state.Target` is a startup error, with nothing\nstating that asymmetry in one place.\n\n`resolveServiceListen(service, flagValue, configured)` (`listen.go`) is already\nname-parameterised. It is the shape the rest should follow.\n\n## Decision\n\n**Key services by name, and carry per-service asymmetry as registration data.**\n\n### 1. Declaration collapses to a map\n\n```go\n// internal/config/services.go\ntype Services map[string]ServiceConnection\n```\n\n`ServiceConnection` is unchanged. The TOML shape is unchanged, so no deployment profile,\n`config.example.toml`, or `install.sh` edit is required. An unrecognised `[services.foo]`\ndecodes without a code edit, satisfying criterion 5.\n\n### 2. Registration is the single entry point\n\n```go\n// internal/config/servicespec.go\nfunc RegisterService(context, name, target, listen, tokenEnv string)\n```\n\nregistered once per service, at package init in `internal/config`:\n\n```go\nRegisterService(\"both\", \"gateway\", \"127.0.0.1:8585\", \"127.0.0.1:8585\", \"GATEWAY_TOKEN\")\nRegisterService(\"client\", \"state\", \"\", \"127.0.0.1:9090\", \"STATE_STORE_TOKEN\")\n```\n\nPositional, five strings, as the story specifies. Each argument is data the current code\nstates in prose or in a branch:\n\n- `context` is `client`, `server`, or `both`, and decides which fields must resolve\n non-empty. `RegisterService` enforces it: an unknown context, or a hosted service with\n no default `listen`, panics at registration. Registrations come from this repository's\n own `init()`, never from operator input, so an invalid one is a programming error with\n no caller positioned to handle an error return. A documented-but-unread `Context` would\n be worse than no field at all.\n- `target` is the **default**, and an empty default means the operator must supply one.\n That single convention replaces the `state.target` required / `gateway.target` defaulted\n asymmetry with no branch: `state` registers `\"\"`, `gateway` registers an address.\n- `listen` is the default listen address, feeding the existing\n flag → `[services.\u003cname\u003e].listen` → default precedence unchanged.\n- `tokenEnv` names the fallback env var, replacing the two copied resolvers with one.\n\n### 3. Defaulting and consumption become loops and lookups\n\n`applyDefaults` iterates the registry instead of naming services. Consumers read\n`cfg.Services.Get(\"state\")`, returning a `ServiceConnection` plus the spec, so\n`stateStoreResolvedToken` and `gatewayResolvedToken` collapse into\n`Services.ResolvedToken(name, secrets)`.\n\n## Criteria satisfied\n\n1. Declared once, in the `RegisterService` call.\n2. No second location can contradict it; the map has no per-service fields to disagree with.\n3. Registration is the entry point, with the five fields a service has.\n4. A new service is one `RegisterService` line: no field, no default branch, no consumer edit.\n5. The set of services is registry data, not a struct definition.\n\n## Scope and risk\n\n- `state-store-contract.md` §10 must be revised in the same change, or the tree contradicts\n its own ratified authority. Its summary (`:7`) already says `[services.\u003cname\u003e]`.\n- A typo'd `[services.gatway]` is **reported** (`UnknownKeys = [services.gatway,\nservices.gatway.target]`), because the struct form leaves it in `toml.MetaData.Undecoded()`.\n A map decodes any section, so preserving that is a requirement, not an optional extra:\n `unregisteredServiceKeys` asks the registry which names exist. This costs nothing in\n extensibility, because the answer comes from registration data rather than a fixed list.\n The typo is not silently ignored either way; the behaviour was measured.\n- Existing tests reference `cfg.Services.State` directly (`state_store_client_test.go`,\n `setup_test.go`, and others). They move to `Get`, mechanically.\n- No wire contract, no RPC, and no persisted format is touched. The 42-RPC State Store\n contract is unaffected.\n\n## Acceptance\n\n- Adding a third service is a one-line diff plus its consumer, proven by a test that\n registers a throwaway service and asserts its defaults and token fallback resolve.\n- `grep -rn 'Services\\.\\(State\\|Gateway\\)' --include='*.go'` returns nothing.\n- The existing `services_test.go` default and precedence assertions pass unchanged in\n behaviour, rewritten only to the new accessor.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/session-memory-curator/",
|
||
"title": "Persistent session-memory curator (reference implementation) -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Persistent session-memory curator (reference implementation) -- decision\n\n**Status:** Approved\n**Date:** 2026-09-02\n**Beads issue:** `archie-core-1786637499114`\n**Parent epic:** `archie-core-1786637500725` (curator engine)\n**Depends on:** `archie-core-1786637499636` (bind curators to memory\nengines, shipped) and the `domain/memory` engine family (shipped)\n\n## Decision\n\nEach pass: find sessions with activity since the last pass, ask the\nmodel to extract a short list of durable observations from each one's\nrecent messages, and write each through the curator's bound memory\nengine. Extraction is **scoped to one session** -- `Observation.Identity`\nis the session ID, not a user or cross-session identity. No\nconsolidation across sessions, no forgetting, no rewriting a prior\nextraction: every pass only adds.\n\nThis is the second contract consumer the epic asks for, and the one that\nexercises the curator/memory-engine binding\n(`archie-core-1786637499636`) for real, unlike the skill curator, which\ndeclares no memory engine at all.\n\n## Why session-scoped, not user- or agent-wide\n\n`docs/architecture/migration-decisions.md#5` lists the four required\nmemory scopes (global, agent-wide, user-wide, agent-user relationship)\nas **still open** -- there is no decided mapping yet from a chat session\nto a durable identity that would outlive that session (a Telegram chat\nID is not necessarily \"the same person\" as a web session; per-platform\nidentity resolution does not exist as a memory concern). Choosing\nsession-as-identity sidesteps a decision this issue has no business\nmaking unilaterally: it is the narrowest scope that is unambiguous\n(`gateway.SessionContext.SessionID` already exists, is already unique,\nand already has no cross-platform identity claim attached to it), and\nit is forward-compatible -- widening to a resolved user identity later\nis a change to what `Identity` is populated with, not to the\nextraction or write path.\n\n## Why a new `curator.ConversationSource`, not `gateway.SessionStore` directly\n\n`internal/domain/curator` is a domain package; `internal/gateway` is not\none of `internal/domain/*`'s dependencies by convention, and\n`gateway.SessionStore`'s full CRUD surface (branch, fork, replace,\nsearch) is far more than a curator needs or should be able to reach --\nthe same reasoning `ToolBuilder`/`SkillStore`/`MemoryEngineSource`\nalready established for every other host service this package depends\non. `curator.ConversationSource` is a two-method read-only interface;\n`internal/app/archied` adapts the real `gateway.SessionStore` to it, the\nsame shape `MemoryEngineSource` already uses for\n`domain/memory.Registry`.\n\n```go\n// ConversationSource is read-only conversation history, narrowed to\n// what a curator needs to review recent activity -- never the full\n// session CRUD/branch/search surface a chat gateway exposes.\ntype ConversationSource interface {\n // RecentSessions returns every session touched at or after since,\n // newest first.\n RecentSessions(ctx context.Context, since time.Time) ([]SessionSummary, error)\n // Messages returns the most recent n messages of one session, in\n // chronological order.\n Messages(ctx context.Context, sessionID string, n int) ([]ConversationMessage, error)\n}\n\ntype SessionSummary struct {\n ID string\n LastActive time.Time\n}\n\ntype ConversationMessage struct {\n Role string // \"user\" or \"assistant\"\n Content string\n At time.Time\n}\n```\n\n`Manifest` gains `Conversations bool`, gated exactly like `Skills`:\n`Registry.validateDeclared` requires a `ConversationSource` on the host\nwhen declared, `Registry.filter` clears it from a curator's view when\nnot. Role is derived the same way `gateway.compressTurnHistory` already\ndoes (`message.From == botUser` -\u003e `\"assistant\"`, else `\"user\"`) --\ncopied logic, not shared code, since the source type\n(`gateway.Message`) is exactly the thing this package must not import.\n\n## Extraction shape\n\n`Pass`:\n\n1. `Conversations.RecentSessions(ctx, effectiveSince(in.Since))` -- see\n below for the zero-`Since` default.\n2. For each session, `Conversations.Messages(ctx, id, 40)` -- the last\n 40 messages, not the full history. A session accumulates far more\n history than a skill file accumulates content; unlike the skill\n curator's \"re-read everything, it's cheap,\" a full-history read here\n is not cheap and is not needed -- durable facts worth keeping surface\n in recent exchanges, and a 40-message tail is deliberately generous\n against the model's own context rather than tuned tightly, since\n getting this number exactly right is not this issue's job.\n3. One model call per session: the message tail, plus an instruction to\n return a JSON array of short strings, each one fact/preference/\n correction worth keeping, or `[]` if nothing is. Capped at 5 items\n per session per pass -- a session that produced more than 5 durable\n facts in one pass is the extraction prompt's problem to get more\n precise about, not a limit this curator should quietly lift.\n4. A response that isn't valid JSON, or that fails to parse, is treated\n as \"nothing extracted\" for that session, not a pass failure -- the\n next pass tries again, and this is a curator, not a build gate.\n5. Each extracted string becomes one `Write` through\n `Registrar.MemoryEngines.Get(\"builtin\")`\n (`Observation{Identity: sessionID, Kind: \"note\", Content: text}`).\n One `Action` per session summarises the count and lists the session\n ID; a session with nothing extracted gets no `Action`, matching the\n skill curator's \"a pass with nothing to report is not an error, and\n not every subject needs an entry every time.\"\n\n`effectiveSince`: `PassInput.Since` zero (first pass, or the runtime\ndeclining to name a horizon) resolves to 24 hours before now, not \"the\nbeginning of time.\" Extracting from a session's entire history on the\nvery first pass is exactly the kind of unbounded-cost-on-cold-start\nmistake `Check` exists to keep cheap, and 24h is long enough to catch\nwhat a daily check-in cadence would have caught anyway.\n\n`Check`: `len(RecentSessions(ctx, effectiveSince(zero-Since))) \u003e 0` --\ncheap (no model call), and answers \"is there anything a pass would find\"\nhonestly rather than always returning true.\n\n## What we deliberately do NOT do\n\n- **No cross-session or cross-user consolidation.** `Identity` is the\n session ID; two sessions from the same human get two disjoint memory\n scopes until the open identity-resolution decision in\n `migration-decisions.md#5` is made.\n- **No re-extraction guard, no deduplication.** A fact mentioned in two\n overlapping passes may be written twice. `domain/memory.Query`\n ranking/relevance is a read-side concern for whatever later consumes\n these observations, not this curator's job to solve by inventing a\n dedup index.\n- **No `Forget`.** Nothing here decides a memory is stale; that is a\n different, harder judgement call than \"is this worth keeping,\" and nothing\n reads these observations back into a chat turn to notice if\n one turned out wrong.\n- **No tool declarations.** `Manifest.Tools` is empty; the model call is\n a plain completion (`ChatRequest.Tools` left nil), not agentic\n tool-calling -- the extraction task does not need tools, and declaring\n none is what keeps `Registrar.Tools` correctly absent from this\n curator's view.\n- **No feeding extracted observations back into the live system\n prompt.** `internal/memory.Manager`'s existing frozen-snapshot\n mechanism is untouched, same disclaimer as the memory-engine-contract\n and builtin-engine issues before this one: the new `domain/memory`\n path and the legacy live path run in parallel by design. Reading these\n observations back into a chat turn is later, separate work.\n\n## Acceptance criteria\n\nRestated from the issue against this shape:\n\n1. Registers and runs through `curator.Registry` with no special-casing\n -- the existing `Register`/`Runtime` path, no new registry method.\n2. Writes through the memory engine contract, never a concrete store:\n every write goes through `Registrar.MemoryEngines.Get(name).Write`,\n resolved by name at call time exactly like the skill curator resolves\n nothing store-specific either.\n3. Extraction runs off the hot path: the model call happens inside\n `Pass`, which the curator runtime already runs off a chat turn's own\n goroutine (existing runtime guarantee, not something this issue\n builds).\n4. What it wrote and why is inspectable: one `Action` per session with a\n count and the session ID, through the existing curator activity\n tracker -- no new observability surface.\n5. Honours the trigger-accounting invariant: `Manifest.OnInput = true`\n wakes it on `events.KindTurnCompleted` via the existing\n `WakeOnPrimaryInput` wiring, which already excludes every\n curator-derived event kind from the forwarded set -- this curator's\n own writes cannot reschedule it, with no new code needed for that\n guarantee.\n\n## Files this change adds\n\n- `internal/domain/curator/registrar.go` -- `ConversationSource`,\n `SessionSummary`, `ConversationMessage`, new `Conversations` field\n- `internal/domain/curator/registry.go` -- `Conversations` gated like\n `Skills` in `validateDeclared`/`filter`\n- `internal/domain/curator/registry_test.go` -- table cases + fakes for\n the new field, mirroring the existing `Skills`/`MemoryEngines` cases\n- `internal/infrastructure/sessioncurator/adapter.go` -- wraps\n `gateway.SessionStore` as `curator.ConversationSource`\n- `internal/infrastructure/sessioncurator/adapter_test.go`\n- `internal/infrastructure/sessioncurator/sessioncurator.go` -- the\n `CuratorEngine`\n- `internal/infrastructure/sessioncurator/sessioncurator_test.go`\n- `internal/app/archied/bootstrap.go` -- register it in `setupCurators`,\n wired to `b.chatSessionStore` and `b.cfg.BotUser`\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/skill-curator/",
|
||
"title": "Skill curator (reference implementation) -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Skill curator (reference implementation) -- decision\n\n**Status:** Approved\n**Date:** 2026-09-02\n**Beads issue:** `archie-core-1786637499206`\n**Parent epic:** `archie-core-1786637500725` (curator engine)\n\n## Decision\n\nThe skill curator's v1 scope is **mechanical validation and safe\nnormalisation only** -- it never rewrites skill prose, never deletes a\nskill, and never calls the model. Each pass:\n\n1. Lists every skill under one configured root.\n2. Reads each `SKILL.md` and attempts to parse its frontmatter\n (`internal/skill.Parse`).\n3. Records an `Action` for any structural problem: a parse failure, or a\n missing `name`/`description` in the frontmatter that parsed.\n4. Normalises trivial whitespace (trailing whitespace per line, exactly\n one trailing newline) and writes back **only** when normalisation\n actually changes the bytes, recording an `Action` for that write.\n\nNothing here needs judgement. \"Review, prune, and improve\" -- the epic's\nframing -- is real for v1 but deliberately narrow: review is structural\nvalidation, improve is whitespace normalisation, and prune is _reporting_\na problem, never removing a skill. `Manifest.Skills = true` still marks\nthis curator agentic (existing `agentic()` rule), so it receives model\naccess from the registrar -- unused but nothing about the contract\nor the runtime needs to change to let a later version reason about\ncontent quality using it.\n\n## Why this scope, not more\n\nAn autonomous process that rewrites or deletes a user's skill content\nis a real risk with no undo path (no draft/approval flow for\ncurator writes exists yet, unlike the workflow engine's PR-based review).\nThe epic's own stated purpose for this issue is proving the curator\ncontract holds for a second, structurally different consumer than the\nsession-memory curator (files-as-skills vs. observations-as-memory), not\nsolving \"what makes a skill good.\" Mechanical validation is real,\nuseful, safe to run unattended, and fully exercises\nList/Read/Write -- every method `curator.SkillStore` declares except\n`Delete`, which this curator's Pass never has a safe reason to call in\nv1 (kept implemented on the store adapter regardless, since the\ninterface requires it and a future consumer may).\n\n## Why `internal/skill` isn't `curator.SkillStore` directly\n\n`internal/skill.Discover` parses every `SKILL.md` under a root and\nreturns an error for the **whole** call if any one of them fails to\nparse -- the read side was built for \"load the catalogue to start the\ndaemon,\" where one bad skill failing the whole boot is arguably correct,\nnot for \"review each skill and report what's wrong with it\nindividually.\" A per-skill parse failure must become one `Action`, not\nabort the pass before any skill is reviewed. So\n`internal/infrastructure/skillcurator` gets its own thin store adapter\nthat lists raw directory names and reads/writes raw `SKILL.md` bytes\nper name; the skill curator itself calls `internal/skill.Parse` on each\none, isolated per skill.\n\n## Root scope\n\nOne root only, resolved the same way `loadWorkflows` already resolves\n`skillsBase` (`cfg.SkillsDir`, falling back to `cfg.WorkDir`) -- not\nevery root `skill.CatalogRoots`/`DefaultRoots` would read for a model\n(which layers a shared dir over the per-profile one). Write and Delete\nneed one unambiguous target; a curator that could write into a shared,\nmulti-tenant skills directory is a bigger decision than this issue\ncovers. The model still sees the full multi-root catalogue through the\nexisting read path -- this curator only maintains the local one.\n\n## Shape\n\n### `internal/infrastructure/skillcurator/store.go`\n\n```go\n// Store implements curator.SkillStore over one root directory's\n// .agents/skills/*/SKILL.md files, tolerating a per-skill parse failure\n// as the curator's problem to report rather than this store's problem\n// to abort on.\ntype Store struct { root string }\n\nfunc NewStore(root string) *Store\n\nfunc (s *Store) List(ctx context.Context) ([]curator.SkillRef, error)\nfunc (s *Store) Read(ctx context.Context, name string) (curator.Skill, error)\nfunc (s *Store) Write(ctx context.Context, sk curator.Skill) error\nfunc (s *Store) Delete(ctx context.Context, name string) error\n```\n\n`Read` best-effort parses frontmatter for `Skill.Description` (empty on\nparse failure -- not this method's job to report that; `Pass` does, via\nits own `skill.Parse` call on `Skill.Content`). `Write` rejects a name\nthat does not already exist as a directory (this store curates existing\nskills; creating a new one is out of scope). `Delete` removes the whole\nskill directory, matching \"delete a skill,\" not just `SKILL.md`.\n\n### `internal/infrastructure/skillcurator/skillcurator.go`\n\n```go\n// Curator implements domain/curator.CuratorEngine. See\n// docs/prds/skill-curator.md for what a pass does and why.\ntype Curator struct { /* unexported: registrar view, interval */ }\n\nfunc New(interval time.Duration) *Curator\n\nfunc (c *Curator) Manifest() curator.Manifest {\n return curator.Manifest{Interval: c.interval, Skills: true}\n}\n\nfunc (c *Curator) Check(ctx context.Context) (bool, error) // len(skills) \u003e 0\nfunc (c *Curator) Pass(ctx context.Context, in curator.PassInput) (curator.PassResult, error)\n```\n\n`Pass` walks every skill via `Bind`'s `Skills` (the filtered\n`SkillStore` the registry already narrows to declared curators, see\n`Registry.filter`), classifying each into exactly one of:\n\n- **parse failure** -- `Action{Type: \"skill.invalid\", Reason: \"frontmatter failed to parse\"}`\n- **missing required field** -- `Action{Type: \"skill.incomplete\", Reason: \"name\" | \"description\"}`\n- **normalised** -- content changed by whitespace cleanup, written back,\n `Action{Type: \"skill.normalized\", Detail: \"\u003cname\u003e: trimmed trailing whitespace\"}`\n- **clean** -- no action recorded (a pass with nothing to report is not\n an error, and not every skill needs an entry every time)\n\nOne pass reviews every skill on disk; there is no\nper-skill \"already reviewed, skip\" state; this errs toward re-checking\nover missing a regression, and the cost is cheap (no model calls, just\nfile reads).\n\n## What we deliberately do NOT do\n\n- **No LLM-authored rewrites of skill content.** Real \"improve\" in the\n sense of better prose is out of scope until there is a review/approval\n path for autonomous content changes to something a human wrote.\n- **No automatic deletion of a skill**, however broken. `Delete` exists\n on the store for a future consumer; this curator's `Pass` never calls\n it.\n- **No dangling-tool-reference checking** (a skill's\n `metadata.archie.tools` naming a tool that no longer exists). The\n mechanism to check \"does this tool exist\" belongs to a tool registry\n this curator has no access to and shouldn't be given for this --\n `curator.ToolBuilder.Build` resolves the curator's _own_ declared\n tools, not an arbitrary existence check against skill-referenced\n names. A real fix here is a separate, better-scoped issue.\n- **No multi-root awareness.** One configured root, matching\n `loadWorkflows`'s existing resolution, not the full layered catalogue a\n chat turn sees.\n- **No new curator-specific webui or chat surface.** Activity is visible\n through the existing curator observability surface\n (`archie-core-1786637489932`) -- `/api/curators` and\n the dashboard Curators page already show any registered curator's\n recent actions.\n\n## Acceptance criteria\n\nRestated from the issue against this shape:\n\n1. Registers and runs through `curator.Registry` with no curator-specific\n special-casing in the runtime -- proven by using the existing\n `Registry.Register`/`Runtime` path, no new registry method.\n2. Its changes to skills are observable and attributable -- every write\n is preceded by an `Action` recording what changed and why, surfaced\n through the existing curator activity tracker.\n3. It operates within its declared tool set and cannot reach tools it\n did not declare -- `Manifest.Tools` is empty; nothing about this\n curator ever touches `Registrar.Tools`.\n\n## Files this change adds\n\n- `internal/infrastructure/skillcurator/store.go` -- `curator.SkillStore` adapter\n- `internal/infrastructure/skillcurator/store_test.go`\n- `internal/infrastructure/skillcurator/skillcurator.go` -- the `CuratorEngine`\n- `internal/infrastructure/skillcurator/skillcurator_test.go`\n- `internal/app/archied/bootstrap.go` -- register it in `setupCurators`, root\n resolved the same way `loadWorkflows` resolves `skillsBase`\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/state-store-contract/",
|
||
"title": "State Store — contract boundary \u0026 transport (ratification)",
|
||
"section": "PRDs",
|
||
"body": "# State Store — contract boundary \u0026 transport (ratification)\n\n**Status:** Approved\n**Revision:** 2e\n**Date:** 2026-09-06\n\nThis is the pre-implementation design document that Phase 2 subtasks `.4.2` (generalise\n`storerpc` as the State Store gRPC contract) and `.4.3` (stand up `archie-state-store`)\nimplement against.\n**Beads milestone:** archie-core-8cda.4.1\n**Parent:** `docs/prds/service-decomposition.md` (open-question Q4 **RESOLVED**)\n\nThis document fixes the **contract boundary** and **transport** for the State Store service.\nIt does not redesign Archie's domain model, does not invent a second store, and does **not**\nsplit `archie.db`. It records method ownership, DTO/wire representation, error semantics,\nlimits, mode/config shape, local-adapter compatibility, and cutover/deletion rules so that\n`.4.2` and `.4.3` have exactly one authoritative surface to build against.\n\n\u003e **Storage superseded.** The store behind this contract is PostgreSQL:\n\u003e `internal/infrastructure/postgres` implements the producer-owned interfaces in\n\u003e `internal/domain/storecontract` and `workflow.Store`, over the database\n\u003e `database_url` names. `internal/store`, the SQLite file and the local adapter\n\u003e are gone; where this document says `archie.db`, `*store.Store` or\n\u003e `internal/store`, read the State Store's PostgreSQL database and\n\u003e `internal/infrastructure/postgres`. The contract, ownership split, error\n\u003e semantics and cutover rules stand.\n\n\u003e **Current state — read before `.4.2` starts.** The §12 step 2 relocation (`archie-core-8cda.\n4.8`) has landed: `Task`/`Status*`/`Source*` and the 3-method `workflow.Store` now live in\n\u003e `internal/domain/workflow`, the six production workflow files import no `internal/store`\n\u003e symbol, and `storerpc`/the NATS transport assert against `workflow.Store` instead of\n\u003e `store.WorkflowStore`. The broader Phase 2 acceptance criterion (State Store as its own\n\u003e service behind gRPC) is still open — that is what `.4.2`/`.4.3` build next.\n\n\u003e **Relationship to migration-decisions §4 — not replaced.** `docs/architecture/\nmigration-decisions.md` §4 (`store.Task → WorkflowExecution`, `mutable Stage →\nStepExecution history`, `RetryCount/Attempt → Attempt records`, versioned Workflow defs)\n\u003e **remains the authoritative, open document for the long-term Workflow migration.** This PRD\n\u003e does **not** replace it. It only pulls the minimal `Task`/`Status`/`Source` type + interface\n\u003e relocation forward as a Phase 2 prerequisite (because the acceptance criterion requires it);\n\u003e the rest of §4 stays its own migration and stays open.\n\n---\n\n## 1. Decision (the invariant everything follows)\n\nArchie is composed such that a single **`archie-state-store`** service owns the one existing\nSQLite file (`archie.db`) and exposes it over **gRPC behind narrow typed contracts**. There is\nexactly one store, one file, one service. No database-per-service, no distributed\ntransactions, no second store abstraction.\n\nTwo adapter families satisfy the contract interfaces:\n\n- **Local adapter** — `*store.Store` (the in-process SQLite implementation).\n- **Remote adapter** — `*staterpc.Client` (wraps the generated gRPC client).\n\nEach adapter carries an explicit compile-time assertion against the contract interfaces it\nserves. Composition picks local vs. remote by the presence of `[services.state].target`\n(empty = local; set = dial the gRPC service). Default is **local** (the State Store is not yet\nextracted), which is the PRD §3 posture; it differs from the gateway's remote-default only\nbecause the gateway is already a separate deployment and the State Store is not.\n\n\u003e **Contract-ownership fork, decided (rev. 2c — resolves the domain contradiction).**\n\u003e Ownership splits by the consumer's _layer_, per the dependency contract's rule #2\n\u003e (\"a domain defines the smallest interfaces required to perform its work\") and rule #7\n\u003e (\"database rows … MUST NOT leak into domain models\"):\n\u003e\n\u003e - The **workflow domain** (`internal/domain/workflow`) is the only domain-layer consumer of\n\u003e store state. It defines its own **consumer-owned** narrow `Store` interface\n\u003e (Update/Transition/InsertEvent) and its own **domain-owned** `Task` type plus\n\u003e `Status`/`Source` vocabulary. It imports **zero** `internal/store` symbols — satisfying the\n\u003e Phase 2 acceptance criterion. This is the minimal bound of migration-decisions §4\n\u003e (`store.Task → WorkflowExecution`), pulled forward **because** the acceptance criterion\n\u003e requires it.\n\u003e - The **daemon / webui / intake** store surfaces (`TaskStore`, `CaptureStore`,\n\u003e `MappingStore`, `BindingStore`, `BindingDispatcher`, `BindingTaskCreator`) stay\n\u003e **producer-owned** in `internal/store`, because those consumers are application and\n\u003e infrastructure layers, not domains — rule #2 does not apply to them.\n\nSo the State Store contract is **not** uniformly producer-owned: the domain-facing workflow\ncontract is consumer-owned (in the domain), while the daemon/webui store surfaces remain in\n`internal/store`. `internal/store` imports `internal/domain/workflow` for the `Task`/`Status`/\n`Source` types it persists (persistence → domain, the correct direction), and its `Store`\nimplementation satisfies `workflow.Store`. This is the single decision that makes the whole\nPhase 2 acceptance criterion and the architecture's rules simultaneously satisfiable.\n\n\u003e **`internal/store` is a TRANSITIONAL compatibility location (rev. 2c).** The repository's target\n\u003e architecture (`docs/architecture/dependencies-and-contracts.md`, `organisation.md`) places\n\u003e persistence implementations under `internal/infrastructure/\u003ccapability\u003e`. `internal/store`\n\u003e holding the task/capture/mapping/binding implementation and ALSO being the home of the\n\u003e producer-owned daemon/webui interfaces is acceptable **only as a transitional location**. The\n\u003e long-term home for the store implementation is `internal/infrastructure/store`; the\n\u003e producer-owned interfaces may move to the same infrastructure package (or an approved\n\u003e contract location) when the store implementation relocates. Agents must not treat\n\u003e `internal/store` as the final infrastructure boundary.\n\u003e\n\u003e **REV. 2d (archie-core-8cda.5.6): the producer-owned interfaces HAVE moved to an approved\n\u003e contract location** — `internal/domain/storecontract` (package `storecontract`), ahead of the\n\u003e store implementation's own relocation. The UI Service deletion gate required the archie-ui\n\u003e process to hold the contracts without linking `internal/store`'s SQL implementation, and\n\u003e `internal/contracts/` itself is ruled out: that tree is the `buf generate` output and\n\u003e `tools/proto.sh check` deletes hand-written files there. `internal/store` keeps type aliases\n\u003e (`type TaskStore = storecontract.TaskStore`, the sentinel `var`s, the value types) so\n\u003e daemon-side callers are unaffected; the sentinel message strings are unchanged and remain\n\u003e the wire contract (§4).\n\n---\n\n## 2. Contract boundary\n\n**Ownership split** (see the fork in §1): the **workflow-domain consumer contract** is\n`workflow.Store` (consumer-owned, in `internal/domain/workflow`); the **daemon/webui/intake\nstore surfaces** are the producer-owned interfaces in `internal/store/interface.go`.\n`store.WorkflowStore` is **superseded** — it is replaced by the domain-owned `workflow.Store`,\nand `internal/store`'s `Store` implementation satisfies `workflow.Store` directly. `TaskStore`\nremains the only composite; every other interface is flat. Narrow slices are split along\nconsumer-capability lines so no consumer acquires a surface it does not need\n(`interfacebloat` cap = 8).\n\n| Interface (owner) | Embeds | Methods | Local impl | Remote impl |\n| ---------------------------------- | --------------------------------------------------------------------------- | ---------- | ---------- | ------------------ |\n| `TaskStore` (store) | `TaskLifecycle`, `TaskEvents`, `TaskQueries`, `TaskArchiver`, `TaskRetryer` | 24 (union) | `*Store` | `*staterpc.Client` |\n| `TaskLifecycle` (store) | — | 8 | `*Store` | `*staterpc.Client` |\n| `TaskEvents` (store) | — | 7 | `*Store` | `*staterpc.Client` |\n| `TaskQueries` (store) | — | 7 | `*Store` | `*staterpc.Client` |\n| `TaskArchiver` (store) | — | 1 | `*Store` | `*staterpc.Client` |\n| `TaskRetryer` (store) | — | 1 | `*Store` | `*staterpc.Client` |\n| `workflow.Store` (workflow domain) | — | 3 | `*Store` | `*staterpc.Client` |\n| `CaptureStore` (store) | — | 2 | `*Store` | `*staterpc.Client` |\n| `MappingStore` (store) | — | 5 | `*Store` | `*staterpc.Client` |\n| `BindingStore` (store) | — | 6 | `*Store` | `*staterpc.Client` |\n| `BindingDispatcher` (store) | — | 3 | `*Store` | `*staterpc.Client` |\n| `BindingTaskCreator` (store) | — | 1 | `*Store` | `*staterpc.Client` |\n| `PlaybookDispatcher` (store) | — | 2 | `*Store` | `*staterpc.Client` |\n\n**Key property of the remote client:** `*staterpc.Client` is a _single_ adapter that wraps one\n`StateStoreClient` and is asserted against **whichever narrow Go interfaces its caller needs**\n(exactly as `gatewayrpc.Client` asserts both `gateway.ChatContract` and\n`gateway.SessionStore`). The Go contracts stay narrow (≤8, the `interfacebloat` cap); the wire\nis one service. `workflow.Store` introduces **no new RPCs** — it is a narrow consumer face over\nthe same `Update`/`Transition`/`InsertEvent` RPCs.\n\n---\n\n## 3. Method ownership\n\n`*Store` remains the single implementation of the daemon/webui store surfaces and of\n`workflow.Store`. Ownership below is about which **process** is the primary consumer of each\nmethod and therefore which methods are genuinely cross-process on the wire vs.\ndaemon-internal.\n\nProcess legend: **DAEMON** = `archied` (incl. in-process webui + gateway runtime +\n`storerpc.Server`); **AGENT** = `archie-agent` worker (no DB connection); **GATEWAY** = store\naccess through daemon-side narrow adapter interfaces.\n\n| Contract | Primary consumers (process) | Remote on the wire? |\n| ------------------------ | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |\n| `TaskLifecycle` (8) | DAEMON (dispatch, claim, action) + **AGENT** (`Transition`, `Update` via storerpc) | `Transition`/`Update` = **YES** (agent); rest DAEMON |\n| `TaskArchiver` (1) | DAEMON (`taskactions` archive) | not yet |\n| `TaskRetryer` (1) | DAEMON (`taskactions` retry) | not yet |\n| `TaskQueries` (7) | DAEMON + GATEWAY (status/list adapters) | not yet (`OpenPRs`/`Tasks`/`StatusCounts` cross to gateway adapters in-process) |\n| `TaskEvents` (7) | DAEMON + **AGENT** (`InsertEvent` via storerpc) | `InsertEvent` = **YES** (agent); rest DAEMON |\n| `workflow.Store` (3) | **AGENT** only | **YES** (the sole genuinely remote consumer) |\n| `CaptureStore` (2) | DAEMON (webhook intake + mapping editor) | not yet |\n| `MappingStore` (5) | DAEMON (dashboard editor + dispatch loop) | not yet |\n| `BindingStore` (6) | DAEMON (dashboard editor + dispatch loop) | not yet |\n| `BindingDispatcher` (3) | DAEMON (dispatch loop) | not yet |\n| `BindingTaskCreator` (1) | DAEMON (dispatch loop) | not yet |\n| `PlaybookDispatcher` (2) | DAEMON (EDA playbook dispatch loop) | **YES** (side-effecting-action idempotency ledger over gRPC) |\n\n**Consequence for migration order:** the **agent's `workflow.Store` is already cross-process\nalready** (via the NATS `storerpc` surrogate), so it is the first contract to move to gRPC. The\ndaemon-internal surfaces (`Capture`/`Mapping`/`Binding`) stay in-process until the daemon\nitself points at `archie-state-store`. `TaskStore` (the large composite) moves last.\n\n**Two orphan methods** — `TaskQueries.ClearTerminalTasks` and\n`TaskQueries.IncrementRetryCount` have **no production consumer** (only `*Store` tests). They\nremain part of the Go `TaskQueries`/`TaskStore` surface, and therefore part of the wire\ncontract, but are flagged for deletion: they must **not** gain dedicated consumers or wire\nmethods in this work. Do not add the rest of the codebase to them.\n\n---\n\n## 4. Domain-facing types \u0026 consumer-owned contract — resolves the contradiction\n\nThe Phase 2 acceptance criterion (\"`internal/domain/workflow` … no longer import\n`internal/store` directly — they call the State Store contract\") **cannot be met while**\n`store.Task`, `store.Status*`, `store.Source*`, and `store.WorkflowStore` live in\n`internal/store`. This section fixes where they move.\n\n**The relocation (a prerequisite that must land before or with `.4.2`):**\n\n| Current symbol (`internal/store`) | Moves to | Notes |\n| ------------------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |\n| `Task` type | `internal/domain/workflow.Task` | The task-execution record the workflow domain operates on. `internal/store` persists this type (persistence → domain). |\n| `Status` + `Status*` constants | `internal/domain/workflow` | e.g. `StatusClosedWontDo`, `StatusMerged`, `StatusPROpen`, `StatusWaitingHuman`, `StatusRunning` — task-lifecycle vocabulary. |\n| `Source` + `Source*` constants | `internal/domain/workflow` | e.g. `SourceChat` — provenance vocabulary, follows `Task`. |\n| `WorkflowStore` interface (3 methods) | `internal/domain/workflow.Store` | **Consumer-owned** (dependency rule #2). Supersedes `store.WorkflowStore`. |\n\n**Resulting dependency direction (correct per the architecture):**\n\n- `internal/domain/workflow` imports **no** `internal/store` symbols → the acceptance\n criterion is satisfied. It consumes `store.Task`'s replacement as `workflow.Task`, uses\n `workflow.Status*`/`workflow.Source*`, and depends on its own `workflow.Store` interface.\n- `internal/store` imports `internal/domain/workflow` for the `Task`/`Status`/`Source` types it\n reads and writes (infrastructure → domain). **NOTE: `internal/store` is a transitional\n compatibility location only** — its long-term home is `internal/infrastructure/store` (see\n the callout in §1). Do not treat `internal/store` as the final infrastructure boundary.\n- Every store test that constructs `store.Task{…}` is updated to the relocated type as part of\n this prerequisite. This is a mechanical rename across ~6 workflow production files\n (`workflow.go`, `steps.go`, `triage.go`, `review.go`, `feasibility.go`, `implement.go`) and\n their tests plus the store tests. `TestRecordDispatchViaExplicitTx` is removed rather than\n updated — its non-nil `*sql.Tx` path has no production consumer (see §5's `RecordDispatch`\n blocker).\n\n**What this does NOT do:** it does not implement the full §4 Workflow migration\n(`mutable Stage → StepExecution history`, `RetryCount/Attempt → Attempt records`,\n`workflow.Registry → versioned Workflow definitions`, etc.). It lands only the minimal\ntask-type + `Status`/`Source` + consumer-owned-interface relocation that the Phase 2\ndomain-decoupling criterion requires. The residue of §4 remains its own migration.\n\n**Forward-compatibility:** because the wire DTOs are derived from the domain `Task`/`Status`\n(§5), when the full §4 migration later renames `workflow.Task` (or refines it into\n`WorkflowExecution`), only `values.go` and the proto field mapping change — the RPC surface and\nconsumer interfaces are unaffected. The wire contract is deliberately type-agnostic at this\nlevel.\n\n---\n\n## 5. DTO / wire representation\n\n### Proto layout\n\n- **Service:** `proto/state/v1/state.proto`, `package state.v1`, **one** `StateStore` service\n (mirrors the gateway's single `ChatService`; proto bypasses the Go `interfacebloat` cap, as\n `ChatContract` does via `//nolint:interfacebloat`).\n- **Codegen:** the existing buf v2 pipeline. `proto/buf.yaml` (module, lint STANDARD, breaking\n FILE against `main`), `proto/buf.gen.yaml` **managed mode** with\n `go_package_prefix → github.com/samcharles93/archie-core/internal/contracts`, plugins\n `protoc-gen-go` + `protoc-gen-go-grpc`, `out: ../internal/contracts`,\n `opt: paths=source_relative`. **No `go_package` in source** — managed mode owns it. Generated\n Go lands in `internal/contracts/state/v1/` (package `statev1`), committed (`proto:check`\n fails on uncommitted/unmodified generated files).\n- **Adapter package:** `internal/infrastructure/staterpc/` with `server.go` (registers the\n generated server over the store), `client.go` (wraps the generated client into `store.*` and\n `workflow.Store` facades), `values.go` (proto ⇄ domain mappers), and `conformance_test.go`\n (bufconn local-vs-grpc dual-adapter suite), mirroring `internal/infrastructure/gatewayrpc/`.\n- **Binary:** `cmd/archie-state-store/main.go`, mirroring `cmd/archie-gateway/main.go` and the\n serve pattern in `internal/app/archied/gateway.go` (`serveGateway`).\n\n### The `StateStore` RPC set (union of the contracts, grouped)\n\nMethods are named after the store methods so the mapping is unambiguous. `workflow.Store` is a\n_view_, not new RPCs — `Update`/`Transition`/`InsertEvent` are shared with\n`TaskLifecycle`/`TaskEvents` and appear once.\n\n| Group | RPCs |\n| ------------------ | -------------------------------------------------------------------------------------------------------------------------- |\n| Lifecycle | `EnqueueIssue`, `EnqueueChatTask`, `ClaimNext`, `ClaimByIssue`, `Transition`, `Update`, `Requeue`, `RecoverStale` |\n| Archive / Retry | `ArchiveTask`, `RetryTask` |\n| Queries | `TaskByIssue`, `OpenTaskByPR`, `TaskByID`, `OpenPRs`, `ClearTerminalTasks`, `Tasks`, `StatusCounts`, `IncrementRetryCount` |\n| Events | `InsertEvent`, `EventsSince`, `TaskEvents`, `WorkflowStats`, `StageStats`, `TokensByDay` |\n| Capture | `InsertCapture`, `ListCaptures` |\n| Mapping | `InsertMapping`, `GetMapping`, `ListMappings`, `UpdateMapping`, `DeleteMapping` |\n| Binding | `InsertBinding`, `GetBinding`, `ListBindings`, `UpdateBinding`, `DeleteBinding`, `ApproveBinding` |\n| Dispatch | `ArmedBindingsForSource`, `RecordDispatch`, `ListUndispatchedCaptures` |\n| Playbook | `RecordPlaybookDispatch`, `DeletePlaybookDispatches` |\n| WorkflowCaller | `EnqueueCallTask`, `WorkflowCallStatus` |\n| Step executions | `StartStep`, `FinishStep` |\n| BindingTaskCreator | `EnqueueBindingTask` |\n| Config snapshot | `PutConfigSnapshot`, `GetConfigSnapshot` |\n| Apply status | `PutApplyStatus`, `ListApplyStatus` |\n| Channel status | `PutChannelStatus`, `ListChannelStatus` |\n| Task log | `ReadTaskLog`, `StreamTaskLogContent` |\n\nThat is **55 unique contract RPCs** across one service (the `TaskEvents.Close` method is dropped).\n`Close()` is **excluded** from the wire (it is server lifecycle, not a client call) — see §11.\n\nThe channel-status pair follows the config snapshot's split for the same reason: the writer is the process that HOSTS the channels -- the Messaging Service, which alone observes whether a channel is starting, running or failed -- and the reader is the UI process. Both RPCs are administrative, so a task-scoped grant reaches neither. A channel's state is runtime state rather than a settings document, which is why it is not a `ControlPlaneService` resource kind: a failed channel is not a document anybody edits. Asking for a reload travels on its own surface rather than through this one; see `docs/architecture/migration-decisions.md`, \"Channel state to the dashboard\".\n\nThe config-snapshot pair was added by `archie-core-ymut` (see\n`docs/architecture/migration-decisions.md`, \"Dashboard configuration page\"). It is the one\nsurface whose writer is the daemon and whose reader is the UI process, and both RPCs are\nadministrative: `authorizesTaskScopedCall` is deny-by-default, so a task-scoped grant reaches\nneither. `ConfigSnapshot.document` is opaque to this contract — the store neither parses nor\nvalidates it, and `schema` names the projection so a reader can refuse a shape it does not\nunderstand.\n\nThe apply-status pair was added by `archie-core-pskb` (see\n`docs/prds/control-plane-apply-status.md`). It is administrative for the same reason as the\nconfig-snapshot pair, and deny-by-default in `authorizesTaskScopedCall` covers it without a\nnew rule. Unlike that pair, the read is a list: `ListApplyStatus` returns every process's\nrecord and an empty slice means nothing has reported yet, so the `found=false` convention for\nabsent single rows does not apply. The rows are keyed on `(process, kind)` and replaced in\nplace, because each process re-stamps its record on an interval and no reader wants a history\nof re-stamps.\n\nThe task-log pair was added by `archie-core-iaqx` (see `docs/prds/ui-service-boundary.md`'s\nroute inventory). It is the one surface whose payload is not a row: the store service reads\nthem out of the state directory it owns with `internal/logging`'s own reader, so the wire\ncarries `logging.Entry`/`logging.Query` values and neither side restates the JSONL format.\n`ReadTaskLog` returns a decoded page and `StreamTaskLogContent` returns the raw bytes\nserver-streamed, because a log file rotates at 20 MB — past gRPC's 4 MiB unary cap — and the\nsame reasoning that made `StreamCaptures` supersede `ListCaptures` applies. `found=false` with\nno error means the attempt has no log; a service with no reader answers `Unavailable` instead,\nand the two answers are load-bearing for the dashboard: only the second is a deployment\ncondition, and the panel that collapsed them told operators task logging was switched off when\nit was not. The Go facade is `storecontract.TaskLogStore` (2 methods), carried by\n`*staterpc.Client` alongside the rest.\n\nThe step-execution pair was added by the execution tree's plan item 3 (see\n`docs/prds/execution-tree-state-machine.md`), the surface a task-scoped grant reaches for\nits own execution beyond the three `task.Store` RPCs and the workflow-caller pair: the\ncontainer records its stages and its own agent calls with the run credential. Both requests\ncarry the caller's execution ID, which the grant check verifies; `FinishStep` additionally\nverifies, handler-side, that the step really is that execution's. The store writes each\ntransition's `stage_start`/`stage_finish` event row in the same transaction and returns it\nfor the caller's post-commit bus publish. `called_execution_id` is stored for the `call`\nkind but not yet carried on the wire: the named consumer is the workflow-call recording\nthat will set it.\n\nThe workflow-caller pair was added by `archie-core-t2db.41` (see everything else from the caller's\nrow and refuses a caller that is not running or a call past the depth limit\n(`ErrCallCallerNotRunning` / `ErrCallDepthExceeded`), and `WorkflowCallStatus`\nre-checks the parent-child relation against the row before answering\n(`ErrCallNotYours`), so the interceptor itself never reads the database. The\nfacade is `workflow/task.Caller` (2 methods), carried by `*staterpc.Client`\nand the PostgreSQL store alike.\n\nThe playbook pair was added by `archie-core-t2db.17`'s implementation (see\n`docs/prds/eda-playbook-engine.md`, execution-time gap 2). It is administrative\nfor the same reason as the config-snapshot and apply-status pairs, and\ndeny-by-default in `authorizesTaskScopedCall` covers it without a new rule. The\nrow is the structural `(playbook_id, playbook_version, event_id, action_id)`\ntuple, written **before** the action's side effect fires (the reverse of\n`binding_dispatches`), so a duplicate returns `store.ErrAlreadyDispatched` and\nthe coordinator skips the invoke rather than repeating a non-revocable side\neffect. `DeletePlaybookDispatches` is the explicit reclamation primitive; nothing\ncalls it automatically, because a playbook file's absence at boot is not a\ndeliberate removal (`eda-playbook-engine.md`, \"Lifetime\"). The Go facade is\n`storecontract.PlaybookDispatcher` (2 methods), carried by `*staterpc.Client`\nalongside the rest.\n\n### Domain types and `values.go` mapping\n\n`values.go` maps proto ⇄ these types. The task model for the **agent** path is the domain-owned\n`workflow.Task` (§4); the daemon/webui store surfaces return the same `workflow.Task` (the store\npersists the domain type).\n\n| Domain type | Package | Notes for `values.go` |\n| ------------------------------------- | -------------------------- | -------------------------------------------------------------------------------------------------------------- |\n| `workflow.Task` | `internal/domain/workflow` | time fields stored as layout strings; convert to/from `google.protobuf.Timestamp` |\n| `events.Event` | `internal/events` | `Timestamp` → `google.protobuf.Timestamp` |\n| `store.CapturedEvent` | `internal/store` | headers/body/remote_addr; body capped 256 KiB |\n| `mapping.Mapping` | `internal/domain/mapping` | JSON-path field bindings |\n| `binding.Binding` | `internal/domain/binding` | secret is **already decrypted** before returning; keep decrypted form out of any durable/logged representation |\n| `workflow.Status` / `workflow.Source` | `internal/domain/workflow` | enum-like; map to proto enum or string, keeping the names stable for compatibility |\n| `store.WorkflowStat` | `internal/store` | aggregate row |\n| `store.StageStat` | `internal/store` | aggregate row |\n| `store.DayTokens` | `internal/store` | aggregate row |\n\n### Wire-contract blocker fixed here: `RecordDispatch`\n\n`store.BindingDispatcher.RecordDispatch(ctx, tx *sql.Tx, bindingID, bindingVersion int64,\ncaptureID int64, taskID int64) error` takes a **`*sql.Tx`** that cannot cross a gRPC boundary.\n\n**Evidence that the `*sql.Tx` is removable:** the dispatch loop calls `RecordDispatch(ctx, nil,\n…)` (`internal/daemon/daemon.go`). The implementation falls back to the store's own\n`*sql.DB` when `tx == nil` (\"best-effort, not atomic\"; the comment and the\n`TestRecordDispatchViaExplicitTx` test are the only consumers of the non-nil `tx` path).\nProduction never opens the explicit transaction.\n\n**Decision:** the contract drops `*sql.Tx` from `RecordDispatch` (the method becomes\n`RecordDispatch(ctx, bindingID, bindingVersion, captureID, taskID) error`). The store service\nowns any transaction boundary internally when atomicity is wanted; the at-most-once\nguarantee is the `INSERT OR IGNORE` dedup row, which is already transactionless in production.\n`TestRecordDispatchViaExplicitTx` is superseded by the store service's own internal-transaction\ntest. **This is a required interface migration that must land before or with `.4.2`** (the\ncurrent `interface.go` signature and test still use the transaction-bearing form).\n\n---\n\n## 6. Agent State Store client handoff\n\nMoving the agent from NATS `storerpc` to gRPC `workflow.Store` requires the agent process to\nreceive a State Store endpoint. The agent already receives its NATS connection settings via\nboot-time env/flags (see `cmd/archie-agent/main.go`: `-nats-url` / `NATS_URL`, `NATS_TOKEN`\nenv), and the daemon injects them into the task container. The State Store handoff follows the\nsame seam.\n\n**Boot handoff contract (additive, not replacing the NATS settings):**\n\n| Field | Where | Source | Notes |\n| ------------------ | ---------------------- | ------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- |\n| `StateStoreTarget` | `agentworker.Settings` | env `STATE_STORE_URL` (or `-state-store-url` flag), set by the daemon in the container env | `host:port` of the State Store gRPC server. Loopback/Docker-bridge address — the agent reaches it the same way it reaches the daemon's NATS. |\n| `StateStoreToken` | `agentworker.Settings` | env `STATE_STORE_TOKEN` (or `-state-store-token` flag) | Per-task bearer token carried in gRPC metadata by a client interceptor (see §9). |\n| `StoreTimeout` | `agentworker.Settings` | a bounded per-call timeout | Mirrors `natsrpc.Client.Timeout` / the existing `rpcTimeout`; bounds each State Store call when the caller `ctx` has no deadline. |\n\n**Wiring change in the transport façade:**\n\n- `internal/infrastructure/agenttransport/nats.Transport` — `Store(timeout) store.WorkflowStore`\n returns `\u0026storerpc.Client{Conn: t.conn, Timeout: timeout}`. It becomes\n `Store(timeout) workflow.Store` and returns a _single long-lived_ `staterpc.Client` dialled to\n `settings.StateStoreTarget` (with the token interceptor and per-call timeout).\n- `internal/app/agentworker/worker.go` `Settings` gains the three fields above; `taskServiceTransport`\n and `taskDependencies` change their `store` type from `store.WorkflowStore` to\n `workflow.Store` (via `domain/workflow.TaskContext{Store: …}`).\n- Reconnect/deadline: `grpc.NewClient` (or the existing dial path) provides transparent reconnect;\n each RPC is bounded by the per-call context deadline (`StoreTimeout`), matching the\n request/reply timeout the NATS path used. No streaming is involved, so no server-sent\n reconnect semantics apply.\n\n**Deletion rule for the NATS path:** the NATS `storerpc` (Server, Client, subjects\n`archie.store.update|transition|insert_event`) is deleted **only after** the agent has fully\nmoved to gRPC and the container env carries `STATE_STORE_URL`. Until then both paths may coexist\n(agent flips when the daemon starts injecting `STATE_STORE_URL`); there is no dual-live window\nacross a single agent process, which is either NATS-backed or gRPC-backed, never both.\n\n### Operational specifics (in-process listener → address → per-task injection)\n\n- **Who binds the in-process State Store listener:** the daemon (`archied`), at bootstrap, when\n the State Store is served in-process/multiplexed (mirroring `serveGateway` in\n `internal/app/archied/gateway.go`): a `grpc.NewServer()` + `RegisterServer` + graceful stop on\n `ctx.Done()`.\n- **Address selection (Docker-bridge reachable):** reuse the host-gateway resolution already used\n for embedded NATS (`internal/container/network.go` `RequireHostGateway` /\n `resolveHostGateway`). Per §9's single listener-topology rule, the listener binds the\n **host-gateway bridge address** when agent containers consume it (the Phase 2 default), or\n loopback-only when the daemon is the sole consumer. The chosen address is stored on the daemon\n as `d.ConnectedStateStore.URL` (analogous to `d.ConnectedNATS.URL`).\n- **Per-task injection:** `containerEnv(task)` (`internal/daemon/daemon.go`) appends\n `STATE_STORE_URL=\u003cd.ConnectedStateStore.URL\u003e` and `STATE_STORE_TOKEN=\u003ctoken\u003e` alongside the\n existing `NATS_URL`/`NATS_TOKEN`. The agent reads them via env (`-state-store-url` /\n `STATE_STORE_URL`, `-state-store-token` / `STATE_STORE_TOKEN`), the same seam as `NATS_URL`. A\n reloaded `[services.state]` must not point new containers at a store the daemon is not serving\n — the same immutability rule as NATS.\n\n---\n\n## 7. Error semantics\n\n### Current state (both boundaries lose error identity)\n\n- **NATS `natsrpc.Envelope`** (`internal/natsrpc/natsrpc.go`) serialises any error to a single\n `err.Error()` string and rehydrates a fresh `errors.New(string)`. The original type, `%w`\n chain, and sentinel identity are lost → `errors.Is(err, store.ErrStaleTransition)` is\n **false** across the boundary.\n- **gRPC `gatewayrpc`** returns raw `err` from handlers; gRPC-Go converts to\n `codes.Unknown` with only the message. The **only** explicit code in the repo is\n `codes.Unavailable` for a missing session store (`server.go`). No status-code mapping for\n store errors exists.\n\n### Decision — structured mapping so `errors.Is` survives\n\nThe `StateStore` service must map the store's sentinel errors to canonical gRPC codes and\nrehydrate them to the _same_ sentinels on the client, so consumer `errors.Is` checks keep\nworking — an improvement over the flattening it replaces, not a regression.\n\n| Store sentinel / outcome | gRPC code | Client rehydrates to |\n| ------------------------------------------------------------------------------------------------- | -------------------------------------------------------- | ---------------------------- |\n| `store.ErrStaleTransition` | `FailedPrecondition` | `store.ErrStaleTransition` |\n| `store.ErrBindingNotFound` | `NotFound` | `store.ErrBindingNotFound` |\n| `store.ErrMappingNotFound` | `NotFound` | `store.ErrMappingNotFound` |\n| `store.ErrBindingOverlap` | `FailedPrecondition` | `store.ErrBindingOverlap` |\n| `store.ErrBindingTransition` | `FailedPrecondition` | `store.ErrBindingTransition` |\n| `store.ErrAlreadyDispatched` | `AlreadyExists` | `store.ErrAlreadyDispatched` |\n| not-found-as-`(nil,nil)` (`TaskByID`, `GetBinding`, `GetMapping`, `ClaimNext`, `ClaimByIssue`, …) | `NotFound` with a `found=false` field (**not** an error) | `(nil, nil)` |\n| infra / wrapped error | `Internal` (**sanitised** message, see below) | wrapped `fmt.Errorf(...%w)` |\n| capability absent | `Unavailable` | context/typed error |\n\nThe `found=false`-not-error convention matters: the repo deliberately returns `(nil, nil)` for\nnot-found in reads and a sentinel for not-found in writes. The wire must preserve **both**\nconventions (a `GoogleRPCStatus` with `NotFound` is an _error_, which would wrongly surface to\ncallers that expect `(nil, nil)`). Implement `found` as an explicit boolean field on read\nresponses, and map `(nil,nil)` ⇄ `found=false` + `OK`.\n\n### Error-detail sanitisation (rev. 2c)\n\nFull internal error messages must **not** be placed in gRPC `details` for remote callers. SQL\npaths, provider/forge details, secrets, and stack traces are sensitive internals. The rule:\n\n- **Sentinel errors** — the client rehydrates the sentinel from the code; the message string in\n `status.Error` is a short, public, stable phrase (e.g. `\"stale transition\"`), not the raw\n wrapped chain.\n- **Infra errors** — the wire carries a **sanitised** public message + the `Internal` code.\n The **full** error (with `%w` chain, SQL, provider detail) is logged **server-side only** by\n the store service. The client rehydrates a generic wrapped error and must not surface the raw\n text to users.\n\n---\n\n## 8. Limits\n\n**Interface bloat (`interfacebloat` cap = 8)** — `.golangci.yml` `interfacebloat: max: 8`,\nwhich overrides the upstream default of 10. The Go consumer facades must stay ≤8 methods.\n`TaskLifecycle` is exactly 8 (at the cap); `TaskStore` is the 24-method composite that the cap\nis why it is decomposed; proto services bypass the cap (like `ChatContract`).\n\n**Payload caps:**\n\n- Capture body: `defaultCaptureMaxBodyBytes = 256 KiB` (may reduce further; the wire message\n must not exceed gRPC's default 4 MiB receive limit for a single capture). A single capture\n fits, but a _batch_ of them did not: the caller-side cap of up to 100 captures at 256 KiB each\n could exceed 4 MiB in one unary `ListCaptures`/`ListUndispatchedCaptures` response. `.4.7` adds\n `StreamCaptures`/`StreamUndispatchedCaptures` (server-streaming, one capture per message) as\n the path both `staterpc.Client` methods actually call; the original unary RPCs stay defined,\n unmodified and deprecated, only so `buf breaking` protects an in-flight rolling deploy.\n- Task `detail`/`park_reason`/event `Detail`: 4000-char hard cap (`clip(s, 4000)`).\n\n**Retention (stays server-side in the store; the wire does not carry pruning decisions):**\n\n- `events` table: **no prune, unbounded growth** — a known risk to revisit, not changed here.\n- `captured_events`: prune-on-write, default 7 days / 5000 events.\n\n**Pagination/arity:** store read methods take a raw `int limit` (no store-side cap); the\ncaller-side caps (task list 100, log 2000, captures 100, capture scan window 5000, binding\ndispatch batch 100, session list 20) remain caller-side. The wire passes `limit int` and returns\n`[]*` slices, mirroring the existing Go signatures — **no cursor/paging protocol is introduced**\nin this work (the current `EventsSince(sinceID, limit)` offset cursor is the only one, and it\nstays as-is).\n\n---\n\n## 9. Transport security boundary\n\nThe gateway precedent is loopback-only, unauthenticated gRPC. The State Store holds the\ncrown-jewel `archie.db` (task/event/capture/mapping/binding data), so its boundary must be\nstated explicitly rather than left implicit.\n\n**The State Store's equivalent boundary (rev. 2c):**\n\n- **Listener topology — single rule (rev. 2c).** One in-process listener; its bind address is\n chosen by the consumer set, never both at once:\n - _Agent containers consume it_ (the Phase 2 driver during multiplexed serving) → bind to the\n **host-gateway bridge address** so containers can reach it, and **token auth is mandatory**\n (a bridge host is reachable by any process on that bridge). A container **cannot reach the\n host's `127.0.0.1`**, so loopback-only is NOT usable for the agent-consumed path.\n - _Only the daemon consumes it_ (no agent) → bind **loopback-only** (`--listen 127.0.0.1`),\n `insecure` is fine because it is not network-reachable; no token required.\n - There is **no dual-listener** design. The practical default during `.4.2` (the reason\n in-process serving exists is to give the agent a gRPC target) is the **bridge address +\n token**. `127.0.0.1` is reserved for the daemon-local-only case.\n- **Agent access → the container bridge.** The agent runs in a task-scoped container on the\n host and reaches the State Store over the Docker bridge gateway (the bind address from the\n topology rule above), exactly as it reaches the daemon's NATS. To close the \"any host\n process could reach the store\" gap, the agent **authenticates with a per-task bearer token**\n carried in gRPC metadata by a client interceptor (`StateStoreToken`, §6). Network\n reachability + the token is the trust boundary for the default bridge topology.\n- **The daemon** uses the local adapter (`*store.Store`) in-process by default, so no network\n boundary exists until `[services.state].target` is set; when it is set, the daemon client\n authenticates with a token supplied alongside the target.\n- **Non-loopback / k8s:** TLS/mTLS is an **operator decision**, not a default. Any non-loopback\n exposure (e.g. `--listen 0.0.0.0`, a CDN, a cross-host service) requires an explicit decision\n and mTLS or token auth — mirroring the gateway's `--listen`/`--enable-cors-header`\n confirmation caveat. The NATS-KV discovery fallback supplies endpoints but does not add\n transport security; that stays the operator's responsibility.\n\n### Token lifecycle (rev. 2c; scoping added in `.4.7`)\n\n- **Generation:** the daemon generates a per-task bearer token when it acquires the container\n (or per incumbence) and injects it via `containerEnv` as `STATE_STORE_TOKEN`. A single token\n may cover a task's container lifetime.\n- **Registration (`.4.7`, post-standalone-cutover):** the State Store is now its own process\n (`.4.3`/`.4.6`), so the daemon can no longer just hold an in-process issued-token set for the\n server to check — it registers the token at the remote authority via the\n `RegisterTaskGrant`/`RevokeTaskGrant` RPCs (`staterpc.GrantIssuer`, `daemon.StateStoreGrantIssuer`),\n admin-token-authenticated calls a task-scoped token itself can never make.\n- **Lifetime:** tied to the task + container grace period; rotated on a new container acquisition\n for the same task; a token for a released container is invalidated (`RevokeTaskGrant`, deferred\n alongside `ContainerPool.Release`). An unrevoked grant still expires server-side (bounded by\n `RegisterTaskGrantRequest.lifetime_seconds`, capped at 7 days) so a daemon crash before revoke\n cannot leave a grant valid indefinitely.\n- **Validation and scope (`.4.7`):** a gRPC interceptor (`staterpc.TaskGrants.UnaryInterceptor`)\n validates the token against the State Store's own issued-grant set; missing/unknown/expired →\n `codes.Unauthenticated`. Unlike the daemon's own administrative token (full access to all RPCs),\n a task-scoped grant additionally authorises only `Update`/`Transition`/`InsertEvent` on\n its own task ID, plus the two workflow-caller RPCs naming that same task ID\n (`EnqueueCallTask`/`WorkflowCallStatus`, `docs/prds/workflow-calls.md` — the one\n sanctioned widening: the callee is the caller's child). Every other RPC, including the\n streaming capture surface and\n `RegisterTaskGrant`/`RevokeTaskGrant` themselves, is `codes.PermissionDenied` for a task-scoped\n caller. The token is carried in gRPC metadata, never in a URL.\n\n### Fail-closed on non-loopback (rev. 2c)\n\nIf `[services.state].target` points to a non-loopback address and **neither** TLS **nor** a token\nis configured, the State Store client (daemon or agent) **fails closed** — it refuses to\nstart/dial rather than fall back to a plaintext, unauthenticated remote connection. This mirrors\nthe gateway's `--listen`/`--enable-cors-header` confinement caveat and prevents a silent\nunauthenticated remote store. The loopback + token default is the only path that starts without\nan explicit operator decision.\n\n---\n\n## 10. Mode / config shape\n\n**Follow the gateway's presence-based `target` seam, not the NATS `mode` enum.**\n\n- **Services are keyed by name, not named as struct fields.** A\n `State ServiceConnection` field per service is the duplication\n `docs/prds/service-registry.md` removes, and this document's summary describes\n the seam as `[services.\u003cname\u003e]`:\n ```go\n type Services map[string]ServiceConnection\n ```\n registered once via `RegisterService(context, name, target, listen, tokenEnv)`. The TOML\n shape below is unchanged, so nothing else in this section is affected; read\n `cfg.Services.Get(\"state\")` wherever it says `cfg.Services.State`.\n- **Empty `Target` → startup error.** The local `*store.Store` adapter default was\n withdrawn when the State Store was extracted (§12 step 7): the standalone `archie-state-store` process\n exclusively owns `archie.db`, so both `archied` and `archie-gateway` now reject an empty\n target (`internal/app/archied/bootstrap.go` and `state_store_client.go`: `services.state.target\nis required`). **Set `Target` → dial gRPC** with `*staterpc.Client`; no client opens the\n database itself. This differs from the gateway, where `Target` is defaulted to\n `127.0.0.1:8585` because `archie-gateway` is already a separate process whose client and\n server share one default.\n- `[services.state]` is present and **uncommented** in `config.example.toml`, with `target`\n marked REQUIRED, because the value is load-bearing for every client. A commented-out\n `target` cannot boot, so the section ships uncommented.\n- Composition root: `internal/app/archied/bootstrap.go` (the daemon) and\n `internal/app/archieui/config.go` (the UI process) both read `cfg.Services.Get(\"state\")` and\n dial the remote State Store. The task store is built only in the standalone\n `archie-state-store` process (`internal/app/archied/state_store.go`), which is the only path\n that opens the task tables.\n\n\u003e **Mode fork, decided.** The gateway uses a single `target` address (no enum). The State Store\n\u003e boundary uses the same seam for symmetry, and **both now require a non-empty `target`**: the\n\u003e State Store's local-store default was withdrawn when the process was extracted (§12 step 7).\n\u003e No `mode` field is added. A companion `STATE_STORE_TOKEN` secret (or a\n\u003e `[services.state] target_token` key) carries authentication.\n\n---\n\n## 11. Local-adapter compatibility + conformance\n\n- `*store.Store` (local) and `*staterpc.Client` (remote) satisfy the **same** contract\n interfaces. Use **explicit** `var _` assertions on **both** adapters (the gateway's local\n adapters use only structural/return-type assertions; the store precedent's explicit `var _`\n style is stronger and should be adopted on both sides).\n ```go\n var _ workflow.Store = (*staterpc.Client)(nil) // first contract to move (agent)\n var _ store.TaskStore = (*staterpc.Client)(nil) // daemon path, added progressively\n var _ store.CaptureStore = (*staterpc.Client)(nil)\n var _ store.MappingStore = (*staterpc.Client)(nil)\n var _ store.BindingStore = (*staterpc.Client)(nil)\n var _ store.BindingDispatcher = (*staterpc.Client)(nil)\n var _ store.BindingTaskCreator = (*staterpc.Client)(nil)\n var _ store.PlaybookDispatcher = (*staterpc.Client)(nil)\n // local side (in store), already present:\n var _ workflow.Store = (*Store)(nil)\n ```\n- **Conformance suite:** a `staterpc/conformance_test.go` bufconn suite runs the **same**\n contract through both `\"local\"` and `\"grpc\"` adapters (mirroring\n `gatewayrpc/conformance_test.go`), asserting error-sentinel fidelity and the\n not-found-as-`(nil,nil)` convention both ways.\n- **`Close()`:** stays on the Go store interfaces (so `TaskStore`/`TaskEvents` remain fully\n satisfiable) but is **not** a `StateStore` RPC. The remote `staterpc.Client` implements\n `Close()` as a **no-op**; the store service owns its own DB lifecycle. When the daemon hands\n the store to `archie-state-store`, `b.cleanup()` no longer calls `store.Close()` on a remote\n client (it is the daemon's own lifecycle only when it owns the store in-process).\n\n---\n\n## 12. Cutover / deletion rules\n\n1. **Ratify the contract (this document).** No code.\n2. **Domain type + interface relocation (prerequisite, coupled with §4).** Move `Task`/`Status`/\n `Source` and the `WorkflowStore` interface into `internal/domain/workflow` (as\n `workflow.Task`/`workflow.Status`/`workflow.Source`/`workflow.Store`), so\n `internal/domain/workflow` imports no `internal/store`. Update the store tests and the 6\n workflow production files. **This must land before or with `.4.2`, because `.4.2` cannot move\n the agent's store contract off `store.WorkflowStore` while the domain still imports it.**\n3. **`.4.2` — generalise `storerpc` to the gRPC contract.** Move the agent's `workflow.Store`\n (`Update`/`Transition`/`InsertEvent`) from NATS JSON `storerpc` to `staterpc.Client` over\n gRPC, using the §6 handoff. The daemon serves the gRPC `StateStore` server **in-process\n (multiplexed)** first so the agent has a gRPC target before the standalone binary exists\n (PRD §3 \"multiplexed in-process is the migration default\"). Bind the in-process listener and\n inject `STATE_STORE_URL`/`STATE_STORE_TOKEN` per task (see §6 \"Operational specifics\"). Also\n land the `RecordDispatch` `*sql.Tx` drop (see §5) in the same span. Flip the agent's\n `Transport.Store` to a gRPC client once `STATE_STORE_URL` is injected.\n4. **Delete NATS `storerpc` only after the agent is fully on gRPC** — delete `internal/storerpc`\n (Server, Client, subjects `archie.store.update|transition|insert_event`), the\n `natsrpc.RegisterAll` registration, and the now-unused `store.WorkflowStore` NATS path in\n `internal/infrastructure/agenttransport/nats`, in the same commit.\n5. **`[services.state].target` stays empty (inproc)** through `.4.2`; the `archie-state-store`\n `cmd/` binary is **`.4.3`** and hosts the same `StateStore` gRPC service.\n6. **Swap daemon-internal contracts one at a time** (only once the daemon points at\n `archie-state-store`): `Capture → Mapping → Binding → TaskStore`. Each swap is independent —\n add the remote-client composition path, run the per-contract conformance suite, flip\n `[services.state].target`, delete the local use. A consumer mixes in-process and remote\n contracts for a contract-code (no dual-live _window_; each contract is either local or remote\n at a time).\n7. **Final flip / deletion of the in-process serving path:** when the daemon no longer owns the\n store (i.e. `TaskStore` has moved), delete the in-process task-store /\n `wireWebStoreSurfaces` path in the daemon in a **same-commit** deletion — no dual-store\n ownership. The single SQLite file remains owned by `archie-state-store`.\n8. **Never** split `archie.db` or add a second store. The gateway keeps its own already-separate\n session SQLite; that is unchanged and out of state-store scope.\n\n---\n\n## 13. Decisions recorded (for the review trail)\n\n- **`internal/store` is a transitional compatibility location** (target\n `internal/infrastructure/store`), not the final infrastructure boundary. The producer-owned\n daemon/webui store surfaces live there now but may relocate with the implementation.\n- **In-process listener + token lifecycle:** the daemon binds the in-process State Store\n listener to the **host-gateway bridge address** when agent containers consume it (the Phase 2\n default) or loopback-only when the daemon is the sole consumer — one listener, one address\n rule, never both (rev. 2c). It selects the bridge address by reusing the embedded-NATS\n host-gateway resolution and injects `STATE_STORE_URL`/`STATE_STORE_TOKEN` per task via\n `containerEnv`. Token is per-incumbence (task + container grace period) and validated by a\n gRPC interceptor → `codes.Unauthenticated` on missing/unknown/expired. Non-loopback targets\n fail closed without TLS or a token.\n- **§12 step 2 (domain type + interface relocation) is done** — `internal/domain/workflow`\n imports no `internal/store` symbol and `store.WorkflowStore` is gone from `TaskContext`. The\n broader Phase 2 acceptance criterion (State Store as its own gRPC service) is still open;\n `.4.2` picks up from here.\n- **Ownership split:** domain-facing workflow contract is **consumer-owned** in the domain\n (`workflow.Store` + `workflow.Task`/`Status`/`Source`, per dependency rules #2 and #7);\n daemon/webui store surfaces stay **producer-owned** in `internal/store`. `store.WorkflowStore`\n is superseded by `workflow.Store`.\n- **One `StateStore` gRPC service** (49 contract RPCs, grouped by contract) + narrow Go consumer facades\n (≤8) on `staterpc.Client` — mirrors the single-`ChatService` precedent.\n- **Domain type relocation** (`Task`/`Status`/`Source` → `internal/domain/workflow`) is a\n Phase 2 prerequisite, pulled forward from migration-decisions §4 (minimal bound,\n `Task → WorkflowExecution`); the rest of §4 stays a separate migration.\n- **`RecordDispatch` drops `*sql.Tx`** (wire-blocker; production passes `nil`) — a\n required interface migration landing before/with `.4.2`.\n- **`Close()` not on the wire**; remote client `Close()` is a no-op.\n- **Structured gRPC error codes** with client rehydration to store sentinels (preserves\n `errors.Is`), a `found=false` + `OK` convention for read not-found, and **sanitised public\n error details** (full detail logged server-side only).\n- **Agent handoff:** `agentworker.Settings` gains `StateStoreTarget`/`StateStoreToken`/\n `StoreTimeout`, injected as `STATE_STORE_URL`/`STATE_STORE_TOKEN` env (mirrors the NATS\n handoff); `Transport.Store` returns a long-lived gRPC `workflow.Store` client.\n- **Transport security + listener topology (rev. 2c):** one in-process listener, bound to the\n host-gateway bridge address when agent containers consume it (the Phase 2 default) or\n loopback-only for a daemon-local-only consumer. `insecure` only on loopback; mandatory\n per-task bearer-token auth on the bridge; TLS/mTLS is an operator choice for any\n non-loopback/k8s topology.\n- **Mode = presence-based `[services.state].target`**, default **local** (not yet extracted);\n add `[services.state]` to `config.example.toml`.\n- **Explicit `var _` assertions on both adapters**; per-contract bufconn conformance.\n- **Migration order:** domain type/interface relocation → `workflow.Store` (agent) → stand up\n `archie-state-store` → `Capture` → `Mapping` → `Binding` → `TaskStore`; NATS `storerpc`\n deleted once the agent is on gRPC; in-process store path deleted in a same-commit flip.\n- **Orphans `ClearTerminalTasks`/`IncrementRetryCount`** retained but not built upon.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/status-health-surface/",
|
||
"title": "/status health surface",
|
||
"section": "PRDs",
|
||
"body": "# /status health surface\n\n**Status:** Finalised\n**Beads issue:** `archie-core-wp9s`\n\nA reported field must have a truthful source. `/status` reports broker\nconnectivity and the last chat-model call; the container pool, last poll and\nchannel state have no source in the process that answers the command, so they\nare left out rather than faked.\n\n## Background\n\n`archie-core-wp9s` (\"/status and /tasks have the wrong division of labour\")\nasked for two things: `/tasks` owning the live per-task work view, and\n`/status` owning daemon health -- \"broker connectivity, provider\nreachability, channel state, container pool, queue depth, last poll,\nversion\" -- with neither duplicating the other or static config available\nelsewhere (`/model` already lists active provider/model).\n\n## Phase 1\n\n- `/tasks`: new command, `gateway.ChatTaskSummary` extended with\n `Stage`/`Attempt`/`ParkReason`/`UpdatedAt` (all already persisted on\n `store.Task`, zero new instrumentation), sorted actionable-first.\n- `/status`: itemised per-status task counts replaced with one aggregate\n `Queue: N in flight (...)` line (running/waiting/parked only -- queued and\n terminal states excluded as not a health concern). Runtime\n (provider/model) is temporarily left in place; see \"Runtime (provider/model)\"\n under Phase 2's \"Truthful or absent\" for where that stands now.\n\nNot touched: `/agents` (`gateway.AgentReader`/`Router.Agents`) is a separate,\npre-existing command that already has the right shape for \"list active\nwork\" but is never wired to a real implementation in any composition-root\ncall site (`bootstrap.go`, `telegram_setup.go`) -- it always answers \"Agent\nlisting is not configured.\" in production. That is a standalone bug\n(`archie-core-mxls`), not folded into this work: deciding whether to wire\nit, repurpose it, or delete it in favour of `/tasks` is Sam's call, not\nsomething to resolve by implication here.\n\n## Phase 2 -- health checks, verified against the actual tree\n\nEach bullet below was checked against real code before being called cheap\nor not; nothing here is guessed. Both open questions were settled (see \"The two\ndecisions\"). The health section reports the broker connection and the last\nchat-model call; the three facts the answering process cannot see are covered\nunder \"Truthful or absent\".\n\n### Verified as cheap against the tree\n\n- **Broker connectivity.** `internal/infrastructure/eventbus/nats.Client`\n already exposes `CoreConn() (*nats.Conn, error)`\n (`client.go`). The underlying `nats.Conn` (nats.go) has its own\n `Status()`/`IsConnected()`. A `Client.Connected() bool` wrapper is a few\n lines.\n- **Container pool.** `container.Pool` already tracks `active int` under\n its own mutex (`pool.go`) but has no public accessor. A\n `Pool.Active() int` getter is a few lines.\n- **Last poll.** Nothing tracks this. The daemon's poll loop\n (`internal/daemon`) needs one `lastPollAt time.Time` field (atomic or\n mutex-guarded, matching the pool's own pattern) set at the top of each\n poll tick, with a getter.\n- **Queue depth.** Covered by Phase 1.\n\n### The two decisions\n\n- **Provider reachability is last-known outcome, never a live probe.** A chat\n command must not perform a blocking network call in its handler, and a probe\n reports the probe's reachability rather than the daemon's. `/status` therefore\n prints what actually happened -- the last chat-model call's model, age and\n error, or \"no calls attempted yet\" when there has been none. Implemented as\n `providerOutcomeRecorder`, written once in `sendChatTurn` (the single point\n every chat-model call in the process passes through) and read back by\n `newStatusHealth`. Nothing on this path dials a provider. The existing live\n model reachability check (`readiness.go`'s `modelReachProbe`) is unchanged and\n stays where it belongs: on `/health/detailed`, off the chat path.\n- **Channel state comes from the existing ChannelManager.** `/status` reads\n `status.Manager.Snapshot()` -- the same ledger the dashboard and the readiness\n probe already use -- rather than re-deriving channel health from each adapter.\n No method was added to the `Channel`/`Gateway` contract.\n\n### Phase 2\n\n- **Broker connectivity.** `internal/infrastructure/eventbus/nats.Client` gained\n `Connected() bool`, which reads the connection the process already holds. The\n standalone Gateway (which dials NATS itself for task actions) reports the same\n fact from that connection.\n- **Container pool.** `container.Pool.Active()` and `Pool.Cap()` read the pool's\n own counter and configured `max_concurrency` -- not a Docker listing, which\n would include containers this pool does not own. A cap of zero means\n unlimited, so `/status` renders the count alone rather than \"1/0 active\".\n- **Last poll.** `daemon.Daemon.LastPollAt()` reports when the most recent poll\n pass _began_; both poll paths (`poll` for single-identity `Run`,\n `pollForIdentity` for `runIdentities`) stamp it. A pass that hangs leaves the\n stamp stale, which is the signal -- stamping completion instead would leave a\n wedged poller looking healthy.\n- **Queue depth.** Covered by Phase 1.\n- **Rendering.** `gateway.formatHealth` renders one line per reported fact\n (`Broker:`, `Containers:`, `Channels:`, `Chat model:`, `Last poll:`) between\n the queue line and the runtime block. A `/status` reply carries `Broker:` and\n `Chat model:`: the Gateway is the only process that serves a Router, and it\n holds neither a container pool, a poll loop nor a channel manager. The other\n three lines render when a report carries them. The broker source is a\n function rather than a captured value because the connection differs by\n composition: the daemon's shared eventbus client in one process, the\n Gateway's own task-actions connection in the other.\n\n### Truthful or absent -- what is deliberately not reported\n\n- **Version.** Still excluded: `/version` owns it, and duplicating it back into\n `/status` is the exact \"answers something available elsewhere in chat\"\n complaint this ticket exists to fix.\n- **Sections the standalone Gateway process cannot see.** The daemon owns the\n poll loop, the container pool and the channel manager; the Gateway process\n owns none of them. Its `/status` reports broker and chat-model health and\n omits pool, polling and channels rather than printing zeroes. Aggregating them\n across processes would need a new daemon RPC -- a new subsystem, out of scope\n here. For the same reason the chat-model line is per process: the daemon\n records the turns it runs (Telegram, email, webhook), the Gateway records the\n web chat's, and the two do not pool outcomes. Each line is true of the process\n answering the command. **Decided: they stay per-process.** Pooling\n them would mean a new daemon↔gateway health RPC for one status line, and a\n per-process line is true where an aggregate would be a claim about a process\n the answering one cannot see. The composition root therefore builds the\n health source from the broker connection and the chat-model recorder alone: a\n producer for a daemon-owned fact would build a section no reply can carry.\n- **A channel's failure reason can outlive the failure.** `status.Manager` keeps\n the last non-empty `Detail` across state changes, and that one field carries\n both a descriptor's standing caveat and a runtime failure reason, so a channel\n that failed and later recovered can still render the old reason. Splitting the\n two belongs at the producer (the manager and its descriptor), not in this\n reader, which reads the field exactly as the dashboard does.\n- **Container pool when Docker is unavailable.** Composition leaves\n `containerPool` nil, so the section is absent. Naming a reason would mean\n carrying Docker's startup error into the health surface.\n- **Runtime (provider/model).** Phase 1 left this block in place; it is still\n here. It is static config that `/model` and `/whoami` already answer, so it is\n the only line in `/status` that this ticket's own rule says should not be\n there -- but the maintainer reads `/status` daily to see the active model at a\n glance, and one command answering both \"is archie well\" and \"what is it\n running on\" is worth that duplication. **Decided: keep it.** It is\n a settled product decision, not an oversight; do not remove it as a\n mechanical follow-up.\n- **`/agents` copy.** The menu still advertises \"List tasks being\n worked\" while `Router.Agents` is never wired (`archie-core-mxls`), so that one\n published description does not match what its command does. Deleting,\n rewiring or redefining `/agents` is that bead's call and is not folded in here.\n\n## Packages this touches (Phase 2)\n\n- `internal/infrastructure/eventbus/nats`: `Client.Connected()`.\n- `internal/container`: `Pool.Active()`, `Pool.Cap()`.\n- `internal/daemon`: `lastPollAt`/`markPoll`/`LastPollAt()`.\n- `internal/gateway`: `health.go` (health contract + renderer), `Router.Health`,\n `formatStatus`.\n- `internal/app/archied`: `status_health.go` (the composition-root bridge),\n `ProviderOutcomes` threaded through `telegramSetup` into every turn runner,\n and `sendChatTurn` recording each call's outcome.\n- `internal/domain/messaging` + `internal/channels/telegram`: `/status` and\n `/tasks` descriptions, which now describe both commands' actual jobs on both\n published surfaces.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/store/",
|
||
"title": "Store",
|
||
"section": "PRDs",
|
||
"body": "# Store\n\n**Status:** Approved\n**Authority:** `docs/architecture/plugins-and-extensions.md`; supersedes\n`docs/prds/extensions-center.md`, whose installed-inventory states are\nadopted below.\n**Compounds with:** `docs/prds/external-agent-harness.md` (harness Kits),\n`docs/prds/pr-review-agent.md` (the first package).\n\n## Decision\n\nOperators get capabilities from a store: browse, install, update and remove\npackages from the dashboard, with no config file editing. A package is an\nOCI image in a registry. A store catalogue is a list of package references.\nInstalling pins a digest. Every package declares the authority it needs, and\nan update that asks for more stops for the operator's approval.\n\n## Packages\n\nTwo kinds, both ordinary OCI images:\n\n- **Harness Kits.** Kit v3 images, resolved as\n `docs/prds/external-agent-harness.md` decides. Their authority is their\n Kit permission surface.\n- **Archie packages.** An image whose manifest annotation\n `dev.archie.package.v1` carries the descriptor and whose single layer\n carries files. A package contributes any of: workflow definitions,\n prompts, skills, MCP server definitions, and default settings for them.\n\nArchie packages are declarative. They carry no Go, Yaegi or script code that\narchie executes on the host. Anything that runs code ships as a Kit and runs\nin a sandbox. An archie package that references a Kit names it by digest,\nand installing the package installs the Kit.\n\nThe descriptor declares: display name, description, version, the families\nit contributes, the packages it requires, and its authority.\n\n## Authority\n\nA package's authority is the set of grants it needs to work:\n\n- credential services (from its Kits and MCP servers);\n- egress hosts (from its Kits);\n- forge permissions its workflows use: read, comment, review, push, open\n pull request;\n- triggers it registers: watched-repository events, schedules;\n- tools it enables for agents.\n\nInstall shows this list and records the operator's acceptance against the\ndigest. An update whose authority is equal or narrower applies under the\npackage's update policy. An update whose authority is wider waits for\napproval, showing only what widened. Nothing a package does at runtime can\nexceed its accepted authority: archie checks forge actions, triggers and\ntool grants against it, and the egress proxy enforces the Kit's grants.\n\n## Catalogues\n\nA catalogue is an OCI artifact listing package references with digests,\nsigned by the catalogue's key. Archie ships with the default catalogue and\nits public key configured. An operator can add catalogues, for example a\nprivate registry, by reference and public key. A catalogue without a key is\nrefused unless the operator marks it unsigned, which the dashboard shows on\nevery package it supplies.\n\nInstall and update verify the catalogue signature and the package digest.\nArchie never installs a tag it has not resolved to a digest through a\nverified catalogue.\n\n## Organisations\n\n`docs/prds/orgs-and-access.md` is the boundary, and the store follows its\nsplit between the instance and each org.\n\n- **Instance.** Catalogues and their keys are instance configuration, set by\n instance admins. Every org browses the same catalogues.\n- **Org.** Installing, updating, removing, accepting authority and setting the\n update policy are org actions. An installed package, its pinned digest and\n its accepted authority belong to one org. Two orgs can run different\n versions of one package.\n- **Contributions** become org resources: workflows are org workflows,\n profiles are org profiles, and credential bindings name org secrets. A\n workspace uses them only once the org grants them to it.\n- **Authority is an upper bound, not a grant.** Accepting a package's\n authority permits the package to ask; the workflow's identity must still\n hold each grant. A package never creates, grants to or widens an identity.\n- Install and authority acceptance are the `create` and `approve` actions on\n the package, decided by the Authorizer. Org approval policy decides who may\n accept, and no member accepts authority they asked for.\n\n## Install, update, remove\n\n- Installed packages, their pinned digests, accepted authority and update\n policy are org resources in the State Store. They apply without a restart.\n- **Update policy** per package: `manual` (default) or `auto`. `auto` applies\n non-widening updates when the catalogue lists them; widening updates always\n wait.\n- **Setup.** A package whose authority includes a credential with no binding\n installs as `needs_setup`, and links to the binding form or the harness\n setup terminal. Its triggers do not fire until setup completes.\n- **Remove** stops its triggers, removes its contributions, and leaves any\n credential bindings for the operator to delete. A package another\n installed package requires cannot be removed first.\n- Rolling back reinstalls the previously accepted digest.\n\n## Inventory\n\nThe installed view lists every package and every bundled capability with\none state, computed on read and never persisted:\n\n| State | Meaning |\n| ------------- | -------------------------------------------------------- |\n| `available` | in a catalogue or bundled, not installed |\n| `needs_setup` | installed, waiting for a credential binding |\n| `active` | installed and running |\n| `degraded` | active, and its family's health check reports a fault |\n| `pending` | an update waiting for approval of widened authority |\n\nCore capabilities (identity, workflow engine, State Store) are listed and\ncannot be removed.\n\n## Out of scope\n\n- Payment, licensing and paid packages.\n- Publishing from the dashboard. Packages are built and pushed with standard\n image tooling.\n- Ratings and reviews.\n\n## Plan\n\n1. Archie package descriptor and its validation, including declarative-only\n content.\n2. Installed-package resources, digest pinning, install and remove for\n archie packages from a local registry.\n3. Authority model: extraction from packages and Kits, acceptance records,\n runtime checks for forge actions, triggers and tools.\n4. Signed catalogues and verification; the default catalogue.\n5. Updates: widening detection, approval, `auto` policy, rollback.\n6. Kit packages through the same flow; `needs_setup`.\n7. Inventory states across packages and bundled capabilities.\n8. The PR review agent published as the first package.\n\n## Verification\n\n- Installing the PR review package from the default catalogue on a fresh\n instance, then binding a credential, reviews a PR without editing\n configuration.\n- An update adding a forge permission or egress host waits as `pending`,\n showing only the added grant, and does not apply under `auto`.\n- A workflow from a package that tries a forge action outside its accepted\n authority fails the step.\n- A tampered package digest or an unsigned catalogue refuses to install.\n- A package containing executable host code is rejected at validation.\n- Rolling back restores the prior digest and its accepted authority.\n- A package installed in org A is invisible to org B, and org B's workflows\n cannot select its profiles.\n- A package whose workflow needs a forge permission the running identity was\n never granted fails that step, even though the org accepted the package's\n authority.\n- A member cannot accept widened authority on an update they requested.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/task-logs-pagination/",
|
||
"title": "task_logs pagination and bounded retrieval -- decision",
|
||
"section": "PRDs",
|
||
"body": "# task_logs pagination and bounded retrieval -- decision\n\n**Status:** Approved\n**Date:** 2026-08-26\n**Beads issue:** `archie-core-vq92`\n**Compounds with:** `archie-core-p1bl` (errors serialise as `{}`)\n\n## Decision\n\n`task_logs` becomes a _cursor-paginated, filterable_ bounded read, not a\none-shot head-of-file window. The reader in `internal/logging` gains a\nsecond entry point (`Page`) that walks the file forward from a byte\noffset; the tool schema in `internal/gateway/task_tools.go` surfaces\nlevel, time-range and cursor parameters; the response carries an explicit\n\"more available\" signal so a model can either keep paging or hand the\nrest to `send_file` when the operator wants a complete archive.\n\nWe do NOT add an unbounded read path. A single response is still capped at\n`MaxTailLines` (2000) entries. What changes is that pagination makes the\ncap reachable in practice instead of silently terminal.\n\n## Why now\n\n`archie-core-vq92` is the right place to fix this: the chat tool is the\nuser-visible failure mode, but the same defect propagates to every caller\nof `internal/logging.Tail` (`internal/webui/api_logs.go` and the\n`chatTaskLogReaderAdapter` in `cmd/archied/main.go`). Patching the\nreader and the tool together is one change in two layers, not two\nchanges; CLAUDE.md's \"package owns its format end-to-end\" rule means the\nreader is the right place for cursor support regardless.\n\nThe compound with `p1bl` matters: a truncated window of uninformative\nerrors leaves an operator with nothing. Cursor pagination lets the model\nwalk the whole file when a level filter does not narrow enough, and the\nresponse's \"more available\" signal means it can honestly say \"I read N of\nM and stopped there\" instead of presenting the head as complete.\n\n## Shape of the API\n\n### Reader (`internal/logging`)\n\nAdd a second entry point next to `Tail`:\n\n```go\n// Page walks the log forward from cursor (a byte offset; 0 starts at\n// the beginning of the scanned window), applying the same filters as\n// Tail. PageResult.Cursor is the file offset just past the last line\n// the walk returned; pass it back as Cursor to continue, and stop when\n// MoreAvailable is false.\nfunc Page(path string, q Query, cursor int64) (PageResult, error)\n```\n\n`PageResult` carries `Entries`, `Truncated` (the existing meaning --\nolder entries exist that were not examined within `maxScanBytes`),\n`Cursor` (the byte offset the next page should resume from), and\n`MoreAvailable` (true iff the scan saw more matching entries beyond the\nreturned page within the same call).\n\n`Cursor` is a byte offset rather than an entry ID for two reasons.\nFirst, `TaskSink` writes JSONL through `slog.NewJSONHandler` directly --\nwithout routing through `FeedHandler` -- so the on-disk entries carry no\n`id` field and a per-task attempt monotonic counter would require\neither a sink-side change or a reader-side reconstruction that is\nbrittle against rotation. Second, the byte offset uniquely identifies\n\"the next unread position\" in a sequential scan, which is exactly what\nthe call pattern needs. A cursor that points at or past the file's\ncurrent size is treated as end-of-stream (empty page, `MoreAvailable`\nfalse).\n\nThe implementation is `readWindow` (reads the tail-most `maxScanBytes`\ninto a single in-memory buffer) plus `readLines` (clamps the cursor into\nthe window, drops a partial first line on the size-cap seek, then\ndelegates) plus `walkWindow` (walks the buffer line-by-line by exact\nslice offset). Offsets are into one contiguous buffer, so they are\ntrivially correct -- unlike an incremental reader over the file, whose\n`bufio.Reader` buffer-ahead would make its underlying file position\nunusable as a cursor.\n\n`Tail` is NOT a wrapper over `Page`: it keeps its legacy ring-buffer\nsemantics (newest `Limit` matches, oldest dropped, `Truncated` set on\nring overflow). They share `readWindow`/`walkWindow` but stay separate\nentry points precisely so Tail's established behaviour is unchanged.\n\n### `Query` gains two optional fields\n\n```go\nSince time.Time // inclusive lower bound on entry.Time\nUntil time.Time // inclusive upper bound on entry.Time\n```\n\n`Levels`, `Component`, `Contains`, `Limit` are unchanged. The existing\nfilter logic and tests move unchanged.\n\n### Tool schema (`internal/gateway/task_tools.go`)\n\n`ChatTaskLogQuery` gains three fields:\n\n- `Level []string` (forwarded to `Query.Levels`)\n- `Since time.Time` / `Until time.Time` (forwarded as `Query.Since/Until`)\n- `AfterID int64` (the cursor; 0 to start at the beginning)\n\n`ChatTaskLogResult` gains two fields:\n\n- `Cursor int64` -- highest ID in this page; pass back as `AfterID` to continue\n- `MoreAvailable bool` -- true iff the scan saw more matching entries that\n did not fit in this page\n\nThe tool description is updated to teach the model the pagination\nprotocol in one sentence: \"pass the previous `cursor` back as `after_id`\nuntil `more_available` is false; for an entire archive, prefer\n`send_file`.\"\n\n### Production adapter (`cmd/archied/main.go`)\n\n`chatTaskLogReaderAdapter` passes the new fields through to the reader.\n`TaskRegistry.Path` is already attempt-scoped, so paging stays inside\none attempt.\n\n## What we deliberately do NOT do\n\n- **No time-range-only read without a cursor.** Time bounds are a filter,\n not a window. A model that wants only \"errors in the last hour\" passes\n both `Levels=[ERROR]` and `Since=now-1h`. The page is still the same\n size; the response's `MoreAvailable` tells the caller whether the hour\n fit.\n- **No \"save the full log\" via the tool.** That is what\n `archie-core-absb`'s `send_file` is for. The PRD description for vq92\n raised this as an open question; the answer is \"use the existing tool\"\n rather than building a parallel archive endpoint.\n- **No reversal or random access.** Paging is forward-only on the byte\n offset. Reverse scan would require either an index or a full file\n pass; neither earns its cost given `Tail` already handles \"show me\n the end.\"\n- **No bumping `MaxTailLines` past 2000.** A single response must not be\n able to exhaust the model's context. Paging makes the cap _reachable_\n but does not relax it.\n- **No raising `maxScanBytes`, and no walking rotated generations.** See\n the ceiling below; lifting it is a separate decision with its own cost.\n\n## The scan-window ceiling, and what it means for \"complete\"\n\n`maxScanBytes` (8 MiB) bounds the tail of the file any single call will\nexamine. Paging does **not** lift that bound: every call re-derives the\nwindow from the file's _current_ size and clamps the incoming cursor up\ninto it, so a paging loop walks the window exhaustively and then stops.\nFor a log larger than 8 MiB the oldest bytes are unreachable through\n`task_logs` at any cursor, and rotated generations (`attempt-N.jsonl.1`,\n`.2`, ...) are separate files the reader never opens.\n\nThis is a deliberate limit, not an oversight, and it is why criterion 1\nbelow says \"paging **or** file delivery\" rather than \"paging\". The\nhonest contract is:\n\n- Within the window: paging reaches every matching entry, in order,\n exactly once.\n- Beyond it: `Truncated` is true, and that is the _only_ correct thing\n to report. An agent that pages to `more_available == false` while\n `truncated` is true has read the window, not the log, and must say so.\n\nLifting the ceiling would mean either buffering an unbounded file or\nteaching the reader to walk rotation history. Neither is needed to close\nvq92 — the issue's own text accepts file delivery as the answer for\n\"save the full logs\" — so both are deliberately out of scope here.\n\n## Acceptance criteria\n\nThe issue's criteria, restated against the new shape:\n\n1. An operator can obtain a task's complete logs through the agent: by\n paging when the log fits the scan window, and by file delivery\n (`send_file`) when it does not.\n2. A bounded response says so explicitly -- `MoreAvailable` for \"more\n matches this call did not return\", `Truncated` for \"the scan cap hid\n older entries\" -- so the agent can report honestly instead of\n implying completeness.\n3. `Levels`, `Since`, `Until` filters exist so a bounded call can answer\n a specific question without paging.\n4. The cap stays enforced: one response cannot exceed `MaxTailLines`,\n and one call cannot examine more than `maxScanBytes`.\n5. Tests cover a log larger than `MaxTailLines` and assert that repeated\n paged calls retrieve every entry without gaps or duplication, plus a\n log larger than `maxScanBytes` asserting the window is walked\n contiguously and `Truncated` is set.\n\n## Files this change touches\n\n- `internal/logging/reader.go` -- add `Page`, expand `Query` with `Since/Until`\n- `internal/logging/reader_test.go` -- new tests for `Page`, `Since/Until`, paging without gaps\n- `internal/gateway/task_tools.go` -- new `ChatTaskLogQuery` and `ChatTaskLogResult` fields, schema description\n- `internal/gateway/task_tools_test.go` -- new tests for the new fields on the tool\n- `cmd/archied/main.go` -- adapter passes through the new fields\n\nThe webui handler in `internal/webui/api_logs.go` is untouched: it is\nalready filter-aware (Levels, Component, Contains, Limit) and is not the\nfailing surface vq92 reports. If the dashboard later wants the same\npagination, it consumes the same reader.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/task-run-detail/",
|
||
"title": "Per-run task detail: attempts, stages, changes, config, debug",
|
||
"section": "PRDs",
|
||
"body": "# Per-run task detail: attempts, stages, changes, config, debug\n\n**Status:** Approved\n**Authority:** the surface contract itself is\n`docs/architecture/observability.md`.\n**Date:** 2026-09-18\n**Compounds with:** `docs/prds/task-logs-pagination.md` (a bounded read says\nwhat it does not show), `docs/prds/status-health-surface.md` (no surface that\nanswers something already answered elsewhere).\n\n## Decision\n\nA task gets one detail page that answers \"what did this run actually do?\" per\n**attempt**: an ordered stage rail, the change each commit/push produced, the\neffective configuration the attempt ran under, the attempt's log, and the raw\nrecord behind all of it.\n\nThe single organising idea is **attempt attribution**. The dashboard\nrenders a task's whole event stream as one flat timeline, so a retried task\nreads as one long run with two of everything. Every new panel is scoped to one\nattempt and says which one.\n\nThe second organising idea is **provenance, not re-derivation**. What a run\nproduced is captured at the producer, while the evidence exists, and persisted\ndurably. Nothing in this surface tries to reconstruct a past run from state\nthat has since moved on.\n\n## The requirement set\n\n| # | Requirement | Mechanism |\n| --- | --------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |\n| R1 | Per-attempt stage rail: stages with name, start, duration, terminal status and error, ordered within one attempt | `GET /api/tasks/{id}/attempts` reads the task's events and groups them by `attempt` |\n| R2 | Events carry an attempt, migrated safely and carried on the wire | `events.attempt` column + presence-gated migrator arm; `Event.attempt = 11` |\n| R3 | Changed files captured at commit/push, persisted per task, exposed, shown with repo/PR links | `changes_captured` durable event written by the commit/push steps; `GET /api/tasks/{id}/changes` |\n| R4 | The effective configuration an attempt ran under, secrets redacted, shown | `config_captured` durable event written once per attempt; read from the events response the page already holds |\n| R5 | Log pane filters by attempt, stage and level; absence stays distinct from an unreadable deployment | `logging.Query.Stage` + `ReadTaskLogRequest.stage = 9`; the pane keeps `disabled` and `found=false` apart |\n| R6 | Retry is reachable from the run view and produces a new attempt shown as a new run | Existing retry action, labelled as a NEW run whose confirm states the previous attempt's commits are discarded |\n| R7 | Raw JSON of the task record and its events for the selected attempt | `GET /api/tasks/{id}/debug` |\n| R8 | Deep-linkable, reload-safe per-task detail page that keeps existing URLs working | `/tasks/{id}` route beside the existing `/tasks?task=N` and `/tasks?status=...` |\n| R9 | Design polish: hierarchy, tokens, both themes, responsive rail, a11y, loading/empty/error states | UI lane; measured at 480px and 700px in both themes |\n| R10 | Documentation of record | This document plus `docs/architecture/observability.md` |\n| R11 | Red-first tests for every new behaviour, full gate, `task ui`, forced `go build ./...`, `go test -race` on touched packages | Per lane; the anti-cost-cutting requirement lives here |\n\n## Attempt attribution: a column, not a guess\n\n`events.Event.Attempt`, persisted as the `events.attempt` column\n(`internal/infrastructure/postgres/migrations/0001_state_store.sql`), is a real column,\n`NOT NULL DEFAULT 0`. It was chosen over segmenting the stream on `task_queued` /\n`task_retried` because a fold would have to guess which side of a boundary an\nevent belongs to when the daemon restarts mid-attempt, when `RecoverStale`\nrequeues, or when two attempts interleave across a crash. A guess rendered as\nstructure is exactly the false fidelity this surface exists to avoid; a column\nis durable, queryable and ordered by construction.\n\n**`0` means \"not attributable\".** It is every row written before this change\nplus the deliberately task-agnostic producers. It is not an attempt number and\nis never rendered as one. The attempts endpoint reports the count of\nunattributed events so the page can say so instead of presenting old history as\nattempt 1.\n\n## Migration against an existing `archie.db`\n\n\u003e Storage superseded: every store now lives in PostgreSQL behind\n\u003e `internal/domain/storecontract`, implemented in `internal/infrastructure/postgres`\n\u003e (`docs/prds/state-store-contract.md`). SQLite names below describe the\n\u003e original design.\n\nThe store's migrator (`migrateTasks` in `internal/store/store.go`) already reads\n`PRAGMA table_info` through `tableColumns` and applies a presence-gated\n`ALTER TABLE ... ADD COLUMN` list. This change adds an `events` arm: a\n`tableColumns(ctx, tx, \"events\")` read beside the other two, and the entry\n`ALTER TABLE events ADD COLUMN attempt INTEGER NOT NULL DEFAULT 0`.\n\nThree consequences a future maintainer must not get wrong:\n\n- `eventsSchema` is `CREATE TABLE IF NOT EXISTS` (`internal/store/events.go`),\n so on an existing database it is a **no-op**. The migrator arm is the only\n thing that adds the column there. This is why the red test builds a raw\n SQLite file with the pre-change `events` DDL and must fail before the arm\n exists: every temp-database test passes either way.\n- **`user_version` stays 3.** The loop is presence-gated, not version-gated;\n bumping it would force an edit to the test that exists to prove a newer\n schema is refused.\n- **No backfill, ever.** Existing rows keep `attempt = 0`. Inferring an attempt\n number for them would be fabrication.\n\n## Changed files: captured at the producer\n\nThe measurement is taken inside the shared step library immediately after a\nsuccessful commit and after a successful push (`StageCommit` and\n`StageCommitPush` in `internal/domain/workflow/steps.go`) — the one place\nwhere the worktree, the branch, the base, the task and the store are all in\nscope, and the only place that knows a push actually happened.\n\nBoth alternatives were rejected on evidence, not taste:\n\n- **Reading the diff on demand** is impossible. The worktree is deleted on\n merge, close and no-PR terminal states (`internal/worktree.(*Manager).Cleanup`,\n called from the daemon's terminal paths) and a retry resets the branch onto\n its base (`internal/worktree.(*Manager).refresh` → `resetOnto`). There is\n nothing left to read.\n- **Reading it from the forge** is impossible. The forge contract exposes\n no changed-file capability, and adding one would cost three implementations\n plus an RPC and would still fail for chat-sourced tasks with no PR.\n\nBinding consequences:\n\n- Capture **degrades, never parks**. A `Trees` implementation without the\n capture seam captures nothing, logs, and lets the run continue. A reporting\n failure must never fail a task.\n- The value types live in `internal/domain/workflow/task` (`changes.go`), not in\n package `workflow` and not in `internal/worktree`. Package `workflow` has no\n import of `internal/worktree`, and this subpackage is not projected into the\n interpreted-stage symbol table, so declaring the types here widens nothing and\n needs no symbol-table regeneration. The capture method stays off that surface\n entirely.\n- `files` is capped at 200 entries because an event's `data` is not\n length-limited by the store. `totals` always covers the **full** set and\n `truncated: true` says when entries were dropped.\n- `binary` means \"no textual hunks\", not \"a binary file\". Rename pairing is\n go-git behaviour the lane pins with a test; where the repository records a\n rename as add + delete, the surface reports what was recorded.\n\n## Per-run configuration: a durable event, not a table\n\n**Decision.** The effective configuration an attempt ran under is persisted as a\n`config_captured` durable event (`internal/events.KindConfigCaptured`), on the\nsame attempt key R2 adds — not in a table, and not behind new RPCs.\n\n**Why.** Three reasons, in order of weight:\n\n1. Events are already this system's durable per-attempt provenance stream, and\n they demonstrably carry aggregation (`StageStats`, `TokensByDay` both\n aggregate over persisted events). R3 puts its diffstat there in this same\n change. A table plus two RPCs would have given one feature **two provenance\n mechanisms for one purpose**.\n2. It adds no RPC, so the ratified state-store surface is untouched and nothing\n in `docs/prds/state-store-contract.md` (rev 2c) changes.\n3. The page already fetches the task's events to build the rail (R1) and to read\n the diffstat (R3), so the document arrives in a fetch it already makes. There\n is nothing to join.\n\nThe single-row `config_snapshot` table is deliberately **left alone**. It\nanswers \"what is the running configuration now\", is replaced on every publish,\nand is pinned to one row by a test. It could not answer \"what did attempt 2 run\nunder\" in any case.\n\n**Payload.** The result of `config.Config.ForTask()` — the non-secret\ntask-runtime subset — plus the schema constant `archie/task-config@1`. Not the\ndashboard's configuration view: that is a larger projection carrying the full\nschema catalogue, and it is the wrong object for this. Stored shape and the tab's\nread rule are in `docs/architecture/observability.md`.\n\n**Binding conditions.** It is emitted through the durable path, never the\nlossy bus; it is written once per attempt, not once per stage; redaction is\nproven **positively** by a canary-secrets test rather than assumed from the\nproducer's contract; the timeline gains one human line for it rather than\ndumping a JSON blob; and an attempt with no such event renders \"not captured for\nthis run\", never an empty table implying defaults applied.\n\n**Documented residual — the honest trust claim.** A task-scoped grant may insert\nevents on its own task ID; that is how `tool_call` and `agent_finish` already\nwork, so a task can in principle write its own `config_captured` event. This\n**widens no capability** — the grant could already insert arbitrary events on\nitself — but it does mean this provenance is exactly as trustworthy as the rest\nof that task's event stream, and no more. It is not tamper-proofing and must\nnever be described as such.\n\n## No new RPC and no new table\n\nThe entire wire change is two fields on existing messages:\n\n```proto\n// proto/state/v1/state.proto\nmessage Event { int64 attempt = 11; }\nmessage ReadTaskLogRequest { string stage = 9; }\n```\n\nNo RPC was added, no store table was added, and no new configuration field was\nintroduced. `docs/prds/state-store-contract.md` is unchanged because no ratified\ncontract changed; the JSON document rides an existing `data` column over an\nexisting RPC.\n\nBoth fields are tolerant in both directions, and the degrade is stated rather\nthan hidden: an old server makes `attempt` read as `0` and `stage` match\nnothing, so the page must say the deployment cannot attribute events to attempts\nrather than show one run labelled \"attempt 1\". An old client shows\n`config_captured` as an unknown kind, and the Config tab must then say \"this\ndeployment cannot read per-run configuration\" — a different claim from \"not\ncaptured for this attempt\".\n\n## Retry is a new run\n\nThe retry action is reachable from the run view and produces a new attempt that\nappears as a new run. It is a **fresh** run: it resets the branch onto\n`origin/\u003cbase\u003e` (`internal/worktree.(*Manager).refresh` → `resetOnto`) and\n**discards the previous attempt's commits**. Labelling it \"restart\" would imply\nresumption that does not happen, so the control and its confirm copy say so.\n\n## Honest limits the surface carries\n\nThese are the claims that would otherwise be overstated. Each is a copy or\nrendering rule, not a caveat in a comment:\n\n- **There is no exit code, so no stage shows pass/fail.** `stage_finish` carries\n a duration and an optional error string and nothing else. A stage `ok` means\n \"returned without error\", not \"the work was correct\". The agent's own verdict\n is shown separately from the rail. A stage that parks after finishing cleanly\n leaves that stage `ok` on a failed task, so attempt status is read together\n with task status.\n- **The log stage filter only matches lines the workflow bound a stage to.**\n Agent and tool output carries no stage and never matches. The pane states that\n coverage limit next to the control rather than presenting a filtered view as\n the whole story.\n- **Absence and unreadability are different answers.** A deployment that cannot\n read task logs says so; an attempt whose log does not exist says that. They\n are never collapsed, in either direction, for logs or for configuration.\n- **An empty changed-files view never means \"nothing changed\".** No capture\n exists for runs predating this change and the forge cannot supply one, so the\n empty state reads \"not captured for this run\".\n- **No run-duration figure is shown.** Per-stage durations only: a run can spend\n days in `waiting_human`, and one number would understate wall time.\n- **No duration is invented** for a stage with no `stage_start`; it reads \"not\n recorded\".\n- **Unattributed history is never presented as an attempt.** Events with\n `attempt = 0` are reported as a count, not guessed into a run.\n- **The configuration shown is the attempt's non-secret task-runtime subset** —\n bot identity, models, limits, budgets, dispatch, diff cap, notifications,\n forge host, tool policy. It is not the full dashboard configuration and must\n never imply it contains providers, repositories, identities or credentials.\n\n## Route and compatibility\n\n`/tasks/{id}` is the detail route; it is deep-linkable and reload-safe.\n`/tasks?task=N` and `/tasks?status=...` keep working and keep their existing\nbehaviour, including the nav highlight. The debug view is why the raw view\nexists at all: the task list response is capped, so a client-side join has no\nrow for an older task, and a debug view that silently hid events would be worse\nthan useless.\n\n## What this deliberately does not do\n\n- **No new RPC, table, store interface or configuration field.** Nothing here\n needed one, and each would have widened a ratified surface.\n- **No exit status, no pass/fail gate badge, no \"steps\" in the CI sense.** The\n reference layout is a CI pipeline; archie's stages are agent runs and the page\n must not borrow CI's claims.\n- **No backfill** of attempts, change captures or configurations onto runs that\n predate the change.\n- **No client-side join for configuration.** It arrives in the events response\n the page already fetches; there is nothing to join.\n- **No change to `docs/prds/state-store-contract.md`.** No ratified contract\n moved.\n\n## Files this change touches\n\n- the State Store schema: the `events.attempt` column.\n- `internal/events/`: `Event.Attempt`, `KindChangesCaptured`,\n `KindConfigCaptured`, `ConfigCapturedSchema`.\n- `internal/domain/workflow/task/changes.go`: the change value types.\n- `internal/worktree/` + `internal/app/agentworker/`: the capture primitive and\n its production forwarding.\n- `internal/domain/workflow/steps.go`: the capture at commit and push.\n- `internal/daemon/`: the per-attempt configuration emit at dispatch.\n- `internal/logging/`: `Query.Stage`.\n- `internal/webui/`: the three read endpoints and the `stage` log parameter.\n- `proto/state/v1/state.proto` + generated stubs: the two field additions.\n- `ui/src/`: the detail page, the tab primitive, the rail, and the panel copy.\n- `docs/`: this document and `docs/architecture/observability.md`.\n\n## Acceptance\n\n1. Two attempts of one task render as two runs, and attempt 1's rail excludes\n attempt 2's stages.\n2. A task with no stages renders \"no stages recorded\", never an unexplained\n empty rail.\n3. A pre-change `archie.db` gains the `attempt` column with `user_version`\n still 3, and its old events are reported as unattributed.\n4. An attempt with a capture shows per-file added/deleted counts, status, repo\n and PR links; an attempt without one says so and never says \"no changes\".\n5. The Config tab shows the attempt's document for the selected attempt only,\n with no secret value present in the serialised document.\n6. Retry from the run view produces a new attempt shown as a new run, with copy\n stating the previous attempt's commits are discarded.\n7. The log stage filter narrows to stage-bound lines and the pane states the\n coverage limit; `disabled` and `found=false` remain distinguishable.\n8. `/tasks/{id}` survives reload, and `/tasks?task=N` / `/tasks?status=...`\n are unchanged.\n9. The debug view returns the task record and its unfiltered events, each\n carrying `attempt`.\n10. Run the gate and the dashboard build: `task check` passes clean, `task ui`\n has been run so `ui/dist` matches the sources, and every touched package\n builds and passes under `-race`.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/task-run-master-detail/",
|
||
"title": "Task run detail as a master-detail split pane",
|
||
"section": "PRDs",
|
||
"body": "# Task run detail as a master-detail split pane\n\n**Status:** Approved\n**Authority:** the panel contracts stay as ratified in\n`docs/prds/task-run-detail.md`; this document changes only the run page's\narrangement of them.\n**Compounds with:** `docs/prds/task-run-detail.md` (attempt attribution, the\nlog stage filter, the per-panel honesty rules).\n\n## Decision\n\nThe run page's five tabs become two panes. The stage rail is the **master**,\nalways visible on the left. The four context panels — log, changed files,\nconfiguration, debug — are the **inspector** on the right, and their tab bar\nmoves into the inspector. The \"Stages\" tab stops existing.\n\nSelecting a stage switches the inspector to the log tab and sets the log\npane's stage filter to that stage. Changing the filter afterwards is the\noperator's choice and is not overridden; rail selection never touches the\nlevel filter. Clicking the selected stage again returns the filter to all\nstages. A stage with no name is rendered but not selectable.\n\nSwitching attempts resets the stage filter to all stages: a selection was\ncommanded on the attempt it was made on, and a different attempt presents a\ndifferent stage list. Changed files, configuration and debug stay\nattempt-scoped; a stage selection never scopes them.\n\n## What does not change\n\n- No new endpoint, RPC, table or configuration field. Every read is one the\n page already makes.\n- The URL contract: `tab` and `attempt` ride in the query string and survive\n a reload. Log filters are not part of the URL; a reload returns the log to\n all stages. `tab=stages` stays a valid URL and renders the log tab; the\n page never generates it again.\n- Attempt attribution, the log stage filter's coverage limit and every\n honesty rule in `task-run-detail.md` still apply: each pane states what it\n cannot show under that document's rendering rules.\n- The task header and the attempt selector sit above both panes.\n\n## Arrangement\n\n- Left pane: fixed-width rail (20rem) with the attempt's stage rows. Status,\n duration, error, the stage's agent report, the attempt's status badge and\n start time, and the attempt-attribution footnotes render in the rail.\n- Right pane: the inspector's tab bar plus the selected panel, filling the\n remaining width.\n- Below the shell's 900px frame breakpoint the rail stacks above the\n inspector; neither pane is hidden.\n- Keyboard: the rail keeps its list semantics and each stage row is a real\n button, so selection and deselection work without a pointer; the inspector\n keeps the WAI-ARIA tabs pattern the tab bar already provides.\n\n## Acceptance\n\n1. The rail is visible without opening any tab, and no \"Stages\" trigger exists.\n2. Clicking a stage opens the log filtered to that stage; clicking it again\n shows all stages.\n3. After a stage selection, a filter change made by the operator is kept, and\n the level filter survives the selection.\n4. Switching attempts resets the stage filter to all stages, and no rail row\n on the new attempt renders as selected unless the operator selected it\n there.\n5. `tab=stages` loads and renders the log tab.\n6. Every inspector tab and attempt deep-link survives a reload.\n7. A stage is selectable and deselectable by keyboard, and the selection\n mirrors the log pane's stage filter.\n8. Below 900px the rail stacks above the inspector.\n9. `task check` passes clean and `ui/dist` matches the sources.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/ui-service-boundary/",
|
||
"title": "UI Service boundary and configuration ownership",
|
||
"section": "PRDs",
|
||
"body": "# UI Service boundary and configuration ownership\n\n**Status:** Approved\n**Revision:** 1\n**Date:** 2026-09-07\n**Beads issue:** `archie-core-8cda.5.1`\n**Parent:** [`service-decomposition.md`](/dev/prds/service-decomposition/)\n**Prerequisites:** Gateway contract extraction and State Store contract ratification\n\nThis document is the authority for Phase 3 UI Service implementation. It fixes\nthe process boundary, configuration ownership, client contracts, security\nposture, readiness behaviour, and migration evidence required by\n`archie-core-8cda.5.2` and later UI extraction work.\n\n## Decision\n\nThe UI Service owns the operator-facing HTTP surface and the compiled SPA. It\nserves `ui/dist` and exposes UI-specific JSON DTOs for dashboard views and\nactions. It does not own conversations, task state, workflow execution,\nchannel lifecycle, forge access, or the daemon's runtime configuration.\n\nThe UI Service consumes two typed remote capabilities:\n\n- Gateway's `gateway.ChatContract` through the versioned Gateway gRPC\n contract, for chat snapshots, messages, turns, streaming, persona changes,\n cancellation, and Gateway task actions.\n- State Store's narrow read/write contracts through the versioned State Store\n gRPC contract, for task views, events, captures, mappings, bindings, and\n configuration data explicitly projected for the dashboard.\n\nThe UI process may compose local adapters during the migration seam, but the\nadapter implements the same wire-safe contract as the remote client. The\nproduction extraction cutover removes the in-process adapter in the same\nchange; a dual-live UI authority is not permitted.\n\n## Boundary and ownership\n\n| Concern | UI Service owns | UI Service may consume | UI Service must not own or receive |\n| ------------------ | -------------------------------------------------------- | --------------------------------------------------------------------------- | ------------------------------------------------------------------------ |\n| HTTP and SPA | routes, auth middleware, UI DTOs, assets, HTTP lifecycle | contract results and health snapshots | domain entities or infrastructure handles |\n| Chat | request validation and response projection | Gateway `ChatContract` | `gateway.Router`, `SessionStore`, turn runner, model/provider runtime |\n| Task dashboard | query/action DTOs and HTTP status mapping | State Store narrow contracts and explicitly ratified task-action contract | `*store.Store`, SQL connections, workflow registry, daemon pointer |\n| Configuration page | secret-free view and update request validation | a UI configuration contract owned by the configuration/application boundary | `config.Config`, `config.Holder`, secret registry, reload implementation |\n| Readiness | UI process liveness and dependency readiness aggregation | bounded health results from Gateway and State Store | direct probing of private databases or providers |\n\nEvery value crossing the process boundary is a versioned DTO or scalar contract\nvalue. `internal/webui` may retain temporary Go-side adapters while the\nmigration is in progress, but handlers must map Gateway/State Store values to\nUI-owned response types at the HTTP edge. A constructor accepting a concrete\nGateway, State Store, daemon, or `config.Holder` is not an approved final\nboundary.\n\nThe existing UI routes remain behaviourally compatible during extraction,\nincluding task actions, SSE event delivery, config views, chat streaming,\ncapture intake, mappings, bindings, update reporting, and health endpoints.\nCompatibility preserves status codes, authentication requirements, redaction,\npagination/limits, event ordering, and failure degradation unless a separately\nratified contract changes them.\n\nThe route families have these target owners. This is the required inventory for\nthe implementation children; a route without an owner is not eligible for\ncutover.\n\n| Route family | Target owner consumed by UI | Migration status |\n| ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `/api/chat/*` | Gateway `ChatContract` and its versioned client | contract seam exists; DTO and stream parity remain |\n| `/api/tasks/*`, `/api/workflows`, `/api/setup` | State Store read/action contracts plus an explicit execution/admin action contract | State Store adapter exists; UI-facing narrow surface remains |\n| `/events` | State Store `EventsSince`, polled by one UI-process pump feeding `Server.Broadcast` | RESOLVED (`archie-core-za9f`): see migration-decisions, \"Dashboard live event delivery\". No new RPC; replay cursor and drop-recovery are the existing `since` watermark in `sse.go` |\n| `/api/logs`, `/api/logs/stream` | daemon diagnostic feed, host-local | unresolved: `logging.Feed` is in-process on the daemon host and has no contract. The per-task log routes moved off this row: see `/api/tasks/*` above |\n| `/api/tasks/{id}/logs`, `/api/tasks/{id}/logs/download` | State Store task-log read contract (`TaskLogStore`: `ReadTaskLog` + `StreamTaskLogContent`), read by the UI process from the service that owns the state directory | RESOLVED (`archie-core-iaqx`): the UI process composes `TaskLogStore` from its own State Store client and serves the attempt log as JSON (`found` distinguishes \"no log for this attempt\" from \"this process cannot read logs\") and as a `Content-Disposition` attachment named `task-\u003cid\u003e-attempt-\u003cn\u003e.log`. The daemon's in-process registry is wired as the server-side reader in `internal/app/archied/state_store.go` only (`staterpc.Deps.TaskLogs`), so no process opens another process's log files |\n| `/api/captures`, `/api/mappings`, `/api/bindings` | State Store contracts, with webhook verification owned by Work Intake/Messaging | RESOLVED (cutover, `archie-core-8cda.5.4`): the UI process composes CaptureStore/MappingStore/BindingStore and mounts the `captureintake.Receiver` from its own State Store client; the daemon's binding-dispatch loop keeps consuming captures from the same store |\n| `/api/config` (read) | daemon-published `ConfigView` snapshot, read over the State Store contract | RESOLVED (`archie-core-ymut`): see migration-decisions, \"Dashboard configuration page\" |\n| `/api/config` (write), `/api/config/reset`, `/api/config/repos/*` | a configuration write contract; the daemon stays the policy owner | DESCOPED for Phase 3 (`archie-core-ymut`), NOT abandoned: 503 in the UI process, SPA hides the controls, editing is config.toml plus reload. Dashboard editing is a tracked feature (`archie-core-1786637498420-327`) and its transport is `archie-core-j28m`; the handlers and their func-field seams on `webui.Server` are kept for it |\n| `/api/channels`, reload, curators, memory, skills, version/update | owning capability contract or an explicitly removed route | owner and failure semantics remain to define |\n| `/`, `/healthz`, `/health`, `/health/detailed` | UI HTTP process; readiness consumes per-service health contracts | current HTTP behaviour is the compatibility baseline |\n\n## Configuration ownership\n\nThe UI Service owns only process-local settings:\n\n- HTTP listen address and read/header/shutdown timeouts;\n- UI asset source or build mode;\n- UI authentication mode and token reference;\n- Gateway endpoint and credential reference;\n- State Store endpoint and credential reference;\n- readiness endpoint and dependency timeout policy.\n\nThese settings are decoded into a UI-owned input DTO by the UI composition root.\nThe DTO contains endpoint references and secret references, never resolved\nsecret values or complete daemon configuration. The UI process resolves its\nown credentials and builds its own clients. It does not read or mutate the\ndaemon's configuration file, runtime overlay, forge credentials, model catalogue,\nworkflow routing, NATS settings, filesystem jail, or agent/container settings.\n\nThe dashboard configuration API is a separate capability. Its secret-free,\nallowlisted field projections remain governed by\n[`config-schema.md`](/dev/prds/config-schema/). During the transition the daemon may\ncontinue to perform validate-persist-publish for those updates through a\nnarrow application contract. The UI process must never implement that policy\nby holding or mutating a shared `config.Holder`.\n\nThe daemon's `config.Holder` sharing was migration residue: `buildDaemon`\nassigned `b.web.Cfg = b.d.Cfg`. Removed by `archie-core-ml30` -- the field no\nlonger exists, and boot owns the single Holder the daemon reads through.\n\nFor Phase 3, configuration ownership is split deliberately. The UI process\nowns its listener, browser-auth, asset, dependency-target, and readiness\nsettings. The daemon/application configuration owner remains authoritative for\nthe dashboard's `/api/config` read, validation, persistence, overlay, reload,\nand audit behaviour. The UI does not become the writer merely because it\nrenders the configuration page.\n\nAmended 2026-09-11 by `archie-core-ml30`: the daemon's `config.Holder` sharing\nis gone. `webui.Server` has no holder field, so the sharing this paragraph\ndescribes cannot be reintroduced by wiring; the daemon owns one Holder and\npublishes a rendered projection.\n\nThere is no single narrow admin contract carrying both the read and the write\n(`archie-core-ymut`). The\nread crosses as a daemon-published `ConfigView` snapshot over the State Store\ncontract, and the write is descoped for Phase 3 rather than contracted, on the\nreasoning recorded in migration-decisions under \"Dashboard configuration\npage\". A future dedicated Configuration Service is the natural owner of a\nwrite contract, and designing one is that feature's work, not this migration's.\n\n## Listen, authentication, and readiness\n\nThe extracted UI follows the current web UI security posture:\n\n- loopback listeners may omit a dashboard token;\n- non-loopback listeners require a non-empty token and fail closed when it is\n absent;\n- `/healthz` and `/health` are unauthenticated liveness probes;\n- `/health/detailed` is authenticated and reports dependency readiness;\n- capture webhooks retain their separately ratified source authentication and\n rate-limit rules; they are not admitted by the dashboard token;\n- Gateway and State Store credentials are service-to-service credentials, never\n browser-visible dashboard tokens.\n\nThe UI is ready only when its HTTP server is serving and each mandatory remote\ncontract has passed its bounded readiness check. Each dependency must expose a\nversioned health result or health endpoint that the UI can consume; the UI must\nnot inspect another service's concrete manager, registry, or database. Optional\ndashboard sections may remain degraded with the current explicit\n`503`/unavailable behaviour. A successful liveness response must never imply\ndependency readiness.\n\n## Migration sequence and evidence\n\nThe implementation must complete these gates in order:\n\n1. Define the UI input DTO, Gateway client surface, State Store client surface,\n and UI response DTOs. Add compile-time contract assertions and mapping\n tests. No handler may depend on a concrete remote implementation.\n2. Make the current in-process web UI consume the same narrow adapters and\n preserve the current route, auth, redaction, stream, SSE, and degradation\n behaviour. Add a composition test proving the selected adapter is the one\n used by production wiring.\n3. Remove direct `internal/gateway` session/router/runtime access from web UI\n handlers. Remove direct store implementation access, SQL access,\n daemon pointers, and broad runtime callback bundles from the UI constructor.\n4. Replace the shared `config.Holder` with the UI-owned configuration DTO and\n narrow configuration application contract. Prove reload/update atomicity\n remains owned by the configuration/application boundary.\n5. Add the standalone UI process and deployment wiring. It must obtain Gateway\n and State Store endpoints from its own configuration, authenticate each\n client independently, start liveness/readiness endpoints, and shut down\n clients and HTTP listeners in bounded order.\n6. Run dual-adapter contract tests and an HTTP parity suite covering every\n retained route. Test remote failures, missing credentials, malformed DTOs,\n stream cancellation, stale State Store data, and readiness degradation.\n7. Cut over the supported composition to the UI process. The old in-process\n `webui.Run` path is deleted in the same change; no fallback or second HTTP\n authority remains.\n\nThe deletion gate is objective: `go list -deps` from the UI process shows no\ndaemon, SQL/store implementation, workflow, forge, channel, model, or secret\nruntime package; production composition contains one UI HTTP listener; and\narchitecture tests reject `config.Holder`, concrete `*store.Store`, concrete\nGateway runtime types, and direct database drivers in the UI service.\n\nRollback is a deployment composition change that points the operator at the\nlast validated UI binary and restores the previous endpoint configuration. It\nmust not restore a second writer or reintroduce shared-holder mutation. UI\nstate is either remote contract state or existing durable state owned by its\ncurrent service; no UI-specific database migration is introduced by this\nboundary decision.\n\n## Current evidence and known delta\n\nCurrent code proves the following migration inputs:\n\n- `internal/app/archied/bootstrap.go:setupObservability` constructs\n `webui.Server` and wires store, channels, events, logs, and config-related\n callbacks.\n- `internal/app/archied/bootstrap.go:buildDaemon` replaces the initial holder\n with `b.d.Cfg`, explicitly sharing the daemon's live configuration.\n- `internal/webui/api_chat.go` calls the Gateway contract but still imports\n Gateway-owned request/value types; handlers must finish the DTO projection.\n- `internal/webui/server.go:Handler` owns the route set, SPA fallback, health\n split, capture bypass, and token wrapper.\n- `internal/app/archied/state_store.go:RunStateStore` owns the task\n database and serves the State Store gRPC contract; the UI must use that contract\n rather than opening the database.\n- `internal/app/archied/gateway.go:RunGateway` serves Gateway gRPC and owns\n Gateway session persistence; the UI must use `ChatContract` rather than\n Gateway internals.\n\nThe current code therefore satisfies the upstream prerequisites but does not\nyet satisfy this boundary. This document is the ratified target; the\nimplementation child must prove each migration gate before claiming Phase 3.\n\n### Phase 3 gates\n\nEvery gate below is proven by test, not by inspection:\n\n| Gate | Evidence |\n| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Deletion | `cmd/archie-ui/architecture_test.go` re-runs `go list -deps` on every `task check` and fails on any banned package, checked in both directions against its two exceptions |\n| Composition | `internal/app/archieui/TestComposeUIServerHoldsNoDaemonState` fails on a concrete `*store.Store` or a Gateway that is not the gRPC client. The holder clause it also carried is enforced by the type: `webui.Server` has no `config.Holder` field (`archie-core-ml30`) |\n| End-to-end | `cmd/archie-ui/main_test.go` builds the binary and drives it over HTTP against a live Gateway and State Store |\n| No dual authority | The dashboard HTTP listener is `internal/app/archieui/run.go` only; the daemon's two `http.Server`s are its own health endpoint and the forge webhook receiver |\n\nAcceptance criterion 1 was amended at the close of Phase 3 to name the UI\nprocess rather than the `internal/webui` package, because the daemon still held\na holder inside its own process to render the published `ConfigView` snapshot.\n`archie-core-ml30` removed that residue, so the criterion holds in its original\nform again: `internal/webui` receives no holder in any process.\n\n### Deletion gate: SATISFIED (archie-core-8cda.5.6, rev. 2)\n\nThe objective gate now holds. `go list -deps ./cmd/archie-ui` links **zero**\nbanned packages: no `modernc.org/sqlite`, no `github.com/jackc/pgx`, no `internal/infrastructure/postgres`, `internal/gateway`,\n`internal/channels`, `internal/memory`, `internal/secret`, `internal/agentexec`,\n`internal/tools`, `internal/skill`, `internal/yaegiutil`, `internal/domain/curator`,\nor the workflow engine (`internal/domain/workflow`). How the runtime handles were\nsevered, each at the producer:\n\n- **Store contracts** moved to `internal/domain/storecontract` (producer-owned\n read contracts, state-store-contract rev. 2d).\n- **Chat contracts** moved to `internal/domain/messaging`; `internal/gateway`\n keeps aliases. The UI process holds `ChatContract`, `SessionStore`, turn and\n stream types without linking the gateway runtime or its session store.\n- **Webui-owned views** replaced concrete runtime handles on `webui.Server`:\n channel lifecycle via `internal/channels/status`, memory via a `MemoryStatus`\n interface, curators via a `CuratorStatus` interface, and the skill catalogue\n via a `SkillCatalog` interface — each supplied by a daemon-side adapter at\n bootstrap, each degrading to an empty page when unwired.\n- **`config.SecretRef`** moved into `internal/config` (it is config vocabulary);\n `internal/secret` keeps an alias and resolves through `Registry.Resolve`.\n- **Task vocabulary** (`Task`, `Store`, `Definition`, status constants) split\n into `internal/domain/workflow/task`; the pipeline engine stays in package\n `workflow`, which keeps aliases. The UI process links the vocabulary only.\n\nThe gate is enforced, not just measured. `cmd/archie-ui/architecture_test.go`\nre-runs `go list -deps` on every `task check` and fails on any package in a\nbanned category, so a single reference to a type sitting beside a runtime\ncannot quietly relink it. Its exception list (`internal/channels/status`,\n`internal/domain/workflow/task`) is checked in both directions: an exception\nthe UI stops needing is reported, so the allowlist cannot decay into an open\nprefix. `internal/app/archieui`'s `TestComposeUIServerHoldsNoDaemonState`\ncovers the composition clauses, failing on a non-nil `config.Holder`, a\nconcrete `*store.Store`, or a Gateway that is not the gRPC client.\n\nOne piece of residue remained visible at the close of Phase 3: `webui.Server`\nstill declared `Cfg *config.Holder`, because the daemon kept a `webui.Server`\nas the renderer that built the published `ConfigView` snapshot. That cleanup\nlanded as `archie-core-ml30`. `webui.Server` no longer has a configuration\nholder, provenance store, reload-status callback or override-listing callback:\nthe projection is built by `webui.BuildConfigView`, a pure function the\nconfiguration owner calls, and the daemon assembles its input from its own\nstate (`boot.configViewInput`). Readiness reads the daemon's own Holder,\nchannel manager and chat contract, and the health registry it produces is a\ndaemon field rather than a field on a server the daemon never serves.\n\n`ConfigView.Editable` moved with it. It describes the rendering process's own\nwrite path, so `handleConfig` sets it from `Server.UpdateConfig` rather than\nany process baking it into the document. A future configuration write contract\n(`archie-core-j28m`) makes the page editable by wiring that one field.\n\n### Real-process evidence (archie-core-8cda.5.6)\n\n`cmd/archie-ui/main_test.go` builds the binary and runs it against a State\nStore and a Gateway on loopback gRPC, then drives it over HTTP: liveness\nwithout a token, 401 for an API request without one, the `?t=` exchange into\nan HttpOnly cookie, task rows with their forge links, the embedded SPA and its\nassets, a chat stream relayed frame by frame from the Gateway, an operator\ntask action crossing to the Gateway (and refused without the CSRF header),\ncapture intake accepted unauthenticated, the ratified degradations, readiness\ndropping when the State Store dies while liveness holds, and a clean SIGTERM.\n\nTwo dashboard values degraded here because the published projection did not\ncarry them: `chat.show_tool_calls` (the chat page never expanded tool calls)\nand the `/api/setup` checklist (the panel was omitted rather than rendered).\nBoth are closed by `archie-core-ml30`: `ConfigView` gained a `chat` section\ncarrying `show_tool_calls`, the operator name, and whether any chat channel is\nconfigured, and both surfaces now read the projection rather than a live\nholder. The end-to-end suite drives the real binary and streams a turn, so the\n`show_tool_calls` assertion is made on the frames a browser receives.\n\nThe checklist stays. The question raised when it was republished -- is a\nread-only setup report wanted in a process that cannot apply the setup it\nreports? -- resolves in favour of keeping it, because the panel never promised\nto apply anything. It renders each step as a title, a detail and a done mark\nwith no control behind it, so it answers \"what is this deployment missing?\"\nrather than \"fix it here\", and the page that CAN act on it (the Configuration\npage) is where an operator goes next, in the daemon's own words. Removing it\nwould delete the one surface that distinguishes a fresh deployment from a\nfinished one for no gain, since the values it reads already cross the boundary\nin `ConfigView` and the read path costs the daemon nothing. The write path\nremains descoped (see `/api/config` (write) in the route table): the checklist\nreports configuration, it does not change it.\n\n`archie-core-ml30` published the channel flag from two of the three chat\nfront-ends, so an email-only deployment read as configured on `/api/channels`\nand unconfigured on the checklist at the same time (GitHub #821). Both now\nderive from `config.ChatConfig.FrontEnds`, the single definition of a configured\nfront-end, and a test pins the two surfaces to the same answer.\n\nA third value degraded for the same reason and is closed by\n`archie-core-pv6t`: in a deployment configuring `[[identities]]` the projection\ncarried the default identity's forge alone, so task rows withheld their links\nrather than pointing every row at the wrong forge. `ConfigView` now also\npublishes each identity's forge coordinates and the repositories it owns\n(`identities`), which is what resolving a task to its owning forge needs —\ncoordinates and repository names only, never a token. The end-to-end suite\ndrives the real binary against a multi-identity projection and asserts each\nrow's issue link is built from the forge that owns it.\n\n## Non-goals and open work\n\nThis ratification does not define a new domain model, move the dashboard's\nallowlisted configuration schema, redesign Gateway or State Store RPCs, choose\nKubernetes versus local discovery, or authorise unrelated package migration.\nThose decisions remain in their existing authority documents. Any new UI\ncapability that requires a wider Gateway or State Store surface must amend that\nservice's contract before implementation.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/vue-migration-checklist/",
|
||
"title": "Preact → Vue 3 + shadcn-vue migration checklist",
|
||
"section": "PRDs",
|
||
"body": "# Preact → Vue 3 + shadcn-vue migration checklist\n\n**Status:** Approved\n**Date:** 2026-09-20\n**Bead:** `archie-core-tavn` (epic), `archie-core-ocfm` (navigation restructure)\n**Bootstrap commit:** `e18fb8b2`\n\nThe Preact dashboard was deleted, not ported incrementally. This is the list of\nwhat has to exist again, what must not be lost on the way, and the contracts\nthat break silently if missed.\n\nDeleted source is recoverable at `git show e18fb8b2^:ui/src/\u003cpath\u003e`. Every\nclaim below was cross-checked against that tree; where a number is a live\nmeasurement rather than something recoverable from git, it says so.\n\n## Invariants that outlive the framework\n\nThese are not features; they are things other processes depend on. Breaking one\ndoes not show up as a broken page.\n\n- [ ] **Embed contract.** `ui/embed.go` embeds a committed `ui/dist`. The build\n must keep unhashed `assets/index.{js,css}`: dist/ lives in git, and\n hashed names orphan a file on every rebuild.\n- [ ] **Route registry.** `internal/gateway/dashboard_tools.go` hand-copies the\n route table and `TestDashboardPagesRegistryCoversEveryRoute` parses\n `ui/src/router/index.ts` to hold them in step. The parser needs\n `const routes = [` on its own line, `path: \"...\"` on a single line per\n route, and `nav: false` on any entry the registry should skip. That guard\n caught a stale `/chat` entry earlier.\n- [ ] **Mutation headers.** Every POST/PATCH/DELETE must send **both**\n `X-Archie-CSRF: 1` and `Content-Type: application/json`. The server\n returns 403 without the first and 415 without the second\n (`internal/webui/api_tasks.go`, `authorizeTaskMutation`). GET sends\n neither. This is the single most likely thing to silently break every\n write after the port.\n- [x] **History routing.** `createWebHistory`, `base: \"/\"`. Legacy `#/path`\n links are rewritten by `src/legacy-hash-redirect.ts`. The chat navigate\n chip should emit `/tasks`; the Go side already stores bare paths.\n- [ ] **Capability gating.** `GET /api/capabilities`\n (`internal/webui/api_capabilities.go`) returns a `sections` map of ten\n booleans: chat, logs, skills, curators, channels, captures, mappings,\n bindings, and workflows/settings which are always true. A route whose\n section is false must be hidden, not rendered empty. **Fail open**: on a\n failed or null response hide nothing — the old client used a bare\n `.catch(() =\u003e {})` for exactly this. A page that says \"unavailable\" is\n recoverable; a nav entry that vanished is not.\n- [ ] **`LOG_LEVELS`** in `base/log-row.jsx` is a wire contract, not a display\n list: the CSV level filter is matched server-side by `splitCSV` in\n `internal/webui/api_tasks_logs.go`. A value mismatch breaks filtering\n with no error.\n- [ ] **SSE lifecycle.** Three streams, all `id: \u003cn\u003e\\ndata: \u003cjson\u003e\\n\\n` except\n chat which omits the id:\n `/events` (task lifecycle, `sse.go`, supports `?since=` and\n `Last-Event-ID` resume), `/api/logs/stream` (`api_logs.go`, same\n resume protocol), `/api/chat/stream` (`api_chat.go`, frames typed\n delta/tool/media/navigate). Each must be closed on unmount or the app\n leaks a connection per navigation. Preact used an `archie:teardown`\n event; Vue should use `onUnmounted`.\n\n\u003e **On tests.** Per Sam's call, the 43-file suite was not ported and tests\n\u003e return when there is behaviour worth pinning. Flagging one exception for when\n\u003e that moment comes: the deleted `ui/test/api-client.test.js` classified every\n\u003e API method as mutation/read/stream/url and asserted the right headers per\n\u003e class. It is the only cheap guard on the CSRF contract above. (It did **not**\n\u003e text-scan for a second `fetch()` — that check was deliberately removed\n\u003e upstream for testing the file's wording rather than its behaviour.)\n\n## Phase 1 — Foundation\n\n- [x] **Theme.** `css/tokens.css` was a complete two-theme semantic token set\n (`--fg-subtle`, `--accent`, `--ok/warn/danger`, surfaces, shadows). Mapped\n onto Tailwind v4's `@theme` in `src/style.css`; the second system was not\n kept.\n- [x] **`color-scheme`.** `[data-theme=\"dark\"] { color-scheme: dark }` and the\n light equivalent. Without it the UA paints light scrollbars and light\n native controls inside the dark app. Fixed; trivially lost.\n- [x] **Contrast values.** The violet palette moved these off the values pinned\n here, so they were recomputed against its own backgrounds instead of\n carried: `--fg-subtle` `#A99DBD` dark (7.67:1, AAA) / `#6E6577` light\n (4.87:1, AA), and `--fg-muted` `#B9AECB` / `#635B6B` (9.28:1 and 5.70:1).\n The invariant the pin existed for holds; the values are the palette's.\n- [x] **API client** (`base/api.jsx` → `src/lib/api.ts`). Keep `ApiError`\n carrying `status`, and the 401/4xx/5xx classification callers branch on\n (`classifyActionError`). `GET /api/bindings/{id}` and\n `GET /api/mappings/{id}` exist server-side but were never called — do not\n add clients for them without a reason.\n- [x] **App shell** — the real source is `main.jsx` (410 lines), not just CSS.\n Beyond the topbar and outlet it carries:\n - [x] **Jump-to search**: Enter navigates to the first nav label matching the\n typed prefix; Escape closes the field and calls `preventDefault` so it\n does not also close the chat panel, which has its own Escape handler.\n - [x] **`soon: true` nav entries** — greyed, labelled, non-navigable. An\n affordance, not dead code.\n - [x] **`loadTaskMeta()` at boot**, which re-renders the mounted route once\n `/api/task-meta` lands so freeze-dried defaults are replaced.\n - [x] **Route keying**: keyed on path + params but deliberately **not** the\n query string, so a query-only change (a task-detail tab switch) does\n not remount and lose state, while two different `:id`s do get fresh\n instances.\n - [x] `.shell` has `backdrop-filter`, making it a containing block for\n `position: fixed` — the chat launcher must render outside it.\n- [ ] **CSS inventory.** `layout.css` was reassessed rather than ported whole.\n The command bar is a Vue component styled with Tailwind like every other\n surface inside the shell, so it emits none of that file's `.topbar`,\n `.nav`, `.topbar-search` or `.icon-btn` classes: only the frame those\n rules bounded is kept (`body` canvas, `.shell`, `.main`, and the 900px\n collapse), and the topbar/nav rules with their 1500px and 1270px\n breakpoints were deleted as unconsumed. The command bar's own responsive\n behaviour is the components' business, so \"port from this file\" applies\n to the frame and, later, the chat dock, not to class names. Still to\n port, when the pages that use them exist: `card.css`\n (`.card`/`.grid-2`/`.grid-4`), `page.css`\n (`.page-head`/`.page-title`/`.page-sub`/`.page-actions`), `pill.css` (the\n ok/warn/danger/info/idle definitions a Badge port must match), and the\n chat dock and morph Phases 3 and 5 describe. `base.css`,\n `responsive.css` and `_main.css`'s import order are ported.\n- [x] **Theme toggle** and its persistence.\n\n## Phase 2 — Shared primitives\n\n| Was | Is |\n| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `base/pill.jsx` | `Badge`, with five status variants (`ok`, `warn`, `danger`, `info`, `idle`) over the palette's status tokens |\n| `base/statTile.jsx` | `base/StatTile.vue`: full `Card` composition (`CardHeader`, `CardTitle`, `CardDescription`, `CardAction`, `CardContent`), `base/Sparkline.vue`, and the `goodDirection` trend logic |\n| `base/icons.jsx` | **Not ported as a module.** Its string-keyed lookup is the pattern shadcn-vue's rules name as wrong, so icons come from `@lucide/vue` at the call site |\n| `base/gauge.jsx` | `base/Gauge.vue` and `base/SegmentBar.vue`, both ported. The gauge's `aria-label` announces the rounded percentage, which closes `86au` |\n| `base/log-row.jsx` | `base/LogRow.vue` over `lib/log.ts` (`LOG_LEVELS`, `levelKind`, `shortTime`, `fmtValue`) |\n| `base/format.jsx` | `lib/format.ts` (`ago`, `compact`) |\n| `base/task-meta.jsx` | `lib/task-meta.ts`; Vue's reactivity replaces the manual boot-time re-render the Preact module needed |\n\nTwo decisions worth not re-litigating:\n\n- The pill's dot did not come across to `Badge`. The Badge's own shape is the\n affordance, and the same variant now covers things that are not status pills\n (a trend indicator), where a dot reads as noise.\n- The five bespoke glyphs (`dashboard`, `mappings`, `bindings`, `curators`,\n `memory`) are not drawn yet because nothing consumes them: the navigation is\n text with tooltips. If a page needs one, it becomes its own component at that\n point rather than a glyph table waiting for a caller.\n\n- [x] **Copy verbatim.** `dashboard/activity-detail.ts`, `chat/command-scroll.ts`,\n `workflows/workflow-rows.ts`, `logs/logs-empty.ts`, and `lib/stream-state.ts`.\n- [x] `base/uuid.jsx` → `src/lib/uuid.ts`.\n- [x] `routing.jsx` is superseded by vue-router. Not ported.\n\n## Phase 3 — Pages\n\nIn dependency order. Each is `git show e18fb8b2^:ui/src/\u003cfolder\u003e/`. File counts\nverified against the deletion diff.\n\nPorted as named components, not as one file per page: see\n`docs/prds/dashboard-port-brief.md#componentisation`. A page file composes and\nholds nothing, shared state lives in a module beside it, and a lookup that maps\na string to a component is not written.\n\n- [x] **Dashboard.** Health row, Throughput tiles, Token outlook, Live activity\n table, the time-of-day `greeting()`, and the click-through from an\n activity row to its task. The setup panel branches on the state machine\n rather than on \"is setup present\", which had shown a 100% checklist where\n the complete state belongs.\n- [x] **Tasks.** List, filters, row actions, detail page with tab bar, stage\n rail, changed files, attempt config, task logs, timeline, debug view.\n Both interaction contracts survive: detail `tab`/`attempt` is written with\n `router.replace` (never a hash, and replace so a tab click adds no history\n entry and does not remount), and the list's reduced-motion-aware\n scroll-to-row reveal for deep links. `1q89`'s column priority is applied.\n One deliberate deviation: the list's inline timeline expansion is gone,\n since the row now opens `/tasks/:id` and that page owns the timeline; the\n phone layout's stacked cards are not reproduced and the table scrolls\n inside its own container instead.\n- [ ] **Chat.** In progress.\n- [x] **Configuration**, split into the six `/system/*` routes the approved\n design names, with the editable-vs-read-only dual mode intact: this\n composition serves a published snapshot, so the pages say they are\n read-only rather than rendering controls that cannot work.\n- [x] **Logs.** Filters, live tail, pause. A 503 on the stream renders as\n \"Log stream unavailable\" with the reason, not as a permanent\n \"reconnecting\".\n- [x] **Event inspector / captures.**\n- [x] **Field mappings**, including the mapping editor. Its `mappingPreview`\n `capture_id` is a number: the wire type is int64 and a JSON string is\n rejected.\n- [x] **Playbook bindings**, including the binding editor.\n- [x] **Workflows**, keeping the \"No stage data yet\" state and omitting the\n Start-work form when the endpoint serves no definitions (`3tij`).\n- [x] **Skills**, **Curators**, **Channels.** Capability-gated and hidden on the\n reference deployment, so all three were verified by their own empty state.\n\nEvery page was rendered against the running daemon at 1440x900 and 720px, with\nno horizontal page scroll at either width.\n\n## Phase 4 — Navigation restructure\n\nBuild it during the port rather than porting the flat bar first.\n\n- [x] Five top-level items: Dashboard, Work, Agent, Events, System.\n- [x] Rename under their group: Inspector, Mappings, Bindings.\n- [x] Per-item hover tooltips carrying a one-line description.\n- [x] Configuration splits into `/system/{status,appearance,tasks,models,repos,advanced}`.\n- [x] `/settings` redirects to `/system/status` — it is in bookmarks and in the\n Go route registry.\n- [x] Add Logs to the nav. It was never in it.\n- [x] Preserve the `soon:` entry treatment.\n- [x] The theme control moves off the topbar and onto `/system/appearance`. Two\n switches for one preference is two places to look and one of them to be\n wrong; `components/topbar/ThemeToggle.vue` is deleted, not orphaned.\n- [x] Update `dashboard_tools.go` for every new route in the same change. The\n registry already carried all sixteen, so its parity test has been passing\n throughout; it is the check that a route cannot be added here and\n forgotten there.\n\n## Phase 5 — Behaviour a naive port loses\n\nThis behaviour exists only in the deleted tree.\nThe implementation source for nearly all of it is\n`git show e18fb8b2^:ui/src/css/layout.css` plus `main.jsx` and `chat/chat.jsx`.\n\n- [ ] **Chat is a FAB, not a route.** No `/chat`, no nav entry, no topbar icon.\n- [ ] **The panel grows out of the launcher**: collapsed to a pill over the\n button, expanding with scale, border-radius and position interpolating\n together; contents fade in after the shape arrives.\n- [ ] **No scrim, no scroll lock.** The page stays readable and clickable.\n- [ ] **No close button** — the FAB is the only toggle, and stays visible on\n mobile where the panel becomes a sheet.\n- [ ] Escape closes the panel **and returns focus to the launcher**; the dock\n carries `aria-expanded`/`aria-controls`/`aria-hidden`.\n- [ ] Header shows the open conversation's name (`panelTitle`).\n- [ ] New chat = `message-circle-plus`, in the header.\n- [ ] Slash palette is the only command surface; it floats above the input and\n scrolls to keep the arrow-key selection visible.\n- [ ] Settings popover closes on outside pointerdown and focus-out.\n- [ ] Stop renders only while streaming; composer stays writeable and focused\n during a send.\n- [ ] Bubbles carry no speaker label.\n- [ ] **Topbar cannot clip.** The nav is the only shrinkable region; search\n collapses below 1500px; nav wraps below 1270px; sheet mode at 720px;\n edge-to-edge at 900px. Previously the utility cluster was unreachable\n between 1221px and ~1582px: measured at 1440x900, `.topbar-end` ended at\n x=1515 against a shell edge of 1409, so 106px carrying the avatar, theme\n toggle and documentation button was clipped with no scroll affordance.\n That figure is a live measurement, not recoverable from git.\n- [ ] **Live activity detail is clamped** to one line with the full payload on\n the title. Unclamped it made the dashboard 7,481px tall.\n- [ ] **HEALTH cards take their own height** (`align-items: start`).\n- [ ] **Workflow rows merge** definitions with statistics.\n- [ ] **Log stream reports `unavailable`** from `readyState`, not a permanent\n \"reconnecting\".\n- [ ] **Unconfigured capabilities say so** (501 → \"Not configured on this\n deployment\") rather than rendering nothing.\n- [ ] **Operator cards on Configuration**: update install/defer, dangerous\n action approvals. Both absent when their endpoint 501s.\n\n## Known-open, carry forward\n\nOpen beads against the old UI that the port should satisfy rather than\nreintroduce: `1q89` (tasks column priority, wrapping row actions), `to2y`\n(three empty-state treatments; Field mappings dead-ends), `gl96` (Logs missing\nfrom nav), `86au` (gauge `aria-label` announces an unrounded percentage),\n`6m8r` (mobile KPI cards push content below the fold; 89 sub-44px targets),\n`ggqo` (config page width — moot once Configuration splits), `3tij`\n(`/api/workflows` serves no definitions, so \"Start work\" never renders),\n`oif3` (Reload Telegram, blocked on `ry4k`).\n\n## Not doing\n\n- Porting the 43 test files. Tests return when there is behaviour worth pinning\n rather than a port to re-assert. `task test:ui` is a `vue-tsc` typecheck\n meanwhile. See the note under Invariants for the one exception worth\n reconsidering first.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/webhook-intake-security/",
|
||
"title": "Webhook intake security -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Webhook intake security -- decision\n\n**Status:** Approved\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-t2db.5`, blocks `archie-core-t2db.4`\n\n## Why this is decided ahead of the code it constrains\n\n`archie-core-t2db` (no-code playbooks) and `archie-core-7d5u` (typed event\nsources) both land a public intake surface whose eventual output is an agent\nwith tool access and a repo. Neither capture (`t2db.1`) nor the intake family\n(`7d5u.3`) has shipped yet, so this decision exists to be a contract the two\nepics build against, not a retrofit once the surface exists. A webhook URL\nthat can start an agent is remote code execution with extra steps if any of\nthe five points below is missing on day one.\n\n## 1. Per-source authentication\n\nReuse the per-route HMAC-SHA256 scheme already in\n`internal/channels/webhook/webhook.go` (`validateHMAC`), unchanged: a source\nregisters with a shared secret; the signature is checked against the raw\nbody. Do not invent a second scheme for intake.\n\n**Capture and triggering split at the signature, not at a separate flag.** An\nevent with a missing or invalid signature is still captured (visible in the\ninspector, marked unauthenticated) so an operator can diagnose a\nmisconfigured sender -- but an unauthenticated event can never be matched\nagainst a binding. Authentication is what promotes an event from \"something\narrived\" to \"something that can run a workflow.\"\n\n## 2. Human approval before a binding arms\n\nA binding has three states: `draft -\u003e pending_approval -\u003e armed`. Only\n`armed` bindings are evaluated against incoming events. Creating or editing a\nbinding's matcher, mapping, or target workflow always sets it to\n`pending_approval`, including edits to an already-armed binding -- an edit is\nnot silently re-armed, or the approval gate is decorative. Approval requires\nan authenticated dashboard session; there is no unattended approval path.\n\nModel: `internal/channels/telegram`'s dangerous-command approval flow\n(`dangerousAction` / `pendingApproval` in `dangerous.go` / `approval.go`) is\nthe existing precedent for \"a human decision gates code from running\" in this\ncodebase -- follow its shape rather than designing a new approval primitive.\n\n## 3. Rate and volume limits, per source\n\nA token-bucket limiter keyed by source ID, not global -- one noisy or hostile\nsender must not exhaust another source's capture budget. Precedent for\nper-key volume capping: `internal/pairing/store.go`'s `MaxPendingPerPlatform`.\n\nOn limit exceeded, the request is rejected (HTTP 429), not silently dropped.\nA silently dropped event is indistinguishable from a sender that never fired\n-- the same invisible-failure shape the update-verification work\n(`archie-core-522`) exists to prevent. The rejection itself is observable in\nthe inspector as a distinct outcome from \"not received.\"\n\n## 4. Blast radius bounded by code, not by prompt wording\n\nA playbook-triggered task runs through exactly the same enforcement as every\nother task: the `[[repos.gate]]` quality gate, `diff_cap_lines`, the\nsandboxed per-task container (`internal/container`), worktree isolation, and\n\"the model never runs git.\" No new relaxation is introduced for\nplaybook-triggered work, and none is needed -- the existing constraints are\nthe control.\n\nMapped payload fields are treated as ordinary user-controlled text inside an\nagent prompt, exactly like an inbound chat message never as trusted\ninstructions. \"The agent should ignore instructions in the payload\" is not a\ncontrol and must not appear as one anywhere in this feature -- per the\nenvironmental-enforcement principle (`docs/architecture/agent-system.md`,\n\"Existing invariants to preserve\"), the boundary is the gate and the sandbox,\nnot a sentence in the system prompt.\n\n## 5. Retention and redaction of captured payloads\n\n**Retention:** captured events (bound or not) are purged after 7 days,\nconfigurable per deployment. Unbound events exist to be mapped against; nothing\njustifies keeping them indefinitely once that window has passed, and an\nunbounded capture store is the same disk-exhaustion risk item 3 addresses at\nthe request-rate level.\n\n**Redaction:** before a captured payload is persisted, apply the same\nkey-name heuristic `internal/gateway/stream.go`'s `sensitiveParameterKey`\nalready uses for tool-call parameters (matches `token`, `secret`, `password`,\n`passwd`, `api_key`, `apikey`, `authorization`, `credential`, `cookie`,\n`private_key` against normalised JSON keys) and replace matching values with\nthe same `[redacted]` marker. This is best-effort, not a guarantee -- a\nsender that names a secret field something the heuristic misses will still\nhave it captured. Document that limitation where captured payloads are\ndisplayed, rather than implying the redaction is complete.\n\n**Visibility:** captured payloads, redacted or not, are visible only to an\nauthenticated dashboard operator -- the same authorisation boundary as every\nother webui surface, not a separate or lower bar.\n\n## What this does not decide\n\nThe intake family's registration interface (`7d5u.3`), the capture storage\nmechanism (`7d5u.2`, embedded NATS), and the binding data model's persistence\nand versioning (`t2db.4`) are separate decisions. This document constrains\nall three: whatever they build must satisfy points 1-5 above.\n"
|
||
},
|
||
{
|
||
"url": "/dev/prds/workflow-calls/",
|
||
"title": "Workflows calling workflows",
|
||
"section": "PRDs",
|
||
"body": "# Workflows calling workflows\n\n**Status:** Draft\n**Date:** 2026-09-26\n**Bead:** `archie-core-t2db.41`\n**Extends:** `docs/prds/event-automation.md` (workflow/binding/profile model), `docs/prds/state-store-contract.md` (two new RPCs), `docs/prds/execution-tree-state-machine.md` (call kind)\n\n## The call, end to end\n\n```\ncaller run (in agent container)\n workflow.call step ──StartCall──▶ State Store EnqueueCallTask(caller, workflow, inputs)\n │ callee task row: org/identity/owner/repo\n │ derived from the caller row server-side\n ▼\n callee task (queued, org of the caller)\n │ daemon claims, pins the callee's OWN\n │ definition → its OWN agent profile\n ▼\n callee run (its own container, own tree)\n wait: true ──poll──▶ State Store WorkflowCallStatus(caller, callee)\n wait: false ──continue\n```\n\nA `workflow.call` step starts the callee as a first-class task and, with\n`wait: true`, waits for its terminal state. The callee is never run \"inside\"\nthe caller: it gets its own task row, its own pinned definition, its own\nprofile, its own container, and its own outcome on its own run\n(`execution-tree-state-machine.md`).\n\n## Save-time checks (the collection is the authority)\n\nAll semantic checks run in `workflow.ValidateDefinitionCollection`, the gate\nevery save and load of the `workflow-definitions` resource passes through.\nA single-definition parse cannot see callees, so it validates shape only.\n\n- **Callee exists.** A call to a workflow outside the collection is refused.\n- **Inputs checked.** A call value is a literal or a reference\n (`inputs.\u003cname\u003e`) to the calling workflow's declared input. References must\n name a declared input; the resolved type must satisfy the callee's declared\n type; unknown callee inputs are refused; a required callee input must be\n assigned.\n- **Self-calls refused.** The call graph over the collection is walked from\n every workflow; a workflow that reaches itself, directly or through other\n workflows, is refused with the cycle path. (Cycles can otherwise deadlock:\n `wait: true` callers hold their containers.)\n- **Repository.** `workflow.call` needs no repository, so it is legal in a\n `repository: none` workflow: the callee inherits the caller's task record\n (owner/repo/identity/org), and the callee's own `repository` mode decides\n whether its run clones.\n- **Org enablement** (bead `archie-core-t2db.42`): the single\n instance-wide collection the current model serves is what \"enabled for\n the caller's org\" resolves against — an undefined callee parks. The\n callee's org is derived from the caller's row, so a callee is never\n visible outside its caller's org. The per-org enable/disable catalogue\n lands with `archie-core-t2db.42` and becomes the one source of this\n predicate.\n\n## Runtime\n\n**Depth limit.** `workflow.MaxCallDepth` (5). The calling stage refuses a call\nthat would exceed it; the store re-checks on insert (callee depth = caller\ndepth + 1), so no path can enqueue past the limit.\n\n**Starting the callee.** The caller runs inside a container whose task-scoped\nState Store grant authorizes only `Update`/`Transition`/`InsertEvent` on its\nown task — it can neither enqueue a task nor read another one. One new\ncontract RPC starts the callee:\n\n- `EnqueueCallTask(caller_task_id, workflow, inputs_json) → task`\n — the store reads the caller row, refuses a caller that is not `running`,\n refuses past the depth limit, and inserts the callee row deriving org,\n workspace, identity, owner, repo and a fresh synthetic issue number from\n the caller (the same org COALESCE `EnqueueIssue`/`InsertChatTask` use). The\n callee row carries `call_parent_task_id` and `call_depth`.\n\n**Waiting on the callee.** One new read RPC returns a call's callee:\n\n- `WorkflowCallStatus(caller_task_id, call_task_id) → (status, detail)` — the\n handler verifies `call_parent_task_id` matches, so a caller reads only its\n own callees, and returns the callee's status plus its latest transition\n detail (the callee's summary — the wait:true \"outputs\" until a structured\n outputs model lands).\n\nBoth RPCs are admitted to task-scoped grants on `caller_task_id == granted\ntask ID` (`authorizesTaskScopedCall` gains two request-shape cases). This is\nthe one sanctioned widening of a task grant: the callee is the caller's child,\nand starting it is the mechanism by which one event becomes several runs. The\nwidenings stay request-shape checks; the parent-child authorization is a\nhandler-side row check, so the interceptor never reads the database.\n\n**Schema.** `tasks` gains `call_parent_task_id` (bigint, default 0) and\n`call_depth` (int, default 0); both cross the wire on the `Task` message.\nColumns default so existing writers and rows read as depth-0 root runs.\n\n**The engine capability.** The workflow domain owns a consumer facade,\n`task.Caller` (2 methods), carried by `*staterpc.Client` and wired into\n`TaskContext.Calls` by the agent worker's composition root — the same shape\nas `workflow.Store`. A runner with no `Calls` capability fails a\n`workflow.call` step with a named error rather than pretending.\n\n**Semantics.**\n\n- `wait: false` — start, record a durable `workflow_call_started` event, and\n continue. The callee's outcome is reported on its own run.\n- `wait: true` — start, then poll `WorkflowCallStatus` (2 s, bounded by the\n task's wall clock) until the callee is terminal. `completed`, `pr_open` and\n `merged` succeed the step; `parked`, `failed`, `rejected`, `dead` and\n `closed_wont_do` fail it with the callee's detail in the error. A durable\n `workflow_call_finished` event carries the terminal status and detail.\n- The caller's own stage/attempt accounting is unchanged: the call is one\n stage that took as long as its callee.\n- A callee inherits the caller's task record but is otherwise independent:\n retry, park and cancel of the callee never rewrite the caller. Cancelling a\n caller cancels its callees through the operator action on each\n (`execution-tree-state-machine.md` \"Cancellation\"); until the tree state\n machine lands, a caller cancelled while waiting fails its stage and leaves\n the callee running — the same semantics as `wait: false`.\n\n## What is deliberately not here\n\n- **Declared outputs.** `wait: true` receives the callee's terminal detail,\n not structured outputs; the outputs model is its own bead.\n- **The StepExecution tree.** The `call` kind is designed for in\n `execution-tree-state-machine.md`; this design lands the behaviour without\n the per-step records.\n- **Per-org enablement** — bead `archie-core-t2db.42`'s catalogue is the\n source once it exists."
|
||
}
|
||
],
|
||
"nav": [
|
||
{
|
||
"title": "Architecture",
|
||
"url": "/dev/architecture/",
|
||
"items": [
|
||
{
|
||
"title": "Adversarial self-review",
|
||
"url": "/dev/architecture/adversarial-review/"
|
||
},
|
||
{
|
||
"title": "Agent System",
|
||
"url": "/dev/architecture/agent-system/"
|
||
},
|
||
{
|
||
"title": "Playbook bindings: model and threat model",
|
||
"url": "/dev/architecture/bindings/"
|
||
},
|
||
{
|
||
"title": "Configuration",
|
||
"url": "/dev/architecture/configuration/"
|
||
},
|
||
{
|
||
"title": "The dashboard's tools for an in-browser agent",
|
||
"url": "/dev/architecture/dashboard-agent-tools/"
|
||
},
|
||
{
|
||
"title": "Dependencies and Contracts",
|
||
"url": "/dev/architecture/dependencies-and-contracts/"
|
||
},
|
||
{
|
||
"title": "Generated Documentation",
|
||
"url": "/dev/architecture/generated-documentation/"
|
||
},
|
||
{
|
||
"title": "Identity",
|
||
"url": "/dev/architecture/identity/"
|
||
},
|
||
{
|
||
"title": "Messaging and Work Intake",
|
||
"url": "/dev/architecture/messaging-and-work-intake/"
|
||
},
|
||
{
|
||
"title": "Architecture Migration Decisions",
|
||
"url": "/dev/architecture/migration-decisions/"
|
||
},
|
||
{
|
||
"title": "Task Observability",
|
||
"url": "/dev/architecture/observability/"
|
||
},
|
||
{
|
||
"title": "Repository Organisation",
|
||
"url": "/dev/architecture/organisation/"
|
||
},
|
||
{
|
||
"title": "Domain and Feature Review Procedure",
|
||
"url": "/dev/architecture/package-review/"
|
||
},
|
||
{
|
||
"title": "Plugins and Extensions",
|
||
"url": "/dev/architecture/plugins-and-extensions/"
|
||
},
|
||
{
|
||
"title": "Policy System",
|
||
"url": "/dev/architecture/policy/"
|
||
},
|
||
{
|
||
"title": "Safe Change and Recovery",
|
||
"url": "/dev/architecture/safe-change-and-recovery/"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"title": "Development guides",
|
||
"url": "/dev/development/",
|
||
"items": [
|
||
{
|
||
"title": "Agent",
|
||
"url": "/dev/development/agent/"
|
||
},
|
||
{
|
||
"title": "Control plane and settings",
|
||
"url": "/dev/development/control-plane/"
|
||
},
|
||
{
|
||
"title": "Daemon",
|
||
"url": "/dev/development/daemon/"
|
||
},
|
||
{
|
||
"title": "Frontend UI",
|
||
"url": "/dev/development/frontend-ui/"
|
||
},
|
||
{
|
||
"title": "Merging work",
|
||
"url": "/dev/development/merging-work/"
|
||
},
|
||
{
|
||
"title": "State Store",
|
||
"url": "/dev/development/state-store/"
|
||
},
|
||
{
|
||
"title": "Processes and wiring",
|
||
"url": "/dev/development/wiring/"
|
||
},
|
||
{
|
||
"title": "Workflows",
|
||
"url": "/dev/development/workflows/"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"title": "PRDs",
|
||
"items": [
|
||
{
|
||
"title": "Architecture Decision Index",
|
||
"url": "/dev/prds/01-project-management/"
|
||
},
|
||
{
|
||
"title": "PRD writing rules",
|
||
"url": "/dev/prds/RULES/"
|
||
},
|
||
{
|
||
"title": "Running action playbooks",
|
||
"url": "/dev/prds/action-playbook-run/"
|
||
},
|
||
{
|
||
"title": "Adversarial self-review stage -- decision",
|
||
"url": "/dev/prds/adversarial-self-review/"
|
||
},
|
||
{
|
||
"title": "Agent2Agent as archie's agent-to-agent protocol — decision",
|
||
"url": "/dev/prds/agent2agent-protocol/"
|
||
},
|
||
{
|
||
"title": "Artifact sender",
|
||
"url": "/dev/prds/artifact-sender/"
|
||
},
|
||
{
|
||
"title": "Binding secret at-rest encryption -- decision",
|
||
"url": "/dev/prds/binding-secret-encryption/"
|
||
},
|
||
{
|
||
"title": "Channel and Forge action positions (resolves `t2db.19`)",
|
||
"url": "/dev/prds/channel-forge-positions/"
|
||
},
|
||
{
|
||
"title": "Code graph over MCP (`tools/codegraph`)",
|
||
"url": "/dev/prds/code-graph-mcp/"
|
||
},
|
||
{
|
||
"title": "Example-config and generated-config drift — decision",
|
||
"url": "/dev/prds/config-example-drift/"
|
||
},
|
||
{
|
||
"title": "Backend-owned configuration schema for the webui -- decision",
|
||
"url": "/dev/prds/config-schema/"
|
||
},
|
||
{
|
||
"title": "Contract conformance audit — decision",
|
||
"url": "/dev/prds/contract-conformance-audit/"
|
||
},
|
||
{
|
||
"title": "Control-plane apply status",
|
||
"url": "/dev/prds/control-plane-apply-status/"
|
||
},
|
||
{
|
||
"title": "Curator sampler wave 1: `Sampler` interface + four cheap strategies -- decision",
|
||
"url": "/dev/prds/curator-sampler-wave1/"
|
||
},
|
||
{
|
||
"title": "Dashboard-aware chat: Archie sees the page, and points the operator there",
|
||
"url": "/dev/prds/dashboard-aware-chat-navigation/"
|
||
},
|
||
{
|
||
"title": "Dashboard live state",
|
||
"url": "/dev/prds/dashboard-live-state/"
|
||
},
|
||
{
|
||
"title": "Dashboard port: what the agent delivers",
|
||
"url": "/dev/prds/dashboard-port-brief/"
|
||
},
|
||
{
|
||
"title": "One source for the dashboard's server vocabularies",
|
||
"url": "/dev/prds/dashboard-vocabulary/"
|
||
},
|
||
{
|
||
"title": "Deep research capability — design",
|
||
"url": "/dev/prds/deep-research-capability/"
|
||
},
|
||
{
|
||
"title": "Dynamically-defined curators -- decision",
|
||
"url": "/dev/prds/dynamic-curators/"
|
||
},
|
||
{
|
||
"title": "Dynamic workflow triage",
|
||
"url": "/dev/prds/dynamic-workflow-triage/"
|
||
},
|
||
{
|
||
"title": "EDA playbook engine -- design",
|
||
"url": "/dev/prds/eda-playbook-engine/"
|
||
},
|
||
{
|
||
"title": "Embedded NATS execution -- decision",
|
||
"url": "/dev/prds/embedded-nats/"
|
||
},
|
||
{
|
||
"title": "Event automation: identified events, typed parameters, purpose-built agents",
|
||
"url": "/dev/prds/event-automation/"
|
||
},
|
||
{
|
||
"title": "Event capture storage -- decision",
|
||
"url": "/dev/prds/event-capture-storage/"
|
||
},
|
||
{
|
||
"title": "Event sources and reactions -- decision",
|
||
"url": "/dev/prds/event-sources-and-reactions/"
|
||
},
|
||
{
|
||
"title": "Execution tree and enforced state machine",
|
||
"url": "/dev/prds/execution-tree-state-machine/"
|
||
},
|
||
{
|
||
"title": "Extensions centre",
|
||
"url": "/dev/prds/extensions-center/"
|
||
},
|
||
{
|
||
"title": "External agent harnesses",
|
||
"url": "/dev/prds/external-agent-harness/"
|
||
},
|
||
{
|
||
"title": "Identity for users, agents and external agents — decision",
|
||
"url": "/dev/prds/identity-decision/"
|
||
},
|
||
{
|
||
"title": "Image capability contract and configuration",
|
||
"url": "/dev/prds/image-capability-contract/"
|
||
},
|
||
{
|
||
"title": "Inline review -- decision",
|
||
"url": "/dev/prds/inline-review/"
|
||
},
|
||
{
|
||
"title": "MCP client and tool-guardrail completion",
|
||
"url": "/dev/prds/mcp-tool-completion/"
|
||
},
|
||
{
|
||
"title": "Memory engine unification: four scopes, CRUD, and one write path -- decision",
|
||
"url": "/dev/prds/memory-engine-unification/"
|
||
},
|
||
{
|
||
"title": "Messaging Service boundary and transport design",
|
||
"url": "/dev/prds/messaging-service-boundary/"
|
||
},
|
||
{
|
||
"title": "Module position -- design",
|
||
"url": "/dev/prds/module-position/"
|
||
},
|
||
{
|
||
"title": "Multi-action playbooks and typed action results",
|
||
"url": "/dev/prds/multi-action-playbooks/"
|
||
},
|
||
{
|
||
"title": "Public news and per-version release notes",
|
||
"url": "/dev/prds/news-and-releases/"
|
||
},
|
||
{
|
||
"title": "Operator-triggered PR review",
|
||
"url": "/dev/prds/operator-pr-review/"
|
||
},
|
||
{
|
||
"title": "Orgs, workspaces and access control",
|
||
"url": "/dev/prds/orgs-and-access/"
|
||
},
|
||
{
|
||
"title": "Payload field mapping -- decision",
|
||
"url": "/dev/prds/payload-field-mapping/"
|
||
},
|
||
{
|
||
"title": "Playbook data-flow and condition syntax",
|
||
"url": "/dev/prds/playbook-expression-syntax/"
|
||
},
|
||
{
|
||
"title": "PR review agent",
|
||
"url": "/dev/prds/pr-review-agent/"
|
||
},
|
||
{
|
||
"title": "PR review comment remediation -- decision",
|
||
"url": "/dev/prds/pr-review-remediation/"
|
||
},
|
||
{
|
||
"title": "Progressive tool disclosure (bridge tools) — wiring",
|
||
"url": "/dev/prds/progressive-tool-disclosure/"
|
||
},
|
||
{
|
||
"title": "Public and development documentation -- design",
|
||
"url": "/dev/prds/public-and-dev-docs/"
|
||
},
|
||
{
|
||
"title": "One release stream, one page per version, and commit rules — decision",
|
||
"url": "/dev/prds/release-stream-and-commit-rules/"
|
||
},
|
||
{
|
||
"title": "Review calibration -- decision",
|
||
"url": "/dev/prds/review-calibration/"
|
||
},
|
||
{
|
||
"title": "Runtime control plane",
|
||
"url": "/dev/prds/runtime-control-plane/"
|
||
},
|
||
{
|
||
"title": "Self-update: artifact install, topology migration, journaled transaction",
|
||
"url": "/dev/prds/self-update-transaction/"
|
||
},
|
||
{
|
||
"title": "Service decomposition -- decision",
|
||
"url": "/dev/prds/service-decomposition/"
|
||
},
|
||
{
|
||
"title": "Service registry — one declaration per service",
|
||
"url": "/dev/prds/service-registry/"
|
||
},
|
||
{
|
||
"title": "Persistent session-memory curator (reference implementation) -- decision",
|
||
"url": "/dev/prds/session-memory-curator/"
|
||
},
|
||
{
|
||
"title": "Skill curator (reference implementation) -- decision",
|
||
"url": "/dev/prds/skill-curator/"
|
||
},
|
||
{
|
||
"title": "State Store — contract boundary \u0026 transport (ratification)",
|
||
"url": "/dev/prds/state-store-contract/"
|
||
},
|
||
{
|
||
"title": "/status health surface",
|
||
"url": "/dev/prds/status-health-surface/"
|
||
},
|
||
{
|
||
"title": "Store",
|
||
"url": "/dev/prds/store/"
|
||
},
|
||
{
|
||
"title": "task_logs pagination and bounded retrieval -- decision",
|
||
"url": "/dev/prds/task-logs-pagination/"
|
||
},
|
||
{
|
||
"title": "Per-run task detail: attempts, stages, changes, config, debug",
|
||
"url": "/dev/prds/task-run-detail/"
|
||
},
|
||
{
|
||
"title": "Task run detail as a master-detail split pane",
|
||
"url": "/dev/prds/task-run-master-detail/"
|
||
},
|
||
{
|
||
"title": "UI Service boundary and configuration ownership",
|
||
"url": "/dev/prds/ui-service-boundary/"
|
||
},
|
||
{
|
||
"title": "Preact → Vue 3 + shadcn-vue migration checklist",
|
||
"url": "/dev/prds/vue-migration-checklist/"
|
||
},
|
||
{
|
||
"title": "Webhook intake security -- decision",
|
||
"url": "/dev/prds/webhook-intake-security/"
|
||
},
|
||
{
|
||
"title": "Workflows calling workflows",
|
||
"url": "/dev/prds/workflow-calls/"
|
||
}
|
||
]
|
||
}
|
||
]
|
||
}
|