MkDocs retires and the Astro app becomes the only public site, so the artifact is the product and the Markdown stays the source -- the split tools/newsgen already states for releases. tools/docsite walks the published set and writes docs/data/generated/docs.json: every page's directory-style url, title, section and raw Markdown body, plus the sidebar tree derived from the layout. Three things the artifact has to get right, each of which was wrong in a draft: - LINKS. The published pages carry relative .md links that MkDocs rewrote at build time and nothing else will. Each is rewritten to the target page's url using the same function that builds urls, so a link and its target cannot disagree; anchors and scheme-qualified links pass through; assets resolve by the same rule and none are referenced. - FRAGMENTS. A link can resolve to a page and still be broken, because a fragment matching no heading lands the reader at the top of a long document. Each is checked against the target's headings, by the slug rule a table of contents uses. - REFUSAL. A link whose target the set holds back, or whose fragment matches no heading, fails the generator naming the file, the line and the target. This found three citations from settled architecture pointing at two PRDs the blanket prds/* exclusion withheld -- links that were already 404s on the published site, so the exclusion broke citations rather than withholding drafts. Both targets are now published, for a reason distinct from the three ratified PRDs already published. The published set moves out of mkdocs.yml into the generator with its reasons: a renderer reading the artifact cannot see why a page is absent. docs/data/generated is accounted for by docsgen's generatedArtifacts, which now names this file beside contracts.json -- one ownership registry for the directory rather than one per writer, so neither tool reports the other's artifact as unaccounted. docs:artifact regenerates; docs:artifact:check reproduces into memory and fails on drift without touching the tree, wired into task check beside news:check. The existing docs:site target is MkDocs and was left alone -- a first attempt at that name silently ran MkDocs and produced nothing. Verified: task check exit 0, docs:artifact:check reporting 23 pages. The artifact carries 23 pages, no archive/ inspiration/ or news/ entry, and the five PRDs the set declares. Coverage: the published set and its count, url derivation, the nav tree, both refusal paths, and check's no-write guarantee.
249 lines
303 KiB
JSON
249 lines
303 KiB
JSON
{
|
||
"pages": [
|
||
{
|
||
"url": "/docs/",
|
||
"title": "Archie",
|
||
"section": "",
|
||
"body": "# Archie\n\nEvent Driven Agentic Automation.\n\n**archied** is a resident daemon. It polls a forge (GitHub, Gitea) for issues\nassigned or labelled to it, works each one in an isolated git worktree through a\nrouted workflow, and opens a pull request for human review.\n\nWork arrives as events. Each task runs in its own sandbox with a task-scoped\ncredential, and the daemon performs all git operations.\n\n## Start here\n\n- [First playbook](/docs/guides/first-playbook/) walks through getting one piece of\n work from an issue to a pull request.\n- [Architecture](/docs/architecture/) is the authority for settled design:\n the package map, the workflow engine, the task lifecycle, and the invariants\n that hold the whole thing together.\n\n## What runs where\n\nA deployment is one daemon plus the processes it talks to. The State Store is a\nstandalone process and owns the database outright; the daemon and gateway never\nopen it directly. Agents execute in containers that receive a task-scoped\ncredential, issued per task and revoked when the worktree is handed back.\n\nConfiguration lives at `${XDG_CONFIG_HOME:-~/.config}/archie/config.toml`.\nNothing in a working tree is load-bearing at runtime.\n\n## A note on these docs\n\nThis site is generated from the `docs/` directory of the\n[archie-core repository](https://github.com/samcharles93/archie-core) on every\nchange to `main`. The repository is the source; this site is a view of it. If\nthe two ever disagree, the repository is right.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/",
|
||
"title": "Architecture",
|
||
"section": "Architecture",
|
||
"body": "# Architecture\n\nArchie is a personal agent platform built around six domain boundaries:\n\n- Identity tracks users and other actors responsible for actions.\n- Agent owns persistent assistants, specialisation, and continuity.\n- Messaging owns conversations, messages, branches, and channel correlation.\n- Workflow owns reusable, versioned behaviour definitions and their executions.\n- Work Intake admits optional durable work without constraining ordinary agent\n conversation or direct capability use.\n- Plugin owns generic extension discovery and lifecycle; each capability family\n owns its typed plugin contract.\n\nThe detailed architecture and migration decisions are maintained under\n`docs/architecture/` in the repository while this generated site is being\nassembled.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/adversarial-review/",
|
||
"title": "Adversarial self-review",
|
||
"section": "Architecture",
|
||
"body": "# Adversarial self-review\n\nArchie runs a fresh adversarial review of the diff before it opens a pull\nrequest, so a PR arrives with a findings record rather than an unreviewed\nclaim. The reviewer provably cannot see the implementer's context: it reviews a\n`.git`-free snapshot of the committed tree, not the worktree, the commit\nhistory, or the implementer's transcript.\n\nPromoted from `docs/prds/adversarial-self-review.md` (the design decision;\n`archie-core-h019`). Describes the stage as shipped.\n\n## Contract (`internal/domain/workflow`)\n\nThe workflow domain owns the contract and the blocking decision; it never names\nthe executor.\n\n- `Reviewer.Review(ctx, ReviewRequest) ReviewReport` — the interface the\n workflow stage consumes.\n- `ReviewRequest` is the reviewer's entire input: `SnapshotDir` (a `.git`-free\n export of the reviewed commit), `Diff` (base...head), `IssueText` (the\n originating issue title/body), and `MaxSteps`.\n- `ReviewFinding` / `ReviewReport` are the structured result. The load-bearing\n rule is `Blocking() == (Verdict == confirmed \u0026\u0026 Level == error)`: only a\n confirmed error-level finding blocks; a plausible finding never blocks\n however severe it claims to be.\n\n`ReviewReport` distinguishes `not_run`, `skipped`, and `completed` (zero\nfindings) structurally, so \"ran and found nothing\" is not the same as \"never\nran\" — collapsing them would make an outage indistinguishable from a clean\nreview.\n\n## Execution (`internal/app/agentworker`)\n\nThe executor is an ai-sdk `agent.Subagent`: a nested synchronous generation in\nthe same worker process, with its own model, system prompt, toolset and step\nbudget.\n\n- **Conversation isolation is structural.** `Subagent.Run` sends only a prompt\n string, never the implementer's message history.\n- **Workspace isolation is the `.git`-free snapshot.** The reviewer's toolset is\n read-only (`read`, `grep`, `find`) and rooted at `SnapshotDir` — never the\n worker's own toolset, which carries the `worktreerpc` publication grant.\n- **Findings return through a typed capture tool** (`record_finding`), validated\n on the way in, so the reviewer reports through the contract rather than free\n text the caller regexes.\n- **Fail-closed.** A reviewer that errors, truncates, or reaches no conclusion\n yields `ReviewStatusNotRun`, which the stage treats as a park — never a silent\n \"PR opened unreviewed\".\n\n## Lifecycle effect\n\n`StageReview` runs in the implement workflow after the deterministic gates and\nbefore `StageOpenPR`, gated by `repo.review_enabled` (opt-in, default off). A\nsurviving blocking finding — or a review that failed to run — parks the task\n(`StatusParked`, not `waiting_human`) with the findings rendered into the park\nreason; `StageOpenPR` never runs. Warn/plausible findings and the zero-finding\ncase leave the outcome unset, so the PR opens with a findings section rendered\nonto its body. `StagePostReviewComments` then runs after `StageOpenPR` and posts\nthe **line-anchored** findings as inline review comments, carrying a one-click\nsuggestion on a confirmed finding the reviewer could state a fix for. That stage\nis best-effort — a comment that fails to post is logged, never a park — because\nthe body section already lists every finding, and it is the record a whole-file\nfinding (no line to anchor to) only ever appears in.\n\n## Model and budget\n\nThe reviewer model is `models.reviewer`, falling back to `models.builder`.\nEffort is the per-stage `MaxSteps` budget, mapped onto `Subagent.MaxSteps`\n(default 10).\n\n## Operator-triggered review\n\nThe same reviewer runs against an *existing* PR via the channel-neutral\n`review_pr` chat command: fetch the PR's head, snapshot it, run the isolated\nreviewer, and return the structured findings. Authorization is identity-scoped\nto the operator's configured repositories; duplicate or concurrent reviews of\nthe same PR are deduplicated.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/agent-system/",
|
||
"title": "Agent System",
|
||
"section": "Architecture",
|
||
"body": "# Agent System\n\n**Status:** Current-state investigation in progress \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Domain framing\n\nThe domain is the Agent System.\n\n`Task` is not a core Agent System concept. Forge tasks, forge issues, Jira\nissues, email, Telegram messages, webhooks, and future external inputs are\nchannel-specific messages. Channels translate those messages into requests to\nthe Agent System; external issue or task models MUST NOT shape execution state.\n\nThe current `store.Task` is a persisted workflow execution. Treating that row as\na task-domain aggregate would incorrectly turn Archie into an agentic issue\nboard.\n\n## Canonical agent interaction model\n\nAn **Agent** is a persistent assistant selected for an interaction. It is not a\nmodel invocation and is not a WorkflowExecution.\n\nEach Agent has its own:\n\n- immutable acting `IdentityID` for ownership and audit;\n- durable memory and learned information about the user;\n- perspective, specialisation, instructions, and communication behaviour;\n- conversation history and channel bindings;\n- available tools, capabilities, and policy;\n- continuity across model or provider changes.\n\nThe underlying model is a replaceable runtime dependency. Changing a model,\nprovider, or model version MUST NOT create a new Agent, discard its memory, or\nchange ownership of its history.\n\nA user MAY create multiple Agents. Each Agent develops its own memory and view\nof the user from the interactions and responsibilities assigned to it. Memories\nMUST NOT be silently pooled merely because Agents belong to the same user or\napplication.\n\nAgent and Identity are separate aggregates. Every Agent references exactly one\nacting IdentityID, and no two Agents share an agent identity. User, system, and\nservice-account identities may exist without an Agent.\n\nAn action initiated through an Agent records the Agent's acting IdentityID and,\nwhen applicable, the initiating user's IdentityID. A future fully automated\nWorkflowExecution MAY use an explicitly selected service-account identity for a\nconcrete unattended use case; this is not the default execution identity.\n\n## Workflow execution identity\n\nA Workflow is owned by an IdentityID. By default, every WorkflowExecution acts\nas the Workflow owner's identity.\n\nFuture delegated execution MAY add an explicit `RunAsIdentityID` assignment to a\nWorkflow. This allows a Workflow owner to select another user or service-account\nidentity for execution without changing Workflow ownership. Until that\ncapability is deliberately introduced, owner identity and execution identity are\nthe same.\n\nWorkflowExecution audit records MUST preserve the Workflow owner, effective\nexecution identity, initiating user when one exists, and any explicit delegation\nthat selected a different run-as identity.\n\n## Memory scopes\n\nThe Agent System requires four distinct durable memory scopes:\n\n- **global shared memory**: information intentionally available across the\n Archie application;\n- **agent-wide memory**: general expertise, operating knowledge, and perspective\n owned by one Agent and safe for that Agent to use across users;\n- **user-wide memory**: information associated with one user and available to\n authorized Agents serving that user;\n- **Agent-user relationship memory**: private history and learned context from\n one particular Agent's relationship with one particular user.\n\nAgent-wide memory is private to its owning Agent by default. Another Agent MUST\nNOT read, inherit, search, or mutate it implicitly.\n\nAgent-user relationship memory is addressed by both AgentID and User IdentityID.\nNeither another Agent serving the same user nor another user speaking to the\nsame Agent may access it implicitly.\n\nArchie is a multi-user system even when initially deployed for one person.\nMultiple users, including family members, may interact with the same configured\nAgents. Per-user memory is therefore an isolation and personalization boundary,\nnot merely an attribution tag on the primary user's records.\n\nAn Agent MUST resolve the initiating user before assembling memory context.\nMemory belonging to one user MUST NOT become visible to another user merely\nbecause both users interact with the same Agent.\n\nShared memory does not erase provenance. Every memory record retains its scope,\nauthor or acting identity, originating user when applicable, source, and\nrevision history. The rules for selecting a write scope, granting access\nto global and user-wide memory, and authorizing explicit cross-scope sharing are\nsettled by [`docs/prds/memory-engine-unification.md`](/docs/prds/memory-engine-unification/)\nand recorded in [`migration-decisions.md`](/docs/architecture/migration-decisions/#5-memory-placement-and-storage)\nsection 5.\n\nAn Agent interaction may answer conversationally, use a tool or capability, read\nor send messages, retain information, or propose durable workflow-backed work.\nNone of these operations inherently requires a WorkflowExecution.\n\n## Agent and Workflow domain boundary\n\nAgent and Workflow are separate domains:\n\n```text\ninternal/domain/agent\ninternal/domain/workflow\n```\n\nThe Agent domain owns persistent assistants, their specialisation, instructions,\nmodel-independent continuity, capability context, and associations with identity\nand memory.\n\nThe Workflow domain owns reusable, user- or Agent-defined behaviour and every\ndurable execution of that behaviour. An Agent may invoke or define a Workflow,\nbut the Workflow is not part of the Agent aggregate and does not require a\nconversational Agent interaction to execute.\n\n## Canonical workflow model\n\nThe Workflow feature uses these canonical terms:\n\n- **Workflow**: a reusable, versioned definition of how an agent behaves and\n performs a kind of work.\n- **WorkflowExecution**: one durable execution of a specific Workflow version.\n- **WorkflowStep**: one defined operation in a Workflow.\n- **StepExecution**: the recorded execution and outcome of one WorkflowStep.\n- **Attempt**: a retry of the same WorkflowExecution, not a new execution.\n\nA Workflow MAY structure any repeatable agent behaviour, including personal\nassistance and scheduled routines. For example, a morning email review may\ncombine a schedule event, message retrieval tool, LLM classification and\nsummarisation, and delivery of the resulting rundown. Workflow is therefore not\nrestricted to forge work, source changes, or requests submitted by a user at\nexecution time.\n\nWorkflows are extensible through Workflow-owned plugins. The Workflow domain\ndefines the typed plugin registration, definition, validation, versioning, and\nexecution contracts. Users and Agents may define reusable Workflows through\nthose contracts.\n\nThe generic plugin contract remains metadata-only. It MUST NOT own Workflow\nsemantics or replace the Workflow registry with generic hooks.\n\n`Workflow` and `WorkflowExecution` are the names used in architecture, generated\nreference, configuration, commands, events, and external contracts. The\nimplementation MAY use package qualification to remain concise:\n\n```go\nworkflow.Definition\nworkflow.Execution\nworkflow.Step\nworkflow.StepExecution\n```\n\nThe unqualified name `Execution` MUST NOT be used in cross-domain contracts or\ndocumentation because it is ambiguous among workflow, tool, command, policy, and\nservice execution.\n\n## Current workflow-execution feature\n\nThe current feature takes a channel request through agent execution and a\ndurable outcome.\n\nCurrent execution state includes source-message references, owning identity,\nrepository, selected workflow and stage, agent output, branch and pull-request\nreferences, token and iteration counts, retry information, human-wait\ninformation, and failure details.\n\nThe current `store.Task` lifecycle uses:\n\n```text\nqueued -\u003e running\nrunning -\u003e waiting_human | pr_open | merged | parked | closed_wont_do\nwaiting_human -\u003e queued | closed_wont_do\nparked -\u003e queued | dead\npr_open -\u003e merged | rejected\nrunning after crash -\u003e queued\n```\n\nWorkflows select and sequence deterministic and agent-run stages. Implement,\nTDD, feasibility, and bootstrap are current Workflow definitions.\nEvery WorkflowExecution is handed wholesale over core NATS to a task-scoped\n`archie-agent` container, which owns the workflow stage loop.\n\n## Current workflow-feature placement\n\nThe behaviour is divided across technical packages:\n\n- `internal/store` defines the current `Task` execution record and status\n strings and persists mutable state and transition history.\n- `internal/daemon` discovers, claims, schedules, retries, recovers, processes,\n waits for human input, and reconciles pull requests.\n- `internal/domain/workflow` routes work, executes stages, mutates tasks, applies\n outcomes, parks failures, and performs forge/worktree operations.\n- `internal/agentexec` defines and runs individual autonomous-stage protocols.\n- `internal/taskrun` hands a complete WorkflowExecution to a container worker.\n- `internal/gate` and repository configuration define execution constraints.\n- `internal/gateway` duplicates task status strings and applies task-control\n rules.\n- forge, worktree, NATS, SQLite, containers, and agent runtimes are concrete\n dependencies mixed into orchestration.\n\n## Current `task` meanings and required replacements\n\nThe existing code uses `task` for unrelated concepts. Migration MUST resolve\neach meaning deliberately; a global text replacement is prohibited.\n\n| Current meaning | Representative locations | Target concept | Requirements |\n| -------------------------------------------------- | --------------------------------------------------------------- | -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Forge issue or external task | `internal/forge`, daemon polling | Channel message | Preserve the channel's native reference and payload. Translate it into an Agent System request without importing forge or issue semantics into Workflow. |\n| Jira issue or other external work item | Messaging infrastructure adapter | Channel message or external artifact | The external system retains its own vocabulary inside its integration. Its record may lead to an accepted work request but is not a Workflow or WorkflowExecution. |\n| Persisted `store.Task` row | `internal/store` | WorkflowExecution | Own identity, Workflow ID and version, lifecycle, source-message reference, outputs, attempts, and outcome. Persistence implements the Agent System repository contract. |\n| Task status and transition history | `store.Status*`, `transitions` | WorkflowExecution lifecycle and domain events | Enforce legal transitions atomically. Domain events and state changes cannot diverge. Remove duplicated status strings from consumers. |\n| Claimed or queued task | `ClaimNext`, `ClaimByIssue`, `RecoverStale` | Scheduled WorkflowExecution | Claim by WorkflowExecution ID and expected version/state. Crash recovery and leases must not silently create a new execution. |\n| `workflow.TaskContext` | `internal/domain/workflow` | WorkflowExecution context | Replace the mutable service bag with execution state plus narrow step-required contracts. Complete application configuration and unrelated services are forbidden. |\n| `taskrun.Request` and `taskrun.Response` | `internal/taskrun`; consumed by `internal/app/agentworker`, decoded by `internal/infrastructure/agenttransport/nats` | WorkflowExecution handoff | This is the versioned worker contract for executing a WorkflowExecution. It carries domain-owned execution data, not store rows or global configuration DTOs. |\n| Per-stage `agentexec.Request.TaskID` | `internal/agentexec` | WorkflowExecutionID correlation | An agent invocation is a bounded WorkflowStep execution. Correlate request/result with WorkflowExecutionID, StepExecutionID, attempt, and protocol version. |\n| NATS `TaskMessage` and `archie.task.*` | `internal/domain/workintake` (subjects, `TaskEnvelope`); `internal/infrastructure/agenttransport/nats` (transport) -- `internal/nats` no longer exists; whether this row's \"current meaning\" itself is now superseded by `TaskEnvelope` needs a closer look, not asserted here | Channel request or WorkflowExecution transport | Discovery messages and execution dispatch are different contracts and subjects. Domain meaning stays outside NATS infrastructure. Identity and source-message idempotency must be explicit. |\n| `taskDispatcher` work item | `internal/daemon` | WorkflowExecution scheduling | Scheduling operates on execution references and declared concurrency policy. Repository serialization is policy, not identity derived from an incomplete task struct. |\n| `container.TaskPayload` and `task.json` | `internal/container`; read by `internal/infrastructure/agentboot` | WorkflowExecution brief | Rename the boot artifact and schema. It identifies the mounted execution and required workspace only; it is not an alternative execution definition. Missing or malformed input must fail startup before the worker subscribes. |\n| `storage.TaskRef` | `internal/storage` | WorkflowExecution workspace lease | Storage receives the minimal execution/workspace reference and policy required to provision and release resources. It does not receive the WorkflowExecution aggregate. |\n| Gateway `TaskCreator` | `internal/gateway` | Accepted work request handoff | A channel may carry a request for durable workflow-backed work, but it does not request or create a WorkflowExecution directly. Work intake admits and routes the request; the Agent System consumes the accepted work request and is the sole owner of selecting a Workflow version and creating the WorkflowExecution. Ordinary agent interactions do not require this handoff. |\n| Gateway `TaskController` and `ChatTaskStatus` | `internal/gateway/tasks.go` | WorkflowExecution commands and query view | Approve, cancel, and status use Agent System contracts. Channel code does not duplicate lifecycle constants or decide transitions. |\n| Gateway `TaskProfile` | `internal/gateway/tasks.go`, `cmd/archied` | Channel/identity execution defaults | This is configuration selecting allowed source targets and default Workflow inputs, not task state. Split it into settings owned by the relevant channel and Agent System features. |\n| `config.TaskConfig` and `TaskForge` | `internal/config` | Worker execution settings and channel/service references | Dissolve with `internal/config`. Each field moves to its owning runtime settings or versioned WorkflowExecution handoff contract. Secrets and external configuration forms do not enter the domain. |\n| `events.Event.TaskID`, task event timelines | `internal/events`, stores, web UI | WorkflowExecutionID and WorkflowExecution events | Authoritative events are Agent System events. Observability and timelines project those events rather than inventing parallel generic task events. |\n| Web UI task lists and counts | `internal/webui`, store query APIs | WorkflowExecution read model | Dashboard status, history, and counts are projections of WorkflowExecution state and events. The UI does not depend on persistence records. |\n| Task-scoped forge/store/worktree RPC | `forgerpc`, `storerpc`, `worktreerpc`, `registerTaskRPCServers` | Execution-scoped capability ports | Worker calls use narrow domain-required service contracts, carry WorkflowExecutionID and IdentityID, and resolve the owning capability binding. Root-client fallback is forbidden. |\n| Per-task worktree, branch, volume, and cleanup | daemon, worktree, storage, container | WorkflowExecution workspace | Every WorkflowExecution receives isolated workspace state. Workspace lifecycle is explicit, identity-aware, recoverable, and controlled outside the model. |\n| Task retry, dead, waiting-human, PR reconciliation | daemon, workflow, gateway | WorkflowExecution lifecycle policies | Retry, waiting, cancellation, reconciliation, and terminal outcomes are Agent System behaviour. Channels and infrastructure report facts or submit commands; they do not mutate state. |\n| Task-run readiness and retry timeouts | daemon taskrun request loop | Worker-dispatch settings and policy | These settings govern WorkflowExecution handoff availability. They are not Workflow data and must be owned by the worker-dispatch capability. |\n\n### Terms that do not migrate\n\n- The `task` command in `Taskfile.yml` is the external Task build tool and keeps\n its name.\n- A forge may call its own records issues or tasks inside its channel adapter.\n- General prose may use “task” informally only when it clearly refers to an\n external system. Agent System contracts and generated reference MUST use the\n canonical Workflow vocabulary.\n\n## Current WorkflowExecution data inventory\n\nThis inventory is the migration ledger for data currently spread across\n`store.Task`, database-only columns, `workflow.TaskContext`, worker messages,\nand surrounding configuration. Implementation SHOULD NOT require another\nrepository-wide search to determine what existing execution data must be\npreserved or deliberately removed.\n\n| Current data | Current representation | Required target |\n| ---------------------- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |\n| Execution identity | `Task.ID`, `task_id`, NATS subjects, logs | Immutable WorkflowExecutionID used consistently by state, events, workers, projections, and capability calls |\n| Owning actor | `Task.Identity` string | Canonical IdentityID |\n| Source type | `Task.Source` (`forge` or `chat`) | Typed channel and SourceMessageReference; no closed source enum in WorkflowExecution |\n| Source address | `Owner`, `Repo`, `IssueNumber`; synthetic chat issue number | Channel-owned source reference with native message/thread/object identifiers; synthetic identifiers removed |\n| Requested work | `Title`, `Body`, `Labels` | Immutable execution input captured from the source request; channel-specific metadata retained in the source envelope |\n| Workflow selection | `Workflow` string | Immutable WorkflowID plus selected WorkflowVersion |\n| Lifecycle | `Status` string | Enforced WorkflowExecution state |\n| Current position | mutable `Stage` string | Derived current WorkflowStep reference plus durable StepExecution history |\n| Attempts | `Attempt`, `RetryCount` | Execution attempt history and retry-policy evidence; distinguish worker delivery retry from workflow retry |\n| Failure | `ParkReason`, transition detail, response errors | Typed failure classification, evidence, retryability, and resulting lifecycle event |\n| Human wait | `WatchCommentID`, `waiting_human` | Channel-neutral wait condition and correlation reference; forge comment IDs stay in the forge channel |\n| Workspace | `Dir`, `Branch`, worktree path, `task.json` | Execution workspace reference and independently managed workspace lifecycle |\n| Change outcome | `BuildSummary`, `BuildNoChanges`, changed files | Step outputs and final WorkflowExecution outcome |\n| Reproduction evidence | `ReproProof` | Typed TDD step evidence retained with the StepExecution and final outcome |\n| Planning output | `Plan` | Typed step output referenced by later steps rather than a special aggregate field |\n| Notes | `Notes`, appended agent notes | Ordered execution annotations with actor, source step, and timestamp |\n| Pull request | `PRNumber`, forge host/repository assumptions | Forge-channel delivery reference and reconciliation state |\n| Agent usage | `TokensUsed`, `Iterations` | StepExecution usage measurements aggregated into execution projections |\n| Decision capture | private feasibility `decision`, capture-tool output | Typed WorkflowStep output defined by that Workflow version |\n| Terminal outcome | mutable `Outcome{Status, Detail}` | Explicit WorkflowExecution outcome command/event validated by the state machine |\n| Skill context | `SkillBody`, `SkillPlugins` | Resolved Workflow version dependencies or execution inputs, with provenance |\n| Agent prompt context | `SystemPrompt`, `ExtraRules`, mission strings | Versioned WorkflowStep definition plus resolved execution context; mutable global prompt callbacks removed |\n| Gates and protection | `Gate`, `Preflight`, `Protection`, repository config | Agent System policy definitions resolved for the WorkflowExecution and enforced outside the agent |\n| Budgets | steps, tokens, wall clock, gate failures | Execution and StepExecution policy values with measured usage and evidence |\n| Tool access | capture tools, registry, guardrails, skill plugins | Resolved step capability set and policy; global registry pointers do not enter execution state |\n| Persistence timestamps | SQL `created_at` and `updated_at` | Domain timestamps and version used for optimistic concurrency and audit |\n| Transition history | SQL `transitions` and generic events | Authoritative WorkflowExecution domain events stored atomically with state |\n\n## Current runtime responsibilities to unify\n\nThe following behaviours currently act on the same WorkflowExecution but are\nimplemented in different packages:\n\n- accept a request from a channel and enforce idempotency;\n- select and version a Workflow;\n- create, queue, claim, schedule, and execute a WorkflowExecution;\n- serialize same-repository execution where policy requires it;\n- resolve identity-specific capabilities;\n- prepare, retain, recover, and clean up the workspace;\n- run deterministic and agent-backed WorkflowSteps;\n- enforce gates, budgets, protected paths, tool constraints, and result review;\n- persist state and StepExecution outputs;\n- publish authoritative lifecycle events;\n- wait for channel input and resume;\n- classify failure, retry, park, mark exhausted, cancel, or recover after crash;\n- deliver and reconcile forge changes;\n- project status, history, counts, and audit for channels and the web UI;\n- hand execution to a worker without moving domain ownership to the worker.\n\n## Current workflow-feature hazards\n\n- There is no enforced WorkflowExecution state machine. `Store.Transition`\n ignores the supplied `from` state when updating the row.\n- State update and transition-history insertion are separate operations, so\n state and audit can diverge.\n- Callers frequently discard persistence and transition errors.\n- Gateway task control duplicates lifecycle status constants by hand.\n- `workflow.TaskContext` is a mutable service bag containing complete\n configuration plus forge, store, worktree, agent, event bus, logging, memory,\n guardrails, skills, and scratch state.\n- Lifecycle changes occur in both daemon and workflow code.\n- Workflow orchestration runs inside task-scoped agent containers while the\n daemon retains lifecycle, admission, and authority-bearing RPC ownership.\n- Forge-backed and chat-created work share one record through source flags and\n synthetic issue numbers.\n- Repository labels currently select workflows and mirror task state, coupling\n domain transitions to forge presentation.\n- Failure generally becomes `parked`, but failure classification and\n retryability are not represented explicitly.\n- Generic observability events are emitted alongside state changes rather than\n being the authoritative domain events.\n\n## Existing invariants to preserve\n\n- The model never performs git operations directly.\n- Deterministic policy and gates cannot be waived by the agent.\n- TDD captures a failing reproduction before implementation and protects the\n test during the fix.\n- Every run ends in an explicit outcome; definition errors and stage errors do\n not disappear silently.\n- Crash recovery requeues interrupted work.\n- Agent execution crosses a versioned data boundary.\n- Forge, persistence, workspace, transport, container, and model runtimes remain\n replaceable infrastructure.\n- Work from the same repository is serialized by default.\n\n## Required migration\n\n- Rename the current persisted `store.Task` concept to WorkflowExecution.\n- Replace task lifecycle terminology in core commands, events, persistence,\n worker contracts, logs, and generated reference.\n- Preserve external source terminology only inside its channel adapter.\n- Replace synthetic issue numbers with explicit source-message references.\n- Associate every WorkflowExecution with an immutable Workflow identifier and\n version.\n- Record StepExecutions explicitly rather than treating one mutable `stage`\n field as complete execution history.\n- Keep Attempt as retry state within the same WorkflowExecution.\n\n## Curator isolation invariant\n\nDecided 2026-08-05\n([#435](https://github.com/samcharles93/archie-core/issues/435) epic,\n[#98](https://github.com/samcharles93/archie-core/issues/98) runtime).\nRecovered 2026-08-09 from the pre-migration issue tracker.\n\n**Curators are peers, not dependencies.** They must never block, delay, or take\ndown chat turns, agent runs, or the daemon.\n\n1. Registry-owned goroutines, one per curator, with per-pass `recover()`,\n per-pass budgets (max steps, tokens, timeout), and a configurable maximum of\n concurrent passes (default 1) so curators cannot saturate the shared LLM\n runtime against chat.\n2. Wake events ride the in-process `events.Bus`, which is mutex fan-out with\n bounded per-subscriber buffers that **drop** on overflow\n (`internal/events/events.go`). A slow curator can never backpressure the chat\n publisher.\n3. Wake is a best-effort nudge. The trigger is a deterministic read of persisted\n primary-input state at pass time, so a dropped wake only delays a run to the\n next check-in and never corrupts trigger accounting\n ([#22](https://github.com/samcharles93/archie-core/issues/22)).\n4. The curator registry lifecycle is independent of the agent loop. Daemon\n shutdown cancels in-flight passes and waits bounded.\n\nChat turns keep their per-session lanes; agent runs keep their workflow\ngoroutines and containers. Rule of thumb: **no code path a curator executes may\never appear on a chat turn's critical path.**\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/bindings/",
|
||
"title": "Playbook bindings: model and threat model",
|
||
"section": "Architecture",
|
||
"body": "# Playbook bindings: model and threat model\n\n**Status:** Approved foundation (documents shipped code: `archie-core-t2db.1`\nthrough `.5`, `.7`, `.9`-`.15`)\n**Date:** 2026-09-03\n\nThis document covers the binding model and the intake threat model portion\nof `archie-core-t2db.6`. It does not cover the `ARCHITECTURE.md` task-lifecycle\naddendum or the operator walkthrough -- those remain separate, open pieces of\nthe same ticket.\n\n## What a binding is\n\nA `binding` (`internal/domain/binding.Binding`) ties four things together:\na **matcher** (which captured events it applies to), a **mapping**\n(`internal/domain/mapping.Mapping`, how fields are pulled out of a payload),\na **workflow** (which registered workflow runs on a match), and a **state\nmachine** governing whether it is live.\n\n```go\ntype Binding struct {\n ID, Name // identity\n Matcher // { Source string } -- the webhook path segment, e.g. \"sentry\"\n MappingID // which Mapping resolves payload fields\n Workflow // registered workflow name to dispatch to\n Version // bumped on every edit\n Status // draft | pending_approval | armed\n Secret // HMAC-SHA256 shared secret, encrypted at rest, never returned by GET\n}\n```\n\nBindings live in the store (SQLite), not `config.toml` -- operators author\nthem from the dashboard while the daemon runs, without a restart.\n\n### Matching\n\n`Matcher.Source` is the only predicate today: the path segment a sender\nPOSTs to (e.g. a webhook hitting `.../sentry` matches bindings with\n`Matcher.Source == \"sentry\"`). The storage column is plain `TEXT`, so adding\npredicates later is non-breaking -- extend `Matcher`, don't invent a second\nmatcher shape.\n\n### Mapping\n\n`mapping.Resolve(fields, payload)` walks a captured JSON payload against a\nlist of typed `Field`s (name, JSON path, expected type), returning resolved\nvalues plus a `[]Failure` for anything that didn't parse or type-match. This\nis schema-by-example in practice: a mapping is authored by pointing at a\nreal captured payload in the dashboard, not by guessing a schema in advance.\n\n### Versioning\n\n`Version` increments on every `UpdateBinding` call. This exists so an\neditor's later change can never silently rewrite the historical provenance\nof a task that already fired -- a dispatched task's binding reference is\npinned to the version that was armed when it fired, not \"whatever the\nbinding currently says.\"\n\n### State machine\n\n```\ndraft ---\u003e pending_approval ---\u003e armed\n ^ ^ |\n | | |\n +--------------+-----------------+\n (any edit to an armed binding drops it back to pending_approval)\n```\n\nModeled directly on Telegram's existing `dangerousAction`/`pendingApproval`\nflow (`internal/channels/telegram/dangerous.go`, `approval.go`) -- not a new\napproval mechanism invented for this feature. **Only `armed` evaluates\nagainst incoming events** (`binding.Matches`, `internal/domain/binding/\nbinding.go`); `draft` and `pending_approval` bindings are inert. An edit to\nan already-armed binding cannot bypass approval by silent mutation -- it\ndrops back to `pending_approval` and requires an explicit `Approve` call to\nre-arm.\n\n## The threat model\n\nThis is a public, unauthenticated-by-default intake surface whose output is\nan agent with a repo, a worktree, and tool access. The design brief\n(`archie-core-t2db.5`, `docs/prds/webhook-intake-security.md`) named five\nnon-negotiables; each is verified below against the actual shipped code,\nnot the design intent.\n\n### 1. Per-source authentication\n\nReuses `internal/channels/webhook`'s existing HMAC-SHA256 scheme --\ncapture accepts unsigned events (visible in the dashboard, marked\nunauthenticated), but **only authenticated events can ever match a\nbinding**. `binding.Matches`' doc comment states this plainly: \"an\nunauthenticated event can never trigger a binding, no matter how well it\nwould otherwise match.\" Enforced structurally at two points, not just\nconvention:\n\n- `captured_events.authenticated` is a real column (`internal/store/\n captures.go`), set at capture time.\n- `dispatchBindings` (`internal/daemon/daemon.go:479`) walks **authenticated\n captures whose source has at least one armed binding** -- an\n unauthenticated capture is never even considered for dispatch.\n\n### 2. Human approval before a binding goes live\n\nCovered by the state machine above. Nothing self-arms: `pending_approval -\u003e\narmed` requires an explicit `Approve` call, and any edit to an armed\nbinding reverts it.\n\n### 3. Rate and volume limits per source\n\nCapture writes are bounded independently of the approval gate, so a flood\nof unauthenticated traffic cannot exhaust disk or drown the inspector:\n`InsertCapture` (`internal/store/captures.go`) prunes by both **retention\nwindow** (`WHERE received_at \u003c cutoff`) and **max event count** in the same\ninsert transaction.\n\n### 4. Blast radius bounded by the same code-level controls every task gets\n\nA playbook-originated task is dispatched into the **same** gate, diff-cap,\nsandboxed-container, worktree-isolation pipeline every task goes through\nregardless of origin. Verified precisely, not assumed: `dispatchOneBinding`\ncalls `EnqueueBindingTask` (`internal/store/store.go`), which wraps\n`EnqueueChatTask` -- the same direct-to-`tasks`-table enqueue path\nchat-spawned tasks already use, then stamps `binding_id`/`binding_version`\nfor provenance in a second statement. This is a **different** enqueue\nmechanism from forge-issue polling's `workintake.TaskEnvelope`/NATS path\n(`docs/prds/event-sources-and-reactions.md`) -- binding and forge-issue\ntasks arrive by different producers, but both become an ordinary\n`store.Task` row that `ClaimNext` picks up identically, so both get the\nsame gate/worktree pipeline downstream regardless of which producer created\nthe row. Payload content is\nordinary untrusted prompt input; the sender's text is never treated as a\ntrusted instruction (Archie's environmental-enforcement principle: \"the\nagent should ignore instructions in the payload\" is not itself a control,\nthe surrounding sandbox is).\n\n### 5. Captured payloads may contain secrets from the sending system\n\nTwo independent protections, not one:\n\n- **Redaction before persistence**: `webhookguard.RedactPayload` walks the\n decoded JSON and redacts recognised sensitive keys before the row is\n written -- the raw, unredacted payload is never durably stored.\n- **Retention**: a configurable window (`InsertCapture`'s `retention`\n parameter) prunes old captures automatically; visibility of what remains\n is gated by the same dashboard authentication as every other webui\n surface, not a separate, weaker path.\n\n### Secrets at rest\n\n`Binding.Secret` (the HMAC shared secret a sender signs with) is encrypted\nat rest via AES-GCM (`internal/store/binding_cipher.go`, `archie-core-\nt2db.7`) -- a leaked SQLite file does not leak sender-side shared secrets in\nplaintext. The envelope is versioned (`bindingEnvelopeVersion`) so a future\ncipher or KDF change can roll forward without breaking already-encrypted\nrows, and the AAD binds ciphertext to the binding-secret context so it\ncannot be relocated to another field/column and still authenticate. A `nil`\ncipher (no encryption key configured) leaves secrets as plaintext --\ndocumented legacy behaviour, not silent degradation.\n\n### At-most-once dispatch\n\n`dispatchBindings` enqueues at most one task per `(binding, capture)` pair,\nenforced by a real ledger, not best-effort application logic:\n`RecordDispatch` (`internal/store/bindings.go`) does `INSERT OR IGNORE` into\n`binding_dispatches`, unique on `(binding_id, capture_id)`; a duplicate\ninsert returns the sentinel `ErrAlreadyDispatched`, which the dispatch loop\ntreats as \"another cycle raced us,\" not an error. This guarantee holds\n**across daemon restarts**, not just within one process's lifetime, because\nthe ledger is a durable table, not in-memory state.\n\nThis ledger is also the direct precedent for the EDA playbook engine's\nexecution-time idempotency, now landed (`archie-core-t2db.17` closed): the\n`playbook_dispatches` table in `internal/store/playbook_dispatches.go` copies\n`binding_dispatches`'s `INSERT OR IGNORE` conventions with a\n`(playbook_id, playbook_version, event_id, action_id)` key\n(`docs/prds/eda-playbook-engine.md` gap 2).\n\n## What this document does not cover\n\n- The `ARCHITECTURE.md` task-lifecycle section noting that a\n playbook-originated task has different provenance from a forge-issue\n task -- open, part of `archie-core-t2db.6`.\n- An end-to-end operator walkthrough (point an external app at archie,\n build a first playbook) -- open, part of `archie-core-t2db.6`.\n- The playbook YAML / CEL / Module/Channel/Forge action-position work\n (`docs/prds/eda-playbook-engine.md`, `module-position.md`,\n `channel-forge-positions.md`) -- a different, newer mechanism than the\n binding model described here; the two are related but not the same\n system, and this document does not attempt to unify them.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/configuration/",
|
||
"title": "Configuration",
|
||
"section": "Architecture",
|
||
"body": "# Configuration\n\n**Status:** Approved; existing-type assignment is ongoing \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Ownership\n\nConfiguration has two separate responsibilities:\n\n1. `internal/infrastructure/configuration` loads files, environment variables,\n secret references, overlays, and compatibility representations.\n2. Each domain and plugin owns the typed runtime settings required for its\n behaviour.\n\n`internal/app` translates external input into those settings, validates\ncross-capability combinations, and supplies each component only its own\nsettings.\n\nThe following are prohibited:\n\n- passing complete application configuration into a domain;\n- making a domain depend on TOML, YAML, environment-variable names, secret\n references, or a configuration library;\n- defining a central runtime configuration type imported across the system;\n- placing plugin options in an unrelated global configuration model.\n\n## Private input document\n\n`internal/infrastructure/configuration` MAY use a complete decoded document\nwhile discovering files, decoding sections, and applying overlays. It is an\ninput-boundary DTO only.\n\nThe document:\n\n- remains private to the configuration infrastructure;\n- is composed from separately defined feature sections;\n- is immediately translated by application composition;\n- is never returned as the application's runtime model;\n- is never passed through runtime components.\n\nExternal DTOs and domain runtime settings are separate types.\n\nPlugins supply external configuration definitions through a narrow extension\ncontract. The core loader MUST NOT gain a field or recognized filename for every\nplugin.\n\nInvalid configuration input MUST NOT replace active valid settings or terminate\nunaffected capabilities. Configuration changes use the candidate validation,\npromotion, audit, and recovery requirements in\n[safe-change-and-recovery.md](/docs/architecture/safe-change-and-recovery/).\n\n## Dissolution of `internal/config`\n\n`internal/config` has no target location. It MUST be dissolved and MUST NOT\nsurvive as a shared package, configuration domain, or renamed global model.\n\nIts loading behaviour moves to `internal/infrastructure/configuration`. Existing\ntypes and methods are reassigned by behaviour. Repository execution policy,\ndispatch policy, retry policy, ecosystem defaults, tool policy, memory settings,\nindexing settings, provider settings, and channel settings move to their\nrespective owners.\n\n## Repository settings\n\nThe existing `config.Repo` MUST NOT move intact into a repository domain or\nshared settings package. A managed repository is an application association:\n\n- work intake owns which repository references an identity monitors;\n- task execution owns gates, preflight, test protection, ecosystem behaviour,\n and change limits;\n- task lifecycle owns retry behaviour;\n- scheduling owns concurrency policy;\n- workspace management owns base branch and persistent-storage requirements.\n\nApplication composition correlates these settings by repository reference. No\ndomain receives a universal repository profile.\n\n## Documentation\n\nGenerated configuration reference describes external sections and their owning\ncapabilities. It MUST NOT establish a global Go configuration structure as the\npublic contract.\n\n## Value classification rule\n\nEvery value that controls behaviour MUST have an explicit owner and MUST be\nclassified as exactly one of:\n\n1. **Invariant** — changing it changes the meaning of the program, a wire\n protocol, or a domain concept. It may be a Go constant.\n2. **Runtime setting** — an operator, identity, domain, plugin, or deployment\n may legitimately choose a different value. It is typed configuration owned by\n the affected capability.\n3. **Policy value** — it expresses what is allowed, limited, routed, retried,\n protected, or gated. Its typed definition belongs to the owning domain and it\n is evaluated through the policy system.\n4. **Derived value** — it is computed from other settings and is not\n independently mutable.\n5. **Runtime state** — it changes as the application operates, such as the\n active model for a conversation. It is changed by a domain command and is not\n configuration merely because it can be changed.\n\nAn unclassified behavioural literal is a defect.\n\n`const` MUST NOT be used merely because a value currently has one default.\nConstants are reserved for actual invariants: enum discriminants, protocol\nversions, fixed compatibility tokens, and values whose alteration would change\nthe program's semantics rather than configure an installation.\n\nIn Go, a constant is a compile-time value and does not represent mutable runtime\nstorage. It generally has no address that can be observed or changed.\nArchitecturally, a constant therefore asserts that the value cannot change for\nthe lifetime of the application. Any value that may legitimately change while\nthe application is running, between application instances, or between\ndeployments is not a constant.\n\nA runtime setting may have a default, but the default is not its only access\npath. The owner exposes a typed settings structure, validation, and a\n`DefaultSettings` constructor or equivalent. Private constants used to implement\ndefaults do not turn those defaults into invariants and MUST NOT be consumed\ndirectly by runtime code.\n\nEach runtime setting and policy value MUST declare:\n\n- its owner;\n- type, units, validation constraints, and safe bounds;\n- default and derivation, if any;\n- whether it is startup-only, live-reloadable, or restart-required;\n- whether it may differ by installation, identity, repository, workflow,\n channel, or plugin instance;\n- whether it is secret, sensitive, or safe to report;\n- its current source, effective value, version, and last-known-good version;\n- the command and identity required to change it.\n\n## Current configuration migration ledger\n\nThis ledger records the current meanings hidden inside `internal/config`. It is\nthe source inventory for dissolving that package; rows are split where the\nexisting structure combines different owners.\n\n| Current values | Current location | Target owner and requirement |\n| ----------------------------------------------------------------------------------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `WorkDir`, `DBPath` | root `Config` | Workspace/storage infrastructure settings. `DBPath` is the prefix for independent task and conversation SQLite files. Defaults derive from the platform data directory. |\n| `SkillsDir`, `PluginDir` | root `Config` | Skill and plugin infrastructure settings, independently typed. |\n| `PollInterval` | root `Config` | Work-intake scheduling setting; not a daemon-wide constant. |\n| `Label`, `BotUser`, `BotEmail` | root `Config` | Identity and dispatch associations. `BotEmail` may be derived from the selected forge account but remains overrideable. |\n| `MaxRetries` | root `Config`, `Repo` | Retry policy owned by the operation being retried. The duplicate global/repository representation must disappear. |\n| `DiffCapLines` | root `Config` | Agent-change policy value. |\n| `MaxSteps`, `WallClock`, `GateMaxFailures` | `Budgets` | Workflow-execution policy, with units and zero/unlimited semantics made explicit. |\n| `Class`, `APIKeyEnv`, `BaseURL` | `Provider` | Provider instance settings. Secret lookup becomes a `SecretRef`, not an environment variable name leaking into the provider domain. |\n| `Mode`, `Command`, `Env` | `LegacyAgent` | Decode-only compatibility input for the removed executor selector. Delete after the config migration window; it has no runtime settings owner. |\n| `Type`, `Host`, `Token` / `TokenEnv` (older form) | `Forge` | Forge adapter settings. The older `TokenEnv` input is accepted only at the loader boundary. |\n| `Trigger`, `AckReaction`, label names | `Dispatch` | Work-intake and forge presentation policy. Label vocabulary is configurable per forge/identity association. |\n| `Engine` | `MemoryConfig` | Memory engine selection. `internal/domain/memory` is the only engine family; per-engine settings remain owned and validated by that engine. |\n| `Name`, `Transport`, `Command`, `Args`, `WorkDir`, `URL` | `MCPServer` | One typed MCP server instance specification. Transport-specific variants must reject fields belonging to other variants. |\n| `MaxResultChars` | `ToolPolicy` | Tool-execution policy values, not generic daemon configuration. |\n| `URL`, `TokenEnv` | `NATSConfig` | NATS transport settings with a secret reference replacing the environment-name field. |\n| `Image`, `MaxConcurrency`, `MaxUptime`, `VolumeTTL`, `PullPolicy`, `Network` | `ContainerConfig` | Mandatory managed-worker infrastructure settings and resource policy. The legacy `enabled` input is decode-only. |\n| model mappings, email, webhook, Telegram | `ChatConfig` | Separate channel instance settings plus channel-routing associations; there is no chat-wide settings owner. |\n| `AllowedUserIDs`, `Token` / `TokenEnv` (older form), update commands | Telegram config | Telegram adapter settings and access policy. `Token` is the preferred `secret.SecretRef`; `TokenEnv` remains a backwards-compatible fallback. |\n| listen/relay addresses | email config | Email adapter instance settings. |\n| webhook target | `Notify` | Notification adapter instance settings. |\n| web listen address | `Web` | Web transport settings. |\n| `listen` under `[services.state]` and `[services.gateway]` | `ServiceConnection` | The bind address of the process that owns the service. Read only by that process, and defaulted to the address its clients dial so the pair cannot drift. |\n| index directory and database path | `Indexing` | Index infrastructure settings, derived from storage settings unless explicitly overridden. |\n| forge, dispatch, models, providers, containers, memory, tools, indexing and repositories | `IdentityConfig` | Existing composite duplicates unrelated settings. Replace it with associations from an `IdentityID` to independently owned configured instances. |\n| owner, name, base, gate, protection, ecosystem, preflight, storage, retry and concurrency | `Repo` | Split across work intake, workspace, workflow execution, policy, gate, and scheduling as described in Repository settings. |\n| `TaskConfig`, `TaskForge` | worker snapshot | Replace with the minimum immutable, versioned WorkflowExecution input assembled by the application. It is not a second configuration model. |\n| `Extra` and arbitrary maps | root/provider config | Allowed only at an extension boundary whose owning plugin/provider supplies schema and validation. Core configuration never consumes an untyped catch-all. |\n\n## Behavioural literals and incorrectly fixed values\n\nThe following current values affect useful behaviour but are not represented as\nowned typed settings or policy. They MUST be classified and migrated before the\nold configuration package is removed. The listed target is the architectural\ndecision; exact setting names are implementation details.\n\n| Capability | Values fixed in current code | Required classification |\n| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Chat agent | maximum steps `8` in `cmd/archied` | Workflow/chat execution policy. |\n| Context compression | enabled, threshold `0.5`, context size `128000`, protect first `3` and last `20`, compression marker, token estimate of four characters per token | Model/context runtime settings. The marker is presentation; the estimate is provider/model strategy, not a universal truth. |\n| Tool guardrails | escalation thresholds `3`, `5`, `10`, `2` | Tool-use policy with named meanings and bounded defaults. |\n| Tool output | per-result cap `50000` characters, no aggregate turn cap, preview `500` characters | Tool-execution resource and presentation settings. Per-result spill/truncation remains; turn continuation is not character- or token-gated. |\n| Pairing/access | code TTL `1h`, maximum pending `3`, rate window `10m`, failures `5`, lockout `1h`, maximum lockout `24h` | Access policy. Code length/alphabet and salt length require an explicit security-constraint decision; they are not automatically valid constants. |\n| NATS daemon consumer | stream `ARCHIE_TASKS`, consumer `archie-daemon`, deduplication `2m`, poll `2s`, deliveries `3`, acknowledgement wait `5m`, inactive threshold `24h`, subjects | NATS topology settings plus delivery policy. Subject grammar or protocol headers may be invariants only if declared as versioned wire contracts. |\n| NATS worker consumer | stream `ARCHIE_TASKS`, consumer `archie-agent`, deduplication `2m`, poll `5s`, acknowledgement wait `30m`, deliveries `3`, inactive threshold `24h`, queue group and retry delay `1s` | Worker transport settings and delivery/retry policy. Shared stream defaults must have one owner rather than two copies. |\n| Worker readiness | readiness timeout `20s`, retry delay `250ms` | Worker lifecycle and retry policy. |\n| Execution/RPC | agent execution `30m`, task-run RPC `60s`, worktree RPC `15m` | Per-operation timeout policy. |\n| Indexing | build timeout `10m`, metadata timeout `5s` | Index operation policy. |\n| Plugin/tool cleanup | plugin rollback `10s`, tool-provider cleanup `10s`, MCP-provider cleanup `10s` | Lifecycle shutdown/rollback policy owned by each engine, with a shared policy vocabulary where appropriate. |\n| Memory read path | per-scope record limit `20`, rendered `\u003cmemory\u003e` block byte cap `8192` | Memory read policy owned by the engine family (`internal/domain/memory`). |\n| Built-in memory store | per-file size limit `100KiB`, query result limit `100` | Split into memory resource policy, input policy, and presentation settings. |\n| Memory safety scanner | compiled prompt-injection and sensitive-data patterns with fixed warn/block consequences | Versioned memory-domain policy definitions. Patterns and consequences must be inspectable and replaceable without editing scanner code. |\n| MCP stdio | initial backoff `500ms`, maximum backoff `30s`, unlimited default retries, send and shutdown `5s` | MCP transport retry, request, and lifecycle policy. Unlimited retries must be explicit rather than a zero-value accident. |\n| MCP SSE/HTTP | reconnect `1s` to `30s`, POST/request timeout `30s` | MCP transport settings and retry policy. |\n| Message framing | MCP frame `4MiB`, worker protocol frame `8MiB`, diagnostic `64KiB`, media/base64 `64MiB` | Resource/security policy unless a published wire protocol mandates the limit. Limits must be configurable only within safe bounds. |\n| Webhooks and web UI | webhook bind `0.0.0.0:8644`, webhook shutdown `5s`, web read-header `5s`, web shutdown `2s` | Transport instance and lifecycle settings. |\n| Telegram | typing refresh `4s`, draft interval `700ms`, update-action TTL `10m` | Telegram adapter presentation and lifecycle settings. |\n| Telegram limits | message length `4000` and callback/update prefixes | Record the upstream platform constraint and version it as an adapter compatibility limit; prefixes may remain wire constants when changing them would invalidate outstanding callbacks. |\n| Containers | stop and pull timeouts `10s` and `30s` | Container lifecycle policy. |\n| HTTP clients | feasibility notification timeout `10s`; other paths use `http.DefaultClient` | Adapter request policy. Every outbound client requires an explicit timeout profile. |\n| Workflow routing | labels `bug`, `feature`, `bootstrap`; workflows `tdd`, `feasibility`, `bootstrap`; fallback `implement` | Agent-system routing policy. Workflow identifiers are references, not compiled routing decisions. |\n| Filesystem/deployment | `.agents/skills`, `.archie/stages`, `.archie/gate.go`, `/data/worktree`, `/data/repo`, cache names/paths and prepared sentinel | Split into deployment settings, repository conventions, and genuine internal protocol markers. Paths an installation may relocate are settings. |\n| Packaged release data | `/usr/share/archie/CHANGELOG.archied.md` and `/usr/share/archie/CHANGELOG.archie.md` | Packaging/deployment settings or paths derived from an installation prefix. |\n| Forge presentation | default label colour `bfd4f2` and state-label colours | Forge adapter presentation settings. |\n| Fallback system prompt | compiled fallback prompt | Agent-system default content, versioned and replaceable through its owning settings; safety constraints remain policy/code. |\n| Default paths and identities | config path, data paths, listen addresses, model roles and workflow names | Typed defaults belonging to their respective owners, never global constants consumed throughout the application. |\n\nThis inventory is intentionally broader than exported Go `const` declarations. A\nliteral hidden in a constructor, timeout, switch, retry loop, path join, prompt,\nlabel map, or client setup restricts behaviour just as effectively as an\nincorrectly used constant.\n\n## Constants that may remain\n\nA value may remain constant only when its owner documents why runtime variation\nwould be invalid. Typical valid cases are:\n\n- domain enum values and state names;\n- a versioned wire-protocol version or framing token;\n- reserved internal headers and callback prefixes that peers must interpret\n identically;\n- compile-time mathematical relationships;\n- sentinel errors and internal type identifiers.\n\nNames such as a stream, consumer, label, workflow, file, directory, network,\nmodel role, timeout, retry count, capacity, size limit, colour, prompt, command,\nor executable are not invariants merely because the current application has only\none of them.\n\nArchitecture tests SHOULD reject exported behavioural constants outside approved\ninvariant packages and SHOULD detect direct consumption of private default\nconstants outside their owning `DefaultSettings` construction.\n\n## Current mutation and reload ledger\n\nThe application currently changes behavioural values through unrelated paths:\n\n| Current change seam | Current behaviour | Required target |\n| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Telegram `/restart` in `cmd/archied` | Re-reads the global document and directly replaces Telegram token and allowed-user IDs; failure retains old values. | Submit a Telegram-settings candidate through the universal change protocol. Validate the complete candidate, promote atomically, audit the acting identity, and retain last-known-good. |\n| Active model selection | `chatModelManager.SetActiveModel` changes a model reference. | A conversation/session command changing runtime state. Validate against configured model instances and audit it; do not rewrite configuration. |\n| Active provider selection | `chatModelManager.SetActiveProvider` changes provider selection. | A conversation/session command changing runtime state, with the same distinction from provider-instance configuration. |\n| Persona selection | `PersonaRegistry.SetActive` mutates an in-memory per-session choice. | Agent-system session state. Persistence and audit requirements are owned by that domain, not the configuration loader. |\n| File and environment overlays | Applied while loading `internal/config`; defaults are filled by mutation in `finalize`. | Configuration-infrastructure input processing that produces separately typed candidates. Defaults and derivations remain visible in provenance. |\n| SIGHUP config reload (2026-08) | Re-resolves the file config, re-applies the database layer over it, validates, then atomically republishes through `config.Holder`. A failed reload, whether the file or the control plane caused it, keeps the running config and records `last_error`/`last_error_at` in `/api/config`. | Candidate promotion with last-known-good retention: the running snapshot is untouched on failure. A bounded observation period and actor audit are not yet implemented. |\n| Control-plane resource command (2026-09) | `ControlPlaneService.Command` validates a whole resource document against its owning feature, then writes it with an audit record (actor, source, request ID, expected version) under optimistic concurrency. Hosted on the State Store's gRPC server; the Web UI and messaging channels are its only clients. See `docs/prds/runtime-control-plane.md`. | Met for validate-stage-promote, actor audit and versioned history. Per-field health observation and validate-before-switch on live apply are not yet implemented. |\n\nAny further mutation seam discovered during migration MUST be added here before\nit is changed. Direct field assignment on a live settings object is prohibited.\n\n## Universal change protocol\n\nChanging configuration is an application command performed by an identity. It\nMUST:\n\n1. resolve the target owner and current version;\n2. parse into that owner's typed settings;\n3. validate syntax, semantic constraints, policy, references, and safe bounds;\n4. stage a candidate without altering the active value;\n5. run owner-defined readiness and health checks;\n6. atomically promote the candidate and record actor, source, reason, diff,\n prior version, and new version;\n7. observe the affected capability for a bounded period;\n8. retain the new version on success, or automatically restore last-known-good\n on failure;\n9. report the rejected or rolled-back candidate without terminating unaffected\n capabilities.\n\nStartup loading uses the same candidate rules. A bad user value therefore does\nnot partially mutate the application or make a previously healthy capability\nunavailable.\n\nSettings that cannot safely reload in-process are marked restart-required. The\nchange may still be validated and versioned immediately, but promotion occurs\nthrough a controlled restart with the same health and rollback semantics.\n\n## Implemented runtime reload and settings writes (2026-09)\n\nTwo mechanisms change behavioural values at runtime: SIGHUP re-reads the file\nconfig, and the control plane owns every database-backed setting. The dashboard\nruntime overlay that used to sit between them was deleted once the control plane\ncovered its surface; `-config-overlay` names a file overlay and is unrelated.\n\n### Published snapshot: `config.Holder`\n\nThe running configuration is a published snapshot held in `config.Holder`\n(`internal/config/holder.go`). Get returns a value under RLock; Set replaces the\nwhole snapshot. A published Config is immutable: reload constructs a fresh one\nand calls Set. `config.Clone` deep-copies every reference-type field so a failed\ndecode cannot mutate the published snapshot's shared maps. `Holder.Get` panics\non a nil receiver — a forgotten `NewHolder` must fail loudly, not boot a daemon\nthat silently runs on an empty config.\n\n### The reloadable-field criterion\n\nThe diff-and-warn on reload asks whether a changed field takes effect without a\nrestart. The criterion is: **a field is reloadable when every consumer re-reads\nit.** That question read as \"what does the daemon re-read\" while the daemon owned\nthe only Holder; it stopped being sufficient when webui began sharing that\nHolder, because webui handlers re-read on every request. The allowlist lives in\n`cmd/archied/reload.go` and is deliberately a deny-by-default list: a field added\nto Config later defaults to requires-restart, forcing whoever adds it to decide\ndeliberately. A reload that changes any non-allowlisted field logs\n\"requires a restart\" with the field names; a fully-applied reload logs Info.\n\n### SIGHUP reload\n\nSIGHUP re-resolves the file config, re-applies the database layer over it\n(`boot.runtimeConfig`, the same call boot makes), validates, and atomically\nrepublishes through the shared Holder. A failed reload keeps the running config\nand records `last_error`/`last_error_at` in `/api/config` so the stale state is\nvisible where the operator is looking. The three poll tickers re-read\n`PollInterval` per iteration and reset on change, so a reloaded interval\ngenuinely takes effect.\n\n### Settings writes go through the control plane\n\nDatabase-backed settings are read and written through\n`ControlPlaneService` on the State Store's gRPC server: catalog, query,\ncommand, watch. Each feature owns its resource's validation and commands;\nstorage writes versions and audit records only. `docs/prds/runtime-control-plane.md`\nis the specification.\n\nTwo properties matter to anyone changing this area:\n\n- **Bootstrap stays file-owned.** The daemon reads the file config first and\n layers database resources over it at boot (`boot.loadRuntimeConfig`). Values\n the daemon needs before it can reach the State Store — the database path,\n listen addresses, credentials — stay file-only by design.\n- **Reload re-reads them.** SIGHUP resolves the file document and then runs the\n same `boot.runtimeConfig` layering boot does, so a reload cannot revert a\n database-owned setting to its file value. A control plane that cannot answer\n fails the reload rather than publishing the file's values over the running\n ones. Workflow execution budgets arrive on a watch rather than in a resource\n query, so boot records the last published settings and the reload re-applies\n them from there.\n\n## Completion criteria\n\nThe configuration rearchitecture is complete only when:\n\n- `internal/config` no longer exists;\n- every behavioural value has one classification and owner;\n- every current field and hard-coded value in the ledgers has a recorded\n disposition;\n- domains and plugins receive only their typed settings;\n- all changes use semantic commands and the universal change protocol;\n- active, candidate, rejected, rolled-back, and last-known-good versions are\n queryable with secret-safe provenance;\n- generated documentation lists every external key, default, constraint,\n mutability class, owner, and deprecation;\n- architecture tests prevent a new central runtime configuration model, direct\n live-setting mutation, and incorrectly fixed behavioural values.\n\n## Startup policy: missing credential degrades, invalid config is fatal\n\nDecided 2026-07-30. Recovered 2026-08-09 from the pre-migration issue tracker.\n\nA missing **credential** disables a capability. Only an **invalid config** stops\nthe daemon. \"Forge is a feature, not a requirement.\"\n\nBefore `518a1a0`, an unresolvable forge token did `return 1` in\n`cmd/archied/main.go`, so a chat-only user who never configured GitHub got no\nchat, no gateway, nothing — and under a systemd unit with `Restart=on-failure`\nand no start limit it crash-looped every five seconds with the error scrolling\npast unread. Forge construction now lives in `resolveForge`, shared by the\nprimary forge and every configured identity, and degrades to `forge.NewNoop`\nwith a warning. The identity path had the same defect and was worse: one\nidentity's missing credential killed all the others.\n\nMalformed configuration still fails fast in `configuration.Validate`, well\nbefore this point. That distinction is the whole design.\n\nApply the same reasoning to any future capability credential, LLM keys\nespecially: start, report the capability as unavailable, let the operator see it.\nGuarded by `TestResolveForgeDegradesInsteadOfFailing`.\n\n**Known consequence, tracked separately:** `worktree.Manager` still receives the\nempty token, so pushes fail late rather than at startup.\n\n## Config generation belongs to `archied setup`, not `install.sh`\n\nDecided 2026-07-30 ([#368](https://github.com/samcharles93/archie-core/issues/368)).\nRecovered 2026-08-09 from the pre-migration issue tracker.\n\n`install.sh` must stop generating `config.toml`; `archied setup` owns it. The\ninstaller hand-rolling TOML has already drifted from the schema once: the\ngenerated-config branch hardcoded `key = ARCHIE_GITHUB_TOKEN` and\n`host = ${GITEA_URL}` regardless of the answers given, so choosing Gitea produced\na config naming a token the installer never wrote (archied then exited 1), and\nchoosing GitHub pointed `host` at `gitea.example.com`. `61a5b35` patched this\nwith a `forge_block` shell helper shared by both config paths — that helper is a\n**workaround**, deleted by\n[#72](https://github.com/samcharles93/archie-core/issues/72). The durable fix is\nthat the code writing the config is the code that reads it.\n\nWhy Go and not Python: the installer must run where nothing is installed yet, and\n`python3` with `tomllib` means 3.11+, which is not safe on LTS distros. `archied`\nis already being built two steps earlier.\n\n**Key constraint for the port:** `BurntSushi/toml` does **not** survive a\ndecode/encode round trip with comments intact, and `config.example.toml` carries\ndocumentation users are meant to hand-edit. Decide the write strategy\n([#412](https://github.com/samcharles93/archie-core/issues/412)) before writing\nany prompting code. Do not port a separate `config.yaml`/`auth.yaml` secret\nsplit — Archie already has `secret.SecretRef{engine,key}`, so setup writes the\nreference to TOML and the value to the secret engine.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/dependencies-and-contracts/",
|
||
"title": "Dependencies and Contracts",
|
||
"section": "Architecture",
|
||
"body": "# Dependencies and Contracts\n\n**Status:** Approved foundation \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Dependency direction\n\n```text\ndomain and shared application contracts \u003c- infrastructure implementations\n ^\n |\n application composition\n```\n\nThe following rules are strict:\n\n1. Domains MUST NOT import `internal/infrastructure`.\n2. A domain defines the smallest interfaces required to perform its work.\n3. Infrastructure implements those interfaces and translates external\n representations at the boundary.\n4. Only application composition chooses concrete implementations.\n5. A domain MUST NOT manipulate another domain's state or call its internal\n services.\n6. Cross-domain work uses an explicitly designed command, event, or approved\n shared contract.\n7. Implementation difficulty is not permission to violate a boundary.\n\nBroker messages, database rows, SDK response types, and configuration DTOs MUST\nNOT leak into domain models.\n\n## Commands and events\n\n- An event records something that happened and is owned by its publishing\n domain.\n- A command requests an action and is owned by the domain expected to handle it.\n- Consumers do not redefine an event's meaning.\n- Senders obey the receiving domain's command contract.\n- There is no generic package containing unrelated application messages.\n\n## Event bus\n\n`internal/eventbus` is an approved shared application contract. It defines only\nthe messaging behaviour Archie requires, including publication, subscription,\ndelivery outcomes, and required lifecycle semantics.\n\nIt MUST remain broker-neutral. NATS subjects, JetStream configuration, consumer\ngroups, acknowledgements, reconnection, and serialization belong under\n`internal/infrastructure/eventbus/nats` unless Archie explicitly requires a\nsemantic guarantee in the shared contract.\n\nMessage schemas stay with their owning domains. The event bus transports\nmessages; it does not own their meaning.\n\n## Wire contracts and generation\n\nCanonical worker and service wire contracts and their authoritative generation\ninputs are owned by the domain or capability that defines their meaning. They\nMUST NOT be centralized in `internal/infrastructure/spec` or another generic\nschema package.\n\nInfrastructure owns:\n\n- generic contract-generation machinery;\n- transport-specific adapters and generated implementations;\n- RPC, REST, NATS, and other external encoding and delivery concerns.\n\nFor example, agent-worker requests and results belong to task execution.\nSubprocess, NATS, or HTTP representations are infrastructure implementations of\nthat contract. The same rule applies independently to forge, persistence,\nworkspace, and other service contracts.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/generated-documentation/",
|
||
"title": "Generated Documentation",
|
||
"section": "Architecture",
|
||
"body": "# Generated Documentation\n\n**Status:** Generator scaffold in place; renderer and publishing removed\n**Date:** 2026-07-28 (revised 2026-09-12)\n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchie's documentation-generation approach is:\n\n- production Go code, registries, configuration, and CLI behaviour remain the\n source of truth for published contracts;\n- a repository-local Go generator extracts and normalizes documentation data;\n- the generator writes deterministic JSON/data files and protocol schemas;\n- the rendering and publishing surface is an **open decision** — the VitePress\n site and its Pages deployment were removed on 2026-09-12 as an unnecessary\n build step (see \"Rendering and publishing\" below).\n\nHugo is not part of the target architecture.\n\nThe documentation system is rooted at:\n\n```text\ntools/\n go.mod\n go.sum\n docsgen/\n main.go\ndocs/\n data/generated/\n public/schemas/\n *.md\n```\n\n### Rendering and publishing — OPEN\n\nSuperseded 2026-09-12. The repository previously carried a VitePress site\n(`docs/.vitepress/config.mts`, `docs/package.json`, a 1620-line\n`docs/pnpm-lock.yaml`, `docs/pnpm-workspace.yaml`) built and deployed to GitHub\nPages by `.github/workflows/docs.yml`. It was removed as an unnecessary build\nstep: the published site was never stood up, and the Pages target is not\nconfigured.\n\n`docs/` is now repository documentation only — markdown read in the repository,\nan editor, or a forge's file view. Nothing in the repository builds, renders, or\npublishes it.\n\nThe renderer for the generated `data/generated/*.json` is therefore **undecided**.\nConsequences that follow, and their current state:\n\n| Concern | State after removal |\n| --- | --- |\n| Reference-page rendering | Undecided; `data/generated/contracts.json` is committed but unrendered |\n| Internal link validation | Not enforced by any build; links are repository-relative (`../architecture/...`) and resolve on the filesystem |\n| Generated-data drift detection | Unchanged — `docsgen check` compares committed output, independent of any renderer |\n| Navigation/sidebar | The authoritative index is `docs/prds/01-project-management.md` |\n| Local authoring server | None; read markdown directly |\n\nA future renderer MUST NOT reintroduce a dead-link-blind build: VitePress\npreviously validated internal links during its production build, and no\nreplacement currently does.\n\n## Single source of truth\n\nEvery published capability has one authoritative machine-readable definition\nowned by its domain or application boundary. Generated documentation derives\nfrom those definitions.\n\nGenerated reference includes:\n\n- domain commands and events;\n- Messages and typed message content;\n- Agent and Workflow contracts;\n- WorkflowSteps and Workflow plugin capability contracts;\n- policies and their typed inputs and outcomes;\n- tools and capabilities;\n- CLI commands, arguments, flags, defaults, and help;\n- runtime settings, types, defaults, validation, and secret metadata;\n- public worker and transport contracts;\n- indexes of registered functionality;\n- deprecation information.\n\nHandwritten documents contain:\n\n- architecture decisions and rationale;\n- tutorials and operational guides;\n- examples and walkthroughs;\n- migration guidance;\n- conceptual explanations.\n\nHandwritten documents link to generated reference rather than copying\nauthoritative tables.\n\n## Target generation architecture\n\n`tools/docsgen` is Archie's documentation extractor and normalizer. It produces\nstructured documentation data and protocol specifications. It does not own\npage layout and MUST NOT generate Markdown pages or fragments.\n\nThe generator is divided internally into reusable stages:\n\n```text\nload registries and types\n -\u003e normalize documentation model\n -\u003e validate ownership and identifiers\n -\u003e serialize data and protocol artifacts\n -\u003e compare or write output\n```\n\nThe normalized documentation model prevents each output renderer from\nindependently interpreting Go types. It contains stable documentation IDs,\nowners, names, descriptions, source references, versions, deprecations, fields,\ndefaults, validation, secret classifications, and relationships.\n\nInitial generator targets:\n\n```text\ndocsgen data\ndocsgen asyncapi\ndocsgen all\ndocsgen check\n```\n\n- `data` writes the normalized JSON reference data.\n- `asyncapi` writes protocol schemas and envelopes.\n- `all` generates every committed artifact.\n- `check` generates into a temporary directory and fails on drift.\n\nThe command may use subcommands or an equivalent explicit `--target` flag. It\nMUST retain one normalization and validation path.\n\n## Target output\n\nDuring adaptation, `docs` remains the staging site. Generated outputs live in\na visibly owned subtree:\n\n```text\ndocs/\n data/\n generated/\n catalog.json\n contracts.json\n docs.json\n cli.json\n configuration.json\n messaging.json\n agents.json\n workflows.json\n policies.json\n plugins.json\n tools.json\n public/\n schemas/\n asyncapi-header.yaml\n archie.yaml\n .tmp/\n```\n\nAll files under `data/generated/` and `public/schemas/` carry generated\nownership metadata where their format permits it and MUST NOT be edited\nmanually. `.tmp/` is generator-owned, ignored, and never committed.\n\n`catalog.json` is the deterministic machine-readable index used for:\n\n- reference navigation and route inputs;\n- duplicate-ID and duplicate-route validation;\n- coverage checks;\n- source-to-data drift diagnostics;\n- future external consumers.\n\nThese files are consumed at render time by whichever renderer is chosen (see\n\"Rendering and publishing\"); the generator's contract is the committed JSON, not\nany particular loader. A renderer reads `data/generated/*.json` directly — Archie\ndoes not use `createContentLoader`-style Markdown collection scanning, because\nthe authoritative inputs are JSON.\n\nThe final move from `docs/` to `docs/` occurs only after Archie content\ncontains intentional Archie pages and the current architecture documents have an\nintentional destination. The staging name MUST NOT leak into public URLs.\n\n## Generation steps\n\n### Step 1: maintain the Archie tools module\n\nKeep `tools/go.mod` isolated from the runtime module:\n\n```go\nmodule github.com/samcharles93/archie-core/tools\n\nrequire github.com/samcharles93/archie-core v0.0.0-00010101000000-000000000000\n\nreplace github.com/samcharles93/archie-core =\u003e ../\n```\n\nKeep generator-only dependencies such as `invopop/jsonschema` and any future\nprotocol-schema serializer in the nested tools module. They MUST NOT enter the\nroot module.\n\n`docsgen` imports Archie-owned contracts and writes normalized Archie data and\nprotocol artifacts.\n\n### Step 2: establish authoritative registries\n\nThe generator consumes domain-owned registries rather than scanning arbitrary\npackages for anything exported.\n\nEach published definition supplies:\n\n- stable documentation ID;\n- owning domain or boundary;\n- name and description;\n- version when versioned;\n- the Go type or declarative definition;\n- lifecycle and deprecation metadata;\n- source reference.\n\nRegistries remain executable application definitions used by production code. A\ndocumentation-only duplicate registry is prohibited.\n\nWhere the migration has not yet created a final domain registry, the generator\nmay use a narrow compatibility adapter over the current registry. That adapter\nhas explicit deletion criteria.\n\nThe initial agent-execution and messaging schema set is such an adapter. It is\ndeleted when those domains expose their production-owned documentation\nregistries; adding another hardcoded type list to `docsgen` is prohibited.\n\n### Step 3: generate AsyncAPI\n\nUse Archie's messaging and worker wire registries as the authoritative inputs.\n\nThe static header continues to own:\n\n- AsyncAPI version;\n- service and protocol description;\n- servers;\n- channels;\n- operations;\n- message envelope placement.\n\nThe generator owns:\n\n- component schemas;\n- discriminated concrete message lists;\n- generated envelopes;\n- rewritten schema references;\n- stable ordering.\n\nThe generated protocol artifact becomes:\n\n```text\ndocs/public/schemas/archie.yaml\n```\n\nThe final owner of each wire registry is the domain or capability defining its\nmeaning. `docsgen` renders the contract; it does not own the message semantics.\n\n### Step 4: add generated documentation data\n\nAdd extractors for the missing reference categories one vertical slice at a\ntime.\n\nRecommended implementation order:\n\n1. CLI commands and flags;\n2. runtime configuration/settings;\n3. Messaging commands, events, and message types;\n4. Agent definitions and public commands;\n5. Workflow definitions, WorkflowSteps, lifecycle, and plugin contracts;\n6. policy definitions and typed outcomes;\n7. tools and capability contracts;\n8. worker and external wire contracts.\n\nEach category extractor contributes:\n\n- stable IDs, owners, routes, and display ordering;\n- fields, types, defaults, and validation metadata;\n- source references;\n- version and deprecation information;\n- relationships to commands, events, policies, and contracts.\n\nThe first vertical slice proves the normalized model and deterministic JSON\nserialization. Later categories reuse it rather than introducing\ncategory-specific ad hoc output formats.\n\n### Step 5: render data (pending renderer decision)\n\nBlocks on \"Rendering and publishing\" above. Once a renderer is chosen, it reads\n`data/generated/*.json`, validates it, and renders reference views.\n\nNavigation and routes derive from `catalog.json`; they MUST NOT be duplicated by\nhand in renderer configuration.\n\nNo renderer is chosen, so this step is unstarted.\n\n### Step 6: implement deterministic drift checking\n\n`docsgen check`:\n\n1. generates all artifacts into a temporary directory;\n2. normalizes line endings and file modes;\n3. compares the complete expected data and schema trees with committed outputs;\n4. reports missing, changed, and obsolete files;\n5. identifies the authoritative definition associated with each mismatch;\n6. exits non-zero without modifying the working tree.\n\nGeneration MUST exclude:\n\n- wall-clock timestamps;\n- absolute filesystem paths;\n- map iteration order;\n- environment-specific values;\n- secrets and resolved credentials.\n\nTwo consecutive `docsgen all` runs MUST be byte-identical.\n\n### Step 7: renderer adaptation — blocked\n\nWas \"adapt the VitePress site\". No renderer is chosen; this step is unstarted.\n\nWhen a renderer is selected it MUST:\n\n- read `data/generated/*.json` rather than re-deriving from source;\n- validate internal links, with dead-link failures enabled (no blanket\n ignore list);\n- state its own local preview and build commands;\n- keep `node_modules/` and any build output out of source control.\n\n### Step 8: developer commands\n\n`task docs:generate` writes the committed artifact; `task docs:check` verifies\nit. `task check` runs `docs:check`, so generated drift is gated rather than\nadvisory: a stale artifact fails the gate and names the authoritative Go\ndefinition behind each mismatch.\n\n```sh\ntask docs:generate\ntask docs:check\n```\n\nBoth wrap the generator, which can still be run directly:\n\n```sh\ngo -C tools run -mod=readonly ./docsgen --repo-root ..\ngo -C tools run -mod=readonly ./docsgen check --repo-root ..\n```\n\n`docsgen all` is still target-only: `docsgen data` and `docsgen asyncapi` do not\nexist. Two artifacts are generated today: `contracts.json` from the Go\ncontract types (`docsgen`) and `docs.json` from the Markdown sources\n(`docsite`, the published set a renderer reads) -- see `task docs:artifact`.\n\n### Step 9: CI\n\nThe original documentation workflow was deleted on 2026-09-12. `deploy.yml` still\ndoes not reference `docs/**`.\n\n`.github/workflows/docs.yml` (added 2026-09-18) builds the MkDocs site with\n`mkdocs build --strict` on changes to `docs/**` or `mkdocs.yml`, so a dead\ninternal link fails before it reaches a deploy. It deliberately does no more\nthan that: it does not run `docsgen`, does not regenerate `contracts.json`, and\ndoes not publish. Deployment is a separate path, run from the Ansible control\nnode by the `vectislabs.automation` `docs_site` role.\n\nGenerated-artifact drift is therefore still gated locally, by `task check`\nrunning `docs:check`, and not in CI. That split is the point. A CI job that\nregenerated output would be able to paper over uncommitted differences instead\nof failing on them, which is the failure this section exists to prevent, so any\nfuture documentation workflow MUST check generated output rather than produce\nit.\n\n### Step 10: cutover — done, in reverse\n\nThe site cutover is moot: the site was removed rather than published. The\ngenerator's outputs (`docs/data/generated/`, `docs/public/schemas/`) remain\ncommitted and are the durable artifacts.\n\nStill standing from the original cutover list:\n\n- remove Markdown, examples, specs, generated schemas, and assets that do not\n describe Archie;\n- adapt retained content and prove it against Archie code;\n- update `docs.go` and its tests to embed only intentional runtime\n documentation;\n- decide which current `docs/prds` remain published architecture pages and which\n remain repository-only planning documents — all of them are repository-only\n now, so the distinction is retired.\n\nDocuments MUST describe Archie. Publishing is a separate, undecided concern.\n\n## Hook and CI policy\n\nGenerated documentation is committed.\n\nNo pre-commit hook is installed in this repository, and no CI step runs the\ngenerator. Step 3 below is enforced by `task check`; steps 1, 2, and 4 still\nrely on the author, because nothing observes a commit:\n\n1. detects authoritative definition or generator changes;\n2. runs the generator;\n3. runs `docsgen check` — enforced by `task check`;\n4. refuses the commit when generated changes are unstaged or validation fails.\n\nA commit MUST NOT be created while generated documentation is stale.\n\n## Validation requirements\n\nGenerator tests cover:\n\n- stable ordering;\n- schema reference rewriting;\n- `$defs` flattening;\n- duplicate IDs and routes;\n- registry coverage;\n- comments and descriptions;\n- secret redaction;\n- obsolete-file removal;\n- byte-identical repeated generation;\n- check-mode diagnostics.\n\nRepository-documentation validation covers:\n\n- internal links resolve on the filesystem (repository-relative paths);\n- absence of unrelated product names and URLs;\n- absence of committed build artifacts and installed dependencies.\n\nRenderer validation (frozen dependency installation, production build, expected\noutput routes) is added with the renderer decision and is currently unenforced.\n\nThe root module remains free of generator-only dependencies.\n\n## Completion criteria\n\nThe documentation migration is complete when:\n\n- `tools` belongs to Archie and builds against the working tree;\n- `docsgen` consumes Archie-owned registries and types;\n- AsyncAPI output describes Archie;\n- required reference categories are generated as committed deterministic data;\n- `catalog.json` matches the generated definitions and rendered routes;\n- two consecutive generations are byte-identical;\n- `docsgen check` detects changed, missing, and obsolete artifacts;\n- build artifacts and installed dependencies are absent from source control;\n- `task docs:generate` and `task docs:check` work from a clean checkout;\n- every document describes Archie and every internal link resolves.\n\nRenderer-dependent criteria (rendered reference pages, local authoring server,\ndeployable build, published site) are removed pending the renderer decision.\n\n## Implementation references\n\n- [GitHub-flavoured Markdown specification](https://github.github.com/gfm/) —\n the dialect repository documentation is written in\n- [pnpm](https://pnpm.io/) — required only if a future renderer chooses a\n Node-based toolchain\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/identity/",
|
||
"title": "Identity",
|
||
"section": "Architecture",
|
||
"body": "# Identity\n\n**Status:** Approved foundation; lifecycle details are deferred \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nIdentity answers which persistent actor owns or performed an action. It supplies\nstable cross-domain attribution and lifecycle semantics.\n\nIdentity kinds include:\n\n- the Archie system identity, such as `System` or `archied`;\n- agent or bot identities;\n- service-account identities;\n- future user identities.\n\nA user is another kind of identity rather than a separate foundation concept.\n\n## Ownership\n\n`internal/domain/identity` owns:\n\n- immutable canonical `IdentityID` values;\n- identity kind;\n- lifecycle state;\n- ownership and attribution semantics;\n- identity-related commands and events.\n\nDomains may record an identity reference on their own state and events when\nownership or attribution is required. They MUST NOT receive a complete\nper-identity application configuration bundle.\n\nPersona and communication style are separate from identity.\n\nEvery Agent has its own immutable acting `IdentityID`. Agent actions use that\nidentity for ownership and attribution in the same way an organisation records\nwhich user accessed a resource, changed a permission, or performed another\nauditable action.\n\nAgent and Identity remain separate aggregates. Identity supplies the stable\nactor reference; Agent owns assistant behaviour, memory, specialisation,\nconversations, and capability context.\n\nAgent identities MUST NOT be shared between Agents. Automated WorkflowExecutions\nMAY eventually act through a dedicated service-account identity when a specific\nunattended automation requires it. Such use is an explicit execution decision\nand audit principal, not the default for Agents or a substitute for recording\nthe initiating user when one exists.\n\nA Workflow is owned by an IdentityID and executes as its owner by default.\nFuture delegated execution MAY explicitly assign a different `RunAsIdentityID`,\nincluding a user or service-account identity. Ownership, effective execution\nidentity, initiating identity, and delegation provenance remain distinct audit\nfacts.\n\n## Canonical identity\n\nEvery identity has an immutable `IdentityID`. It is the only value used for\ncross-domain ownership, attribution, routing, persistence, commands, and events.\n\nIdentity kind is explicit:\n\n- system;\n- bot;\n- service account;\n- user.\n\nDisplay names are mutable and MUST NOT be used as identifiers. Forge usernames,\nchannel bot names, Git authors, provider account names, and other external\nidentifiers are bindings associated with an identity rather than identity\nitself.\n\n`System` is represented by a real identity and `IdentityID`. The empty string\nMUST NOT retain special meaning as a legacy system identity.\n\nConfiguration refers to identities by `IdentityID` and supplies external\nbindings and capability settings separately. Renaming an identity or changing an\nexternal account MUST NOT change ownership or historical attribution.\n\nSecret-bearing channel bindings remain configuration concerns rather than\nidentity data. In particular, `[chat.telegram].token` is a\n`secret.SecretRef` resolved by the application through the configured secret\nregistry; the older `token_env` field remains a compatibility fallback. Neither\nform changes the identity that owns or performs a chat action.\n\n## Persistence\n\nIdentity records are durable domain state. The Identity domain's repository is\nthe source of truth; infrastructure implements its persistence contract.\n\nConfiguration MAY bootstrap a missing identity through an explicit, audited\ncreation operation and MAY propose runtime capability bindings for an existing\n`IdentityID`. Configuration is not the identity registry.\n\nOmitting an identity from configuration stops assembling the omitted runtime\ncapabilities. It MUST NOT delete the identity record, erase history, or make\nhistorical references unresolvable.\n\nDisabling or retiring an identity requires an explicit Identity-domain command.\nRetired identities remain resolvable for historical attribution. A failed\nexternal binding affects capability availability without destroying or silently\nreplacing the identity.\n\n## Lifecycle\n\nIdentity lifecycle is deliberately small:\n\n- `active`: may participate in runtime composition;\n- `suspended`: deliberately prevented from initiating new work while ownership\n and history remain;\n- `retired`: terminal, unable to initiate work, and permanently retained for\n attribution.\n\nAllowed transitions are:\n\n```text\ncreate -\u003e active\nactive -\u003e suspended\nsuspended -\u003e active\nactive -\u003e retired\nsuspended -\u003e retired\n```\n\n`retired` has no outgoing transition.\n\nCapability health is separate from identity lifecycle. Forge, model, channel,\nworkspace, and other bindings report health independently. An active identity\nmay be partially degraded without becoming suspended or retired. Runtime\nsupervision isolates and restores affected bindings through the safe-change and\nrecovery protocol.\n\n## System identity\n\nArchie has exactly one built-in system identity. It is created during initial\nstore bootstrap, is strictly always active, and cannot be suspended or retired.\nIts `IdentityID` and kind are immutable.\n\nStartup, migration, supervision, remediation, and rollback actions performed by\nthe application use the system identity. The system identity MUST NOT be\ndelegated to individual identities. Every user, bot, service account, or other\nactor mutation records the identity that actually performed it.\n\nSystem capabilities may independently become unhealthy or unavailable without\nchanging the system identity.\n\n## Target placement and contracts\n\nThe Identity domain lives at `internal/domain/identity`.\n\nIt owns:\n\n- `IdentityID`, identity kind, display name, lifecycle, version, and audit\n timestamps;\n- create, rename, suspend, reactivate, and retire commands;\n- corresponding lifecycle events;\n- the repository contract required for durable identity records;\n- lookup of current and historical identities.\n\nExternal account bindings and capability settings remain with the domains that\nuse them and reference `IdentityID`.\n\n## Migration constraints\n\nMigration from the current string model MUST:\n\n- bootstrap the permanent system identity;\n- assign stable IDs to configured identities without losing their names;\n- replace empty-string legacy ownership with the system identity;\n- migrate tasks and sessions to canonical identity references;\n- include identity where task uniqueness and transport deduplication require\n independent ownership;\n- route container RPC through the task owner's capability bindings;\n- retain historical resolution after rename, suspension, or retirement;\n- remove identity-string equality from authentication or access semantics.\n\nThis completes the foundation required to restructure the current Identity\nbehaviour. Exact storage schemas and Go method signatures belong to\nimplementation design, not this architecture review.\n\n## Lifecycle\n\nIdentity lifecycle is deliberately small:\n\n- `active`: may participate in runtime composition;\n- `suspended`: deliberately prevented from initiating new work while ownership\n and history remain;\n- `retired`: terminal, unable to initiate work, and permanently retained for\n attribution.\n\nAllowed transitions are:\n\n```text\ncreate -\u003e active\nactive -\u003e suspended\nsuspended -\u003e active\nactive -\u003e retired\nsuspended -\u003e retired\n```\n\n`retired` has no outgoing transition.\n\nCapability health is separate from identity lifecycle. Forge, model, channel,\nworkspace, and other bindings report health independently. An active identity\nmay be partially degraded without becoming suspended or retired. Runtime\nsupervision isolates and restores affected bindings through the safe-change and\nrecovery protocol.\n\n## Explicit non-responsibilities\n\nIdentity is not authentication and is not access control.\n\nThe current architecture does not require authentication or authorization. Those\ncapabilities MAY be introduced later if deployment or product needs justify\nthem. They MUST remain separate from the identity model.\n\nAn identity reference therefore proves attribution within Archie's trusted\nsystem model; it does not by itself prove credentials or permission.\n\n## Per-identity configuration\n\nValues selected for an identity remain external configuration values. At\ncomposition time they are split into the runtime settings owned by the affected\ndomains and correlated by identity reference.\n\nThe existing `IdentityConfig` aggregate MUST be dissolved. Forge accounts,\nrepositories, models, providers, workspace behaviour, polling, budgets,\nnotifications, and transport namespaces retain their respective owners.\n\n## Audit\n\nConfiguration changes and other state changes are actions attributed to an\nidentity. System-initiated changes use the system identity. Their audit records\nMUST preserve the acting identity, the target, and the resulting change.\n\nThe command ownership and persistence mechanism for configuration changes is a\nseparate configuration-management decision.\n\n## Current implementation\n\nIdentity is not currently implemented as a domain. It is a collection of string\nfields and process-local associations:\n\n- `config.IdentityConfig.Name` is the daemon routing key.\n- `config.IdentityConfig.BotUser` is the external forge and Git author name.\n- `daemon.IdentityRunner` bundles the configured name with a forge client,\n worktree manager, repository list, configuration subset, logger, and polling\n goroutine.\n- `store.Task.Identity` persists the configured name on tasks.\n- NATS task messages copy the same configured name.\n- gateway routing accepts an identity name and uses it to select a repository\n profile.\n- gateway sessions persist `BotUser`, not the configured identity name.\n\nThere is no identity registry, identity record, identity lifecycle state,\nidentity persistence, or identity command/event model. Removing an identity from\nconfiguration makes existing task ownership unresolvable.\n\nAn empty identity string selects the legacy single-identity path. This gives the\nempty string behavioural meaning rather than representing a real system\nidentity.\n\n## Current enforcement and isolation\n\nThe current implementation uses identity equality as an authorization check for\nchat task approval and cancellation. This conflicts with the approved rule that\nidentity is attribution rather than authentication or access control and must be\nreassigned to an appropriate task-ownership or future access policy.\n\nFocused tests verify that:\n\n- a task resolves the forge client and repository list associated with its\n configured identity name;\n- cross-identity repository profiles do not leak;\n- chat task creation propagates the selected identity;\n- task persistence retains the identity string;\n- chat task control rejects a different identity string.\n\n## Current hazards\n\n- SQLite task uniqueness is `(owner, repo, issue_number)` and excludes identity.\n Two identities discovering the same forge issue cannot own distinct task\n records.\n- NATS deduplication uses `owner/repo/issue` and excludes identity, so discovery\n by a second identity can be discarded as a duplicate.\n- Container-mode forge and worktree RPC servers are wired to the root clients,\n not the task owner's clients. The source documents this as an unresolved\n identity leak.\n- The shared task queue claims the oldest task without identity partitioning and\n relies on later string lookup to recover the correct runner.\n- Sessions use forge bot usernames where tasks use configured identity names, so\n there is no canonical cross-domain identity reference.\n- Identity lifecycle is process lifetime only; there is no disabled, degraded,\n retired, or unavailable state and no audit history.\n\nThe focused identity tests pass. Broader daemon and NATS suites could not open\nembedded listener ports in the current sandbox; their failures occurred before\nidentity behaviour executed.\n\n## Three-layer identity model\n\nAgreed 2026-07-30; the frame for SOUL\n([#68](https://github.com/samcharles93/archie-core/issues/68)) and the curator\n([#435](https://github.com/samcharles93/archie-core/issues/435)).\nRecovered 2026-08-09 from the pre-migration issue tracker.\n\n**Layer 1 — invariants.** Instruction precedence, core rules, and the `\u003ctools\u003e`\nand `\u003cenv\u003e` blocks live in `internal/gateway/templates/archie.md.tpl` and are\nunreachable at runtime by anyone.\n\n**Layer 2 — SOUL.** User-authored identity: agent name, register, warmth.\nEditable without a rebuild.\n\n**Enforcing the Layer 1 / Layer 2 boundary.** SOUL is rendered as the\n`Persona` field of `SystemPromptConfig` (`internal/gateway/prompt.go`) into a\n`\u003csoul purpose=\"identity_and_style\" trust=\"data\"\u003e` block, XML-escaped through\nthe same `xml` template helper used for tool and env metadata, and placed\n*before* `\u003cinstruction_precedence\u003e` in `archie.md.tpl`. Escaping means SOUL\ntext can never forge a live `\u003ccore_rules\u003e`, `\u003cinstruction_precedence\u003e`,\n`\u003ctools\u003e` or `\u003cenv\u003e` tag -- an attempt renders as inert escaped text\n(`\u0026lt;core_rules\u0026gt;...`), and `\u003cinstruction_precedence\u003e` explicitly tells the\nmodel the `\u003csoul\u003e` block is reference data, not an authority. This is\nguarded by `TestBuildSystemPromptSoulCannotOverrideInvariants` in\n`internal/gateway/prompt_test.go`, which renders a hostile SOUL that attempts\nto countermand `core_rules` and asserts the real invariant tags stay singular\nand their text stays intact. Keep this test in sync if the template's escaping\nor block structure changes -- do not relax it to make a future SOUL feature\n(e.g. curator-proposed edits) fit.\n\n**Layer 3 — representation.** Curator-derived knowledge about the user. It may\nonly **propose** changes to SOUL and never apply them. If derived conclusions can\nsilently rewrite tone, personality drifts and the user loses the control SOUL\nexists to give them. A proposal queue also makes the visibility requirement fall\nout for free, since pending conclusions are inherently reviewable.\n\n**Why SOUL comes first:** tone currently lives in Go string constants\n(`DefaultPersonas` in `internal/gateway/persona.go`), so fixing a rude reply on\n2026-07-29 required editing Go and rebuilding. SOUL is small, fixes a real\ndefect, and defines the boundary the curator must respect.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/messaging-and-work-intake/",
|
||
"title": "Messaging and Work Intake",
|
||
"section": "Architecture",
|
||
"body": "# Messaging and Work Intake\n\n**Status:** Current state researched; foundation decisions in progress \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchie is a personal assistant and general agentic system. Its interactions\ninclude conversation, questions, notes, reminders, goals, message handling,\ntroubleshooting, infrastructure operations, server management, source changes,\nand open-ended future capabilities.\n\nA message to an agent is therefore not inherently a work request, and invoking\nan agent is not inherently a WorkflowExecution.\n\nMessaging and work intake are separate from the Agent System's durable workflow\nexecution model:\n\n- messaging preserves interactions with people and external systems;\n- an agent turn interprets and responds to an interaction using its available\n capabilities;\n- work intake handles the optional promotion of an interaction into durable,\n workflow-backed work;\n- the Agent System owns Workflow definitions and creates WorkflowExecutions.\n\nForge issues, Jira issues, Telegram messages, emails, webhooks, and future\nchannels remain peer external message or artifact sources. None becomes an\nexecution model merely because it can lead to work.\n\n## Confirmed boundary\n\nWork intake MUST NOT create a WorkflowExecution directly.\n\nInbound interactions normally route to the Agent selected by the channel\nbinding. The selected Agent retains its own memory, user knowledge, conversation\ncontinuity, specialisation, and capability context. This continuity is\nindependent of the underlying model or provider.\n\nEach selected Agent acts through its own immutable IdentityID. Interaction and\nresulting action records preserve both that acting identity and the initiating\nuser identity when one exists.\n\nAgent memory is private by default. Interaction context MAY additionally use\nuser-wide memory and intentionally global shared memory, but another Agent's\nagent-wide memory is not part of the selected Agent's context.\n\nMultiple users may interact with the same Agent. The channel binding and inbound\nmessage MUST resolve both AgentID and initiating User IdentityID before\nconversation or memory context is loaded. Sharing an Agent MUST NOT implicitly\nshare one user's private memory with another user.\n\nAn interaction may assemble memory from four scopes: global shared, the selected\nAgent's agent-wide memory, the initiating user's authorized user-wide memory,\nand the private Agent-user relationship memory addressed by both IDs.\n\n## Conversation identity\n\nEvery conversation is identified by the complete tuple:\n\n```text\nAgentID\n+ UserIdentityID\n+ ChannelBindingID\n+ ExternalConversationID\n+ ThreadID\n```\n\n`ThreadID` has an explicit empty value for channels without threads. No element\nof this key may be replaced by a display name, configured bot username, channel\ntype, or other mutable label.\n\nAgent and user memory continue across conversations according to their memory\nscope. Immediate conversation history and delivery correlation remain isolated\nto the identified channel conversation or thread. Moving between Telegram,\nemail, or another channel does not create a new Agent or user, but it does\ncreate a distinct conversation unless an explicit linking operation relates\nthem.\n\nConversation and message events support projections for channel-entry metrics,\nuse cases, volumes, outcomes, latency, capability usage, and other product or\noperational analysis. Metrics are derived read models; they MUST NOT become\nmutable counters embedded in the conversation aggregate or change message\nrouting semantics.\n\n## Canonical messages\n\nEvery inbound, assistant, tool-call, and tool-result message has an immutable\napplication-generated MessageID. Platform identifiers are external correlation,\nnot substitutes for the canonical ID.\n\nMessages are addressable because an interaction is more than text. A message may\ncontain assistant content, reasoning data subject to its retention policy, one\nor more tool calls, a tool result with authoritative execution metadata,\nattachments, reply relationships, edits, or other typed content. Stable\nMessageIDs also allow sessions to branch from an exact prior point.\n\nThe canonical session and message implementation must supply:\n\n- application-generated stable message IDs;\n- user, assistant, and tool roles;\n- assistant tool calls and correlated tool-result messages;\n- authoritative tool-result status, timing, error, truncation, and size\n metadata;\n- session lifecycle and active-request state;\n- session-to-agent-instance and parent-session lineage;\n- durable SQLite round trips and timestamp validation.\n\nThese semantics must use the confirmed Agent, user, channel-binding, and\nconversation ownership model.\n\nThe canonical message store MUST preserve messages as immutable records and\nrepresent later edits, deletions, and branches through related records or\nevents.\n\nArchie requires exact branch-point correlation using stable MessageIDs.\n\n## Conversation branches\n\nA branch is a continuation of a previous conversation at an exact message. A\nchild conversation records:\n\n```text\nParentConversationID\nForkMessageID\n```\n\nMessages remain an append-only linear sequence within each conversation. The\nchild's effective initial context is the parent's history through\n`ForkMessageID`; messages after that point in the parent are not inherited.\n\nAfter the fork, parent and child are isolated:\n\n- child messages, tool calls, tool results, capability actions, and errors do\n not appear in the parent conversation;\n- later parent messages do not silently enter the child;\n- returning to the parent resumes its unchanged history;\n- the child may itself be branched using the same rules;\n- branch lineage remains queryable for navigation, audit, and metrics.\n\nThis supports building substantial context about a subject and then performing\none or more contextual subtasks without polluting the original conversation. For\nexample, a user may branch from a project discussion, create Jira issues in the\nchild using inherited context, and then return to the project discussion without\nadding the Jira tool transcript to the parent.\n\nBranching does not copy mutable message rows. History is resolved through the\nimmutable parent conversation and fork MessageID, followed by the child's own\nmessage sequence.\n\nBranch isolation applies to conversation history, not to Agent learning or\nmemory. A memory action performed in a branch has exactly the same semantics as\nthe same action performed in any other conversation. It writes to its selected\nglobal, agent-wide, user-wide, or Agent-user relationship scope.\n\nMemory provenance MAY retain the source ConversationID and MessageIDs for\nexplanation and audit. Retrieval and effect are determined by memory scope and\naccess rules, not by the conversation from which the memory originated.\nReturning to a parent does not insert the child's tool transcript into the\nparent history, but scoped memories learned in the child remain available to\nfuture retrieval wherever that scope is visible.\n\nChanging sessions or branching therefore controls immediate conversational\ncontext without preventing the Agent from learning durable knowledge.\n\n## Target ownership\n\nThe Messaging domain lives at `internal/domain/messaging`. It owns:\n\n- canonical Messages and typed message content;\n- Conversations, branches, and lineage;\n- channel bindings and external source correlation;\n- replies, acknowledgements, and outbound delivery attempts;\n- message and conversation lifecycle events;\n- domain-defined repository and delivery service contracts;\n- projections required for conversation navigation and metrics.\n\nWork intake is a separate cohesive domain at `internal/domain/workintake`. It\nowns admission, validation, deterministic routing, and accepted work requests\nfor the optional transition into durable workflow-backed execution.\n\nTelegram, email, webhook, forge, Jira, and future messaging systems are\ninfrastructure adapters implementing Messaging contracts. They are users of the\ndomain; they do not own Messages, Conversations, work admission, or\nWorkflowExecution semantics.\n\nThe Agent decides whether an ordinary interaction is answered directly, uses a\ncapability, changes or retrieves Agent-owned information, or proposes durable\nworkflow-backed work.\n\nExplicitly configured automation MAY enter work intake without a conversational\nAgent turn when the external trigger and admission rule already unambiguously\nrequest durable work. This avoids spending an LLM turn to rediscover a\ndeterministic routing decision.\n\nWhen an interaction requires durable workflow-backed execution:\n\n1. a channel adapter produces a channel-neutral inbound message;\n2. the acting agent handles or interprets the interaction;\n3. work intake validates, admits, and routes a proposed work request;\n4. work intake emits an accepted work request with its source correlation,\n acting identity, requested outcome, inputs, and routing evidence;\n5. the Agent System consumes that request, selects a specific Workflow version,\n and creates the WorkflowExecution.\n\nThe accepted work request is a cross-domain handoff, not an alternate execution\naggregate. The Agent System is the sole authority that creates a valid\nWorkflowExecution.\n\nAn interaction MAY instead be completed directly by an agent turn, delegated to\na tool or capability, recorded as a note or reminder, answered conversationally,\nor retained for later context. Those paths MUST NOT require a Workflow or\nWorkflowExecution.\n\nA Workflow MAY add reusable, versioned structure to any of those behaviours.\nScheduled personal-assistant routines, message triage, reminders, infrastructure\noperations, and other capability sequences are valid Workflow definitions. The\ndistinction is whether durable structured execution is useful, not whether the\nsubject originated from a forge.\n\n## Canonical execution language\n\n- **Workflow**: reusable, versioned behaviour definition.\n- **WorkflowExecution**: durable execution of a specific Workflow version.\n- **WorkflowStep**: a defined operation in a Workflow.\n- **StepExecution**: the recorded execution and outcome of a WorkflowStep.\n- **Attempt**: a retry within the same WorkflowExecution.\n\nExternal issues, messages, requests, goals, and notes are not synonyms for these\nconcepts.\n\n## Current implementation\n\nCurrent intake is fragmented:\n\n- Telegram, email, and webhook adapters translate payloads into\n `gateway.Message`, while an unused richer `MessageEvent` contract exists\n alongside it.\n- `gateway.Router` combines local channel commands, conversational LLM routing,\n and direct task lifecycle operations.\n- Telegram is the only channel with production LLM and durable conversation\n wiring. Email and webhook free text currently have no LLM responder.\n- Forge polling bypasses the gateway and directly enqueues store rows or NATS\n task messages.\n- Chat `/spawn` directly creates a `store.Task` and invents a synthetic forge\n issue number to satisfy the existing persistence key.\n- Forge replies and chat approval commands implement separate waiting-human\n paths.\n- Message delivery, acknowledgement, retry, deduplication, and source\n correlation are platform-specific side effects rather than one durable model.\n- `store.Task` collapses an external source reference and durable execution\n state.\n\n## Current hazards\n\n- Treating every agent interaction as a work request would remove Archie's\n personal-assistant role and force conversation and immediate capability use\n through an inappropriate workflow lifecycle.\n- Treating every accepted work request as a WorkflowExecution would let intake\n or channel code select versions and construct invalid execution state outside\n the Agent System.\n- Source identity is incomplete: sessions use configured bot usernames, tasks\n use identity strings, and external message IDs are generally absent.\n- SQLite and NATS deduplication are based on forge coordinates rather than a\n channel-neutral source-message reference.\n- Replies and acknowledgements have no durable delivery-attempt lifecycle, so\n partial success and retry behaviour differ by adapter.\n- Production code has two competing channel-neutral message contracts.\n- The existing Archie session implementation lacks the required tool-call\n semantics and exact branch lineage and MUST be brought to the canonical\n contract.\n\n## Decisions still required\n\n- the cohesive owner and final location of messaging behaviour;\n- the cohesive owner and final location of work-intake admission and routing;\n- the canonical inbound message, conversation, and source-correlation model;\n- the boundary between an agent turn, direct capability use, and a proposed\n durable work request;\n- the exact relationship between an Agent, its acting IdentityID, its user, and\n its channel bindings;\n- outbound response, acknowledgement, delivery-attempt, retry, deduplication,\n and failure semantics;\n- the smallest accepted-work-request contract consumed by the Agent System;\n- migration of forge polling, replies, chat commands, email, webhook, Telegram,\n and Jira integrations.\n\n## Non-goal: semantic search over chat messages\n\nSemantic/vector search over chat messages is an **explicit non-goal** until usage\ndata shows a real need.\n\nMessage search is lexical only: session-scoped SQLite FTS5 queries paged via\n`MessageQuery{Query,Limit,Offset}` → `MessagePage`. Do not add embeddings\nconfiguration or vector columns to message records.\n\nThe rationale is cost without a demonstrated use case. Measured 2026-08-03 with\n`embeddinggemma:300m` — note this was never an infrastructure problem, the model\nwas already pulled and `ai-sdk/provider/ollama` already has `embed.go`:\n\n- 121 ms median warm per message on the write path, 12.5 s cold;\n- 768 dims = 3072 bytes per vector, roughly 15× a typical message payload;\n- chronological reads would carry vector data they do not use on the hot path\n of every turn;\n- similarity search requires an explicit indexing and retention design.\n\nIf this is revisited, vectors belong in a sibling table so chronological reads\nnever load them.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/migration-decisions/",
|
||
"title": "Architecture Migration Decisions",
|
||
"section": "Architecture",
|
||
"body": "# Architecture Migration Decisions\n\n**Status:** Active migration inventory \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nThis document records what remains to be decided before Archie's current\nimplementation can be migrated into the approved domain-oriented structure.\n\nThe product and core domain model are not being redesigned. Existing Archie\nbehaviour and the existing session implementation are the migration\nbaseline. Remaining decisions MUST be grounded in current packages, consumers,\ntests, persistence, and runtime wiring.\n\n## Confirmed domain packages\n\n```text\ninternal/domain/\n identity/ (approved, not yet materialised as a package)\n agent/ (approved, not yet materialised as a package)\n workflow/\n messaging/ (canonical Message/Conversation types only; gateway migration pending, see section 2)\n workintake/\n plugin/ (approved, not yet materialised as a package)\n curator/\n mapping/\n scheduling/\n```\n\n- `identity` owns persistent actors, lifecycle, ownership, attribution, and\n audit identity.\n- `agent` owns persistent assistants, specialisation, instructions,\n model-independent continuity, capability context, and associations with\n identity and memory.\n- `workflow` is a separate domain. It owns reusable, versioned Workflow\n definitions, WorkflowSteps, WorkflowExecutions, StepExecutions, Attempts,\n outcomes, and typed Workflow plugin contracts.\n- `messaging` owns canonical Messages, Conversations, branches, channel and\n source correlation, replies, acknowledgements, delivery attempts, events, and\n projections.\n- `workintake` owns admission, validation, deterministic routing, and accepted\n work requests for the optional transition into durable workflow-backed\n execution.\n- `plugin` owns generic plugin identity, discovery, compatibility, and lifecycle\n mechanics. It remains metadata-only; capability semantics and typed\n registration belong to the owning domain.\n- `curator` owns the curator engine family: long-running agent loops that\n reason over memory and perform maintenance tasks on a declared shape\n (check-in interval, tool set, attached memory engine). See epic\n archie-core-yp9.\n- `mapping` owns the payload field-mapping vocabulary: binding named fields to\n JSON paths inside a captured webhook payload, and detecting drift. See\n `docs/prds/payload-field-mapping.md`.\n- `scheduling` owns the ticker engine: deciding when a scheduled job runs and\n with what concurrency. It declares `JobSource`/`Runner` contracts and owns\n no storage or delivery. See epic archie-core-1786637496320-249-be6d9a20.\n\n**Reconciliation (2026-08-31):** `curator`, `mapping`, and `scheduling` existed\nin the tree without being added to this list when their epics landed. They are\nconfirmed as legitimate, narrowly-scoped domains under the same rules as the\noriginal six — each owns one capability family, declares typed contracts, and\nhas no cross-domain reach. This list was out of date, not the tree.\n\nConversely, `identity`, `agent`, `messaging`, and `plugin` are approved target\ndomains that do not yet exist as packages. Their behaviour currently lives in\npre-migration packages (see \"Complete package destination map\" below for the\nfull inventory). Do **not** create empty stub packages for them ahead of an\nactual migration PR — an unused stub is dead code and gives no test coverage.\nEach materialises when its corresponding migration decision below is resolved\nand implemented in the same change.\n\nMemory's final package placement remains undecided pending focused review of the\nexisting implementation.\n\nThe following are approved shared or composition packages, not domains:\n\n```text\ninternal/eventbus/\ninternal/policy/\ninternal/infrastructure/\ninternal/app/archied/\ninternal/app/agentworker/\n```\n\n## Remaining migration decisions\n\n### 1. Complete package destination map\n\nEvery current package MUST receive one explicit disposition:\n\n- move into a confirmed domain;\n- move under infrastructure as an implementation of a domain-owned contract;\n- move into application composition;\n- merge with an identified duplicate;\n- remain as an approved shared application contract;\n- move to a plugin, extra, deployment, example, or generated artifact;\n- be deleted after its replacement is proven.\n\nThe largest unreviewed areas include memory, tools and capabilities, workspace\nand worktree behaviour, gates, storage, events, web UI, containers, RPC\npackages, release management, and scheduling.\n\nThe destination matrix MUST name the behaviour owner, target path, dependencies\nto remove, state and contracts to preserve, migration prerequisites, and\ndeletion criteria.\n\n**`internal/webui` / UI Service — RATIFIED Phase 3 disposition (2026-09-07).**\nThe current owner is the in-process HTTP/SSE dashboard: SPA hosting, operator\nauthentication, UI response projection, configuration reads and mutations,\ntask/control APIs, and direct adapters to State Store, channel, event/log,\nmemory, curator, workflow, forge, and Gateway runtime values. The target owner\nis a standalone UI application process; `internal/webui` remains its HTTP\ntransport adapter, while Gateway, State Store, configuration, Work Intake,\nMessaging, and other capability owners expose the contracts behind those\nroutes. The migration preserves retained route schemas, auth/CSRF behavior,\ntask/capture/mapping/binding/session continuity, SSE replay semantics, and\ncommitted generated SPA assets. Prerequisites are the ratified UI boundary,\nroute-owner inventory, narrow remote adapters, configuration/auth/readiness\ncontracts, and process/conformance tests. Remove the shared live\n`config.Holder`, direct event/log bus, daemon stopper, forge client, channel\nmanager, capability registries, composite store access, and in-process UI\nlistener only after the focused criteria in\n[`docs/prds/ui-service-boundary.md`](/docs/prds/ui-service-boundary/) pass in\nproduction composition. The old construction and listener are deleted in the\nsame cutover change; no second UI authority remains.\n\n### 2. Session and Messaging migration\n\nThe existing session implementation is the migration baseline. The migration must\ndecide:\n\n- which session and message components are retained or adapted;\n- how Agent, user, channel, Conversation, and branch ownership are added;\n- how Archie's parallel gateway session implementation is replaced;\n- how canonical Messages become immutable persisted records;\n- how existing sessions and messages are migrated;\n- how outbound delivery, retry, acknowledgement, deduplication, and failure\n semantics are represented;\n- how current Telegram, email, webhook, forge, and Jira integrations migrate to\n Messaging infrastructure adapters.\n\nThe following are already decided and MUST NOT be reopened without contradictory\nimplementation evidence:\n\n- the composite Conversation identity;\n- immutable MessageIDs;\n- typed tool-call and tool-result messages;\n- exact branch lineage through ParentConversationID and ForkMessageID;\n- parent and child transcript isolation;\n- normal scoped-memory behaviour regardless of the originating Conversation;\n- Messaging and Work Intake ownership.\n\n**Decision (2026-08-31), resolving the open questions above:**\n\n`internal/gateway/` is the \"parallel gateway session implementation\" this\nsection names. It is retained as-is for now — a big-bang rewrite of live\nsession/compression/approval/branch logic across Telegram, web UI, email, and\nwebhook channels in one change is exactly the \"convoluted to satisfy edge\ncases\" failure mode CLAUDE.md's Revert-on-Flail rule warns about, and would\nrisk silent behavioural regressions in the one path every user-facing message\ntravels through.\n\nInstead, `internal/domain/messaging/` is materialised now with the pieces\nthat are already decided and don't require touching gateway: the canonical\n`Message` and `Conversation` types (composite Conversation identity, immutable\n`MessageID`, typed tool-call/tool-result message variants, branch lineage via\n`ParentConversationID`/`ForkMessageID`). These are pure domain types with no\ngateway dependency — a new package, not a rewrite of an existing one.\n\nMigrating `internal/gateway/`'s session, compression, approval, and branch\nlogic onto these types is real, cross-channel-risk work and is **not** done in\nthis pass. It is tracked as separate, appropriately-scoped bd issues (see\n`bd list --label messaging-migration`), sequenced channel-by-channel so each\nlands with its own behavioural verification instead of one large diff no one\ncan safely review.\n\n**Amended 2026-09-15.** `internal/infrastructure/gatewayrpc.StoreClient` is the\nadapter this migration will need: it implements `messaging.SessionStore` over\nthe chat service's session-store RPCs and is proven by\n`TestStoreClientPreservesCanonicalRecords`. It is deliberately unwired.\nProduction `archied` dials the standalone gateway for `ChatContract` only\n(`composeChatContract` → `gatewayrpc.Dial` returns a `*Client`, not a\n`*StoreClient`), and the daemon's session store is in-process SQLite via\n`makeTelegramSessionStore`. The test comment that says it mirrors production\ndescribes the target, not today's tree. Kept rather than deleted because the\nmigration it serves is deferred by this very section, and re-deriving a tested\nwire adapter once that lands is pure cost.\n\n**Phase 4 Messaging Service extraction (2026-09-19, `archie-core-8cda.6.1`).**\nThe process boundary for external channels (Telegram, email, webhook) is ratified\nin [`docs/prds/messaging-service-boundary.md`](/docs/prds/messaging-service-boundary/).\nMessaging is extracted into `cmd/archie-messaging` / `internal/app/archiemessaging`,\ncalling Gateway's `messaging.ChatContract` exclusively over gRPC. Channel frontends\nhold no direct database handles, daemon state, or model runtimes.\n\n**Telegram operator surface after extraction (2026-09-19, `archie-core-8cda.6.6`).**\n`setupTelegramGateway` wired eight `telegram.Gateway` seams the daemon could\nsatisfy from its own process. Each is resolved by where the fact it needs is\nactually owned, not by where it used to be built:\n\n| Seam | Resolution |\n|---|---|\n| `Version` | Sourced from the Gateway via `ChatSnapshot.Version`. The Messaging Service never stamps or infers a build version; `cmd/archie-messaging` deliberately takes no version ldflags, so it cannot report a partial upgrade as a matched one. |\n| `Updates` (`/update`) | Built in the Messaging Service from `[chat.telegram].update_check_command` / `update_install_command` — its own configuration. `Enrich` is dropped: `componentInstallTypeEnricher` reads `[nats]`, which this service must not decode, so the check command's own reported install type stands unmodified. |\n| `UpdateReportPath` | A local state file under `work_dir`, keyed by the same `sha256(bot_user)[:8]` scheme the daemon used, so a pending report is found rather than written afresh. |\n| `ReleaseAnnouncements` | **Deliberately left nil**, for the same reason as `RunningVersions`. `releaseannounce.Announcer` announces nothing unless each `Component` carries a parseable version, and the only versions this service can obtain are the Gateway's own build stamps, not archied's. Wiring the announcer without them yields a no-op that reads as configured, so it stays unwired until component self-reporting exists. |\n| `SetShowToolCalls` | Projected from `[chat].show_tool_calls`. |\n| `Reload` | Local: re-resolves the token and allowlist from this service's own config file and overlay. Reload was never a daemon fact. |\n| `RunningVersions` | **Deliberately left nil.** It exists to turn an installer's claim into a checked one, and only a component's own compiled-in build can vouch for it. The Messaging Service knows neither archied's build nor the observed archie-agent version (`daemon.AgentStatus`), and the Gateway's own `gatewayVersion` is a different binary's stamp — reporting it as the daemon's would manufacture exactly the false success the check exists to catch. Update reports therefore relay as unverified claims until a component self-report contract exists. |\n| `Dangerous` | Stays nil. No daemon composition ever set it; `/rollback` and `/stop` reported \"not configured\" before the extraction and still do. |\n\nThe Messaging Service consequently reads `work_dir`, `bot_user`, `[chat]` and\n`[health]` in addition to the PRD's `[chat.*]` / `[services.gateway]` list.\n`[health]` is the daemon's health URL, which `releaseupdate.CommandInstaller`\npolls from outside to confirm an update came back up; the rest of the PRD's\nprohibition list is untouched.\n\n**Phase 4 messaging severance: what must move, and the decision the PRD does\nnot cover (2026-09-22, `archie-core-1ng1`).** The PRD's deletion gate cannot\npass today. `cmd/archie-messaging/architecture_test.go` does not exist, and the\ncheck that section specifies -- `go list -deps ./cmd/archie-messaging` links\nzero banned runtime packages -- reports four:\n\n| banned package | PRD category |\n| --- | --- |\n| `modernc.org/sqlite` | State Store |\n| `internal/store` | State Store |\n| `internal/domain/workflow` | Workflow engine |\n| `internal/agentexec` | Workflow engine, via `internal/domain/workflow/agent.go` |\n\nAll four arrive through one import: `internal/app/archiemessaging/run.go`\nimports `internal/app/controlplane`, whose own dependency set is exactly those\nfour. The Messaging Service uses three symbols from that package\n(`NewRPCClient`, `RuntimeChatConfig`, `ChannelSettingsKind`); the package also\ncarries the store-backed `ResourceStore` server, the `Definition` registry and\nthe workflow step vocabulary. Sever the package. Amending the gate to tolerate\n`modernc.org/sqlite` and `internal/store` is rejected: a boundary you whitelist\naround is no longer a boundary.\n\n**What moves.**\n\n- Every kind constant. They are strings with no dependency, and the client names\n them.\n- The channel-settings vocabulary and its decode: `channelSettings`,\n `telegramSettings`, `webhookChannelSettings`, `emailSettings`,\n `rateLimitSettings`, `channelDuration`, and `decodeRenamingLegacyKeys`.\n- The generic client: `Client`, `NewRPCClient`, `RuntimeConfig` with\n `runtimeConfigFrom`, `runtimeChatConfigFrom`, the other `runtime*From`\n projections, and the `resourceReader` interface.\n- The persona watch (`WatchPersonas`, `AppliedPersonas`), which needs only\n `internal/domain/agent`.\n- `executionSettingsDocument`, the document alone.\n\n**What stays, and why.**\n\n- The store-backed server: `Server`, `NewServer`, `ResourceStore`,\n `ImportConfig`, `SeedSkip`, the `Definition` registry, and every other kind's\n seed, validator and `Normalize` hook.\n- `WorkflowDefinitionsClient` and `stepRegistry`, which decode through\n `internal/domain/workflow`.\n- `WatchWorkflowExecutionSettings` and `decodeSettings`: `decodeSettings`\n returns and validates `workflow.ExecutionSettings`, so any package holding it\n links the workflow domain and therefore `internal/agentexec`. This is why the\n client cannot move as one package -- see open question 2.\n- The schedules document, whose type is `cronstore.JobSpec` in\n `internal/infrastructure`. A domain vocabulary package may not import\n infrastructure.\n- The schema derivation, which describes the documents the server advertises.\n\n**Placement as landed (2026-09-22, `archie-core-1ng1`).** One new package,\n`internal/infrastructure/controlplanerpc`, holding both the vocabulary (kinds, the\nchannel-settings document and its decode) and the client\n(`NewRPCClient`, `Query`, `RuntimeChatConfig`, the exported `ResourceReader`, the\nwire sentinels and `ClientError`). A second `internal/domain/controlplane` package\nfor the vocabulary was planned and then measured out of the design: the vocabulary\nhas exactly one consumer, the client that projects it, and the store-backed side\nreads the same definitions through type aliases rather than a copy -- so a separate\ndomain package would have been a package with one consumer and a second name for\none thing. What made that safe is a measurement, not a preference:\n`internal/app/controlplane` was the ONLY direct import of the Messaging Service\nthat dragged a banned runtime (`gatewayrpc`, `staterpc`, `applystatus`, `messaging`,\n`configuration`, `readiness` and `health` were all clean), so severing that one\ndependency is sufficient, and the severance needed no third package. Promoting the\nvocabulary to a domain package stays open for the day a second consumer appears.\n\n**Settled by the implementation.** Two of the four open questions this decision\noriginally left are answered, in both cases by measuring rather than by preference:\n\n- The client did NOT become two packages. `WatchWorkflowExecutionSettings` and\n `WatchPersonas` stayed on `internal/app/controlplane`'s `Client`, which now\n *embeds* the new one, so the daemon's validated watch is still defined once and\n the read path still has one implementation -- which is what the second package\n was for -- with one fewer package to keep in step.\n- The schedules and workflow documents stayed put, and\n `internal/domain/workflow/task` is linked deliberately: it carries the Task\n status and view vocabulary, its own dependency set holds none of the banned\n runtimes, and `cmd/archie-messaging/architecture_test.go` carries it as an\n exception checked in both directions (the same entry, for the same reason, as\n `cmd/archie-ui/architecture_test.go`).\n\n**Still open.**\n\n1. Does the server also leave `internal/app/`? `organisation.md` reserves\n `internal/app/` for wiring and composition, and a store-backed gRPC server is\n infrastructure -- but the gate does not require the move, and it would multiply\n the diff across archied's wiring and the State Store binary. Tracked as\n `archie-core-8f4n`.\n2. Does `internal/domain/workflow`'s contract/engine split happen, and when? A\n contract type (`WorkflowDefinitionCollection`) drags `internal/agentexec` only\n because `agent.go` sits beside it. Independent of this severance, tracked as\n `archie-core-inh6`, and the shape `archie-core-8cda.7` will meet again.\n3. Does the vocabulary move to a domain package when a second consumer appears?\n\n**Both consumers keep compiling.** archied imports the new vocabulary for the\nkinds it names and the new client package for `RuntimeConfig` and the persona\nwatch, and keeps `internal/app/controlplane` for the server, the registry,\n`ErrVersionConflict` and the workflow-aware pieces. The State Store binary\ncontinues to call `NewServer` and `ResourceStore` unchanged. The Messaging\nService then imports only `internal/infrastructure/controlplanerpc`,\n`gatewayrpc` and `staterpc`, so `go list -deps ./cmd/archie-messaging` loses all\nfour banned packages and the gate can land green.\n\n**Channel state to the dashboard: one administrative surface, following the\nconfig-snapshot precedent (2026-09-22, `archie-core-8cda.6.8`).** The Messaging\nService now records its channels' lifecycle -- `internal/channels/status.Manager`,\none entry per composed channel, written by each channel's own report through\n`channels.Lifecycle` -- and nothing reads that across a process boundary.\n`/api/channels` answers with nothing because `webui.Server.Channels` has no\nproduction source, and POST `/api/channels/{id}/reload` answers 501 for the same\nreason. The carrier is the State Store, and the shape is the one\n`archie-core-ymut` already established for config snapshots\n(`docs/prds/state-store-contract.md`, \"the one surface whose writer is the daemon\nand whose reader is the UI process\"): a pair of administrative RPCs, deny-by-default\nunder `authorizesTaskScopedCall`, so a task-scoped grant reaches neither.\n\nWhat that decides, so the implementation need not invent it:\n\n- A channel's state is **runtime** state, not a settings document. It does not\n belong in the `ControlPlaneService` resource kinds, which are validated,\n versioned documents seeded from config; a channel that is failed is not a\n document anybody edits.\n- The reader is the UI process, which already dials the State Store and pumps its\n cursor, so the surface needs no new transport in either direction.\n- Adding the pair changes the contract's RPC count, so\n `docs/prds/state-store-contract.md`'s grouped table gains a group in the same\n change that adds the RPCs -- the table is the contract's own index, and a\n surface it does not list is a surface nobody finds.\n\n**Reload gets its own surface, with its own lifetime (decided 2026-09-22).** Reload\nruns the other way -- the UI asks the service to reload a channel -- and the two\ncandidates were a request surface the service observes and clears, or a field on\nthe status surface the service watches. It is the first. The reasoning, recorded\nbecause it would look arbitrary later:\n\n- A request queue and a status report do not share a lifetime, so they should not\n share a surface. A pending request is transient and has an outcome; a channel's\n state is durable and has a current value.\n- A \"please reload\" field on the status resource makes READING status capable of\n RETRIGGERING an action, and makes a transient request indistinguishable from\n state the moment anything polls it.\n- It would also invert the writer/reader relationship the precedent establishes:\n the service would have to clear a field on an object whose whole contract is\n that the service reports it.\n\nSo the status surface stays writer-is-service, and the request carrier is its own\nobject with its own states.\n\n`archie-core-8cda.6.9`'s component self-report is the **same shape**: a process\nreporting something about itself for the UI to read. Whether it shares this\nsurface or gets its own is a decision for `6.9`, and the one outcome to avoid is\ntwo homes for \"what a component reports about itself\".\n\n### 3. Identity data migration\n\nThe migration must define:\n\n- the durable identity schema and repository implementation;\n- bootstrap of the permanent system identity;\n- stable IdentityIDs for existing configured Agents;\n- migration from configured identity names, empty-string ownership, and bot\n usernames;\n- preservation of historical attribution;\n- identity-aware uniqueness and transport deduplication;\n- correction of container and RPC capability resolution so execution uses the\n owning identity's bindings;\n- removal of identity-string equality from access-control behaviour.\n\nExact schema and method signatures are implementation decisions. The approved\nIdentity lifecycle and attribution semantics are fixed migration constraints.\n\n### 4. Workflow migration\n\nThe current implementation must be transformed deliberately:\n\n```text\nstore.Task -\u003e WorkflowExecution\nmutable Stage -\u003e StepExecution history\nRetryCount / Attempt -\u003e Attempt records\nworkflow.Registry -\u003e versioned Workflow definitions\ncurrent stages -\u003e WorkflowStep definitions or Workflow plugins\n```\n\nThe migration must decide:\n\n- Workflow definition and version persistence;\n- the smallest accepted-work-request contract consumed from Work Intake;\n- the enforced WorkflowExecution state machine;\n- atomic state and domain-event persistence;\n- migration of existing task rows and transition history;\n- disposition of current skill, Yaegi, and stage-plugin implementations;\n- the versioned worker-handoff contract;\n- scheduler, claim, retry, crash-recovery, and waiting-human cutover.\n\nWorkflow remains a separate domain from Agent. A user or Agent may define or\ninvoke a Workflow, but Workflow semantics and plugins remain Workflow-owned.\n\n### 5. Memory placement and storage\n\nThe four required scopes are fixed:\n\n- global shared;\n- Agent-wide;\n- user-wide;\n- Agent-user relationship.\n\nA focused review of `internal/memory`, its providers, consumers, persistence,\nand tests had these decisions:\n\n- `internal/domain/memory` versus ownership within the Agent domain — settled\n (2026-09-02, below);\n- the authoritative memory record and revision model — settled by the PRD\n (2026-09-16, below);\n- provider and infrastructure boundaries — settled by the PRD (2026-09-16,\n below);\n- retrieval and access enforcement — settled by the PRD (2026-09-16, below);\n ranked retrieval remains open (below);\n- migration of existing memory data — open: existing data is not migrated;\n- provenance representation — settled by the PRD (2026-09-16, below).\n\nConversation branches do not create another memory scope. A memory action has\nthe same scoped effect regardless of which Conversation originated it.\n\n**`internal/domain/memory` ownership — DECIDED (2026-09-02).** The typed\nengine family lives at `internal/domain/memory`\n(archie-core-1786637490069-11-e245c170), mirroring `internal/domain/curator`'s\ncontract/registrar/registry split. Not owned within the Agent domain: memory\nis consumed by more than agent turns (curators, per archie-core-1786637499636)\nand the plugin engine rule requires its own family, not a capability bolted\nonto another domain's contract.\n\n**Authoritative record and revision model for the builtin engine — DECIDED\n(2026-09-02),** scoped to unblock archie-core-1786637499480 (ship the\nfile-writing store as the default engine). This decides only the builtin\nengine's mapping onto the new `Record{ID}`/`Forget(id)` contract, not the\nmemory model in general or a future non-file engine's model:\n\n- `internal/memory/builtin.Store` is unchanged. It has no concept of a\n stable per-block ID — blocks are addressed by content substring — and nothing\n in this decision requires changing that; existing `internal/memory` tests\n keep passing unmodified, per that issue's acceptance criteria.\n- A new infrastructure adapter (`internal/infrastructure/memory`, per\n `organisation.md`'s \"infrastructure implements domain contracts\" rule — not\n under `internal/domain/memory`, which stays contract-only) wraps one\n `builtin.Store` per identity.\n- **Identity → directory:** `hex(sha256(identity))`, not the identity string\n itself. Closes the path-traversal question without an allowlist regex, and\n sidesteps collisions from identities that differ only in characters a\n filesystem treats as equivalent.\n- **Record ID:** assigned once at `Write`, independent of content, so a later\n edit elsewhere in the file can never orphan a `Forget`. Realized as a\n leading marker line (`\u003c!--mem:\u003culid\u003e--\u003e`) on the block `builtin.Store.Add`\n writes; `Query`/`List` strip it back out of `Record.Content` before\n returning, and `Forget(id)` locates the block by that unique marker via the\n store's existing substring-based `Remove`. The marker is a pragmatic,\n reversible choice — plain-text records stay human-readable and\n human-editable, at the cost of one marker line an operator editing the file\n by hand should leave alone. Revisit if/when provenance representation\n (still open, below) settles on something richer.\n- **Kind → section:** `Observation.Kind` (default `\"general\"` when empty) is\n the section name, reusing `builtin.ValidateSectionName` — so an\n invalid `Kind` fails `Write` with the same validation error `memory_edit`\n already returns for an invalid section today, not a new error shape.\n\n**Memory engine: scopes, CRUD with retained revisions, and store placement —\nDECIDED (2026-09-16).** Supersedes the 2026-09-02 \"Authoritative record and\nrevision model for the builtin engine\" entry above, whose `\u003c!--mem:\u003culid\u003e--\u003e`\nlive marker cannot carry provenance. Authority:\n[`docs/prds/memory-engine-unification.md`](/docs/prds/memory-engine-unification/)\n(Approved); the contract is `internal/domain/memory/contract.go`. This register\nsupersedes; it does not erase.\n\n- **Four typed scopes, caller-named.** `internal/domain/memory` is addressed by\n `Scope{Kind, Agent, User}` over `global`, `agent`, `user` and `agent-user`,\n with `Validate` rejecting a component present where it is not required. A\n caller names the read and write sets (`Subject.Scopes()` /\n `WritableScopes()`; global is never model-writable) and the engine applies no\n policy of its own: **access control is naming a scope, and nothing else.** An\n engine that enforced policy would need to know about channels, sessions and\n bindings, and every backend would reimplement it. `Scope.Key()` length-prefixes\n each component, so two scopes can never share a key even when an opaque\n channel-native id embeds the separator.\n- **The caller resolves identity; the engine never does.** Resolution happens in\n the gateway turn path, where the native sender id and the session are both in\n hand, and is supplied per channel by the composition. A webhook's sender id is\n a route path, not a person, and treating it as one would give a URL an\n identity. An absent user id yields global and agent scopes only: \"show none of\n it\", never a wider fallback.\n- **CRUD supersedes, never overwrites.** `Store` is `Create`/`Get`/`Query`/\n `List`/`Update`/`Forget`/`Revisions` over the four scopes. `Update` appends the\n superseded state to the scope's `HISTORY.md` before replacing the live block;\n `Update.Expected` names the revision being replaced and returns\n `ErrStaleRevision` on mismatch, making a lost update between two processes\n sharing one scope file detectable. `Forget` records a deleted revision before\n removing the live block, so a record's provenance outlives its content; `Get`\n and `Forget` take a scope, because an id alone does not locate one.\n- **Provenance representation.** The live marker becomes JSON, since a\n `\u003c!--mem:\u003culid\u003e--\u003e` line cannot carry provenance, and `Record` retains the\n provenance `agent-system.md` requires — scope, author, originating user,\n source, and revision history. `Metadata` is dropped: it never round-tripped.\n- **Store and scanner placement.** The markdown store relocates to\n `internal/infrastructure/memory/builtin`, its correct home under\n `organisation.md`'s \"a package owns its on-disk format end-to-end\". The content\n scanner moves into `internal/domain/memory` and is applied in `Create` and\n `Update` — the single choke point every producer crosses, so curator-extracted\n content is covered too.\n\n**Open in this section.** Ranked retrieval is deferred: `Query.Text` stays in\nthe contract, unused by the prompt path. Existing memory data is not migrated:\n`\u003cworkDir\u003e/memory` and `\u003cworkDir\u003e/memory-engine` are left on disk, unreferenced.\nThe PRD leaves these undetermined and they remain open here: group-chat\nextraction, where the curator fails closed on mixed-sender sessions; two\nprocesses (`archied` and `archie-gateway`) holding one scope file, where\n`Update.Expected` detects but does not prevent a lost block; unbounded\n`HISTORY.md` with no compaction policy; whether the per-scope size bound is\nright; and whether the dashboard should carry a user identity.\n\n### 6. Runtime and process boundaries — DECIDED\n\n`archied` is the native resident orchestration and interactive-chat process.\nEvery autonomous repository workflow crosses one full-task core-NATS handoff\ninto a task-scoped `archie-agent` container. Embedded versus external NATS\nchanges broker deployment only; it never selects an executor. The former host\nin-process runner, per-stage NATS protocol, and subprocess runner are removed.\n\nForge, store, and worktree authority remains in `archied` and is exposed to the\ntask container through scoped RPC services. The worker runs the workflow and\nits worker-local ai-sdk loop; it does not receive forge credentials or direct\nstore access. See `docs/prds/embedded-nats.md`.\n\n**UI Service boundary — RATIFIED (2026-09-07).** Phase 3 has a focused\nauthority record in [`docs/prds/ui-service-boundary.md`](/docs/prds/ui-service-boundary/).\nThe UI Service owns HTTP/SPA delivery, UI DTOs, browser authentication, and\nits own endpoint/credential/readiness settings. It consumes Gateway and State\nStore contracts and does not receive `config.Holder`, daemon pointers, SQL or\nstore implementations, workflow registries, forge/channel/model runtimes, or\nresolved secrets. The current `internal/webui` implementation remains inside\n`archied` until the ratified migration gates prove route parity, adapter\nselection, failure/readiness behavior, and deletion of the shared-holder and\ndirect-access paths. The extraction cutover removes the in-process UI\nlistener in the same change; there is no dual-live UI authority.\n\n**Dashboard live event delivery — DECIDED (2026-09-09, `archie-core-za9f`).**\nThe extracted UI receives live activity by polling the State Store's existing\n`EventsSince` cursor, not by a new push-subscription RPC. No proto change, no\ncontract amendment beyond this record.\n\nThe UI process runs one event pump. It primes its watermark by paging\n`EventsSince` to the end at startup without delivering, then polls\n`EventsSince(watermark, limit)` on an interval and hands each new event to\n`webui.Server.Broadcast`, exactly where `internal/app/archied/main.go`'s\n`persistAndBroadcastEvents` hands events to it today. One pump serves every\nconnected browser, so cost is one indexed query per interval regardless of\nviewer count, and `Broadcast` is a no-op when nobody is watching.\n\nThree properties make the cursor sufficient, and a push hub unnecessary:\n\n- **The cursor cannot skip.** `internal/store/store.go` opens the database with\n `SetMaxOpenConns(1)`, so writes serialise and a row's `id` is assigned in\n commit order. There is no window in which a lower id becomes visible after a\n higher one, which is the failure mode that normally rules out polling an\n autoincrement cursor. SSE catch-up already depends on this same property.\n- **A broadcast is a wakeup, not the payload.** `sseStream.drain` treats each\n broadcast event as a signal to fill the persisted gap ahead of it via\n `catchUp`, then delivers the event only if catch-up did not already cover\n it, deduplicating on the `since` watermark. Substituting a poller for the\n in-process bus therefore changes no SSE semantics, and needs no change to\n `internal/webui/sse.go`. It also means a broadcast dropped to a stalled\n client self-heals on the next one.\n- **The table is the only complete fan-out point.** Events reach the store\n through several paths that never touch the daemon's bus, including the\n audit event `store.ArchiveTask` writes inside its transaction\n (`internal/store/store.go`) and the binding-dispatch failure\n `internal/daemon/daemon.go` inserts directly. A push hub would have to be\n hooked at every write site and at post-commit, and would silently miss any\n site added later that forgets to notify. Reading the table misses nothing.\n\nThe cost is one poll interval of latency on the activity feed, default 1s and\noperator-tunable. That is accepted: the feed reports task lifecycle history,\nand an operator action already gets its own synchronous HTTP response.\n\nThis resolves the `/events` row of the route-owner inventory in\n[`ui-service-boundary.md`](/docs/prds/ui-service-boundary/). `/api/logs` and\n`/api/logs/stream` are not resolved by it, they read the daemon's in-process\ndiagnostic feed and remain host-local (see `archie-core-8cda.5.4`).\n\nRevisit only if sub-second feed latency becomes a requirement, or if a second\nconsumer needs the same stream, at which point the fan-out belongs in the State\nStore process behind a `SubscribeEvents` RPC and this record is superseded.\n\n**Dashboard configuration page — DECIDED (2026-09-09, `archie-core-ymut`).**\nThe configuration page becomes read-only in the UI process for Phase 3. Reads\ncross the boundary as a published snapshot; writes are descoped, not\ncontracted. This amends the ratified boundary, which assumed a single admin\ncontract carrying both.\n\n**Reads.** `webui.ConfigView` is already the whole answer. It is a UI-owned,\nsecret-free value projection built in `handleConfig`: `ProviderView` carries\nthe API key's environment variable name and a `Configured` boolean, never a\nvalue, and `RepoView` is repository metadata only. It is already the exact\nshape the browser receives. So the daemon publishes that projection, plus\nprovenance, reload status and the overridden-key list, as a snapshot the UI\nreads; nothing new has to be designed, only moved. Two State Store RPCs carry\nit, a `PutConfigSnapshot` restricted to the administrative token and a\n`GetConfigSnapshot` the UI calls. The daemon republishes on boot and after\neach successful reload, which is precisely when the projection changes.\n\nThe State Store is the right holder because a published configuration snapshot\nis shared state, it is already dialled by the UI, and it costs no new listener,\nport, token or interceptor. The alternative, a config admin service on\n`archied`, would make the daemon a gRPC server for the first time. Today it is\npurely a client of Gateway and State Store; `grpc.NewServer` appears only in\n`RunGateway` and `RunStateStore`, which are the standalone binaries' entry\npoints, not the daemon's.\n\n**Writes.** `PATCH /api/config`, `POST /api/config/reset` and\n`PATCH /api/config/repos/{owner}/{name}` answer 503 in the UI process, and the\nSPA hides those controls rather than offering a button that fails. Editing is\n`config.toml` plus reload, which is already the only path for every setting\noutside the runtime-tunable allowlist.\n\nThis is a deliberate feature reduction and worth naming as one. The write path\nis not hard because of the transport; it is hard because the policy is\ndaemon-local and stays that way: `installUpdateConfigHandler` denies\nnon-runtime-tunable keys, applies a dotted overlay to a clone of the published\nconfig, validates the materialised result, writes overlay rows, and rebuilds\nprovenance. Contracting that means either moving the policy, which the\nboundary forbids, or exposing it verbatim as remote procedure calls, which\nbuilds a Configuration Service inside a migration bead. Neither is work this\nphase should be doing to keep one page's buttons alive.\n\nRevisit when dashboard configuration editing is picked up as a feature in its\nown right (`archie-core-1786637498420-327`, still open). At that point it gets\nan admin contract designed as a feature, and the Configuration Service this\ndocument already anticipates is its natural owner. Until then the UI renders\nconfiguration and does not write it, which is what the boundary always said\nabout ownership even when it assumed a wider contract.\n\nAmended 2026-09-11. This descope is a deferral, not a decision that the\ndashboard should be read-only. Name it that way in review: the daemon-side\npolicy was built and tested, then deleted by the cutover (`e037b95` removed\n`internal/app/archied/config_update.go` and its tests), so reviving it is\nrecoverable work, not a rewrite. `archie-core-j28m` carries the remaining\nquestion -- which process serves the policy and what transport carries the\nwrite -- and blocks the feature bead. The handlers, their error sentinels and\nthe `UpdateConfig`/`ResetConfig`/`UpdateRepoField` seams on `webui.Server` are\nkept deliberately for it; `ConfigView.Editable` follows whichever process holds\n`UpdateConfig`, so wiring that one field makes the page editable\n(`archie-core-ml30`).\n\nImplementation belongs to `archie-core-8cda.5.4`, the cutover that removes the\ndaemon's own dashboard. Nothing here is built yet.\n\n**Superseded 2026-09-21 by the runtime control plane.** The open question this\ndescope left -- which process serves the policy and what transport carries the\nwrite -- was answered by `docs/prds/runtime-control-plane.md`: neither a daemon\nadmin service nor a revival of the daemon-local policy. `ControlPlaneService` on\nthe State Store's existing gRPC server owns every database-backed setting, and\neach feature validates its own resource, so there is no daemon-local policy left\nto contract. `archie-core-j28m` is closed.\n\nConsequently the three write seams described above no longer exist.\n`PATCH /api/config`, `POST /api/config/reset` and\n`PATCH /api/config/repos/{owner}/{name}`, the `UpdateConfig`/`ResetConfig`/\n`UpdateRepoField` fields on `webui.Server`, and `ConfigView.Editable` were\ndeleted in `31622ec5` along with the runtime overlay store they wrote to.\n`GET /api/config` remains a read of the published projection, as this decision\nalways intended.\n\n### 7. Shared mechanics\n\nThe following cross-domain mechanics require exact contracts before dependent\npackages migrate:\n\n- event-bus delivery and failure semantics;\n- policy evaluation API, composition, and precedence;\n- configuration candidate preparation, health validation, atomic promotion,\n observation, and rollback;\n- service-specific health, bounded remediation, isolation, and escalation;\n- changelog association and grouping.\n\nThese shared mechanisms do not acquire ownership of domain meaning. Domains\ncontinue to own their commands, events, policies, settings, and consequences.\n\nThe Go documentation generator, its output placement, and drift checking are\ndefined in `generated-documentation.md`. The VitePress site and its Pages\ndeployment were removed on 2026-09-12 as an unnecessary build step; `docs/` is\nrepository documentation only. The rendering and publishing surface is an open\ndecision, and so are the renderer's page templates and any developer commands\nor CI sequence that would accompany one. Adapting the generator and implementing\nArchie's normalized documentation model remain migration work rather than open\nproduct architecture.\n\n**Amended 2026-09-15.** `internal/infrastructure/servicediscovery/nats`\nimplements the `servicediscovery.ServiceRegistry` contract (Connect, Register,\nResolve, Watch) over NATS KV and is fully tested, but no production code\nconstructs it and no config surface exposes it. This is a deliberate deferral,\nnot an oversight: service-to-service addressing belongs to the runtime and\nprocess boundary work in section 6, which currently pins container addressing\nby injecting the daemon's startup NATS URL rather than resolving through a\nregistry. The scaffold is kept for that work; re-derive it only if the\naddressing model changes.\n\n### 8. Cutover sequence\n\nThe final migration plan must order:\n\n- compatibility adapters;\n- schema migrations and data backfills;\n- any required dual-read or transitional paths;\n- domain-by-domain package replacement;\n- production wiring changes;\n- legacy package removal;\n- architecture tests preventing dependency regression;\n- focused, integration, race, and full quality gates.\n\nEach legacy package requires objective removal criteria. No compatibility path\nmay become an indefinite second implementation.\n\n\n## Completion criteria\n\nMigration design is complete when:\n\n- every current package has one approved owner and destination;\n- every mutable record has one authoritative owner and migration path;\n- all external and cross-process contracts have versioned owners;\n- existing behaviour and data have explicit parity requirements;\n- process boundaries have evidence-backed justification;\n- the cutover order preserves a working application;\n- legacy deletion criteria and deterministic architecture tests are defined;\n- no unresolved decision requires an implementer to invent domain semantics.\n\nExact Go APIs, SQL statements, and mechanical refactoring steps MAY be finalized\nduring implementation when they do not change these semantics.\n\n## Recorded migration position (2026-07-29)\n\nRecovered 2026-08-09 from the pre-migration issue tracker. This section records\ndecisions the maintainer approved that the repository did not otherwise capture.\n\n**Completed:** `internal/infrastructure/eventbus/nats` (rebuilt, not relocated);\n`internal/infrastructure/configuration` (loading moved out of `internal/config`,\nwhich is now types-only and does no I/O); `internal/eventbus` (broker-neutral\ncontract); `internal/domain/workintake` (the first `internal/domain/` package,\nowning `TaskEnvelope`, the routing `Kind`, the label vocabulary, and task\nsubjects).\n\n**DECISION — private input DTO deferred (approved).**\n`configuration.Document.Config` is still `internal/config.Config` rather than the\nprivate per-domain input DTO this document mandates. Building an ~800-line\nparallel type tree while every consumer still reads `config.Config` would mean\ntwo shapes kept in sync by hand, with no consumer for the new one. It is named as\na field so the compromise is visible at each use site, with the deletion\ncondition recorded in that package's `doc.go`. Do this **with** the\n`internal/config` dissolution, not ahead of it.\n\n**Agreed next: `internal/app/{archied,agentworker}`.** `run()` in\n`cmd/archied/main.go` is ~460 lines, against `organisation.md`'s rule that\n`cmd/*` must not contain substantive wiring or act as a service locator. It is\nalso the last non-infrastructure holder of the NATS SDK, and it is where the\nconfig DTO translation will land, so it unblocks the dissolution above.\n\n**Method constraint.** Do not write a separate recorded package-review document\nper area when the decisions are already dictated by\n`dependencies-and-contracts.md`. That is the ceremony pattern this project\ndeliberately avoids.\n\n## OPEN — `[chat.*]` config section naming\n\n`configuration.md:160` already states the position: `ChatConfig` holds \"separate\nchannel instance settings... there is no chat-wide settings owner\". The config\nsection name has not followed, so a settings block named for a product surface\nowns what the architecture calls channel state.\n\nTwo candidates, neither settled: `[channel.*]` and `[messaging.channel.*]`. The\nsecond reads as a domain path, and no config section in this repo is currently\nnamed after a domain package, so adopting it sets a precedent for every other\nsection rather than fixing one name.\n\nBlocked on a maintainer decision, not on evidence. This is a rename with an\noperator-visible TOML break, so it should land with the `internal/config`\ndissolution recorded above rather than on its own.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/observability/",
|
||
"title": "Task Observability",
|
||
"section": "Architecture",
|
||
"body": "# Task Observability\n\n**Status:** Implemented. The per-run detail surface was added on 2026-09-18 and\nis described below.\n**Date:** 2026-08-09 (revised 2026-09-18)\n\n## Historical problem\n\narchied's own daemon logging (`internal/logging/`) was already solid: a\nrotating file sink, a live `Feed` the dashboard subscribes to, and a reader\nthe `/api/logs` endpoint serves from. Before this implementation, none of\nthat reached inside a task.\n\nA task's actual work happens in a task-scoped `archie-agent` container,\ndispatched as one complete workflow over NATS.\nBefore this design was implemented, five things destroyed the evidence of what\nthat container did:\n\n1. **Worker startup** constructed an stderr-only logger and never replaced its\n handler after discovering a dedicated boot task, so runtime records could\n not leave the container.\n2. **`internal/container/pool.go`** creates every task container with\n `AutoRemove: true`. The container — and its stderr — is gone at exit,\n success or failure alike.\n3. **`internal/domain/workflow/implement.go`, `StageBaselineGate`** captures\n `go build`/`go vet`/`go test` output via `cmd.CombinedOutput()`, then\n clips it to 200 characters into a `Warn` log line and discards the rest.\n The eventual park reason is built from `res.Status` alone — the actual\n compiler error is never persisted anywhere a human or Archie's own chat\n tools can read it back.\n4. **`agentexec.Result`** (`internal/agentexec/protocol.go`) carries only\n `Summary`/`Detail` — no step-by-step transcript (tool calls, durations,\n errors). There is no transcript to recover even in principle.\n5. **`agentexec.SubjectForSystem`** is documented as the return channel for\n exactly this — *\"log dumps, health, and PII warnings... the daemon reads\n these for observability and never forwards them\"* — but has no publisher\n and no consumer anywhere in the codebase except a subject-format\n assertion in `nats_test.go`. The designed return path was never wired.\n\nThe practical consequence: a task parks with\n`stage baseline: baseline red -- go build ./... fails and builder could not\nauto-fix (status: parked)` and there is no way — not from the dashboard, not\nfrom Telegram chat, not by SSH'ing into the deployment host — to see *why*\n`go build` failed. The container that knew is already gone.\n\nThis matters beyond debugging convenience: supported deployments can combine\na resident daemon with ephemeral Docker containers that run unattended for\nweeks between operator logins, and the system is\nintended to be usable by people other than the original operator. A system\nthat is only debuggable by an operator who happens to be watching `docker ps`\nin real time does not meet that bar.\n\n## Decision: NATS is the sole autonomous handoff\n\nThe agent image must reach NATS in every deployment, because the broker is the\nonly autonomous task handoff and the only channel carrying task results back\nto the daemon. `archied` has no workflow runner or host fallback. A container\nthat dies before establishing a NATS connection is a distinct, narrower\nfailure (connectivity/config) from the one this document addresses (a connected\ncontainer's own output being unrecoverable), and is out of scope here.\n\n## Implemented design\n\nThe key property: **logs leave the container while it is alive**, so its\ndeath at `AutoRemove` no longer matters. Persistence happens in the daemon\nprocess, which already owns durable state.\n\n### Sink: extend `internal/logging`, not a new package\n\n`internal/logging` already owns its on-disk format end to end per\n`organisation.md` (\"a package owns its own format end to end\"). Add a\ntask-scoped writer producing the same `logging.Entry` JSONL shape at\n`\u003cstate_dir\u003e/logs/tasks/\u003ctask_id\u003e/attempt-\u003cn\u003e.jsonl`. `logging.Tail`,\n`logging.Query`, and `logging.Components` operate on this unchanged;\n`internal/webui/api_logs.go` stays transport-only, as it does today.\n\n### Agent side: make the designed subject real\n\n`internal/app/agentworker` replaces the stderr-only `slog.Handler` with one\nthat tees to stderr (kept for container-level operator debugging when a\nshell is attached) and publishes each record to\n`agentexec.SubjectForSystem(taskID)`, which already exists for this purpose.\nThe application layer obtains only a narrow publisher from\n`internal/infrastructure/agenttransport/nats`; raw core-NATS types remain in\ninfrastructure. `internal/infrastructure/agentboot.TaskID` reads the task ID\nfrom `\u003cworktree\u003e/.git/task.json` per the container/pool.go trap documented in\n`CLAUDE.md`. Publishing is fire-and-forget: log shipping must never block or\nfail the run it is reporting on.\n\n### Daemon side\n\n`cmd/archied/main.go` registers `subscribeSystemLogs` once at startup on\n`agentexec.SubjectSystemWildcard`. It demultiplexes by task ID, appends to that task's\nsink, and publishes into the existing `logging.Feed` so live task output\nappears on the dashboard exactly like daemon output. The approved target is\nfor this composition to move to `internal/app/archied`; that package does not\nyet exist, so this document does not claim the migration is complete.\n\nThe daemon is also a writer in its own right for the decisions only it makes.\nEvery park records its reason into that attempt's sink (`daemon.recordPark`),\non the daemon side after the guarded transition lands and in\n`workflow.park` on the agent side, because \"why did this park?\" is the question\nthe log exists to answer and the container path cannot answer a run that never\nreached a container. The entry is written only once the park actually\nhappened, so a transition another actor won does not leave a log claiming an\noutcome the task board contradicts.\n\n### Gate output: stop discarding it\n\n`StageBaselineGate` and the equivalent TDD-stage gate runs write full\n`cmd.CombinedOutput()` to the task's sink under a `gate` component, and put a\nbounded tail (~2KB, not 200 bytes) into the park reason. This is the cheapest\nfix and independently valuable — it alone would have made the `go build`\nfailure diagnosable without any NATS plumbing.\n\n### Transcript\n\nStep-level events (tool name, argument digest, duration, error) are emitted\nas log entries on the same NATS system subject rather than added as new\nfields on `agentexec.Result`. This keeps the wire protocol\n(`internal/agentexec/protocol.go`) stable and the transcript lands in the\nsink automatically, with no separate delivery path to keep consistent.\n\n### Retention\n\nReuse the existing rotation (`MaxSizeMB`/`Keep`) semantics per task\ndirectory. Prune a task's log directory when `store.ArchiveTask` archives\nit, so task logs share the task's own\nlifecycle rather than accumulating independently.\n\n### Surfaces\n\n- `/api/tasks/{id}/logs`, built on `logging.Reader` the same way\n `/api/logs` is today.\n- A chat tool so Archie can answer \"why did task 4 park?\" directly from\n Telegram or the webui chat — this is the surface that actually matters\n for the deployment model in `CLAUDE.md`, where an operator may not be\n looking at a dashboard at all.\n- The per-attempt read surface described in the next section, on top of the\n same task-scoped stream.\n\n## Per-run task detail (attempts, stages, changes, config)\n\n**Status:** landed, together with the change set that carries this revision\n(workflow run `76f8e877`). The provenance half is in the tree: `events.Event.Attempt`,\nthe `changes_captured` and `config_captured` event kinds, and the two protobuf field\nadditions. So is the read half: the three per-attempt endpoints\n(`/api/tasks/{id}/attempts`, `/changes`, `/debug`) and the dashboard run-detail page\nthat consumes them. The decision record, the rejected alternatives and the honest\nlimits are `docs/prds/task-run-detail.md`.\n\nThe task log surface above answers *why did this park?*. This surface answers\n*what did this run actually do?*, per attempt.\n\n### Attempt attribution\n\nOne task can run many times, and the task's event stream is the durable record\nof all of them. `events.Event.Attempt` attributes each event to one run; `0`\nmeans **not attributable** — every event written before this change, plus the\ndeliberately task-agnostic producers. An unattributed event is reported as a\ncount and is never presented as an attempt.\n\nAttribution is a column, not a fold over `task_queued`/`task_retried`: a fold\nwould have to guess which side of a boundary an event belongs to across a daemon\nrestart, a stale-run recovery, or two interleaved attempts, and a guess rendered\nas structure is exactly the false fidelity this surface exists to avoid.\n\nThe column reaches an existing database through the presence-gated arm in\n`internal/store`'s `migrateTasks` — `eventsSchema` is `CREATE TABLE IF NOT\nEXISTS`, so on an existing database the migrator is the *only* thing that adds\nit. `PRAGMA user_version` is unchanged because the loop is presence-gated\nrather than version-gated, and nothing is backfilled.\n\n### Read surface\n\nThe routes are registered beside the existing task routes in\n`internal/webui`'s `registerTaskRoutes` and all resolve the task through the\nstore's `TaskByID`; a task that does not exist is a `404`, never a `200` with an\nempty body.\n\n- `GET /api/tasks/{id}/attempts` — every attempt with its stages inline, so the\n rail and the attempt selector can never disagree. Stages are ordered by\n occurrence within their attempt. An attempt with no stages is legitimate and\n renders as \"no stages recorded\", never as an unexplained empty rail.\n- `GET /api/tasks/{id}/changes?attempt=N` — the change captures for one\n attempt, ordered by capture time; more than one per attempt is normal (a\n mid-workflow commit such as TDD's `commit-repro`, then `commit-push`, then a\n final `open-pr` capture that carries the pull-request number).\n `found: false` means no capture was recorded for that attempt, which is\n **not** \"no files changed\"; a capture that was recorded but could not be read\n is reported as exactly that, and never as a diffstat of zeroes.\n- `GET /api/tasks/{id}` — unchanged: the raw event array, each event now\n carrying `attempt`. The per-attempt configuration is read from this response,\n selected by kind **and** attempt.\n- `GET /api/tasks/{id}/logs` — gains `stage` beside its existing filters. The\n match is exact and case-insensitive, and an entry that records no stage never\n matches (see `logging.Query.Stage`).\n- `GET /api/tasks/{id}/debug?attempt=N` — the stored task record and the task's\n events, deliberately unfiltered, so an operator can attribute everything the\n task did even when a panel has nothing to show.\n\nThe `attempt` parameter follows one rule on every route: absent or `0` selects\nthe task's current attempt, and the resolved attempt is echoed in the response.\n\n### Stage status\n\n`stage_start` and `stage_finish` are the only inputs, so the derivation is\nexplicit and each case is pinned by a test:\n\n| Events seen for the stage | Reported |\n| --- | --- |\n| `stage_finish` with `data.error` | `failed`, with the error text |\n| `stage_finish` with `data.interrupted` | `interrupted` |\n| `stage_finish`, neither | `ok` — \"returned without error\" |\n| `stage_start` unmatched, and this is the task's current in-flight attempt | `running` |\n| `stage_start` unmatched on any dead attempt | `interrupted` — the run ended without finishing it |\n| `stage_finish` with no matching start | the stage, with no start time invented |\n| no stage information at all for that attempt | `unknown` |\n\n**There is no exit code.** `stage_finish` carries a duration and an optional\nerror string and nothing else, so no stage can be shown as pass/fail and the\npage must not imply verification. A stage that finished cleanly and then parked\nstays `ok` on a failed task, which is why the attempt status is read together\nwith the task status rather than on its own.\n\n### Changed files\n\nThe change an attempt produced is read from the worktree at the moment it is\ncommitted or pushed, and persisted as a `changes_captured` durable event — plus\none final capture when `OpenPR` records the pull-request number, which is the\nonly point at which a capture can carry one. It has to be captured there: the\nworktree is deleted on merge, close and no-PR terminal states, a retry resets\nthe branch onto its base, and no forge contract exposes a changed-file read.\n`internal/domain/workflow/task/changes.go` owns the value types and the status\nstrings they persist.\n\n- One capture carries per-file `added`/`modified`/`deleted`/`renamed`/\n `typechange` status with added and deleted line counts, and totals.\n- Each capture names its schema (`archie/task-changes@1`, in `internal/events`)\n and a reader refuses a payload whose schema or key set it does not know,\n rather than rendering the zero value of every field it could not find. That\n refusal surfaces as \"a capture was recorded but could not be read\", which is\n a different statement from \"no capture was recorded\".\n- `base_sha` is the merge base the file list was diffed against, not the base\n branch's tip, so the pair describes one change set even when the base branch\n moved during the run.\n- `files` is capped; `totals` always covers the **full** set, and a truncated\n capture says so rather than silently dropping entries.\n- `binary` means \"no textual hunks\" — never a claim about the file's type on\n disk — and a path the repository recorded as a rename pair is reported as the\n repository recorded it.\n- Capture **degrades and never parks**: an implementation without the capture\n seam captures nothing, logs, and lets the run continue.\n\n### Per-attempt configuration\n\nThe effective configuration an attempt ran under is persisted as a once-per-\nattempt `config_captured` durable event carrying the schema constant\n`archie/task-config@1` and the non-secret task-runtime subset from\n`config.Config.ForTask()`. It is not stored in a table and has no endpoint of\nits own: the dashboard reads it out of the events response it already fetches.\nThe single-row `config_snapshot` table is left alone — it answers what the\nrunning configuration is now and is replaced on every publish.\n\nRedaction is proven positively by a test that builds a full configuration\ncontaining canary values and asserts none of them survive serialising, rather\nthan being inferred from the producer's contract. The timeline carries one\nhuman line for this kind; the document itself is rendered by the panel.\n\n**Trust boundary, stated honestly.** A task-scoped grant may insert events on\nits own task ID — that is how `tool_call` and `agent_finish` already work — so a\ntask could in principle write its own `config_captured` event. This widens no\ncapability, because the grant could already insert arbitrary events on itself,\nbut it does mean this provenance is exactly as trustworthy as the rest of that\ntask's event stream, and no more. It is not tamper-proofing and must never be\ndescribed as such.\n\n### The wire\n\nThe whole wire change is two fields on existing messages: `Event.attempt`\n(field 11) and `ReadTaskLogRequest.stage` (field 9) in\n`proto/state/v1/state.proto`. No RPC, no table and no configuration field was\nadded, so the ratified state-store contract in\n`docs/prds/state-store-contract.md` rev 2c is unchanged.\n\nVersion skew degrades instead of failing, and each degrade is a statement the\npage has to make rather than hide: an old server makes `attempt` read as `0` and\n`stage` match nothing, so the page says it cannot attribute events to attempts\ninstead of labelling the whole stream \"attempt 1\"; an old client sees\n`config_captured` as an unfamiliar kind, and the panel must then say it cannot\nread per-run configuration — a different claim from \"not captured for this\nattempt\".\n\n### Rules the surface keeps\n\n- \"No record\" and \"this deployment cannot read this\" stay different answers,\n for logs and for configuration alike.\n- No run-duration figure is shown. A run can spend days in `waiting_human`, so\n one number would understate wall time; per-stage durations only.\n- No duration is invented for a stage with no start; it reads \"not recorded\".\n- The stage filter is a narrowing, not a coverage claim: agent and tool output\n carries no stage and is never matched, and the pane says so beside the\n control.\n- An attempt predating the change shows \"not captured for this run\" for its\n configuration and its change capture, never an empty view that implies\n defaults applied or nothing changed.\n\n### Failure and rollback\n\nEvery part is additive. A capture or a configuration emit that cannot happen\nleaves the run alone — reporting must never park a task. Reverting the change\nset removes the endpoints and the page; the extra column and the extra event\nkinds are inert to the previous revision (SQLite ignores a column no statement\nnames, and an unfamiliar event kind already renders as unfamiliar), and there is\nno data transformation to undo.\n\n## Dependency direction\n\n`internal/logging` remains cross-cutting per `organisation.md` — it imports\nno domain, infrastructure, or app package, so extending it to be task-scoped\ndoes not change its classification. The workflow domain writes through the\ncross-cutting logging contract. On the agent side,\n`internal/app/agentworker` composes the narrow publisher supplied by\n`internal/infrastructure/agenttransport/nats`. On the daemon side,\n`cmd/archied/main.go` currently composes the NATS subscription with the task\nsink until the approved `internal/app/archied` migration is implemented.\n\n## Implementation sequence (completed)\n\n1. Gate-output capture (~30 lines, no NATS changes) — fixes the immediate\n symptom that prompted this document.\n2. Task log sink in `internal/logging`.\n3. Agent-side publisher on `SubjectForSystem`.\n4. Daemon-side subscriber + `logging.Feed` integration + retention on\n archive.\n5. `/api/tasks/{id}/logs` + chat tool.\n\nTracked as GitHub epic/subtasks per the issue list linked from this\ndocument's PR.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/organisation/",
|
||
"title": "Repository Organisation",
|
||
"section": "Architecture",
|
||
"body": "# Repository Organisation\n\n**Status:** Approved \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Target structure\n\nArchie MUST be organised around cohesive application responsibilities:\n\n```tree\ncmd/\n archied/\n archie-agent/\n\ninternal/\n domain/\n \u003carea\u003e/\n eventbus/\n policy/\n infrastructure/\n configuration/\n eventbus/\n nats/\n forge/\n persistence/\n \u003cexternal-service\u003e/\n app/\n archied/\n agentworker/\n\nplugins/\n \u003cplugin\u003e/\ndeployments/\n \u003csupported-assembly\u003e/\ndocs/\n guides/\nexamples/\n \u003ccopyable-example\u003e/\nextras/\n \u003coptional-extension\u003e/\n```\n\nDomain names are chosen from application behaviour during package review.\nExisting names MUST NOT be retained merely to reduce migration work.\n\n## Domain areas\n\nA domain owns one cohesive job performed by Archie. It contains that job's\nlanguage, state, rules, operations, commands, events, policy implementations,\nruntime settings, and required service contracts.\n\nCode that changes together SHOULD live together. Empty or ceremonial `entities`,\n`repositories`, or `services` layers are prohibited.\n\n## Application composition\n\n`internal/app`:\n\n- constructs domains and infrastructure implementations;\n- translates external configuration into domain and plugin settings;\n- connects commands, events, and shared contracts;\n- starts services in dependency order;\n- owns health aggregation and shutdown ordering.\n\n`cmd/*` parses only process-level input and invokes the relevant application\ncomposition package. It MUST NOT contain substantive wiring or act as a service\nlocator.\n\n## Process boundaries\n\nArchie begins as a modular monolith even when capabilities run in separate\nprocesses. A domain becomes a separately deployed service only for a concrete\nneed such as security isolation, independent scaling, failure containment, or\nindependent operation.\n\nPossible future extraction does not justify premature network protocols or\nduplicated service scaffolding.\n\nA new command is not a reason for a new binary. `cmd/` entries are process\nboundaries: a binary earns its place when its surface fails independently of the\nfleet, or when one process may need to run in more than one location, such as a\nhigh-availability pair or across a NAT boundary. Operational commands over a\nsurface an existing binary already owns are subcommands of that binary. Offline\ndatabase maintenance belongs to `archie-state-store`, which owns the file and\nits path layout, not to a new recovery binary.\n\n## Optional and operational material\n\n- `extras/\u003cextension\u003e` contains a maintained optional extension with real\n behaviour. Core Archie MUST NOT import it, and removing `extras/` MUST NOT\n prevent the core application from building or running.\n- `deployments/\u003cassembly\u003e` contains a supported assembly, including deployment\n definitions and concise operational instructions.\n- `docs/guides/` contains handwritten explanation and links to generated\n reference material.\n- `examples/` contains copyable samples with explicit purpose, prerequisites,\n and support expectations.\n\nSupported assemblies SHOULD demonstrate that capabilities can be selected\nindependently.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/package-review/",
|
||
"title": "Domain and Feature Review Procedure",
|
||
"section": "Architecture",
|
||
"body": "# Domain and Feature Review Procedure\n\n**Status:** Active \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchitecture work proceeds one domain or feature at a time. Before any design\nquestion, the selected area is researched across its complete live\nimplementation, consumers, tests, state, configuration, and runtime wiring.\n\nExisting packages are evidence, not the unit that drives the interview.\nPlacement follows behaviour and ownership, not the existing directory name.\n\nNo restructuring implementation begins until the relevant package decisions\nare agreed and recorded in the focused domain or requirement document.\n\n## Required research\n\nBefore design questions, the current-state report MUST identify:\n\n- entry points and runtime composition;\n- state, persistence, and lifecycle;\n- public and internal contracts;\n- commands, events, and policy enforcement;\n- configuration and defaults;\n- all consumers and cross-domain interactions;\n- process and infrastructure boundaries;\n- tests, failure behaviour, and known invariants;\n- duplicated or conflicting implementations;\n- documentation that disagrees with live code.\n\nDiscoverable facts MUST be investigated rather than presented to the user as\ndesign questions.\n\n## Required decisions\n\nEach package review records:\n\n| Decision | Required result |\n|---|---|\n| Owned behaviour | The cohesive job the code performs |\n| Domain ownership | The owner of the behaviour and vocabulary |\n| Final location | Its target path |\n| Public contracts | The smallest interfaces, commands, and events exposed |\n| Required services | Domain-defined interfaces implemented by infrastructure |\n| State ownership | The owner of each mutable record and transition |\n| Dependencies | Allowed imports and dependencies to remove |\n| Duplication | Code to consolidate, delete, or keep separate |\n| Plugin status | Core domain, shared contract, infrastructure, plugin, or extra |\n| Process boundary | In-process, or the concrete reason for isolation |\n| Configuration | Settings owned by the domain or plugin |\n| Policy | Definitions, evaluators, consequences, and shared mechanics used |\n| Documentation | Authoritative definitions and generated reference |\n| Migration constraints | Behaviour and data that must remain correct |\n\n## Interview method\n\nFor every selected domain or feature:\n\n1. Declare the single area under review.\n2. Complete the current-state research.\n3. Explain actual behaviour in plain technical language.\n4. Separate facts from design decisions and concerns.\n5. Resolve obvious consequences directly and ask only questions whose answer\n materially changes behaviour or structure.\n6. Record each confirmed decision in its focused document.\n7. Complete that area's handoff before selecting another.\n8. Keep unrelated domains outside the implementation scope.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/plugins-and-extensions/",
|
||
"title": "Plugins and Extensions",
|
||
"section": "Architecture",
|
||
"body": "# Plugins and Extensions\n\n**Status:** Approved foundation \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Plugin framework\n\nThe core plugin domain lives under `internal/domain/plugin`. It owns:\n\n- plugin identity and metadata;\n- discovery and registration contracts;\n- lifecycle rules;\n- capability registration;\n- validation and compatibility rules.\n\nThe generic plugin contract remains metadata-only. Functionality is exposed\nthrough narrow typed capability contracts with an owning registry or manager.\n\nWorkflow plugins belong to the Workflow capability family. The Workflow domain\nat `internal/domain/workflow` owns their typed registration, validation,\nversioning, definition, and execution contracts. The generic plugin domain may\nprovide plugin identity, discovery, compatibility, and lifecycle mechanics, but\nit MUST NOT define Workflow semantics or expose untyped Workflow hooks.\n\n## Plugin implementations\n\nFirst-party plugins are self-contained vertical modules under\n`plugins/\u003cplugin\u003e`.\n\nA plugin:\n\n- owns its implementation, settings, templates, assets, and tests;\n- receives only explicitly supplied contracts and services;\n- MUST NOT receive daemon internals, a service locator, or unrestricted\n application access;\n- MUST NOT import arbitrary infrastructure or application composition code;\n- MAY use approved domain and shared application contracts;\n- MUST remain removable without changes to unrelated domains.\n\nPlugins may be internally cohesive like small services. They do not become\nnetwork services without a concrete security or operational reason.\n\n## Event sources and reactions\n\n`archie-core-7d5u.1` asked whether extensions should react to events\nin-process through typed capability families (this document's model) or\nout-of-process like a generic `DispatchEvent` dispatcher. Decided 2026-08-22\nin `docs/prds/event-sources-and-reactions.md`: in-process typed families,\nconfirming this document's model rather than amending it. The forge webhook\nreceiver (`internal/forge/webhook`, see `ARCHITECTURE.md`'s \"Webhook intake\"\nsection) and the EDA playbook engine's Module action position\n(`docs/prds/eda-playbook-engine.md`, `internal/domain/eda/module`) are the\nshipped instances of that decision -- each is a narrow typed contract with an\nowning registry, not a generic event hook. The playbook engine's Channel and\nForge action positions are designed but not yet implemented (open\ninvestigation `archie-core-t2db.19`); when they land they follow the same\nmodel, not a new one.\n\n## Curator surprisal sampling\n\n**Decided 2026-08-05, wave 1 landed 2026-09-05** — the curator engine family\n(epic [#435](https://github.com/samcharles93/archie-core/issues/435)) selects which memories deserve agentic attention via\nsurprisal-based sampling: score memories by how surprising they are and\nspend the sampled reasoner's agentic budget on the most surprising items.\n\nThe prerequisites are built: the embeddings capability\n([#436](https://github.com/samcharles93/archie-core/issues/436), `internal/domain/embedding` contract +\n`internal/infrastructure/embedding` implementation, config-driven via\n`models.embedding`) and the wave-1 `Sampler` strategy seam\n(`internal/domain/sampling` -- not `internal/curator`, see\n`docs/prds/curator-sampler-wave1.md` for the settled design and tracking)\nwith four\ncheap, embedding-free strategies (recency, random, all, staleness proxy).\nNo curator consumes a `Sampler` yet, and the embeddings client is likewise\ndeliberately inert: `setupEmbeddings` constructs it at boot and holds it on\n`b.embeddings`, but no production code reads that field yet, so a resolved\n`models.embedding` role currently produces a client whose only observable\neffect is a boot log line. Both are prerequisites waiting on #437, not\nunwired defects.\n\nStill deferred: [#437](https://github.com/samcharles93/archie-core/issues/437), the embedding-backed surprisal strategy, which\ndepends on the above and on [#407](https://github.com/samcharles93/archie-core/issues/407). Requirements for that deferred work,\nso it can be picked up without re-deriving the design:\n\n- A `Sampler` implementation behind the same interface as the cheap\n strategies; selection deterministic given fixed inputs.\n- Algorithm: sample up to a candidate cap; build a k-NN graph over embedding\n vectors; row-normalize the adjacency with self-loops; power-iterate to the\n stationary distribution; surprisal = −log(stationary probability of the\n nearest point); select the highest-surprisal items.\n- Degradation: content without an embedding is skipped; an embedding failure\n aborts sampling and falls back to free exploration — a sampling failure\n must never fail or crash a curator pass.\n- Cost bound: the graph build is quadratic in the candidate cap; the cap must\n be a named constant with a documented bound.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/policy/",
|
||
"title": "Policy System",
|
||
"section": "Architecture",
|
||
"body": "# Policy System\n\n**Status:** Approved foundation; exact API is deferred \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nRetry limits, authorization, gates, commands, protected paths, tool\navailability, and similar rules define or constrain what Archie may do. They\nneed consistent consumption, composition, evidence, and audit without erasing\ntheir different domain meanings.\n\n## Ownership\n\n`internal/policy` is an explicitly approved shared application contract. It is\nnot infrastructure and it is not a generic domain that owns every rule.\n\nThe shared policy system owns:\n\n- policy identity, version, scope, and applicability;\n- typed evaluation invocation;\n- deterministic composition and precedence;\n- common evaluation states such as satisfied, violated, indeterminate, and not\n applicable;\n- reasons and evidence;\n- evaluation audit records;\n- registration, discovery, and lifecycle.\n\nEach consuming domain owns:\n\n- its policy vocabulary and typed inputs;\n- its concrete policy definitions;\n- its evaluators;\n- the meaning and consequence of an evaluation.\n\nAccess therefore owns principals, resources, actions, and authorization\nconsequences. Task lifecycle owns retry eligibility and exhaustion. Task\nexecution owns gate requirements. Workspace mutation owns protected-path\neffects. Tool execution owns tool and command constraints.\n\nThese policies use consistent evaluation mechanics without pretending their\ndecisions are semantically interchangeable.\n\n## Infrastructure\n\nInfrastructure MAY load, persist, distribute, and audit external policy\nrepresentations. It MUST NOT define what Archie allows or assign the meaning of\na domain policy result.\n\nThe exact Go API, composition operators, precedence rules, and representation\nformat require a focused policy design pass.\n"
|
||
},
|
||
{
|
||
"url": "/docs/architecture/safe-change-and-recovery/",
|
||
"title": "Safe Change and Recovery",
|
||
"section": "Architecture",
|
||
"body": "# Safe Change and Recovery\n\n**Status:** Requirements approved; mechanics are under design. Database-backed settings are implemented through the control plane (2026-09): see the degrade/recovery section below, which records a gap rather than a solution. The full runtime-supervision protocol (bounded observation, versioned audit trail) remains under design.\n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchie is built and improved by agents as well as humans. The architecture MUST\nmake a change easy to locate, understand, test, audit, extend, and reverse\nwithout requiring its author to understand unrelated parts of the application.\n\nSeparation of concerns is therefore an operational safety boundary, not only a\ncode-organisation preference.\n\n## Required outcomes\n\n- An agent can improve a focused feature without receiving authority over\n unrelated domains.\n- Every event, function, action, setting, and policy has an identifiable owner\n and provenance.\n- Source and runtime changes pass deterministic checks outside the actor making\n the change.\n- A rejected or unhealthy candidate cannot replace the last known working\n version.\n- Invalid user configuration does not terminate an otherwise healthy\n application.\n- Configuration and service failures are observable, attributed, and eligible\n for bounded remediation or rollback.\n- The application continues serving unaffected capabilities when one\n capability is misconfigured or unhealthy.\n- Recovery failure is reported and contained rather than hidden in an\n unbounded retry loop.\n\n## Source changes\n\nAgent-authored source changes occur through the task-execution boundary in an\nisolated workspace. The agent does not directly mutate the running\ninstallation.\n\nPromotion is controlled by deterministic repository-owned policy, including\ntests, generated-reference drift checks, protected paths, change scope, and\nother required gates. The actor proposing a change cannot waive these checks.\n\nRepository and domain separation SHOULD allow a task to load and modify only\nthe governing requirements and code for its declared scope.\n\n## Runtime changes\n\nConfiguration and other runtime changes are proposed as candidates rather than\nmutating the active state in place.\n\nThe affected domain owns:\n\n- the command describing the semantic change;\n- validation of its typed settings;\n- health signals that demonstrate correct operation;\n- domain-specific remediation and rollback consequences.\n\nShared policy supplies consistent evaluation, evidence, and audit mechanics.\nIdentity supplies actor attribution. Infrastructure supplies external\nconfiguration storage, service control, and other operational adapters.\n\n## Runtime change protocol\n\nLast-known-good preservation and candidate promotion are the universal runtime\nchange protocol.\n\n1. An identity proposes an immutable candidate version.\n2. Infrastructure parses it without changing active state.\n3. Each affected domain validates its typed settings.\n4. Shared policy evaluates cross-cutting constraints.\n5. Each affected capability prepares the candidate without destroying its\n active instance.\n6. Capability-specific health checks prove the candidate works.\n7. Runtime supervision atomically promotes the candidate.\n8. A bounded observation period watches for delayed failure.\n9. Failure restores the last-known-good version and records the reason,\n evidence, actor, and involved versions.\n10. Repeated remediation failure stops retrying, isolates the capability, and\n reports the fault.\n\nThe affected domain owns the semantic change command, typed validation, health\nsignals, and remediation consequences. Shared runtime supervision owns the\npromotion transaction. Infrastructure performs storage and service-control\noperations.\n\nThe protocol's exact Go contracts, storage model, atomicity boundary, and\nobservation rules require a focused runtime-supervision design pass.\n\n## Failure isolation\n\nInvalid input MUST leave the active valid configuration unchanged and produce a\ndiagnostic record.\n\nA capability that cannot start or reload SHOULD fail independently where its\nabsence does not violate a declared application invariant. Unaffected\ncapabilities continue operating.\n\nRemediation MUST be bounded, observable, and policy-driven. Recovery MUST NOT\nsilently oscillate between versions or retry forever.\n\n## Database settings: degrade paths and recovery (2026-09)\n\nThe control plane owns runtime-tunable settings and stores them in `archie.db`.\nThis replaced a dashboard overlay store that degraded to file config on any\nfailure and had three documented ways back. The replacement has none, and the\nposture is the opposite: **the daemon fails closed.**\n\n### Degrade paths\n\n| Failure | Boot behaviour | Where the operator sees it |\n| --- | --- | --- |\n| State Store unreachable | `boot.loadRuntimeConfig` returns the dial error and `archied` exits non-zero | stderr: `runtime settings unavailable` |\n| A stored resource fails validation | Same: `validate database settings` wraps the error and the daemon exits | Same |\n| SIGHUP reload with a bad file, or a control plane that cannot be read | Running config is kept; the reload records `last_error` / `last_error_at`. The reload re-applies the database layer, so it never publishes file values over the running database-owned ones | `/api/config` → `reload.last_error` → dashboard banner |\n\nFailing closed is deliberate for settings the database owns: falling back to the\nfile values would restore the second source of truth the control plane exists to\nremove. The cost is that a bad stored value stops the daemon starting, and the\nvalue can only be corrected through the daemon's own API.\n\n### Recovery: offline commands on the State Store\n\nNothing skips the control plane at boot, and the only writer for a resource is\n`ControlPlaneService.Command` on a running State Store -- which is exactly what\nthe daemon fails closed without. `archie-state-store` therefore carries the\noffline recovery commands (`archie-core-6zw0`). They work on the task database\nfile directly -- the configured `db_path` with `-tasks.sqlite` appended, not\n`db_path` itself -- and need neither `archied` nor the Web UI running: `validate`\nreads the configuration file the daemon boots with, which is what makes its\nverdict the daemon's verdict.\n\n| Command | What it does | State Store |\n| --- | --- | --- |\n| `backup -db F -out F` | writes a transactionally consistent snapshot (`VACUUM INTO`), verified as a usable store before it is published | may be running |\n| `restore -db F -from F` | replaces the database with a snapshot and drops the replaced file's `-wal`/`-shm` | must be stopped |\n| `validate -db F [-config C]` | runs the checks that stop archied starting: the file is readable, uncorrupted and no newer than the binary; every stored resource decodes under the definition that owns it; and `configuration.Validate` accepts the settings the daemon would read -- the stored ones, or the seed the State Store writes for a kind the file does not hold yet -- layered onto `C`, the check the daemon reports as `validate database settings` | may be running |\n| `rollback -db F -kind K [-revision N]` | replays an earlier revision of a stored resource through the ordinary replace, recording the rollback as a new revision | must be stopped |\n\nThe two validation layers `validate` reports are not the same check, and it\nreports both rather than merging them: the write path's own decode is stricter\nabout shape (it rejects unknown fields boot's decode ignores), while boot is\nstricter about meaning (only boot checks `dispatch.trigger`, and a positive\n`poll_interval`). Either one refusing is a store somebody has to look at.\n\nNo rollback RPC exists and none is needed: a revision carries its own value, so\nrestoring one is the same replace the dashboard and the channels already use.\nThe offline command is that operation with the store stopped -- the one state in\nwhich the daemon's own API cannot serve it.\n\n\"Must be stopped\" is checked rather than assumed. The serving process takes an\nexclusive flock on `\u003cdb\u003e.lock` for its whole life, and `restore` and `rollback`\nrefuse with `task database is owned by another process` while another process\nholds it. The read-only commands -- `backup` and `validate` -- take no lock and\nare meant to run against a serving store: `backup` because the update installer\nruns inside the process it is replacing and cannot stop the State Store, and\n`validate` because an operator checks a running deployment with it. That\n`backup` exemption is only sound while backup never replaces the database, so\nnaming the database as its own `-out` is refused. `scripts/archie-update-install`\ntakes its pre-flight snapshot through this subcommand rather than through its\nown `sqlite3` shell, so the installer and an operator run one implementation.\n\n`rollback` is the surgical route back from a bad stored value; `restore` is the\nblunt one, and is also the way back from a schema migration a release cannot\ngrow out of.\n\n## Gateway restart: two constraints learned the hard way\n\nTelegram `/restart` (`internal/channels/telegram/restart.go` plus the `Start`\nsupervisor loop). Recovered 2026-08-09 from the pre-migration issue tracker.\nBoth constraints must survive any refactor.\n\n**1. Deadlock.** The `/restart` handler runs *on* the bot instance the supervisor\nis about to stop. Tearing down inline from the handler deadlocks the restart it\njust asked for. So the handler sends on a **buffered** channel (`restartCh`,\ncapacity 1) and returns immediately; the supervisor loop in `Start` does the\nteardown. Duplicate requests hit the default branch and are dropped, not queued.\n\n**2. Lockout.** A failed config reload must **not** be fatal. It logs and\nrelaunches with the previous in-memory settings. If a bad config edit killed the\ngateway, it would also destroy the only means of fixing it remotely — which is\nthe entire reason `/restart` exists.\n\n**Scope is gateway-only by design.** The daemon keeps running so in-flight agent\ntasks survive. Exiting the process would also restart under a `unless-stopped`\npolicy, but would kill running work. `Reload` (supplied by `cmd/archied/main.go`)\nre-reads config from disk and refreshes **only** the token and `AllowedUserIDs`;\neverything else is wired into the daemon at construction and still needs a full\nrestart.\n\n**Authorisation is inherited:** `authorizedMessage` rejects non-allowlisted\nsenders before any handler runs, so all gateway commands are admin-only by\nconstruction. Do not add per-command auth without checking that invariant still\nholds.\n"
|
||
},
|
||
{
|
||
"url": "/docs/guides/first-playbook/",
|
||
"title": "Building your first playbook binding",
|
||
"section": "Guides",
|
||
"body": "# Building your first playbook binding\n\nThis walks through the whole no-code path the event-capture epic promises:\npoint an external app's webhook at archie, see the real payload it sends,\nmap the fields you care about, bind them to a workflow, and watch the next\nmatching event turn into a task -- no code, no redeploy, no restart. See\n`docs/architecture/bindings.md` for the model and threat model behind each\nstep below.\n\n## 1. Send a test event\n\nPoint whatever external app you want archie to react to at:\n\n```\nPOST http://\u003cyour-archied-host\u003e:8484/webhooks/capture/\u003csource\u003e\n```\n\n`\u003csource\u003e` is a name you choose for this sender (e.g. `sentry`, `github-ci`,\n`stripe`) -- it's just a URL path segment, not a preset list. Send one real\nevent however is easiest for a first test, e.g.:\n\n```bash\ncurl -X POST http://localhost:8484/webhooks/capture/sentry \\\n -H \"Content-Type: application/json\" \\\n -d '{\"issue\": {\"title\": \"NullPointerException in checkout\"}, \"level\": \"error\"}'\n```\n\nThis event isn't signed yet, so it's captured but marked **unauthenticated**\n-- visible for you to design against, but incapable of triggering anything\n(`docs/architecture/bindings.md`'s threat-model section explains why: an\nunauthenticated event can never match a binding, no matter how it looks).\n\n## 2. Inspect the captured payload\n\nOpen the dashboard's **Event inspector** (`/captures`). Your test event\nappears there with its real body -- this is schema-by-example: you're\nlooking at an actual payload from the actual sender, not guessing at a\nschema in advance.\n\n## 3. Map the fields you care about\n\nOpen **Field mappings** (`/mappings`) → **New mapping**. Pick your captured\nevent from the dropdown, then click through its payload tree on the right --\nevery key and value is clickable. Clicking a value binds it as a field\n(name, JSON path, and an inferred type you can change). Add whichever fields\nyour workflow needs (e.g. the issue title), mark the ones that must be\npresent as **Required**, hit **Preview** to confirm they resolve correctly\nagainst the real payload, then **Save**.\n\n## 4. Create the binding\n\nOpen **Playbook bindings** (`/bindings`) → **New binding**:\n\n- **Name** -- anything descriptive.\n- **Source** -- the same `\u003csource\u003e` segment from step 1 (e.g. `sentry`).\n- **Field mapping** -- the one you just saved.\n- **Workflow** -- which registered workflow should run on a match.\n- **Repo pin** (optional) -- leave both blank to use the single configured\n repo; set owner/repo explicitly if you run a multi-repo deployment.\n- **Shared secret** -- a real secret (16+ bytes), never `test1234`. The\n sender signs every request body with this via HMAC-SHA256 so archie can\n tell a genuine event from anyone who finds the URL.\n\nSave. The binding starts in **draft** -- nothing fires yet.\n\n## 5. Approve it\n\nA **draft** or freshly-**edited** binding needs an explicit human approval\nbefore it can ever match a live event -- this is deliberate, not a missing\nstep: an operator authoring a binding is a considered act, an event arriving\nis not, and nothing should self-arm (`docs/architecture/bindings.md`'s\nthreat-model section, point 2). Click **Approve**. Status becomes **armed**.\n\n## 6. Fire it for real\n\nSend the same kind of event again, this time **signed** with the binding's\nsecret:\n\n```bash\nBODY='{\"issue\": {\"title\": \"NullPointerException in checkout\"}, \"level\": \"error\"}'\nSECRET=\"the-secret-you-set-in-step-4\"\nSIG=$(printf '%s' \"$BODY\" | openssl dgst -sha256 -hmac \"$SECRET\" | sed 's/^.* //')\n\ncurl -X POST http://localhost:8484/webhooks/capture/sentry \\\n -H \"Content-Type: application/json\" \\\n -H \"X-Hub-Signature-256: sha256=$SIG\" \\\n -d \"$BODY\"\n```\n\nThis event is captured as **authenticated**, matched against your armed\nbinding, and dispatched into a new task through the exact same\ngate/worktree/sandbox pipeline every other task goes through -- see\n`ARCHITECTURE.md`'s Task Lifecycle section, \"Provenance by origin.\" Check\nthe **Tasks** page; a new task should appear, its body built from the fields\nyour mapping resolved.\n\n## Editing a live binding\n\nAny edit to an armed binding -- even a typo fix -- drops it back to\n**pending_approval** automatically. This can't be bypassed by editing; it\nhas to be re-approved. When editing, leave the **Shared secret** field blank\nto keep the existing secret; only fill it in if you're deliberately\nrotating it.\n"
|
||
},
|
||
{
|
||
"url": "/docs/prds/config-schema/",
|
||
"title": "Backend-owned configuration schema for the webui -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Backend-owned configuration schema for the webui -- decision\n\n**Status:** Approved\n**Date:** 2026-09-02\n**Beads issue:** `archie-core-b6ew` (epic), with children `.1`-`.5`\n\n## Decision\n\nThe dashboard's Configuration page keeps reading a secret-free,\nhand-picked view of `config.Config` -- that allowlist is a safety\nboundary and stays exactly as it is. What changes is who defines the\npage around it: `ui/src/settings/settings.js` and\n`config-row.js` hardcode every row's label, section, type, and\neditability by hand; after this, the backend attaches that metadata to\neach field it already allows through, and the frontend renders\nsections and fields generically from what the backend sends.\n\nThis is **not** a reflection-based schema over `config.Config`.\n`internal/config/config.go`'s own package doc states it is \"scheduled\nfor dissolution: its types and methods are to be reassigned to the\ndomains whose behaviour they describe\" -- building a generic\nschema-from-struct-tags mechanism over a struct on its way out would be\nbuilding on a foundation this repo has already decided to remove.\nInstead, every field already in `ConfigView` (`internal/webui/api_config.go`)\ngot there because someone deliberately vetted it as safe to show; the\nschema adds a hand-authored descriptor to that same deliberate act,\nnot a new mechanism that re-derives safety automatically.\n\n## Why now\n\nTask #6's investigation (`/9/26`) found the concrete failure this\nproduces: `Repo.AllowConcurrent` and `Repo.MaxRetries` already reach\n`GET /api/config` (`RepoView` in `api_config.go`), but the\nrepositories card renders a static read-only table that never shows\nthem, so the fix for \"why isn't a second agent running\" required an\nagent tracing Go source instead of a checkbox on the dashboard.\n`Repo.ReviewEnabled` is worse: it never reaches the API at all. Both\nare symptoms of the same root cause -- the frontend decides what a\nfield means and whether it's shown, and nothing keeps that in sync\nwith what the backend actually allows.\n\n## Shape\n\n```go\n// internal/webui/config_schema.go (new)\n\ntype ConfigFieldType string\n\nconst (\n FieldString ConfigFieldType = \"string\"\n FieldInt ConfigFieldType = \"int\"\n FieldBool ConfigFieldType = \"bool\"\n FieldDuration ConfigFieldType = \"duration\"\n FieldEnum ConfigFieldType = \"enum\"\n FieldStructured ConfigFieldType = \"structured\" // repositories, models, providers\n)\n\ntype ConfigField struct {\n Key string `json:\"key\"` // dotted path, matches UpdateConfig's key space\n Label string `json:\"label\"`\n Description string `json:\"description,omitempty\"`\n Type ConfigFieldType `json:\"type\"`\n Value any `json:\"value\"`\n Editable bool `json:\"editable\"`\n LockedReason string `json:\"locked_reason,omitempty\"`\n Overridden bool `json:\"overridden\"`\n Options []string `json:\"options,omitempty\"` // enum only\n RestartRequired bool `json:\"restart_required\"`\n}\n\ntype ConfigSection struct {\n ID string `json:\"id\"`\n Label string `json:\"label\"`\n Fields []ConfigField `json:\"fields\"`\n}\n```\n\n`handleConfig` builds `[]ConfigSection` from the existing per-field\nvalues it already assembles, plus `overlay.DeniedKeys` for\n`LockedReason` and the existing `Overridden` list -- both already\ncomputed just re-attached per field instead of returned as\nside-channel maps the frontend cross-references by key string.\n`Locked`/`Overridden`/`Reload`/`Provenance` stay as top-level\n`ConfigView` fields; they describe runtime state (what happened on\nthe last reload, which keys the overlay refuses), not the field\ncatalog, and duplicating that logic per-field would be the same drift\nrisk this change exists to remove.\n\n`FieldStructured` fields (repositories, models, providers) carry\n`Value` as structured JSON and are rendered by dedicated\neditors, not the generic renderer -- see \"What this does not do\".\n\n## Sections\n\nMatches the cards the settings page already has: `identity`,\n`repositories`, `models`, `providers`, `budgets`, `storage`,\n`containers`, `web`. No new sections; this is metadata on existing\nrows, not new configuration surface.\n\n## Frontend\n\n`config-row.js`'s `row(label, value, opts)` becomes the renderer's\nprivate implementation detail, called generically:\n\n```js\nfor (const section of schema.sections) {\n const rows = section.fields\n .filter(f =\u003e f.type !== \"structured\")\n .map(f =\u003e row(f.label, f.value, {\n key: f.key, type: f.type, options: f.options,\n locked: f.locked_reason, overridden: f.overridden,\n restartRequired: f.restart_required,\n }));\n // structured fields render through their dedicated editor (b6ew.4)\n}\n```\n\n`settings.js` stops knowing that `containers.max_concurrency` is an\n`int`, belongs in the \"Sandboxed containers\" card, or requires a\nrestart. It knows how to render a section and a field of a given type.\n\n## What this does not do\n\n- **Does not marshal `config.Config` wholesale or add reflection over\n it.** The allowlist boundary in `api_config.go` is unchanged; this\n adds a label next to a value that was already being shown.\n- **Does not make every backend field appear automatically.** A field\n still requires someone to write its descriptor, same as it requires\n someone to add it to `ConfigView` -- the fix is one\n source of truth instead of two, not zero review.\n- **Does not solve structured editing generically.** Repositories and\n maps get their own typed editor (`archie-core-b6ew.4`), not a\n generic \"array of objects\" renderer -- a repo's fields have their\n own validation shape (owner/name pair identity, gate command\n argv) that a generic structured-field renderer would either get\n wrong or reinvent per-type anyway.\n- **Does not change what `UpdateConfig`/`overlay.DeniedKeys` accept.**\n `Editable` on a field descriptor mirrors that existing policy; it\n does not grant new write access.\n\n## Acceptance criteria (epic-level; see child issues for per-task detail)\n\n1. Every field rendered as a hardcoded row in `settings.js`\n has a backend-authored descriptor with type, label, and section.\n2. `Repo.ReviewEnabled` reaches the API and the dashboard, alongside\n `AllowConcurrent` and `MaxRetries`.\n3. Adding a new scalar field to the schema requires one backend\n descriptor and zero frontend changes to appear correctly labeled,\n sectioned, and editable.\n4. `TestHandleConfigNeverLeaksSecrets`-style coverage extends to the\n schema response; no descriptor exists for a secret-bearing field.\n5. A schema-fixture-driven frontend test asserts the generic renderer\n needs no field-specific branch to render every scalar field type.\n\n## Work breakdown\n\n- `archie-core-b6ew.1` -- descriptor types + hand-authored section/field\n metadata for every existing `ConfigView` field.\n- `archie-core-b6ew.2` -- `handleConfig` returns the schema-driven\n shape (depends on `.1`).\n- `archie-core-b6ew.3` -- generic frontend renderer for scalar/bool/\n int/enum/duration fields (depends on `.2`).\n- `archie-core-b6ew.4` -- structured editors: repositories (incl. the\n `ReviewEnabled` gap), models, providers (depends on `.2`).\n- `archie-core-b6ew.5` -- contract tests for metadata completeness,\n secret-safety, and generic-renderer coverage (depends on `.3`, `.4`).\n"
|
||
},
|
||
{
|
||
"url": "/docs/prds/memory-engine-unification/",
|
||
"title": "Memory engine unification: four scopes, CRUD, and one write path -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Memory engine unification: four scopes, CRUD, and one write path -- decision\n\n**Status:** Approved\n**Date:** 2026-09-16\n**Findings:** `memory-4` and its dependants in the adversarial sweep\n(`.local/issue-tracker`); the real defect is the split described below, for\nwhich `memory-4` is a symptom\n**Supersedes:** the draft `memory-turn-context.md` (its synchronous-read\nargument survives and is reused; its `Manager` method and consumer-owned port do\nnot)\n**Settles:** four open items in `docs/architecture/migration-decisions.md#5`\n(listed at the end)\n\n## Problem\n\nThere are **two live memory stores for one assistant**, split by which component\nwrites them, and the chat turn reads neither.\n\n| Family | Written by | Root | Format |\n|---|---|---|---|\n| `internal/memory` (`Manager`, `builtin.Provider`) | the chat agent's `memory_edit` tool via `memorytoolprovider` (`bootstrap.go`) | `\u003cworkDir\u003e/memory` | `MEMORY.md` / `USER.md` |\n| `internal/domain/memory` + `internal/infrastructure/memory` | the session curator (`sessioncurator.go`) | `\u003cworkDir\u003e/memory-engine` | identity-keyed blocks |\n\nThey share no code, no format and no scope vocabulary. Neither reaches a prompt:\n`Manager.SystemPromptBlock` (`internal/memory/manager.go`) has no production\ncaller anywhere, so the agent can write memory through a tool and never read it\nback.\n\nThree further defects fall out of the same split:\n\n1. **The engine's contract is not CRUD.** `Store` is `Write`/`Query`/`List`/\n `Forget` — create, read, delete, and **no Update**. The retired file provider\n *does* have `handleAdd`/`handleReplace`/`handleRemove`, so the family being\n deleted is operationally richer than its replacement.\n2. **There is no scope model.** The contract carries one opaque `Identity\n string`, and the curator fills it with a **session id** (`sessioncurator.go`)\n — which is none of the four scopes `docs/architecture/agent-system.md`\n requires, and stops meaning anything when the session ends.\n3. **The chat turn has no user identity on half the channels.** Telegram and\n email populate `SenderID` with a person; the dashboard populates it with\n nothing, and a webhook populates it with nothing either — a route path names\n a source, not a person, so `webhook.go` leaves `SenderID` empty and carries\n the route in `Inbound.BudgetKey` for rate limiting. `sender_id` is persisted\n (`session_store_sqlite.go`), so whatever a channel puts there reaches the\n consumers that read it.\n\n`agent-system.md` says the write-scope selection and cross-scope sharing\nrules \"require a focused memory design decision\". This is that decision.\n\n## Decision\n\n### 1. One engine, addressed by four typed scopes\n\n`internal/domain/memory` is the only memory engine. `Observation.Identity`,\n`Query.Identity`, `Record.Identity` and `List(identity)` are deleted, not\nextended: a field whose meaning depends on the caller cannot carry a scope.\n\n```go\ntype ScopeKind string\n\nconst (\n ScopeGlobal ScopeKind = \"global\" // shared across the application\n ScopeAgent ScopeKind = \"agent\" // private to one Agent\n ScopeUser ScopeKind = \"user\" // one user, across their Agents\n ScopeAgentUser ScopeKind = \"agent-user\" // one Agent's relationship with one user\n)\n\ntype Scope struct {\n Kind ScopeKind\n Agent AgentID // required for ScopeAgent, ScopeAgentUser\n User IdentityID // required for ScopeUser, ScopeAgentUser\n}\n\nfunc (s Scope) Validate() error // rejects a component present where it is not required\nfunc (s Scope) Key() string // canonical, collision-free storage key\n```\n\n`AgentID` and `IdentityID` are opaque. This package never resolves or\ncanonicalises them.\n\nA turn's resolved identity is one value:\n\n```go\ntype Subject struct {\n AgentID AgentID\n UserID IdentityID // \"\" when the channel carries no per-person identity\n}\n\nfunc (s Subject) Scopes() []Scope // readable, most specific first\nfunc (s Subject) WritableScopes() []Scope // never includes ScopeGlobal\n```\n\n**Access control is naming a scope, and nothing else.** The read path names\nexactly the scopes of the resolved `Subject`; the engine never unions beyond what\nit is handed and applies no policy of its own. An engine that enforced policy\nwould need to know about channels, sessions and bindings, and every backend would\nreimplement it. \"Agent-wide is private to its owning Agent by default\"\n(`agent-system.md`) falls out mechanically, because `ScopeAgent{a}` and\n`ScopeAgent{b}` are different storage keys.\n\n`ScopeGlobal` is never model-writable. Writing it is an operator action; nothing\nproduces one yet, but the contract models it because the read set includes it.\n\n### 2. CRUD, with revisions retained\n\n```go\ntype Store interface {\n Create(ctx context.Context, in NewRecord) (Record, error)\n Get(ctx context.Context, scope Scope, id RecordID) (Record, error) // ErrNotFound\n Query(ctx context.Context, q Query) ([]Record, error) // heads only\n List(ctx context.Context, scope Scope) ([]Record, error) // heads only\n Update(ctx context.Context, in RecordUpdate) (Record, error) // ErrNotFound, ErrStaleRevision\n Forget(ctx context.Context, scope Scope, id RecordID) error // idempotent\n Revisions(ctx context.Context, scope Scope, id RecordID) ([]Revision, error)\n}\n```\n\n`Get` and `Forget` take a scope. The current `Forget` scans every identity's\nstore to find a marker (`builtin_engine.go`) purely because the contract\ngave it nothing better.\n\n`Record` carries what `agent-system.md` requires be retained: `Scope`,\n`Kind`, `Content`, `Revision`, `Author`, `OriginUser`, `Source`, `CreatedAt`,\n`UpdatedAt`. `Metadata` is dropped — nothing sets it and the parser never\nrestores it, so it is a field that silently does not round-trip.\n\n`Update` supersedes rather than overwrites. The superseded state is appended to\nthe scope's `HISTORY.md` **before** the live block is replaced, so a crash\nbetween the two leaves an extra history entry and an unchanged record —\nrecoverable — never a record whose previous state was lost. `Update` carries no\n`Scope`: a record may never move between scopes, and the API makes that\nunrepresentable rather than validating it. `Expected` optionally names the\nrevision being replaced and returns `ErrStaleRevision` on mismatch, which matters\nbecause two processes (`archied` and `archie-gateway`) can hold the same scope's\nfile.\n\n`Forget` records a deleted revision before removing the live block, so a record's\nprovenance outlives its content.\n\n### 3. The caller resolves identity; the engine never does\n\nResolution happens in the gateway turn path, where the native sender id and the\nsession are both in hand, and it is supplied **per channel by the composition**:\n\n```go\n// TurnRunnerConfig\n// UserIdentity resolves the initiating user's identity for one inbound message.\n// false means the channel carries no per-person identity, and the turn gets\n// global and agent scopes only. Nil means the same.\nUserIdentity func(msg messaging.Message) (IdentityID, bool)\n```\n\nThis must be injectable rather than a blanket `channel + \":\" + SenderID`\nconcatenation, because a channel's sender id is not necessarily a person: a\nwebhook's route path names a source, and treating it as an identity would give\na URL one -- which is why `webhook.go` leaves `SenderID` empty and carries the\nroute in `Inbound.BudgetKey` instead. Telegram and email supply their sender\nid; the dashboard and webhook resolve nothing.\n\n**Fail closed.** When `UserID` is empty the turn proceeds with global and agent\nscopes only, and a write targeting `user` or `agent-user` fails with a clear\nerror. `agent-system.md` forbids showing one user's memory to another, so\n\"we do not know who this is\" must mean \"show none of it\", never \"fall back to a\nwider scope\".\n\n### 4. The chat read path\n\nSynchronous and total. It runs inside `prepareTurn` (`turn.go`) before the\nprompt is built, never returns an error to the caller, and degrades to an empty\nblock on any engine failure or provider panic. A fire-and-forget hook would race\nthe prompt build in the same turn, which is why this is not a lifecycle hook.\n\nThe read is a scope-only `Query` over the subject's scopes — **not** ranked\nsearch, which remains open in `migration-decisions.md#5`. A ranked read on the\nhot path with no ranking implementation would be a lie.\n\n`SystemPromptConfig` gains a `Memory` field, rendered into\n`templates/archie.md.tpl` between the `\u003ctools\u003e` and `\u003cenv\u003e` blocks:\n\n```\n\u003cmemory purpose=\"durable_context\" trust=\"data\"\u003e\n```\n\n`trust=\"data\"` and inclusion in the precedence clause follow the `\u003ctools\u003e`/`\u003cenv\u003e`\nprecedent exactly: memory content is model-written from untrusted input and must\nbe framed as data, never instructions. The rendered block carries each record's\nshort id, because the write path addresses records by id — one source of truth\nfor both directions.\n\nBoth a per-scope record limit and a byte cap on the rendered block are named\nconstants. The cap is about size, not latency; a local read needs no timeout.\n\n**Prerequisite:** `setupGateways` runs before `setupMemoryAll`\n(`main.go` vs `:428`), and `startGatewayRuntime` before `setupMemory`\n(`gateway.go` vs `:115`), so the engine is nil at every turn-runner\nconstruction site. The ordering is the root-cause fix and nothing in either\nmemory step reads what those later steps produce.\n\n### 5. The chat write path\n\nThe memory tool is built **per turn**, not registered once at boot, because it\nmust write into a scope derived from the turn's resolved subject:\n\n| action | engine call |\n|---|---|\n| `create` | `Create` |\n| `update` | `Update` (by id) |\n| `delete` | `Forget` (by id) |\n| `list` | `List` |\n\n**The model chooses the scope kind; it never chooses the ids.** Agent and user\nids come from the turn's resolution and there is no schema field to put anything\nelse in — even a model that emits another user's id has nowhere to write it. That\nis the root-cause form of `agent-system.md`, matching this system's other\nnon-waivable boundaries (the model never runs git; gates cannot be waived by the\nagent). `global` is not offered at all.\n\nThe tool takes a record **id** where the retired provider took a content\nsubstring (`handleReplace`/`handleRemove`). The substring addressing existed\nbecause a markdown file was hand-edited and the model had no stable handle on a\nblock; with ids rendered into the prompt it disappears, and one addressing scheme\nreplaces two.\n\n### 6. The curator's write\n\n`sessioncurator.reviewOne` stops writing `Identity: sessionID` and writes\n`ScopeAgentUser{AgentID: \u003csession's agent\u003e, UserID: \u003cparticipant\u003e}`. Relationship\nscope is the only defensible home: agent-wide is explicitly unsafe for user facts\nand would leak user A's extracted facts to user B through the same agent;\nuser-wide would pool facts across agents; and a session id cannot survive the\nsession ending, which is the whole requirement.\n\nThis needs data the tree does not persist: the SQLite `messages` table\ngains `sender_id`, and `curator.SessionSummary` / `ConversationMessage` gain the\nagent and participant. The participant is derived from the session's own\nuser-role messages: exactly one distinct non-empty sender writes; zero (a\ndashboard or webhook session) or several (a group chat) writes nothing and\nrecords why. Ambiguity fails closed rather than guessing.\n\nThe payoff is the point of the exercise: chat-written and curator-derived memory\nnow occupy the same scopes, so a future turn's prompt contains both, and a\ncurator's bad note becomes id-addressable and therefore correctable by the agent.\n\n### 7. The content scanner moves to the engine\n\n`internal/memory/scanner.go` and its installation at `bootstrap.go` are what\nmake `configuration.md`'s \"Memory safety scanner\" row describe a control that\nactually runs. Deleting the package would silently remove prompt-injection\nscanning on memory writes. The scanner moves into the engine and is applied in\n`Create` and `Update`, which is the single choke point every producer crosses —\nso the curator's model-extracted content is covered too, not just the chat tool's.\n\n### 8. `internal/memory` is deleted\n\nNot shimmed, not partially, not behind a flag: `manager.go`, `provider.go`,\n`lifecycle.go`, `sync.go`, `scanner.go` (moved), `schema.go`, `backup.go`,\n`config.go`, `builtin/provider.go`, and their tests. Its markdown store\n(`builtin/store.go`) is the one survivor, relocated to\n`internal/infrastructure/memory/builtin/` — `organisation.md`'s \"a package owns\nits on-disk format end-to-end\" makes infrastructure its correct home, and the\nengine already parses that format.\n\nAlso deleted: `internal/tools/provider/memory`'s manager wrapping (rewired to the\nengine), `config.Memory.{Provider,ProviderConfig}` and the\n`validateMemory` rejection that exists only to complain about `Provider`\n(`validate.go`), and `webui.MemoryStatus`/`MemoryProviderHandle` — never\nimplemented in production, so `/api/memory` reports empty forever unless it is\nrepointed at the engine registry.\n\nExisting data at `\u003cworkDir\u003e/memory` is left on disk, unreferenced. No migration.\n`\u003cworkDir\u003e/memory-engine` is likewise orphaned: its directories were keyed by\n`sha256(sessionID)` and no scope key can hash to one.\n\n## What we deliberately do NOT do\n\n- **No cross-scope sharing or grants in this wave.** Promoting a record between\n scopes is a future explicit command, and `Update` cannot change scope, so it\n cannot happen by accident.\n- **No canonical user identity.** `SenderID` is channel-native, so one person on\n Telegram and email is two identities. Unifying needs an Identity aggregate that\n does not exist (`internal/domain` has no identity package and `IdentityID`\n appears nowhere in Go). The contract does not change when it lands.\n- **No ranked retrieval.** `Query.Text` stays, unused by the prompt path.\n- **No data migration**, per the decision above.\n- **No prompt-memory timeout mechanism.** The read is bounded by construction.\n- **No `Metadata` field.** It is write-only; a caller needing it later\n needs one field, not a speculative contract.\n\n## Acceptance criteria\n\n1. A record created through the memory tool in turn 1 appears in turn 2's prompt,\n asserted on content the test planted, with its id.\n2. The same record is **absent** from a different user's prompt on the same\n channel and agent. Isolation asserted end to end, not at the engine boundary.\n3. An agent-scope record is visible to both of that agent's users; a global\n record is visible to a second agent; an agent-user record is not.\n4. A curator pass over a session writes a record a later turn reads, and which\n survives that session being deleted.\n5. `create` → prompt shows it; `update` → prompt shows new content and\n `Revisions` returns the prior content with author and time; `delete` → gone\n from prompt and `List`. Round-trip is asserted as `parse(render(r)) == r`,\n never against a fixture string.\n6. A panicking engine and a failing engine both complete the turn with no memory\n block.\n7. A channel with no resolvable user renders only global and agent scopes, and\n the tool refuses `user`/`agent-user` with a reason. A webhook route path never\n becomes a user identity.\n8. A write that would exceed the file cap fails loudly rather than silently\n dropping content.\n9. `internal/memory` is gone, `task check` is green, and no document describes a\n manager that no longer exists.\n\nEvery criterion asserts behaviour or planted content. None asserts on the text of\na hand-written file, and none pins that a method was called. In particular none\nasserts the composition calls setup in a given order: if the ordering were\nswapped back, criteria 1 and 4 would fail, and that is the assertion.\n\n## Sequencing\n\nEach step is independently verifiable and lands green on its own.\n\n1. **Persist the participant.** `sender_id` column on `messages`,\n insert/select/migration, `curator.SessionSummary.AgentID` and\n `ConversationMessage.SenderID`. Touches no memory contract and is shippable\n alone — nothing consumes it yet.\n2. **Scope, CRUD and revisions.** The `internal/domain/memory` contract, the\n builtin engine with `HISTORY.md`, the store relocation, the scanner move, and\n the curator rewritten onto agent-user scope (it is the engine's only live\n consumer, so it belongs in this slice). *Depends on 1 for the curator half.*\n3. **Chat read path.** The `Memory` seam, the template block, subject resolution,\n and the composition-ordering fix. *Depends on 2.*\n4. **Chat write path.** The tool adapter onto CRUD and the turn's subject on the\n context. *Depends on 3, because a write is only observable through the read.*\n5. **Delete `internal/memory` and the dead config/dashboard surface**, and\n rewrite `ARCHITECTURE.md`'s \"Memory engine family\" section to describe the\n engine family rather than a manager it deleted. *Depends on 3 and 4.*\n\nSteps 3 and 4 are separable from each other; step 1 is separable from everything.\n\n## Settles these open items in `migration-decisions.md` §5\n\n| §5 open item | Settled by |\n|---|---|\n| authoritative memory record and revision model | §1, §2 — the live marker becomes JSON, since the ratified `\u003c!--mem:\u003culid\u003e--\u003e` cannot carry provenance; superseded states live in a per-scope `HISTORY.md` |\n| provider and infrastructure boundaries | §2 — contract in `internal/domain/memory`, builtin format in `internal/infrastructure/memory/builtin` |\n| retrieval and access enforcement | §1 — caller-named scope sets, no engine-side policy |\n| provenance representation | §2 |\n\n## Not determined\n\n- **Group chats produce no derived memory**, because the curator fails closed on\n mixed-sender sessions. Whether extraction should instead become\n per-participant is a product call.\n- **Two processes, one scope file.** After step 3 both `archied` and\n `archie-gateway` hold an engine over the same root, and the engine caches a\n store per scope and rewrites the whole file, so concurrent writers on one scope\n can lose a block. `Update.Expected` makes that detectable for one record; it\n does not make the file safe. A reload-before-mutate or advisory lock is the fix\n and is not designed here.\n- **Unbounded history.** `HISTORY.md` grows without limit; no compaction policy.\n- **Size bounds.** `defaultMaxFileBytes` is now per scope rather than per\n identity; whether that is right is unexamined.\n- **Whether the dashboard should carry a user identity.** It has one bearer\n token, not users. Until it has users, web turns are correctly limited to global\n and agent scope.\n"
|
||
},
|
||
{
|
||
"url": "/docs/prds/messaging-service-boundary/",
|
||
"title": "Messaging Service boundary and transport design",
|
||
"section": "PRDs",
|
||
"body": "# Messaging Service boundary and transport design\n\n**Status:** Approved\n**Revision:** 1 \n**Date:** 2026-09-19 \n**Beads issue:** `archie-core-8cda.6.1` \n**Parent:** [`service-decomposition.md`](/docs/prds/service-decomposition/) \n**Prerequisites:** Gateway contract extraction (Phase 1), State Store extraction (Phase 2), UI extraction (Phase 3)\n\nThis document is the authoritative specification for Phase 4: Messaging Service extraction. It fixes the process boundary, configuration ownership, client contracts, security posture, readiness behavior, channel invariants, and deletion criteria required by `archie-core-8cda.6` and its child tasks.\n\n## Decision\n\nThe Messaging Service (`cmd/archie-messaging`, composed via `internal/app/archiemessaging`) owns external channel connections (Telegram, email, webhook) and dispatches inbound messages and commands to the standalone Gateway Service over gRPC via `messaging.ChatContract`.\n\nIt is a peer service to the UI Service and the daemon:\n- It serves external chat participants across configured communication platforms.\n- It consumes Gateway's `messaging.ChatContract` exclusively for chat turn execution, streaming, session queries, cancellation, persona selection, and task control actions.\n- It holds no direct database connections (`*store.Store` or SQLite), owns no LLM execution or model provider credentials, and has no dependency on the daemon runtime or the UI HTTP server.\n\n## Boundary and Ownership\n\n| Concern | Messaging Service owns | Messaging Service consumes | Messaging Service must not own or link |\n|---|---|---|---|\n| **Channel transports** | Telegram bot long-polling, Email IMAP/SMTP loops, Inbound Webhook HTTP listener | Gateway `ChatContract` | Direct LLM providers, model runtimes, tokenizer libraries |\n| **Turn dispatch** | Inbound message packaging, typing indicators, live streaming updates | `ChatContract.Route`, `ChatContract.Stream` | `gateway.Router`, `runtime.Runtime`, agent prompt builders |\n| **Sessions \u0026 History** | Platform-to-session key mapping, conversation thread ID translation | `ChatContract.GetSession`, `RecentMessages`, `RecentTurns` | `*store.Store`, `SessionStore` SQLite handles, direct database files |\n| **Control \u0026 Actions** | Platform command parsing (`/status`, `/tasks`, `/cancel`, `/stop`, `/persona`) | `ChatContract.Snapshot`, `ApplyTaskAction`, `Cancel`, `SetPersona` | `daemon.Daemon`, workflow execution engine, worktree operations |\n| **Task Approvals** | Channel interactive keyboards, approval callbacks | `ChatContract.ApplyTaskAction` | Direct task status mutations, forge comment posters |\n| **Configuration** | Platform bot tokens, allowed sender/user IDs, webhook listen ports | `[chat.*]` channel configs, Gateway target/token | `config.Holder`, forge tokens, repo configs, agent container specs |\n| **Readiness** | Messaging process liveness, channel connectivity, Gateway probe | Gateway `ChatContract.Snapshot` | Internal daemon subsystem probes |\n\n## Target Service Shape\n\n1. **Binary entry point**: `cmd/archie-messaging/main.go` parses CLI flags (`-config`, `-config-overlay`, `-gateway-target`, `-gateway-token`) and hands execution to `internal/app/archiemessaging.Run`.\n2. **Application composition**: `internal/app/archiemessaging/` wires:\n - Config loading scoped strictly to `[chat.*]` and `[services.gateway]`.\n - Secret resolution for channel credentials (e.g. Telegram bot token).\n - Gateway gRPC client (`internal/infrastructure/gatewayrpc.Client`).\n - Channel lifecycle managers (`internal/channels/status`).\n - Channel instances (`internal/channels/telegram`, `email`, `webhook`).\n3. **Turn and Stream Execution**:\n - Channels adapt inbound platform messages to `messaging.Inbound`.\n - Streaming channels (Telegram) consume `ChatContract.Stream(ctx, inbound)` which returns `\u003c-chan messaging.ChatEvent`, mapping `started`, `delta`, `tool`, `media`, `done`, and `error` events directly to live message frames.\n - Non-streaming channels (email, synchronous webhooks) consume `ChatContract.Route(ctx, inbound)`.\n\n## Load-Bearing Channel Invariants\n\nThe Messaging Service preserves existing invariants defined in `AGENTS.md` and channel documentation:\n1. **Telegram long-polling**: Must call `dropPendingUpdates(ctx, b)` before `b.Start(ctx)` and on `/restart`.\n2. **Telegram command menus**: Must register command menus simultaneously to `default`, `all_private_chats`, and `all_group_chats` via `commands.go` to prevent menu shadowing.\n3. **No UI detour**: Turns and task actions route directly to Gateway's `ChatContract`; no HTTP request may ever be sent to `archie-ui` or `internal/webui`.\n4. **Sender Confinement**: Telegram sender checks against `allowed_user_ids` must continue to fail closed before initiating any turn dispatch.\n\n## Configuration Ownership\n\n`cmd/archie-messaging` reads only:\n- `[services.gateway].target` and `[services.gateway].target_token` (or CLI flags).\n- `[chat.telegram]` (token, allowed user IDs).\n- `[chat.email]` (IMAP/SMTP host, ports, credentials).\n- `[chat.webhook]` (listen address, secret).\n- `[secrets]` (for resolving token secret references).\n\nIt does not read or decode `[forge]`, `[repos]`, `[models]`, `[providers]`, `[runners]`, `[containers]`, or `[nats]`.\n\n## Deletion Gate and Verification\n\nThe extraction is guarded by `cmd/archie-messaging/architecture_test.go`, verifying that `go list -deps ./cmd/archie-messaging` links zero banned runtime packages:\n- `modernc.org/sqlite` and `internal/store` (State Store)\n- `internal/daemon`, `internal/app/archied`, `internal/container`, `internal/worktree` (Daemon runtime)\n- `internal/domain/workflow`, `internal/agentexec`, `internal/taskrun` (Workflow engine)\n- `internal/forge`, `internal/forgerpc` (Forge client)\n- `internal/gateway` runtime (only `internal/domain/messaging` and `internal/infrastructure/gatewayrpc` are permitted)\n- `internal/webui` (Dashboard)\n"
|
||
},
|
||
{
|
||
"url": "/docs/prds/service-decomposition/",
|
||
"title": "Service decomposition -- decision",
|
||
"section": "PRDs",
|
||
"body": "# Service decomposition -- decision\n\n**Status:** Draft\n**Date:** 2026-09-05\n**Beads epic:** archie-core-8cda\n**Prior art:** `docs/inspiration/2026-09-05-awx-service-decomposition-and-ansible-task-model.md`,\n`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`,\n`docs/inspiration/service-discovery-research-2026-09-05.md`,\n`/work/artefacts/archie-core-current-state-archify.html`,\n`/work/artefacts/archie-core-ideal-state-archify.html`\n\nThe Phase 3 UI boundary and configuration decision is ratified in\n[`ui-service-boundary.md`](/docs/prds/ui-service-boundary/). That focused document is\nthe implementation authority for `archie-core-8cda.5.2`; this parent remains\nthe broader service-distribution proposal.\n\n## Decision\n\narchie-core moves from one Go binary (`archied`) hosting every domain\nin-process to a set of independently deployable services, each exposing\nexactly one typed contract. Consumers depend on the contract; they never\nreach into another service's internal types, session state, or config.\ngRPC is the synchronous transport; NATS remains for asynchronous fan-out\n(status, telemetry) rather than being replaced.\n\nThis is a **distribution decision**, not a rewrite of Archie's domain\nmodel. The six-domain split already approved in\n`docs/architecture/index.md` and `docs/architecture/organisation.md`\n(Identity, Agent, Messaging, Workflow, Work Intake, Plugin) is unchanged;\nthis PRD is about how the *processes* running that domain logic get\npackaged, contracted, and discovered -- not about redrawing domain\nboundaries.\n\n### Target services\n\n| Service | Contract it owns | Existing code it replaces/wraps |\n|---|---|---|\n| UI Service | thin client only; talks to Gateway and State Store contracts | `internal/webui` (SPA hosting stays; API surface becomes typed contract clients, not a peer with direct access) |\n| Gateway Service | routing / turn / session contract | `internal/gateway` |\n| Messaging Service | channel-plugin contract (Telegram, email, Discord, Teams, ...) | `internal/channels` |\n| Curator Service | curator contract; **optional**, installs as a Gateway plugin | `internal/domain/curator` |\n| Scheduler Service | cron/ticker contract | `internal/domain/scheduling` |\n| Execution Environment | execution-dispatch contract | `internal/domain/workflow`, task dispatch paths |\n| Runner | isolated worktree/container execution | `internal/container`, `internal/agentexec`, `internal/worktree` |\n| State Store | state-access contract | `internal/store` |\n\n### Discovery\n\nPer `docs/inspiration/service-discovery-research-2026-09-05.md`:\n\n- **Primary:** Kubernetes-native discovery (Service + headless-Service/DNS\n + EndpointSlice, served by CoreDNS), delivered as a Helm chart + Operator,\n on k3s for single-host installs and a real cluster for multi-host or a\n hosted offering.\n- **Fallback:** NATS-based discovery (micro `$SRV` announce + JetStream KV\n registry) for no-K8s single-host installs, with a custom gRPC resolver\n reading `host:port` from the KV bucket.\n- Both implementations sit behind one internal interface:\n\n ```go\n type ServiceRegistry interface {\n Resolve(service string) []Endpoint\n Watch(service string) \u003c-chan Event // join/leave\n }\n ```\n\n- An optional service (Curator; others later) that is not installed\n must resolve to a distinct `NotInstalled` outcome, never a `Down`/\n `Unhealthy` one. Under K8s this falls out of DNS returning no record;\n under the NATS fallback it requires an explicit \"installed\" marker in the\n KV registry, not just absence-implies-down.\n\n### Distribution model this supports\n\nBoth self-hosted (customer runs the Helm chart or the NATS-fallback\nbinaries themselves) and a managed/hosted offering (we run the same Helm\nchart) are in scope -- this is the RHEL / Ansible Automation Platform\nmodel: one architecture, sold both ways. This PRD does not decide the\nbusiness model; it decides that the architecture must not foreclose\neither shape.\n\n## Why this shape\n\n`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`\nestablished that Archie's existing domain split already does what AWX's\nservice-decomposition post argues for -- boundaries exist. What didn't\nexist was deployability: `internal/webui` was not a passive frontend. The\nre-baselined coupling inventory is below, and the ratified boundary is\n[`ui-service-boundary.md`](/docs/prds/ui-service-boundary/).\n\n- Telegram task actions now call Gateway's `ChatContract` directly through\n `chatTaskActorAdapter`; the old Telegram-to-webui HTTP detour is historical.\n- `internal/webui/api_chat.go` now holds `gateway.ChatContract`; direct\n `SessionStore`, `Router`, and `TurnHistory` access is historical.\n- The third coupling is closed. UI extraction shipped (`archie-core-8cda.5`,\n closed 2026-09-11): `cmd/archie-ui` serves the dashboard against remote\n Gateway and State Store contracts, holds no `config.Holder`, and relinks no\n daemon runtime (`cmd/archie-ui/architecture_test.go` enforces that on every\n `task check`). The `b.web.Cfg = b.d.Cfg` alias left in `buildDaemon` is now\n inside the daemon's own process -- it renders the published `ConfigView`\n snapshot and holds the readiness probes -- so it no longer crosses a service\n boundary. That cleanup is `archie-core-ml30` (P2); the UI boundary is\n ratified in [`ui-service-boundary.md`](/docs/prds/ui-service-boundary/).\n\n## Non-goals\n\n- Redesigning the domain model (Identity/Agent/Messaging/Workflow/Work\n Intake/Plugin boundaries are settled elsewhere and unchanged).\n- Replacing NATS. It remains the async transport; this PRD only adds gRPC\n for synchronous request/response and (optionally) K8s-native discovery.\n- Deciding the business model (self-hosted vs. managed vs. both). Kept\n open deliberately; the architecture must serve either.\n- Config relocation (`config.toml` -\u003e ConfigMaps/Secrets) is real work\n implied by a K8s-native deployment but is its own decision, not decided\n here. Flagged in the discovery research as a phased concern.\n- Any concrete adoption of Pi (or another external agent harness) as an\n execution runner. Raised in conversation alongside this decomposition,\n but it is a separate decision (agent execution as a strict data\n boundary, `ARCHITECTURE.md`) and does not gate or get gated by this PRD.\n\n## Open questions\n\nAll ten are resolved; each carries its resolution inline. Three phases have\nsince shipped against them (Gateway, State Store, UI), so the framing here is\nhistorical record rather than an outstanding gate.\n\n1. **Extraction order.** Which service gets pulled out of the monolith\n first? The webui entanglement findings above suggest Gateway Service's\n `ChatContract` should exist before UI Service is extracted, since UI is\n the most entangled consumer. Not yet sequenced.\n\n **RESOLVED.** Gateway Service first, then State Store,\n then UI, then Messaging, then Execution/Runner/Scheduler, then Curator\n (optional) — preceded by an in-process \"contract seams\" (modular\n monolith) phase. Confirms the hypothesis and strengthens it: the\n Gateway is already store-decoupled (it uses narrow contract adapters,\n not `internal/store`) and already owns an isolated session SQLite, so it\n is both the stickiest capability *and* the one with a ready seam and\n isolated data — extractable alone, first. Full phase list and reasoning:\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#1`.\n2. **Contract definition mechanism.** Protobuf/gRPC service definitions\n need a home (a new `proto/` or `contracts/` tree, versioning scheme,\n codegen wiring into `Taskfile.yml`). Not yet designed.\n\n **RESOLVED.** buf v2 driving `protoc-gen-go` +\n `protoc-gen-go-grpc` (grpc-go, not connect-go — connect-go's net/http\n client breaks the discovery research's gRPC resolver plumbing for K8s\n DNS and NATS-KV). One committed `proto/\u003cservice\u003e/v1/` tree, generated\n Go committed to `internal/contracts/\u003cservice\u003e/v1/`, directory\n versioning (breaking change -\u003e new `vN`, enforced by `buf breaking` in\n the gate), and `task proto:generate`/`proto:lint`/`proto:check` wired\n into `task check` and mirrored into `.github/workflows/deploy.yml`\n (CI does not run `task check`). Full toolchain comparison, layout\n rationale, and scratch-verified `buf`/`buf breaking`/`connect-go`\n results:\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#2`.\n3. **In-process vs. out-of-process during migration.** Whether services\n can run multiplexed inside one binary behind the same `ServiceRegistry`\n interface during a transition period (so extraction is incremental and\n testable) or whether each service must be a separate binary from day\n one. Strongly recommend the former for a solo maintainer, but not\n decided here.\n\n **RESOLVED.** Multiplexed in-process is the migration\n default -- this is `docs/architecture/organisation.md`'s standing\n \"Process boundaries\" rule, not a new lean. Mechanism: one wire-safe Go\n contract interface per service; a local adapter and a generated gRPC\n client adapter, both satisfying the same interface via a compile-time\n assertion (the `var _ workflow.Store = (*Client)(nil)` idiom;\n `store.WorkflowStore` is superseded by the consumer-owned\n `workflow.Store`); composition (not `ServiceRegistry`, which stays\n network-only) picks the adapter by the presence of a\n `[services.\u003cname\u003e].target` address -- empty = local (in-process) adapter,\n set = dial the remote gRPC client. This replaced the earlier\n `mode = \"inproc\" | \"remote\"` enum, which was dropped (see the gateway's\n `services.gateway.target`). The adapter is chosen by presence of the\n target; Phase 2 then deleted every local fallback, so remote is now the\n only mode and an empty `[services.state].target` is a startup error\n (docs/prds/state-store-contract.md §12 step 7). A\n per-contract conformance suite runs against both adapters over\n `bufconn`; the flip to a real process is a same-commit deletion of the\n in-process path, no dual-live window. Consequence: Gateway's Phase 0\n (`ChatContract` seam) ships in-process first, with the network protocol\n staying out of the production path until Phase 1 -- no amendment to Q1's\n phase list needed. Full mechanism and evidence:\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#3`\n (the State Store's concrete ratified config shape is\n `docs/prds/state-store-contract.md` §10).\n4. **State Store's contract shape.** Whether every service gets its own\n storage (true microservice isolation) or several services share the\n State Store service's contract (less isolation, less migration risk).\n Not yet decided.\n\n **RESOLVED.** One State Store service owns the existing\n single SQLite file behind narrow typed contracts (`TaskStore`,\n `CaptureStore`, `MappingStore`, `BindingStore`, `WorkflowStore`) over\n gRPC. *(Ratified 2026-09-06: the workflow contract is now the\n domain-owned `workflow.Store`, which supersedes\n `internal/store`'s `WorkflowStore`. See note below and\n `docs/prds/state-store-contract.md`.)*\n No physical database-per-service: the solo-maintainer / team-\n topology rule of thumb plus the low value of per-service autonomy for\n one person rule it out, and one store means no distributed-transactions\n problem. The Gateway keeps its own already-separate session SQLite.\n Concrete migration path (generalise `storerpc`, swap consumers one\n contract at a time, stand up the store service, split the file only on\n evidence):\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#4`.\n\n **RATIFIED (2026-09-06, beads archie-core-8cda.4.1).** The authoritative\n contract boundary \u0026 transport design for this is fixed in\n `docs/prds/state-store-contract.md` (rev. 2c): one `StateStore` gRPC service,\n 42 RPCs; **ownership split** — consumer-owned `workflow.Store` +\n `workflow.Task`/`Status`/`Source` in the workflow domain (dependency rules\n #2/#7), producer-owned daemon/webui store surfaces in `internal/store`;\n the `Task`/`Status`/`Source` + interface relocation is a Phase 2\n prerequisite pulled from migration-decisions §4 (`store.Task →\n WorkflowExecution`); `RecordDispatch` drops `*sql.Tx`; `Close` off-wire;\n structured gRPC error codes with sanitised details; explicit agent State\n Store gRPC handoff (`STATE_STORE_URL`/`STATE_STORE_TOKEN`); loopback+bearer\n transport-security boundary; presence-based `[services.state].target`\n defaulting local; migration order `workflow.Store -\u003e archie-state-store -\u003e\n Capture -\u003e Mapping -\u003e Binding -\u003e TaskStore`. `.4.2` and `.4.3` implement\n against that doc.\n5. **Helm chart / Operator ownership.** Net-new work with no existing\n analog in this repository (`deployments/` holds TOML profiles\n and a `docker-compose.yml`, not Kubernetes manifests). Scoping not yet\n started.\n\n **RESOLVED.** Three charts: an `archie-library` chart\n (shared Deployment/Service templates, named ports), an `archie` chart\n for the 7 core services, and an `archie-curator` chart + Curator\n Operator (CRD, controller, RBAC). The Curator reconciler is a\n level-triggered loop that fetches the CR, builds Deployment + Service,\n sets owner refs, applies idempotently (`CreateOrUpdate`), watches via\n `Owns()`, and reports a `Ready` status condition. Smallest real\n milestone (M1): install a Curator CR → Operator creates Deployment +\n Service → CoreDNS serves `curator` → Gateway's resolver flips it from\n `NotInstalled` to Installed with no redeploy; uninstall flips it back.\n Full scoping and first bead-issue list:\n `docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#5`.\n\n## Completion criteria\n\nNot applicable yet -- this PRD records the target shape and the discovery\ndecision. It does not authorize implementation. Before work begins: file\na beads epic, resolve the open questions above (at minimum #1-#3), and\nget explicit go-ahead given the scale of this change relative to\n`AGENTS.md`'s \"smallest workable change\" default for this solo project.\n"
|
||
},
|
||
{
|
||
"url": "/docs/prds/ui-service-boundary/",
|
||
"title": "UI Service boundary and configuration ownership",
|
||
"section": "PRDs",
|
||
"body": "# UI Service boundary and configuration ownership\n\n**Status:** Approved\n**Revision:** 1\n**Date:** 2026-09-07\n**Beads issue:** `archie-core-8cda.5.1`\n**Parent:** [`service-decomposition.md`](/docs/prds/service-decomposition/)\n**Prerequisites:** Gateway contract extraction and State Store contract ratification\n\nThis document is the authority for Phase 3 UI Service implementation. It fixes\nthe process boundary, configuration ownership, client contracts, security\nposture, readiness behavior, and migration evidence required by\n`archie-core-8cda.5.2` and later UI extraction work.\n\n## Decision\n\nThe UI Service owns the operator-facing HTTP surface and the compiled SPA. It\nserves `ui/dist` and exposes UI-specific JSON DTOs for dashboard views and\nactions. It does not own conversations, task state, workflow execution,\nchannel lifecycle, forge access, or the daemon's runtime configuration.\n\nThe UI Service consumes two typed remote capabilities:\n\n- Gateway's `gateway.ChatContract` through the versioned Gateway gRPC\n contract, for chat snapshots, messages, turns, streaming, persona changes,\n cancellation, and Gateway task actions.\n- State Store's narrow read/write contracts through the versioned State Store\n gRPC contract, for task views, events, captures, mappings, bindings, and\n configuration data explicitly projected for the dashboard.\n\nThe UI process may compose local adapters during the migration seam, but the\nadapter implements the same wire-safe contract as the remote client. The\nproduction extraction cutover removes the in-process adapter in the same\nchange; a dual-live UI authority is not permitted.\n\n## Boundary and ownership\n\n| Concern | UI Service owns | UI Service may consume | UI Service must not own or receive |\n|---|---|---|---|\n| HTTP and SPA | routes, auth middleware, UI DTOs, assets, HTTP lifecycle | contract results and health snapshots | domain entities or infrastructure handles |\n| Chat | request validation and response projection | Gateway `ChatContract` | `gateway.Router`, `SessionStore`, turn runner, model/provider runtime |\n| Task dashboard | query/action DTOs and HTTP status mapping | State Store narrow contracts and explicitly ratified task-action contract | `*store.Store`, SQL connections, workflow registry, daemon pointer |\n| Configuration page | secret-free view and update request validation | a UI configuration contract owned by the configuration/application boundary | `config.Config`, `config.Holder`, secret registry, reload implementation |\n| Readiness | UI process liveness and dependency readiness aggregation | bounded health results from Gateway and State Store | direct probing of private databases or providers |\n\nEvery value crossing the process boundary is a versioned DTO or scalar contract\nvalue. `internal/webui` may retain temporary Go-side adapters while the\nmigration is in progress, but handlers must map Gateway/State Store values to\nUI-owned response types at the HTTP edge. A constructor accepting a concrete\nGateway, State Store, daemon, or `config.Holder` is not an approved final\nboundary.\n\nThe existing UI routes remain behaviorally compatible during extraction,\nincluding task actions, SSE event delivery, config views, chat streaming,\ncapture intake, mappings, bindings, update reporting, and health endpoints.\nCompatibility preserves status codes, authentication requirements, redaction,\npagination/limits, event ordering, and failure degradation unless a separately\nratified contract changes them.\n\nThe route families have these target owners. This is the required inventory for\nthe implementation children; a route without an owner is not eligible for\ncutover.\n\n| Route family | Target owner consumed by UI | Migration status |\n|---|---|---|\n| `/api/chat/*` | Gateway `ChatContract` and its versioned client | contract seam exists; DTO and stream parity remain |\n| `/api/tasks/*`, `/api/workflows`, `/api/setup` | State Store read/action contracts plus an explicit execution/admin action contract | State Store adapter exists; UI-facing narrow surface remains |\n| `/events` | State Store `EventsSince`, polled by one UI-process pump feeding `Server.Broadcast` | RESOLVED (`archie-core-za9f`): see migration-decisions, \"Dashboard live event delivery\". No new RPC; replay cursor and drop-recovery are the existing `since` watermark in `sse.go` |\n| `/api/logs`, `/api/logs/stream` | daemon diagnostic feed, host-local | unresolved: `logging.Feed` is in-process on the daemon host and has no contract. The per-task log routes moved off this row: see `/api/tasks/*` above |\n| `/api/tasks/{id}/logs`, `/api/tasks/{id}/logs/download` | State Store task-log read contract (`TaskLogStore`: `ReadTaskLog` + `StreamTaskLogContent`), read by the UI process from the service that owns the state directory | RESOLVED (`archie-core-iaqx`): the UI process composes `TaskLogStore` from its own State Store client and serves the attempt log as JSON (`found` distinguishes \"no log for this attempt\" from \"this process cannot read logs\") and as a `Content-Disposition` attachment named `task-\u003cid\u003e-attempt-\u003cn\u003e.log`. The daemon's in-process registry is wired as the server-side reader in `internal/app/archied/state_store.go` only (`staterpc.Deps.TaskLogs`), so no process opens another process's log files |\n| `/api/captures`, `/api/mappings`, `/api/bindings` | State Store contracts, with webhook verification owned by Work Intake/Messaging | RESOLVED (cutover, `archie-core-8cda.5.4`): the UI process composes CaptureStore/MappingStore/BindingStore and mounts the `captureintake.Receiver` from its own State Store client; the daemon's binding-dispatch loop keeps consuming captures from the same store |\n| `/api/config` (read) | daemon-published `ConfigView` snapshot, read over the State Store contract | RESOLVED (`archie-core-ymut`): see migration-decisions, \"Dashboard configuration page\" |\n| `/api/config` (write), `/api/config/reset`, `/api/config/repos/*` | a configuration write contract; the daemon stays the policy owner | DESCOPED for Phase 3 (`archie-core-ymut`), NOT abandoned: 503 in the UI process, SPA hides the controls, editing is config.toml plus reload. Dashboard editing is a tracked feature (`archie-core-1786637498420-327`) and its transport is `archie-core-j28m`; the handlers and their func-field seams on `webui.Server` are kept for it |\n| `/api/channels`, reload, curators, memory, skills, version/update | owning capability contract or an explicitly removed route | owner and failure semantics remain to define |\n| `/`, `/healthz`, `/health`, `/health/detailed` | UI HTTP process; readiness consumes per-service health contracts | current HTTP behavior is the compatibility baseline |\n\n## Configuration ownership\n\nThe UI Service owns only process-local settings:\n\n- HTTP listen address and read/header/shutdown timeouts;\n- UI asset source or build mode;\n- UI authentication mode and token reference;\n- Gateway endpoint and credential reference;\n- State Store endpoint and credential reference;\n- readiness endpoint and dependency timeout policy.\n\nThese settings are decoded into a UI-owned input DTO by the UI composition root.\nThe DTO contains endpoint references and secret references, never resolved\nsecret values or complete daemon configuration. The UI process resolves its\nown credentials and builds its own clients. It does not read or mutate the\ndaemon's configuration file, runtime overlay, forge credentials, model catalog,\nworkflow routing, NATS settings, filesystem jail, or agent/container settings.\n\nThe dashboard configuration API is a separate capability. Its secret-free,\nallowlisted field projections remain governed by\n[`config-schema.md`](/docs/prds/config-schema/). During the transition the daemon may\ncontinue to perform validate-persist-publish for those updates through a\nnarrow application contract. The UI process must never implement that policy\nby holding or mutating a shared `config.Holder`.\n\nThe daemon's `config.Holder` sharing was migration residue: `buildDaemon`\nassigned `b.web.Cfg = b.d.Cfg`. Removed by `archie-core-ml30` -- the field no\nlonger exists, and boot owns the single Holder the daemon reads through.\n\nFor Phase 3, configuration ownership is split deliberately. The UI process\nowns its listener, browser-auth, asset, dependency-target, and readiness\nsettings. The daemon/application configuration owner remains authoritative for\nthe dashboard's `/api/config` read, validation, persistence, overlay, reload,\nand audit behavior. The UI does not become the writer merely because it\nrenders the configuration page.\n\nAmended 2026-09-11 by `archie-core-ml30`: the daemon's `config.Holder` sharing\nis gone. `webui.Server` has no holder field, so the sharing this paragraph\ndescribes cannot be reintroduced by wiring; the daemon owns one Holder and\npublishes a rendered projection.\n\nThere is no single narrow admin contract carrying both the read and the write\n(`archie-core-ymut`). The\nread crosses as a daemon-published `ConfigView` snapshot over the State Store\ncontract, and the write is descoped for Phase 3 rather than contracted, on the\nreasoning recorded in migration-decisions under \"Dashboard configuration\npage\". A future dedicated Configuration Service is the natural owner of a\nwrite contract, and designing one is that feature's work, not this migration's.\n\n## Listen, authentication, and readiness\n\nThe extracted UI follows the current web UI security posture:\n\n- loopback listeners may omit a dashboard token;\n- non-loopback listeners require a non-empty token and fail closed when it is\n absent;\n- `/healthz` and `/health` are unauthenticated liveness probes;\n- `/health/detailed` is authenticated and reports dependency readiness;\n- capture webhooks retain their separately ratified source authentication and\n rate-limit rules; they are not admitted by the dashboard token;\n- Gateway and State Store credentials are service-to-service credentials, never\n browser-visible dashboard tokens.\n\nThe UI is ready only when its HTTP server is serving and each mandatory remote\ncontract has passed its bounded readiness check. Each dependency must expose a\nversioned health result or health endpoint that the UI can consume; the UI must\nnot inspect another service's concrete manager, registry, or database. Optional\ndashboard sections may remain degraded with the current explicit\n`503`/unavailable behavior. A successful liveness response must never imply\ndependency readiness.\n\n## Migration sequence and evidence\n\nThe implementation must complete these gates in order:\n\n1. Define the UI input DTO, Gateway client surface, State Store client surface,\n and UI response DTOs. Add compile-time contract assertions and mapping\n tests. No handler may depend on a concrete remote implementation.\n2. Make the current in-process web UI consume the same narrow adapters and\n preserve the current route, auth, redaction, stream, SSE, and degradation\n behavior. Add a composition test proving the selected adapter is the one\n used by production wiring.\n3. Remove direct `internal/gateway` session/router/runtime access from web UI\n handlers. Remove direct `internal/store` implementation access, SQL access,\n daemon pointers, and broad runtime callback bundles from the UI constructor.\n4. Replace the shared `config.Holder` with the UI-owned configuration DTO and\n narrow configuration application contract. Prove reload/update atomicity\n remains owned by the configuration/application boundary.\n5. Add the standalone UI process and deployment wiring. It must obtain Gateway\n and State Store endpoints from its own configuration, authenticate each\n client independently, start liveness/readiness endpoints, and shut down\n clients and HTTP listeners in bounded order.\n6. Run dual-adapter contract tests and an HTTP parity suite covering every\n retained route. Test remote failures, missing credentials, malformed DTOs,\n stream cancellation, stale State Store data, and readiness degradation.\n7. Cut over the supported composition to the UI process. The old in-process\n `webui.Run` path is deleted in the same change; no fallback or second HTTP\n authority remains.\n\nThe deletion gate is objective: `go list -deps` from the UI process shows no\ndaemon, SQL/store implementation, workflow, forge, channel, model, or secret\nruntime package; production composition contains one UI HTTP listener; and\narchitecture tests reject `config.Holder`, concrete `*store.Store`, concrete\nGateway runtime types, and direct SQLite imports in the UI service.\n\nRollback is a deployment composition change that points the operator at the\nlast validated UI binary and restores the previous endpoint configuration. It\nmust not restore a second writer or reintroduce shared-holder mutation. UI\nstate is either remote contract state or existing durable state owned by its\ncurrent service; no UI-specific database migration is introduced by this\nboundary decision.\n\n## Current evidence and known delta\n\nCurrent code proves the following migration inputs:\n\n- `internal/app/archied/bootstrap.go:setupObservability` constructs\n `webui.Server` and wires store, channels, events, logs, and config-related\n callbacks.\n- `internal/app/archied/bootstrap.go:buildDaemon` replaces the initial holder\n with `b.d.Cfg`, explicitly sharing the daemon's live configuration.\n- `internal/webui/api_chat.go` calls the Gateway contract but still imports\n Gateway-owned request/value types; handlers must finish the DTO projection.\n- `internal/webui/server.go:Handler` owns the route set, SPA fallback, health\n split, capture bypass, and token wrapper.\n- `internal/app/archied/state_store.go:RunStateStore` owns the task SQLite\n file and serves the State Store gRPC contract; the UI must use that contract\n rather than opening the file.\n- `internal/app/archied/gateway.go:RunGateway` serves Gateway gRPC and owns\n Gateway session persistence; the UI must use `ChatContract` rather than\n Gateway internals.\n\nThe current code therefore satisfies the upstream prerequisites but does not\nyet satisfy this boundary. This document is the ratified target; the\nimplementation child must prove each migration gate before claiming Phase 3.\n\n### Phase 3 gates\n\nEvery gate below is proven by test, not by inspection:\n\n| Gate | Evidence |\n| --- | --- |\n| Deletion | `cmd/archie-ui/architecture_test.go` re-runs `go list -deps` on every `task check` and fails on any banned package, checked in both directions against its two exceptions |\n| Composition | `internal/app/archieui/TestComposeUIServerHoldsNoDaemonState` fails on a concrete `*store.Store` or a Gateway that is not the gRPC client. The holder clause it also carried is enforced by the type: `webui.Server` has no `config.Holder` field (`archie-core-ml30`) |\n| End-to-end | `cmd/archie-ui/main_test.go` builds the binary and drives it over HTTP against a live Gateway and State Store |\n| No dual authority | The dashboard HTTP listener is `internal/app/archieui/run.go` only; the daemon's two `http.Server`s are its own health endpoint and the forge webhook receiver |\n\nAcceptance criterion 1 was amended at the close of Phase 3 to name the UI\nprocess rather than the `internal/webui` package, because the daemon still held\na holder inside its own process to render the published `ConfigView` snapshot.\n`archie-core-ml30` removed that residue, so the criterion holds in its original\nform again: `internal/webui` receives no holder in any process.\n\n### Deletion gate: SATISFIED (archie-core-8cda.5.6, rev. 2)\n\nThe objective gate now holds. `go list -deps ./cmd/archie-ui` links **zero**\nbanned packages: no `modernc.org/sqlite`, no `internal/store`, `internal/gateway`,\n`internal/channels`, `internal/memory`, `internal/secret`, `internal/agentexec`,\n`internal/tools`, `internal/skill`, `internal/yaegiutil`, `internal/domain/curator`,\nor the workflow engine (`internal/domain/workflow`). How the runtime handles were\nsevered, each at the producer:\n\n- **Store contracts** moved to `internal/domain/storecontract` (producer-owned\n read contracts, state-store-contract rev. 2d); `internal/store` keeps aliases.\n- **Chat contracts** moved to `internal/domain/messaging`; `internal/gateway`\n keeps aliases. The UI process holds `ChatContract`, `SessionStore`, turn and\n stream types without linking the gateway runtime or its SQLite session store.\n- **Webui-owned views** replaced concrete runtime handles on `webui.Server`:\n channel lifecycle via `internal/channels/status`, memory via a `MemoryStatus`\n interface, curators via a `CuratorStatus` interface, and the skill catalogue\n via a `SkillCatalog` interface — each supplied by a daemon-side adapter at\n bootstrap, each degrading to an empty page when unwired.\n- **`config.SecretRef`** moved into `internal/config` (it is config vocabulary);\n `internal/secret` keeps an alias and resolves through `Registry.Resolve`.\n- **Task vocabulary** (`Task`, `Store`, `Definition`, status constants) split\n into `internal/domain/workflow/task`; the pipeline engine stays in package\n `workflow`, which keeps aliases. The UI process links the vocabulary only.\n\nThe gate is enforced, not just measured. `cmd/archie-ui/architecture_test.go`\nre-runs `go list -deps` on every `task check` and fails on any package in a\nbanned category, so a single reference to a type sitting beside a runtime\ncannot quietly relink it. Its exception list (`internal/channels/status`,\n`internal/domain/workflow/task`) is checked in both directions: an exception\nthe UI stops needing is reported, so the allowlist cannot decay into an open\nprefix. `internal/app/archieui`'s `TestComposeUIServerHoldsNoDaemonState`\ncovers the composition clauses, failing on a non-nil `config.Holder`, a\nconcrete `*store.Store`, or a Gateway that is not the gRPC client.\n\nOne piece of residue remained visible at the close of Phase 3: `webui.Server`\nstill declared `Cfg *config.Holder`, because the daemon kept a `webui.Server`\nas the renderer that built the published `ConfigView` snapshot. That cleanup\nlanded as `archie-core-ml30`. `webui.Server` no longer has a configuration\nholder, provenance store, reload-status callback or override-listing callback:\nthe projection is built by `webui.BuildConfigView`, a pure function the\nconfiguration owner calls, and the daemon assembles its input from its own\nstate (`boot.configViewInput`). Readiness reads the daemon's own Holder,\nchannel manager and chat contract, and the health registry it produces is a\ndaemon field rather than a field on a server the daemon never serves.\n\n`ConfigView.Editable` moved with it. It describes the rendering process's own\nwrite path, so `handleConfig` sets it from `Server.UpdateConfig` rather than\nany process baking it into the document. A future configuration write contract\n(`archie-core-j28m`) makes the page editable by wiring that one field.\n\n### Real-process evidence (archie-core-8cda.5.6)\n\n`cmd/archie-ui/main_test.go` builds the binary and runs it against a State\nStore and a Gateway on loopback gRPC, then drives it over HTTP: liveness\nwithout a token, 401 for an API request without one, the `?t=` exchange into\nan HttpOnly cookie, task rows with their forge links, the embedded SPA and its\nassets, a chat stream relayed frame by frame from the Gateway, an operator\ntask action crossing to the Gateway (and refused without the CSRF header),\ncapture intake accepted unauthenticated, the ratified degradations, readiness\ndropping when the State Store dies while liveness holds, and a clean SIGTERM.\n\nTwo dashboard values degraded here because the published projection did not\ncarry them: `chat.show_tool_calls` (the chat page never expanded tool calls)\nand the `/api/setup` checklist (the panel was omitted rather than rendered).\nBoth are closed by `archie-core-ml30`: `ConfigView` gained a `chat` section\ncarrying `show_tool_calls`, the operator name, and whether any chat channel is\nconfigured, and both surfaces now read the projection rather than a live\nholder. The end-to-end suite drives the real binary and streams a turn, so the\n`show_tool_calls` assertion is made on the frames a browser receives.\n\nThe checklist stays. The question raised when it was republished -- is a\nread-only setup report wanted in a process that cannot apply the setup it\nreports? -- resolves in favour of keeping it, because the panel never promised\nto apply anything. It renders each step as a title, a detail and a done mark\nwith no control behind it, so it answers \"what is this deployment missing?\"\nrather than \"fix it here\", and the page that CAN act on it (the Configuration\npage) is where an operator goes next, in the daemon's own words. Removing it\nwould delete the one surface that distinguishes a fresh deployment from a\nfinished one for no gain, since the values it reads already cross the boundary\nin `ConfigView` and the read path costs the daemon nothing. The write path\nremains descoped (see `/api/config` (write) in the route table): the checklist\nreports configuration, it does not change it.\n\n`archie-core-ml30` published the channel flag from two of the three chat\nfront-ends, so an email-only deployment read as configured on `/api/channels`\nand unconfigured on the checklist at the same time (GitHub #821). Both now\nderive from `config.ChatConfig.FrontEnds`, the single definition of a configured\nfront-end, and a test pins the two surfaces to the same answer.\n\nA third value degraded for the same reason and is closed by\n`archie-core-pv6t`: in a deployment configuring `[[identities]]` the projection\ncarried the default identity's forge alone, so task rows withheld their links\nrather than pointing every row at the wrong forge. `ConfigView` now also\npublishes each identity's forge coordinates and the repositories it owns\n(`identities`), which is what resolving a task to its owning forge needs —\ncoordinates and repository names only, never a token. The end-to-end suite\ndrives the real binary against a multi-identity projection and asserts each\nrow's issue link is built from the forge that owns it.\n\n## Non-goals and open work\n\nThis ratification does not define a new domain model, move the dashboard's\nallowlisted configuration schema, redesign Gateway or State Store RPCs, choose\nKubernetes versus local discovery, or authorize unrelated package migration.\nThose decisions remain in their existing authority documents. Any new UI\ncapability that requires a wider Gateway or State Store surface must amend that\nservice's contract before implementation.\n"
|
||
}
|
||
],
|
||
"nav": [
|
||
{
|
||
"title": "Archie",
|
||
"url": "/docs/"
|
||
},
|
||
{
|
||
"title": "Architecture",
|
||
"url": "/docs/architecture/",
|
||
"items": [
|
||
{
|
||
"title": "Adversarial self-review",
|
||
"url": "/docs/architecture/adversarial-review/"
|
||
},
|
||
{
|
||
"title": "Agent System",
|
||
"url": "/docs/architecture/agent-system/"
|
||
},
|
||
{
|
||
"title": "Playbook bindings: model and threat model",
|
||
"url": "/docs/architecture/bindings/"
|
||
},
|
||
{
|
||
"title": "Configuration",
|
||
"url": "/docs/architecture/configuration/"
|
||
},
|
||
{
|
||
"title": "Dependencies and Contracts",
|
||
"url": "/docs/architecture/dependencies-and-contracts/"
|
||
},
|
||
{
|
||
"title": "Generated Documentation",
|
||
"url": "/docs/architecture/generated-documentation/"
|
||
},
|
||
{
|
||
"title": "Identity",
|
||
"url": "/docs/architecture/identity/"
|
||
},
|
||
{
|
||
"title": "Messaging and Work Intake",
|
||
"url": "/docs/architecture/messaging-and-work-intake/"
|
||
},
|
||
{
|
||
"title": "Architecture Migration Decisions",
|
||
"url": "/docs/architecture/migration-decisions/"
|
||
},
|
||
{
|
||
"title": "Task Observability",
|
||
"url": "/docs/architecture/observability/"
|
||
},
|
||
{
|
||
"title": "Repository Organisation",
|
||
"url": "/docs/architecture/organisation/"
|
||
},
|
||
{
|
||
"title": "Domain and Feature Review Procedure",
|
||
"url": "/docs/architecture/package-review/"
|
||
},
|
||
{
|
||
"title": "Plugins and Extensions",
|
||
"url": "/docs/architecture/plugins-and-extensions/"
|
||
},
|
||
{
|
||
"title": "Policy System",
|
||
"url": "/docs/architecture/policy/"
|
||
},
|
||
{
|
||
"title": "Safe Change and Recovery",
|
||
"url": "/docs/architecture/safe-change-and-recovery/"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"title": "Guides",
|
||
"items": [
|
||
{
|
||
"title": "Building your first playbook binding",
|
||
"url": "/docs/guides/first-playbook/"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"title": "PRDs",
|
||
"items": [
|
||
{
|
||
"title": "Backend-owned configuration schema for the webui -- decision",
|
||
"url": "/docs/prds/config-schema/"
|
||
},
|
||
{
|
||
"title": "Memory engine unification: four scopes, CRUD, and one write path -- decision",
|
||
"url": "/docs/prds/memory-engine-unification/"
|
||
},
|
||
{
|
||
"title": "Messaging Service boundary and transport design",
|
||
"url": "/docs/prds/messaging-service-boundary/"
|
||
},
|
||
{
|
||
"title": "Service decomposition -- decision",
|
||
"url": "/docs/prds/service-decomposition/"
|
||
},
|
||
{
|
||
"title": "UI Service boundary and configuration ownership",
|
||
"url": "/docs/prds/ui-service-boundary/"
|
||
}
|
||
]
|
||
}
|
||
]
|
||
}
|