Files
archie/docs/prds/service-decomposition.md
T
sam ac7ffe343a docs(eda): split the expression decision out and cut the playbook PRD to its design
The playbook engine PRD had grown to 972 lines against CLAUDE.md's one-page
bar, because four documents were living in it.

The expression decision was the worst of it: an H2 nested inside a numbered
list item, 270 lines with twelve H3s and its own Status and Date fields. That
broke the outline and gave the document a second, non-canonical status. It is
now docs/prds/playbook-expression-syntax.md, named flat and linked by a Parent
field the way module-position.md already relates to this epic. The parent
keeps the decision and a pointer.

Repetition, against the state-each-fact-once rule: reject-at-load was stated
ten times, the plugin engine rule nine, producer-only seven. The producer-only
argument ran 32 lines and closed by restating itself; it is eight lines and
stated once. One heading read "confirmed non-issue (stated once)" above the
third statement of the point.

Open question 3 was closed by module-position.md and still listed here as
open. The section holding four decisions and one note is no longer called
"Open questions", and the note is a standing constraint, which is what it is.

Also removed: two list summaries written before their lists, ten dated
attributions, two progress claims, three orphan line citations, and the
undefined { Call: ... } notation.

Five findings in other PRDs came from tightening the guard, not from this
split.
2026-09-22 01:07:05 +10:00

14 KiB

Service decomposition -- decision

Status: Draft Date: 2026-09-05 Beads epic: archie-core-8cda Prior art: docs/inspiration/2026-09-05-awx-service-decomposition-and-ansible-task-model.md, docs/inspiration/module-reference-awx-decomposition-vs-archie.md, docs/inspiration/service-discovery-research-2026-09-05.md, /work/artefacts/archie-core-current-state-archify.html, /work/artefacts/archie-core-ideal-state-archify.html

The Phase 3 UI boundary and configuration decision is ratified in ui-service-boundary.md. That focused document is the implementation authority for archie-core-8cda.5.2; this parent remains the broader service-distribution proposal.

Decision

archie-core moves from one Go binary (archied) hosting every domain in-process to a set of independently deployable services, each exposing exactly one typed contract. Consumers depend on the contract; they never reach into another service's internal types, session state, or config. gRPC is the synchronous transport; NATS remains for asynchronous fan-out (status, telemetry) rather than being replaced.

This is a distribution decision, not a rewrite of Archie's domain model. The six-domain split already approved in docs/architecture/index.md and docs/architecture/organisation.md (Identity, Agent, Messaging, Workflow, Work Intake, Plugin) is unchanged; this PRD is about how the processes running that domain logic get packaged, contracted, and discovered -- not about redrawing domain boundaries.

Target services

Service Contract it owns Existing code it replaces/wraps
UI Service thin client only; talks to Gateway and State Store contracts internal/webui (SPA hosting stays; API surface becomes typed contract clients, not a peer with direct access)
Gateway Service routing / turn / session contract internal/gateway
Messaging Service channel-plugin contract (Telegram, email, Discord, Teams, ...) internal/channels
Curator Service curator contract; optional, installs as a Gateway plugin internal/domain/curator
Scheduler Service cron/ticker contract internal/domain/scheduling
Execution Environment execution-dispatch contract internal/domain/workflow, task dispatch paths
Runner isolated worktree/container execution internal/container, internal/agentexec, internal/worktree
State Store state-access contract internal/store

Discovery

Per docs/inspiration/service-discovery-research-2026-09-05.md:

  • Primary: Kubernetes-native discovery (Service + headless-Service/DNS

    • EndpointSlice, served by CoreDNS), delivered as a Helm chart + Operator, on k3s for single-host installs and a real cluster for multi-host or a hosted offering.
  • Fallback: NATS-based discovery (micro $SRV announce + JetStream KV registry) for no-K8s single-host installs, with a custom gRPC resolver reading host:port from the KV bucket.

  • Both implementations sit behind one internal interface:

    type ServiceRegistry interface {
        Resolve(service string) []Endpoint
        Watch(service string) <-chan Event // join/leave
    }
    
  • An optional service (Curator; others later) that is not installed must resolve to a distinct NotInstalled outcome, never a Down/ Unhealthy one. Under K8s this falls out of DNS returning no record; under the NATS fallback it requires an explicit "installed" marker in the KV registry, not just absence-implies-down.

Distribution model this supports

Both self-hosted (customer runs the Helm chart or the NATS-fallback binaries themselves) and a managed/hosted offering (we run the same Helm chart) are in scope -- this is the RHEL / Ansible Automation Platform model: one architecture, sold both ways. This PRD does not decide the business model; it decides that the architecture must not foreclose either shape.

Why this shape

docs/inspiration/module-reference-awx-decomposition-vs-archie.md established that Archie's existing domain split already does what AWX's service-decomposition post argues for -- boundaries exist. What didn't exist was deployability: internal/webui was not a passive frontend. The re-baselined coupling inventory is below, and the ratified boundary is ui-service-boundary.md.

  • Telegram task actions now call Gateway's ChatContract directly through chatTaskActorAdapter; the old Telegram-to-webui HTTP detour is historical.
  • internal/webui/api_chat.go now holds gateway.ChatContract; direct SessionStore, Router, and TurnHistory access is historical.
  • The third coupling is closed. UI extraction shipped (archie-core-8cda.5, closed 2026-09-11): cmd/archie-ui serves the dashboard against remote Gateway and State Store contracts, holds no config.Holder, and relinks no daemon runtime (cmd/archie-ui/architecture_test.go enforces that on every task check). The b.web.Cfg = b.d.Cfg alias left in buildDaemon is now inside the daemon's own process -- it renders the published ConfigView snapshot and holds the readiness probes -- so it no longer crosses a service boundary. That cleanup is archie-core-ml30 (P2); the UI boundary is ratified in ui-service-boundary.md.

Non-goals

  • Redesigning the domain model (Identity/Agent/Messaging/Workflow/Work Intake/Plugin boundaries are settled elsewhere and unchanged).
  • Replacing NATS. It remains the async transport; this PRD only adds gRPC for synchronous request/response and (optionally) K8s-native discovery.
  • Deciding the business model (self-hosted vs. managed vs. both). Kept open deliberately; the architecture must serve either.
  • Config relocation (config.toml -> ConfigMaps/Secrets) is real work implied by a K8s-native deployment but is its own decision, not decided here. Flagged in the discovery research as a phased concern.
  • Any concrete adoption of Pi (or another external agent harness) as an execution runner. Raised in conversation alongside this decomposition, but it is a separate decision (agent execution as a strict data boundary, ARCHITECTURE.md) and does not gate or get gated by this PRD.

Open questions

All ten are resolved; each carries its resolution inline. Three phases have since shipped against them (Gateway, State Store, UI), so the framing here is historical record rather than an outstanding gate.

  1. Extraction order. Which service gets pulled out of the monolith first? The webui entanglement findings above suggest Gateway Service's ChatContract should exist before UI Service is extracted, since UI is the most entangled consumer. Not yet sequenced.

    RESOLVED. Gateway Service first, then State Store, then UI, then Messaging, then Execution/Runner/Scheduler, then Curator (optional) — preceded by an in-process "contract seams" (modular monolith) phase. Confirms the hypothesis and strengthens it: the Gateway is already store-decoupled (it uses narrow contract adapters, not internal/store) and already owns an isolated session SQLite, so it is both the stickiest capability and the one with a ready seam and isolated data — extractable alone, first. Full phase list and reasoning: docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#1.

  2. Contract definition mechanism. Protobuf/gRPC service definitions need a home (a new proto/ or contracts/ tree, versioning scheme, codegen wiring into Taskfile.yml). Not yet designed.

    RESOLVED. buf v2 driving protoc-gen-go + protoc-gen-go-grpc (grpc-go, not connect-go — connect-go's net/http client breaks the discovery research's gRPC resolver plumbing for K8s DNS and NATS-KV). One committed proto/<service>/v1/ tree, generated Go committed to internal/contracts/<service>/v1/, directory versioning (breaking change -> new vN, enforced by buf breaking in the gate), and task proto:generate/proto:lint/proto:check wired into task check and mirrored into .github/workflows/deploy.yml (CI does not run task check). Full toolchain comparison, layout rationale, and scratch-verified buf/buf breaking/connect-go results: docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#2.

  3. In-process vs. out-of-process during migration. Whether services can run multiplexed inside one binary behind the same ServiceRegistry interface during a transition period (so extraction is incremental and testable) or whether each service must be a separate binary from day one. Strongly recommend the former for a solo maintainer, but not decided here.

    RESOLVED. Multiplexed in-process is the migration default -- this is docs/architecture/organisation.md's standing "Process boundaries" rule, not a new lean. Mechanism: one wire-safe Go contract interface per service; a local adapter and a generated gRPC client adapter, both satisfying the same interface via a compile-time assertion (the var _ workflow.Store = (*Client)(nil) idiom; store.WorkflowStore is superseded by the consumer-owned workflow.Store); composition (not ServiceRegistry, which stays network-only) picks the adapter by the presence of a [services.<name>].target address -- empty = local (in-process) adapter, set = dial the remote gRPC client. This replaced the earlier mode = "inproc" | "remote" enum, which was dropped (see the gateway's services.gateway.target). The adapter is chosen by presence of the target; Phase 2 then deleted every local fallback, so remote is now the only mode and an empty [services.state].target is a startup error (docs/prds/state-store-contract.md §12 step 7). A per-contract conformance suite runs against both adapters over bufconn; the flip to a real process is a same-commit deletion of the in-process path, no dual-live window. Consequence: Gateway's Phase 0 (ChatContract seam) ships in-process first, with the network protocol staying out of the production path until Phase 1 -- no amendment to Q1's phase list needed. Full mechanism and evidence: docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#3 (the State Store's concrete ratified config shape is docs/prds/state-store-contract.md §10).

  4. State Store's contract shape. Whether every service gets its own storage (true microservice isolation) or several services share the State Store service's contract (less isolation, less migration risk). Not yet decided.

    RESOLVED. One State Store service owns the existing single SQLite file behind narrow typed contracts (TaskStore, CaptureStore, MappingStore, BindingStore, WorkflowStore) over gRPC. (Ratified 2026-09-06: the workflow contract is now the domain-owned workflow.Store, which supersedes internal/store's WorkflowStore. See note below and docs/prds/state-store-contract.md.) No physical database-per-service: the solo-maintainer / team- topology rule of thumb plus the low value of per-service autonomy for one person rule it out, and one store means no distributed-transactions problem. The Gateway keeps its own already-separate session SQLite. Concrete migration path (generalise storerpc, swap consumers one contract at a time, stand up the store service, split the file only on evidence): docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#4.

    RATIFIED (2026-09-06, beads archie-core-8cda.4.1). The authoritative contract boundary & transport design for this is fixed in docs/prds/state-store-contract.md (rev. 2c): one StateStore gRPC service, 42 RPCs; ownership split — consumer-owned workflow.Store + workflow.Task/Status/Source in the workflow domain (dependency rules #2/#7), producer-owned daemon/webui store surfaces in internal/store; the Task/Status/Source + interface relocation is a Phase 2 prerequisite pulled from migration-decisions §4 (store.Task → WorkflowExecution); RecordDispatch drops *sql.Tx; Close off-wire; structured gRPC error codes with sanitised details; explicit agent State Store gRPC handoff (STATE_STORE_URL/STATE_STORE_TOKEN); loopback+bearer transport-security boundary; presence-based [services.state].target defaulting local; migration order workflow.Store -> archie-state-store -> Capture -> Mapping -> Binding -> TaskStore. .4.2 and .4.3 implement against that doc.

  5. Helm chart / Operator ownership. Net-new work with no existing analog in this repository (deployments/ holds TOML profiles and a docker-compose.yml, not Kubernetes manifests). Scoping not yet started.

    RESOLVED. Three charts: an archie-library chart (shared Deployment/Service templates, named ports), an archie chart for the 7 core services, and an archie-curator chart + Curator Operator (CRD, controller, RBAC). The Curator reconciler is a level-triggered loop that fetches the CR, builds Deployment + Service, sets owner refs, applies idempotently (CreateOrUpdate), watches via Owns(), and reports a Ready status condition. Smallest real milestone (M1): install a Curator CR → Operator creates Deployment + Service → CoreDNS serves curator → Gateway's resolver flips it from NotInstalled to Installed with no redeploy; uninstall flips it back. Full scoping and first bead-issue list: docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#5.

Completion criteria

Not applicable yet -- this PRD records the target shape and the discovery decision. It does not authorize implementation. Before work begins: file a beads epic, resolve the open questions above (at minimum #1-#3), and get explicit go-ahead given the scale of this change relative to AGENTS.md's "smallest workable change" default for this solo project.