Files
archie/docs/prds/service-decomposition.md
T
sam ff00de9519 docs(eda): split the expression decision out and cut the playbook PRD to its design
The playbook engine PRD had grown to 972 lines against CLAUDE.md's one-page
bar, because four documents were living in it.

The expression decision was the worst of it: an H2 nested inside a numbered
list item, 270 lines with twelve H3s and its own Status and Date fields. That
broke the outline and gave the document a second, non-canonical status. It is
now docs/prds/playbook-expression-syntax.md, named flat and linked by a Parent
field the way module-position.md already relates to this epic. The parent
keeps the decision and a pointer.

Repetition, against the state-each-fact-once rule: reject-at-load was stated
ten times, the plugin engine rule nine, producer-only seven. The producer-only
argument ran 32 lines and closed by restating itself; it is eight lines and
stated once. One heading read "confirmed non-issue (stated once)" above the
third statement of the point.

Open question 3 was closed by module-position.md and still listed here as
open. The section holding four decisions and one note is no longer called
"Open questions", and the note is a standing constraint, which is what it is.

Also removed: two list summaries written before their lists, ten dated
attributions, two progress claims, three orphan line citations, and the
undefined { Call: ... } notation.

Five findings in other PRDs came from tightening the guard, not from this
split.
2026-09-22 01:07:05 +10:00

250 lines
14 KiB
Markdown

# Service decomposition -- decision
**Status:** Draft
**Date:** 2026-09-05
**Beads epic:** archie-core-8cda
**Prior art:** `docs/inspiration/2026-09-05-awx-service-decomposition-and-ansible-task-model.md`,
`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`,
`docs/inspiration/service-discovery-research-2026-09-05.md`,
`/work/artefacts/archie-core-current-state-archify.html`,
`/work/artefacts/archie-core-ideal-state-archify.html`
The Phase 3 UI boundary and configuration decision is ratified in
[`ui-service-boundary.md`](ui-service-boundary.md). That focused document is
the implementation authority for `archie-core-8cda.5.2`; this parent remains
the broader service-distribution proposal.
## Decision
archie-core moves from one Go binary (`archied`) hosting every domain
in-process to a set of independently deployable services, each exposing
exactly one typed contract. Consumers depend on the contract; they never
reach into another service's internal types, session state, or config.
gRPC is the synchronous transport; NATS remains for asynchronous fan-out
(status, telemetry) rather than being replaced.
This is a **distribution decision**, not a rewrite of Archie's domain
model. The six-domain split already approved in
`docs/architecture/index.md` and `docs/architecture/organisation.md`
(Identity, Agent, Messaging, Workflow, Work Intake, Plugin) is unchanged;
this PRD is about how the *processes* running that domain logic get
packaged, contracted, and discovered -- not about redrawing domain
boundaries.
### Target services
| Service | Contract it owns | Existing code it replaces/wraps |
|---|---|---|
| UI Service | thin client only; talks to Gateway and State Store contracts | `internal/webui` (SPA hosting stays; API surface becomes typed contract clients, not a peer with direct access) |
| Gateway Service | routing / turn / session contract | `internal/gateway` |
| Messaging Service | channel-plugin contract (Telegram, email, Discord, Teams, ...) | `internal/channels` |
| Curator Service | curator contract; **optional**, installs as a Gateway plugin | `internal/domain/curator` |
| Scheduler Service | cron/ticker contract | `internal/domain/scheduling` |
| Execution Environment | execution-dispatch contract | `internal/domain/workflow`, task dispatch paths |
| Runner | isolated worktree/container execution | `internal/container`, `internal/agentexec`, `internal/worktree` |
| State Store | state-access contract | `internal/store` |
### Discovery
Per `docs/inspiration/service-discovery-research-2026-09-05.md`:
- **Primary:** Kubernetes-native discovery (Service + headless-Service/DNS
+ EndpointSlice, served by CoreDNS), delivered as a Helm chart + Operator,
on k3s for single-host installs and a real cluster for multi-host or a
hosted offering.
- **Fallback:** NATS-based discovery (micro `$SRV` announce + JetStream KV
registry) for no-K8s single-host installs, with a custom gRPC resolver
reading `host:port` from the KV bucket.
- Both implementations sit behind one internal interface:
```go
type ServiceRegistry interface {
Resolve(service string) []Endpoint
Watch(service string) <-chan Event // join/leave
}
```
- An optional service (Curator; others later) that is not installed
must resolve to a distinct `NotInstalled` outcome, never a `Down`/
`Unhealthy` one. Under K8s this falls out of DNS returning no record;
under the NATS fallback it requires an explicit "installed" marker in the
KV registry, not just absence-implies-down.
### Distribution model this supports
Both self-hosted (customer runs the Helm chart or the NATS-fallback
binaries themselves) and a managed/hosted offering (we run the same Helm
chart) are in scope -- this is the RHEL / Ansible Automation Platform
model: one architecture, sold both ways. This PRD does not decide the
business model; it decides that the architecture must not foreclose
either shape.
## Why this shape
`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`
established that Archie's existing domain split already does what AWX's
service-decomposition post argues for -- boundaries exist. What didn't
exist was deployability: `internal/webui` was not a passive frontend. The
re-baselined coupling inventory is below, and the ratified boundary is
[`ui-service-boundary.md`](ui-service-boundary.md).
- Telegram task actions now call Gateway's `ChatContract` directly through
`chatTaskActorAdapter`; the old Telegram-to-webui HTTP detour is historical.
- `internal/webui/api_chat.go` now holds `gateway.ChatContract`; direct
`SessionStore`, `Router`, and `TurnHistory` access is historical.
- The third coupling is closed. UI extraction shipped (`archie-core-8cda.5`,
closed 2026-09-11): `cmd/archie-ui` serves the dashboard against remote
Gateway and State Store contracts, holds no `config.Holder`, and relinks no
daemon runtime (`cmd/archie-ui/architecture_test.go` enforces that on every
`task check`). The `b.web.Cfg = b.d.Cfg` alias left in `buildDaemon` is now
inside the daemon's own process -- it renders the published `ConfigView`
snapshot and holds the readiness probes -- so it no longer crosses a service
boundary. That cleanup is `archie-core-ml30` (P2); the UI boundary is
ratified in [`ui-service-boundary.md`](ui-service-boundary.md).
## Non-goals
- Redesigning the domain model (Identity/Agent/Messaging/Workflow/Work
Intake/Plugin boundaries are settled elsewhere and unchanged).
- Replacing NATS. It remains the async transport; this PRD only adds gRPC
for synchronous request/response and (optionally) K8s-native discovery.
- Deciding the business model (self-hosted vs. managed vs. both). Kept
open deliberately; the architecture must serve either.
- Config relocation (`config.toml` -> ConfigMaps/Secrets) is real work
implied by a K8s-native deployment but is its own decision, not decided
here. Flagged in the discovery research as a phased concern.
- Any concrete adoption of Pi (or another external agent harness) as an
execution runner. Raised in conversation alongside this decomposition,
but it is a separate decision (agent execution as a strict data
boundary, `ARCHITECTURE.md`) and does not gate or get gated by this PRD.
## Open questions
All ten are resolved; each carries its resolution inline. Three phases have
since shipped against them (Gateway, State Store, UI), so the framing here is
historical record rather than an outstanding gate.
1. **Extraction order.** Which service gets pulled out of the monolith
first? The webui entanglement findings above suggest Gateway Service's
`ChatContract` should exist before UI Service is extracted, since UI is
the most entangled consumer. Not yet sequenced.
**RESOLVED.** Gateway Service first, then State Store,
then UI, then Messaging, then Execution/Runner/Scheduler, then Curator
(optional) — preceded by an in-process "contract seams" (modular
monolith) phase. Confirms the hypothesis and strengthens it: the
Gateway is already store-decoupled (it uses narrow contract adapters,
not `internal/store`) and already owns an isolated session SQLite, so it
is both the stickiest capability *and* the one with a ready seam and
isolated data — extractable alone, first. Full phase list and reasoning:
`docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#1`.
2. **Contract definition mechanism.** Protobuf/gRPC service definitions
need a home (a new `proto/` or `contracts/` tree, versioning scheme,
codegen wiring into `Taskfile.yml`). Not yet designed.
**RESOLVED.** buf v2 driving `protoc-gen-go` +
`protoc-gen-go-grpc` (grpc-go, not connect-go — connect-go's net/http
client breaks the discovery research's gRPC resolver plumbing for K8s
DNS and NATS-KV). One committed `proto/<service>/v1/` tree, generated
Go committed to `internal/contracts/<service>/v1/`, directory
versioning (breaking change -> new `vN`, enforced by `buf breaking` in
the gate), and `task proto:generate`/`proto:lint`/`proto:check` wired
into `task check` and mirrored into `.github/workflows/deploy.yml`
(CI does not run `task check`). Full toolchain comparison, layout
rationale, and scratch-verified `buf`/`buf breaking`/`connect-go`
results:
`docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#2`.
3. **In-process vs. out-of-process during migration.** Whether services
can run multiplexed inside one binary behind the same `ServiceRegistry`
interface during a transition period (so extraction is incremental and
testable) or whether each service must be a separate binary from day
one. Strongly recommend the former for a solo maintainer, but not
decided here.
**RESOLVED.** Multiplexed in-process is the migration
default -- this is `docs/architecture/organisation.md`'s standing
"Process boundaries" rule, not a new lean. Mechanism: one wire-safe Go
contract interface per service; a local adapter and a generated gRPC
client adapter, both satisfying the same interface via a compile-time
assertion (the `var _ workflow.Store = (*Client)(nil)` idiom;
`store.WorkflowStore` is superseded by the consumer-owned
`workflow.Store`); composition (not `ServiceRegistry`, which stays
network-only) picks the adapter by the presence of a
`[services.<name>].target` address -- empty = local (in-process) adapter,
set = dial the remote gRPC client. This replaced the earlier
`mode = "inproc" | "remote"` enum, which was dropped (see the gateway's
`services.gateway.target`). The adapter is chosen by presence of the
target; Phase 2 then deleted every local fallback, so remote is now the
only mode and an empty `[services.state].target` is a startup error
(docs/prds/state-store-contract.md §12 step 7). A
per-contract conformance suite runs against both adapters over
`bufconn`; the flip to a real process is a same-commit deletion of the
in-process path, no dual-live window. Consequence: Gateway's Phase 0
(`ChatContract` seam) ships in-process first, with the network protocol
staying out of the production path until Phase 1 -- no amendment to Q1's
phase list needed. Full mechanism and evidence:
`docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#3`
(the State Store's concrete ratified config shape is
`docs/prds/state-store-contract.md` §10).
4. **State Store's contract shape.** Whether every service gets its own
storage (true microservice isolation) or several services share the
State Store service's contract (less isolation, less migration risk).
Not yet decided.
**RESOLVED.** One State Store service owns the existing
single SQLite file behind narrow typed contracts (`TaskStore`,
`CaptureStore`, `MappingStore`, `BindingStore`, `WorkflowStore`) over
gRPC. *(Ratified 2026-09-06: the workflow contract is now the
domain-owned `workflow.Store`, which supersedes
`internal/store`'s `WorkflowStore`. See note below and
`docs/prds/state-store-contract.md`.)*
No physical database-per-service: the solo-maintainer / team-
topology rule of thumb plus the low value of per-service autonomy for
one person rule it out, and one store means no distributed-transactions
problem. The Gateway keeps its own already-separate session SQLite.
Concrete migration path (generalise `storerpc`, swap consumers one
contract at a time, stand up the store service, split the file only on
evidence):
`docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#4`.
**RATIFIED (2026-09-06, beads archie-core-8cda.4.1).** The authoritative
contract boundary & transport design for this is fixed in
`docs/prds/state-store-contract.md` (rev. 2c): one `StateStore` gRPC service,
42 RPCs; **ownership split** — consumer-owned `workflow.Store` +
`workflow.Task`/`Status`/`Source` in the workflow domain (dependency rules
#2/#7), producer-owned daemon/webui store surfaces in `internal/store`;
the `Task`/`Status`/`Source` + interface relocation is a Phase 2
prerequisite pulled from migration-decisions §4 (`store.Task →
WorkflowExecution`); `RecordDispatch` drops `*sql.Tx`; `Close` off-wire;
structured gRPC error codes with sanitised details; explicit agent State
Store gRPC handoff (`STATE_STORE_URL`/`STATE_STORE_TOKEN`); loopback+bearer
transport-security boundary; presence-based `[services.state].target`
defaulting local; migration order `workflow.Store -> archie-state-store ->
Capture -> Mapping -> Binding -> TaskStore`. `.4.2` and `.4.3` implement
against that doc.
5. **Helm chart / Operator ownership.** Net-new work with no existing
analog in this repository (`deployments/` holds TOML profiles
and a `docker-compose.yml`, not Kubernetes manifests). Scoping not yet
started.
**RESOLVED.** Three charts: an `archie-library` chart
(shared Deployment/Service templates, named ports), an `archie` chart
for the 7 core services, and an `archie-curator` chart + Curator
Operator (CRD, controller, RBAC). The Curator reconciler is a
level-triggered loop that fetches the CR, builds Deployment + Service,
sets owner refs, applies idempotently (`CreateOrUpdate`), watches via
`Owns()`, and reports a `Ready` status condition. Smallest real
milestone (M1): install a Curator CR → Operator creates Deployment +
Service → CoreDNS serves `curator` → Gateway's resolver flips it from
`NotInstalled` to Installed with no redeploy; uninstall flips it back.
Full scoping and first bead-issue list:
`docs/inspiration/service-decomposition-open-questions-research-2026-09-05.md#5`.
## Completion criteria
Not applicable yet -- this PRD records the target shape and the discovery
decision. It does not authorize implementation. Before work begins: file
a beads epic, resolve the open questions above (at minimum #1-#3), and
get explicit go-ahead given the scale of this change relative to
`AGENTS.md`'s "smallest workable change" default for this solo project.