servicediscovery/nats and gatewayrpc.StoreClient are implemented and fully tested, but neither has a production caller and no config surface exposes either. Both were previously unrecorded, which made them indistinguishable from oversight. The register now names them: StoreClient under the session migration that will consume it, service discovery under shared mechanics, alongside the runtime boundary that pins container addressing by injected NATS URL instead. Kept rather than deleted because re-deriving tested wire code later is pure cost.
27 KiB
Architecture Migration Decisions
Status: Active migration inventory
Date: 2026-07-28
Tracking issue: #73
Purpose
This document records what remains to be decided before Archie's current implementation can be migrated into the approved domain-oriented structure.
The product and core domain model are not being redesigned. Existing Archie behaviour and the existing session implementation are the migration baseline. Remaining decisions MUST be grounded in current packages, consumers, tests, persistence, and runtime wiring.
Confirmed domain packages
internal/domain/
identity/ (approved, not yet materialised as a package)
agent/ (approved, not yet materialised as a package)
workflow/
messaging/ (canonical Message/Conversation types only; gateway migration pending, see section 2)
workintake/
plugin/ (approved, not yet materialised as a package)
curator/
mapping/
scheduling/
identityowns persistent actors, lifecycle, ownership, attribution, and audit identity.agentowns persistent assistants, specialisation, instructions, model-independent continuity, capability context, and associations with identity and memory.workflowis a separate domain. It owns reusable, versioned Workflow definitions, WorkflowSteps, WorkflowExecutions, StepExecutions, Attempts, outcomes, and typed Workflow plugin contracts.messagingowns canonical Messages, Conversations, branches, channel and source correlation, replies, acknowledgements, delivery attempts, events, and projections.workintakeowns admission, validation, deterministic routing, and accepted work requests for the optional transition into durable workflow-backed execution.pluginowns generic plugin identity, discovery, compatibility, and lifecycle mechanics. It remains metadata-only; capability semantics and typed registration belong to the owning domain.curatorowns the curator engine family: long-running agent loops that reason over memory and perform maintenance tasks on a declared shape (check-in interval, tool set, attached memory engine). See epic archie-core-yp9.mappingowns the payload field-mapping vocabulary: binding named fields to JSON paths inside a captured webhook payload, and detecting drift. Seedocs/prds/payload-field-mapping.md.schedulingowns the ticker engine: deciding when a scheduled job runs and with what concurrency. It declaresJobSource/Runnercontracts and owns no storage or delivery. See epic archie-core-1786637496320-249-be6d9a20.
Reconciliation (2026-08-31): curator, mapping, and scheduling existed
in the tree without being added to this list when their epics landed. They are
confirmed as legitimate, narrowly-scoped domains under the same rules as the
original six — each owns one capability family, declares typed contracts, and
has no cross-domain reach. This list was out of date, not the tree.
Conversely, identity, agent, messaging, and plugin are approved target
domains that do not yet exist as packages. Their behaviour currently lives in
pre-migration packages (see "Complete package destination map" below for the
full inventory). Do not create empty stub packages for them ahead of an
actual migration PR — an unused stub is dead code and gives no test coverage.
Each materialises when its corresponding migration decision below is resolved
and implemented in the same change.
Memory's final package placement remains undecided pending focused review of the existing implementation.
The following are approved shared or composition packages, not domains:
internal/eventbus/
internal/policy/
internal/infrastructure/
internal/app/archied/
internal/app/agentworker/
Remaining migration decisions
1. Complete package destination map
Every current package MUST receive one explicit disposition:
- move into a confirmed domain;
- move under infrastructure as an implementation of a domain-owned contract;
- move into application composition;
- merge with an identified duplicate;
- remain as an approved shared application contract;
- move to a plugin, extra, deployment, example, or generated artifact;
- be deleted after its replacement is proven.
The largest unreviewed areas include memory, tools and capabilities, workspace and worktree behaviour, gates, storage, events, web UI, containers, RPC packages, release management, and scheduling.
The destination matrix MUST name the behaviour owner, target path, dependencies to remove, state and contracts to preserve, migration prerequisites, and deletion criteria.
internal/webui / UI Service — RATIFIED Phase 3 disposition (2026-09-07).
The current owner is the in-process HTTP/SSE dashboard: SPA hosting, operator
authentication, UI response projection, configuration reads and mutations,
task/control APIs, and direct adapters to State Store, channel, event/log,
memory, curator, workflow, forge, and Gateway runtime values. The target owner
is a standalone UI application process; internal/webui remains its HTTP
transport adapter, while Gateway, State Store, configuration, Work Intake,
Messaging, and other capability owners expose the contracts behind those
routes. The migration preserves retained route schemas, auth/CSRF behavior,
task/capture/mapping/binding/session continuity, SSE replay semantics, and
committed generated SPA assets. Prerequisites are the ratified UI boundary,
route-owner inventory, narrow remote adapters, configuration/auth/readiness
contracts, and process/conformance tests. Remove the shared live
config.Holder, direct event/log bus, daemon stopper, forge client, channel
manager, capability registries, composite store access, and in-process UI
listener only after the focused criteria in
docs/prds/ui-service-boundary.md pass in
production composition. The old construction and listener are deleted in the
same cutover change; no second UI authority remains.
2. Session and Messaging migration
The existing session implementation is the migration baseline. The migration must decide:
- which session and message components are retained or adapted;
- how Agent, user, channel, Conversation, and branch ownership are added;
- how Archie's parallel gateway session implementation is replaced;
- how canonical Messages become immutable persisted records;
- how existing sessions and messages are migrated;
- how outbound delivery, retry, acknowledgement, deduplication, and failure semantics are represented;
- how current Telegram, email, webhook, forge, and Jira integrations migrate to Messaging infrastructure adapters.
The following are already decided and MUST NOT be reopened without contradictory implementation evidence:
- the composite Conversation identity;
- immutable MessageIDs;
- typed tool-call and tool-result messages;
- exact branch lineage through ParentConversationID and ForkMessageID;
- parent and child transcript isolation;
- normal scoped-memory behaviour regardless of the originating Conversation;
- Messaging and Work Intake ownership.
Decision (2026-08-31), resolving the open questions above:
internal/gateway/ is the "parallel gateway session implementation" this
section names. It is retained as-is for now — a big-bang rewrite of live
session/compression/approval/branch logic across Telegram, web UI, email, and
webhook channels in one change is exactly the "convoluted to satisfy edge
cases" failure mode CLAUDE.md's Revert-on-Flail rule warns about, and would
risk silent behavioural regressions in the one path every user-facing message
travels through.
Instead, internal/domain/messaging/ is materialised now with the pieces
that are already decided and don't require touching gateway: the canonical
Message and Conversation types (composite Conversation identity, immutable
MessageID, typed tool-call/tool-result message variants, branch lineage via
ParentConversationID/ForkMessageID). These are pure domain types with no
gateway dependency — a new package, not a rewrite of an existing one.
Migrating internal/gateway/'s session, compression, approval, and branch
logic onto these types is real, cross-channel-risk work and is not done in
this pass. It is tracked as separate, appropriately-scoped bd issues (see
bd list --label messaging-migration), sequenced channel-by-channel so each
lands with its own behavioural verification instead of one large diff no one
can safely review.
Amended 2026-09-15. internal/infrastructure/gatewayrpc.StoreClient is the
adapter this migration will need: it implements messaging.SessionStore over
the chat service's session-store RPCs and is proven by
TestStoreClientPreservesCanonicalRecords. It is deliberately unwired.
Production archied dials the standalone gateway for ChatContract only
(composeChatContract → gatewayrpc.Dial returns a *Client, not a
*StoreClient), and the daemon's session store is in-process SQLite via
makeTelegramSessionStore. The test comment that says it mirrors production
describes the target, not today's tree. Kept rather than deleted because the
migration it serves is deferred by this very section, and re-deriving a tested
wire adapter once that lands is pure cost.
3. Identity data migration
The migration must define:
- the durable identity schema and repository implementation;
- bootstrap of the permanent system identity;
- stable IdentityIDs for existing configured Agents;
- migration from configured identity names, empty-string ownership, and bot usernames;
- preservation of historical attribution;
- identity-aware uniqueness and transport deduplication;
- correction of container and RPC capability resolution so execution uses the owning identity's bindings;
- removal of identity-string equality from access-control behaviour.
Exact schema and method signatures are implementation decisions. The approved Identity lifecycle and attribution semantics are fixed migration constraints.
4. Workflow migration
The current implementation must be transformed deliberately:
store.Task -> WorkflowExecution
mutable Stage -> StepExecution history
RetryCount / Attempt -> Attempt records
workflow.Registry -> versioned Workflow definitions
current stages -> WorkflowStep definitions or Workflow plugins
The migration must decide:
- Workflow definition and version persistence;
- the smallest accepted-work-request contract consumed from Work Intake;
- the enforced WorkflowExecution state machine;
- atomic state and domain-event persistence;
- migration of existing task rows and transition history;
- disposition of current skill, Yaegi, and stage-plugin implementations;
- the versioned worker-handoff contract;
- scheduler, claim, retry, crash-recovery, and waiting-human cutover.
Workflow remains a separate domain from Agent. A user or Agent may define or invoke a Workflow, but Workflow semantics and plugins remain Workflow-owned.
5. Memory placement and storage
The four required scopes are fixed:
- global shared;
- Agent-wide;
- user-wide;
- Agent-user relationship.
A focused review of internal/memory, its providers, consumers, persistence,
and tests must decide:
internal/domain/memoryversus ownership within the Agent domain;- the authoritative memory record and revision model;
- provider and infrastructure boundaries;
- retrieval and access enforcement;
- migration of existing memory data;
- provenance representation.
Conversation branches do not create another memory scope. A memory action has the same scoped effect regardless of which Conversation originated it.
internal/domain/memory ownership — DECIDED (2026-09-02). The typed
engine family lives at internal/domain/memory
(archie-core-1786637490069-11-e245c170), mirroring internal/domain/curator's
contract/registrar/registry split. Not owned within the Agent domain: memory
is consumed by more than agent turns (curators, per archie-core-1786637499636)
and the plugin engine rule requires its own family, not a capability bolted
onto another domain's contract.
Authoritative record and revision model for the builtin engine — DECIDED
(2026-09-02), scoped to unblock archie-core-1786637499480 (ship the
file-writing store as the default engine). This decides only the builtin
engine's mapping onto the new Record{ID}/Forget(id) contract, not the
memory model in general or a future non-file engine's model:
internal/memory/builtin.Storeis unchanged. It has no concept of a stable per-block ID — blocks are addressed by content substring — and nothing in this decision requires changing that; existinginternal/memorytests keep passing unmodified, per that issue's acceptance criteria.- A new infrastructure adapter (
internal/infrastructure/memory, perorganisation.md's "infrastructure implements domain contracts" rule — not underinternal/domain/memory, which stays contract-only) wraps onebuiltin.Storeper identity. - Identity → directory:
hex(sha256(identity)), not the identity string itself. Closes the path-traversal question without an allowlist regex, and sidesteps collisions from identities that differ only in characters a filesystem treats as equivalent. - Record ID: assigned once at
Write, independent of content, so a later edit elsewhere in the file can never orphan aForget. Realized as a leading marker line (<!--mem:<ulid>-->) on the blockbuiltin.Store.Addwrites;Query/Liststrip it back out ofRecord.Contentbefore returning, andForget(id)locates the block by that unique marker via the store's existing substring-basedRemove. The marker is a pragmatic, reversible choice — plain-text records stay human-readable and human-editable, at the cost of one marker line an operator editing the file by hand should leave alone. Revisit if/when provenance representation (still open, below) settles on something richer. - Kind → section:
Observation.Kind(default"general"when empty) is the section name, reusingbuiltin.ValidateSectionName— so an invalidKindfailsWritewith the same validation errormemory_editalready returns for an invalid section today, not a new error shape.
6. Runtime and process boundaries — DECIDED
archied is the native resident orchestration and interactive-chat process.
Every autonomous repository workflow crosses one full-task core-NATS handoff
into a task-scoped archie-agent container. Embedded versus external NATS
changes broker deployment only; it never selects an executor. The former host
in-process runner, per-stage NATS protocol, and subprocess runner are removed.
Forge, store, and worktree authority remains in archied and is exposed to the
task container through scoped RPC services. The worker runs the workflow and
its worker-local ai-sdk loop; it does not receive forge credentials or direct
store access. See docs/prds/embedded-nats.md.
UI Service boundary — RATIFIED (2026-09-07). Phase 3 has a focused
authority record in docs/prds/ui-service-boundary.md.
The UI Service owns HTTP/SPA delivery, UI DTOs, browser authentication, and
its own endpoint/credential/readiness settings. It consumes Gateway and State
Store contracts and does not receive config.Holder, daemon pointers, SQL or
store implementations, workflow registries, forge/channel/model runtimes, or
resolved secrets. The current internal/webui implementation remains inside
archied until the ratified migration gates prove route parity, adapter
selection, failure/readiness behavior, and deletion of the shared-holder and
direct-access paths. The extraction cutover removes the in-process UI
listener in the same change; there is no dual-live UI authority.
Dashboard live event delivery — DECIDED (2026-09-09, archie-core-za9f).
The extracted UI receives live activity by polling the State Store's existing
EventsSince cursor, not by a new push-subscription RPC. No proto change, no
contract amendment beyond this record.
The UI process runs one event pump. It primes its watermark by paging
EventsSince to the end at startup without delivering, then polls
EventsSince(watermark, limit) on an interval and hands each new event to
webui.Server.Broadcast, exactly where internal/app/archied/main.go's
persistAndBroadcastEvents hands events to it today. One pump serves every
connected browser, so cost is one indexed query per interval regardless of
viewer count, and Broadcast is a no-op when nobody is watching.
Three properties make the cursor sufficient, and a push hub unnecessary:
- The cursor cannot skip.
internal/store/store.goopens the database withSetMaxOpenConns(1), so writes serialise and a row'sidis assigned in commit order. There is no window in which a lower id becomes visible after a higher one, which is the failure mode that normally rules out polling an autoincrement cursor. SSE catch-up already depends on this same property. - A broadcast is a wakeup, not the payload.
sseStream.draintreats each broadcast event as a signal to fill the persisted gap ahead of it viacatchUp, then delivers the event only if catch-up did not already cover it, deduplicating on thesincewatermark. Substituting a poller for the in-process bus therefore changes no SSE semantics, and needs no change tointernal/webui/sse.go. It also means a broadcast dropped to a stalled client self-heals on the next one. - The table is the only complete fan-out point. Events reach the store
through several paths that never touch the daemon's bus, including the
audit event
store.ArchiveTaskwrites inside its transaction (internal/store/store.go) and the binding-dispatch failureinternal/daemon/daemon.goinserts directly. A push hub would have to be hooked at every write site and at post-commit, and would silently miss any site added later that forgets to notify. Reading the table misses nothing.
The cost is one poll interval of latency on the activity feed, default 1s and operator-tunable. That is accepted: the feed reports task lifecycle history, and an operator action already gets its own synchronous HTTP response.
This resolves the /events row of the route-owner inventory in
ui-service-boundary.md. /api/logs and
/api/logs/stream are not resolved by it, they read the daemon's in-process
diagnostic feed and remain host-local (see archie-core-8cda.5.4).
Revisit only if sub-second feed latency becomes a requirement, or if a second
consumer needs the same stream, at which point the fan-out belongs in the State
Store process behind a SubscribeEvents RPC and this record is superseded.
Dashboard configuration page — DECIDED (2026-09-09, archie-core-ymut).
The configuration page becomes read-only in the UI process for Phase 3. Reads
cross the boundary as a published snapshot; writes are descoped, not
contracted. This amends the ratified boundary, which assumed a single admin
contract carrying both.
Reads. webui.ConfigView is already the whole answer. It is a UI-owned,
secret-free value projection built in handleConfig: ProviderView carries
the API key's environment variable name and a Configured boolean, never a
value, and RepoView is repository metadata only. It is already the exact
shape the browser receives. So the daemon publishes that projection, plus
provenance, reload status and the overridden-key list, as a snapshot the UI
reads; nothing new has to be designed, only moved. Two State Store RPCs carry
it, a PutConfigSnapshot restricted to the administrative token and a
GetConfigSnapshot the UI calls. The daemon republishes on boot and after
each successful reload, which is precisely when the projection changes.
The State Store is the right holder because a published configuration snapshot
is shared state, it is already dialled by the UI, and it costs no new listener,
port, token or interceptor. The alternative, a config admin service on
archied, would make the daemon a gRPC server for the first time. Today it is
purely a client of Gateway and State Store; grpc.NewServer appears only in
RunGateway and RunStateStore, which are the standalone binaries' entry
points, not the daemon's.
Writes. PATCH /api/config, POST /api/config/reset and
PATCH /api/config/repos/{owner}/{name} answer 503 in the UI process, and the
SPA hides those controls rather than offering a button that fails. Editing is
config.toml plus reload, which is already the only path for every setting
outside the runtime-tunable allowlist.
This is a deliberate feature reduction and worth naming as one. The write path
is not hard because of the transport; it is hard because the policy is
daemon-local and stays that way: installUpdateConfigHandler denies
non-runtime-tunable keys, applies a dotted overlay to a clone of the published
config, validates the materialised result, writes overlay rows, and rebuilds
provenance. Contracting that means either moving the policy, which the
boundary forbids, or exposing it verbatim as remote procedure calls, which
builds a Configuration Service inside a migration bead. Neither is work this
phase should be doing to keep one page's buttons alive.
Revisit when dashboard configuration editing is picked up as a feature in its
own right (archie-core-1786637498420-327, still open). At that point it gets
an admin contract designed as a feature, and the Configuration Service this
document already anticipates is its natural owner. Until then the UI renders
configuration and does not write it, which is what the boundary always said
about ownership even when it assumed a wider contract.
Amended 2026-09-11. This descope is a deferral, not a decision that the
dashboard should be read-only. Name it that way in review: the daemon-side
policy was built and tested, then deleted by the cutover (e037b95 removed
internal/app/archied/config_update.go and its tests), so reviving it is
recoverable work, not a rewrite. archie-core-j28m carries the remaining
question -- which process serves the policy and what transport carries the
write -- and blocks the feature bead. The handlers, their error sentinels and
the UpdateConfig/ResetConfig/UpdateRepoField seams on webui.Server are
kept deliberately for it; ConfigView.Editable follows whichever process holds
UpdateConfig, so wiring that one field makes the page editable
(archie-core-ml30).
Implementation belongs to archie-core-8cda.5.4, the cutover that removes the
daemon's own dashboard. Nothing here is built yet.
7. Shared mechanics
The following cross-domain mechanics require exact contracts before dependent packages migrate:
- event-bus delivery and failure semantics;
- policy evaluation API, composition, and precedence;
- configuration candidate preparation, health validation, atomic promotion, observation, and rollback;
- service-specific health, bounded remediation, isolation, and escalation;
- changelog association and grouping.
These shared mechanisms do not acquire ownership of domain meaning. Domains continue to own their commands, events, policies, settings, and consequences.
The Go documentation generator, its output placement, and drift checking are
defined in generated-documentation.md. The VitePress site and its Pages
deployment were removed on 2026-09-12 as an unnecessary build step; docs/ is
repository documentation only. The rendering and publishing surface is an open
decision, and so are the renderer's page templates and any developer commands
or CI sequence that would accompany one. Adapting the generator and implementing
Archie's normalized documentation model remain migration work rather than open
product architecture.
Amended 2026-09-15. internal/infrastructure/servicediscovery/nats
implements the servicediscovery.ServiceRegistry contract (Connect, Register,
Resolve, Watch) over NATS KV and is fully tested, but no production code
constructs it and no config surface exposes it. This is a deliberate deferral,
not an oversight: service-to-service addressing belongs to the runtime and
process boundary work in section 6, which currently pins container addressing
by injecting the daemon's startup NATS URL rather than resolving through a
registry. The scaffold is kept for that work; re-derive it only if the
addressing model changes.
8. Cutover sequence
The final migration plan must order:
- compatibility adapters;
- schema migrations and data backfills;
- any required dual-read or transitional paths;
- domain-by-domain package replacement;
- production wiring changes;
- legacy package removal;
- architecture tests preventing dependency regression;
- focused, integration, race, and full quality gates.
Each legacy package requires objective removal criteria. No compatibility path may become an indefinite second implementation.
Completion criteria
Migration design is complete when:
- every current package has one approved owner and destination;
- every mutable record has one authoritative owner and migration path;
- all external and cross-process contracts have versioned owners;
- existing behaviour and data have explicit parity requirements;
- process boundaries have evidence-backed justification;
- the cutover order preserves a working application;
- legacy deletion criteria and deterministic architecture tests are defined;
- no unresolved decision requires an implementer to invent domain semantics.
Exact Go APIs, SQL statements, and mechanical refactoring steps MAY be finalized during implementation when they do not change these semantics.
Recorded migration position (2026-07-29)
Recovered 2026-08-09 from the pre-migration issue tracker. This section records decisions the maintainer approved that the repository did not otherwise capture.
Completed: internal/infrastructure/eventbus/nats (rebuilt, not relocated);
internal/infrastructure/configuration (loading moved out of internal/config,
which is now types-only and does no I/O); internal/eventbus (broker-neutral
contract); internal/domain/workintake (the first internal/domain/ package,
owning TaskEnvelope, the routing Kind, the label vocabulary, and task
subjects).
DECISION — private input DTO deferred (approved).
configuration.Document.Config is still internal/config.Config rather than the
private per-domain input DTO this document mandates. Building an ~800-line
parallel type tree while every consumer still reads config.Config would mean
two shapes kept in sync by hand, with no consumer for the new one. It is named as
a field so the compromise is visible at each use site, with the deletion
condition recorded in that package's doc.go. Do this with the
internal/config dissolution, not ahead of it.
Agreed next: internal/app/{archied,agentworker}. run() in
cmd/archied/main.go is ~460 lines, against organisation.md's rule that
cmd/* must not contain substantive wiring or act as a service locator. It is
also the last non-infrastructure holder of the NATS SDK, and it is where the
config DTO translation will land, so it unblocks the dissolution above.
Method constraint. Do not write a separate recorded package-review document
per area when the decisions are already dictated by
dependencies-and-contracts.md. That is the ceremony pattern this project
deliberately avoids.