Commit Graph
981 Commits
Author SHA1 Message Date
sam 69a71d032f fix(archieui): resolve service tokens from env and correct the stale State Store config comment
D1: the UI read [services.*].target_token verbatim, but the config loader does
not expand environment variables. A deployment that presents the gateway and
state tokens by env (as config.example.toml documents for remote consumers)
would leave archie-ui with an empty token, and staterpc.Dial then refuses a
non-loopback start. Mirror the daemon's stateStoreResolvedToken /
gatewayResolvedToken: explicit key, then GATEWAY_TOKEN / STATE_STORE_TOKEN.

D5: internal/config/services.go claimed the State Store service 'is not yet
extracted into its own process' and that target_token 'remains dead
configuration', both false since the daemon made an empty services.state.target
a hard startup error. State current truth.

D3: document that Web.Listen = 'off' is read with opposite meanings by the
dsaemon (disables the dashboard) and the UI process (falls through to the
default listener), pinning it as the one overloaded key an operator preparing
the split would set.
2026-09-09 02:38:16 +10:00
sam 82ebd3974a test(webui): cover dashboard data paths over the remote State Store read contracts
The UI Service serves task, capture, mapping and binding routes against a
*staterpc.Client, not a concrete *store.Store, but every existing webui test
fed a local store directly. Add remote coverage that stands up a real SQLite
store behind staterpc.RegisterServer, dials it, and asserts the handlers serve
over the wire. This surfaced two composition facts: the server Deps must wire
Captures/Mappings/Bindings/BindingDispatcher or those list endpoints fail
closed with err*Unavailable, and the timeline route's PathValue only resolves
through Server.Handler().

Log and health are asserted as deliberate degradation (log feed and health
registry are nil in a remote composition), matching the route inventory that
no State Store log/health contract exists.

Refs: archie-core-8cda.5.3
2026-09-09 01:32:27 +10:00
sam d45c7cf974 merge(gateway): flip channel boundary onto messaging.Message (archie-core-9g5e) 2026-09-09 01:15:02 +10:00
sam e3b1a859a8 feat(gateway): flip channel boundary onto messaging.Message
Replaced gateway.Message as the currency at the channel-facing boundary
with the canonical internal/domain/messaging.Message. Channel adapters
now build the record directly (RoleUser, ConversationID, Sender) and the
Router hands it back unchanged, so nothing converts between two shapes in
process. gateway.Message is deleted; the dashboard route that had no home
in the record lives in a new Inbound envelope (Message + transport-only
Page), which reaches the system prompt and is never persisted.

Role is now set at construction instead of derived at the store boundary,
and the assistant reply is a direct messaging.Message (RoleAssistant,
Sender=botUser). The gRPC proto is unchanged: pb.Message still carries no
role, and both sides keep deriving it from the owning session's bot
identity via gateway.RoleForSender (ToStoredMessage/FromStoredMessage are
deleted).

Adds tests that the old shape could not express: Page reaches the system
prompt but not the persisted record, and a channel-built Inbound round-trips
with the Role the channel set.

Refs: archie-core-9g5e
2026-09-09 01:05:36 +10:00
sam 4c9c7cac55 feat(gateway): authenticate the gRPC contract and allow off-loopback listeners
The Gateway had no transport boundary: RunGateway refused any non-loopback
listen address and serveGateway registered no interceptor, so archie-ui was
colocation-only while the State Store already had a token-authenticated
equivalent.

gatewayrpc gains a gateway-token metadata credential with constant-time
comparison and both unary and stream interceptors on each side. Stream
coverage is required here and not in staterpc: the Gateway's Stream RPC is
server-streaming. Dial fails closed on a non-loopback target with no token,
mirroring staterpc.Dial. gatewayServerOpts applies the same rule server-side:
loopback stays insecure, non-loopback requires a token, malformed errors.

The daemon resolves [services.gateway].target_token then GATEWAY_TOKEN;
archie-ui passes the token its config projection already carried, so a
distributed dashboard is now possible.

Closes: archie-core-juqx
2026-09-09 00:27:03 +10:00
sam fc18892324 docs(agents): list every step task check actually runs
The gate also runs proto:lint, proto:check and lint (Taskfile.yml:187-200);
the summary line omitted all three. A session that reads this line and
assembles go vet + go build + go test + gofumpt as an equivalent stand-in
misses golangci-lint entirely, which is how main last went red.
2026-09-09 00:26:54 +10:00
sam 126474a8b7 merge(archieui): standalone UI Service process composition (archie-core-8cda.5.2) 2026-09-08 22:59:59 +10:00
sam c00f8fc8db fix(gateway): restore the lint gate after the messaging store migration
89d8dc0 landed with golangci-lint red: scanSearchMessages was an exact copy
of scanMessages bar two error strings (dupl), and the pre-role schema
migration test seeded its fixture with db.Exec (noctx). task check has been
failing on main since.

Collapse the two scanners into one -- SearchMessages already selects the same
column order the history reads do, which the deleted function's own comment
said -- and seed the fixture through ExecContext.

No behaviour change: the gateway suite passes unchanged.
2026-09-08 22:59:39 +10:00
sam a3e8718d19 feat(archieui): standalone UI process over remote Gateway and State Store
Add cmd/archie-ui and internal/app/archieui: a second composition root that
serves the dashboard and the compiled SPA against the two contracts the daemon
already dials, so the UI stops needing the daemon's in-process objects.

The composition root lives outside internal/app/archied deliberately. That
package pulls 88 internal packages (daemon, forge, containers, NATS, secrets,
workflow); reusing its boot the way RunGateway and RunStateStore do would make
the PRD's deletion gate unreachable. archieui links 41, and cmd/archie-ui 42.

Configuration ownership follows docs/prds/ui-service-boundary.md: the process
reads the shared file, but only [services.gateway].{target,target_token},
[services.state].{target,target_token} and [web].{listen,
trust_forwarded_headers}, with flags taking precedence. Options has no field
for anything else in that file to land in, and no config.Holder.

compose() sets only the contract-backed and process-local fields. Every daemon
runtime handle stays nil on purpose and its route degrades through the
existing 501/503/empty paths. Captures, CaptureLimiter and BindingDispatcher
are withheld for the duration of the seam: the daemon's dashboard still serves
POST /webhooks/capture/{source}, and two intakes are not permitted.

Readiness is the UI's own registry over its two dependencies via a new bounded
readiness.ContractProbe. The existing GatewayProbe was unusable here: with no
channel manager to inject it reports "no gateway channels configured", i.e.
permanently degraded.

The State Store's fail-closed non-loopback rule and the two dial paths move to
staterpc.Dial/TargetIsLoopback and gatewayrpc.Dial, so the server side and
every client share one implementation instead of three copies.

The Gateway remains loopback-only with no auth interceptor, so this process is
scoped to being colocated with it; filed as archie-core-juqx.

Refs: archie-core-8cda.5.2
2026-09-08 22:54:26 +10:00
sam 89d8dc0e7f feat(messaging): migrate gateway session persistence onto domain types
SessionStore message history now stores/loads domain/messaging.Message:
sender attribution, derived role, and conversation address are canonical
record fields. SQLite gains a role column with open-time backfill for
pre-migration databases. gateway.Message remains the channel-facing
transport; conversion lives in one boundary module. ChatContract,
gatewayrpc proto, and dashboard JSON are unchanged; the RPC client
splits into ChatContract (Client) and SessionStore (StoreClient) views.

Sessions stay SessionContext: UUID keys and platform/bot/title/branch
metadata have no lossless home in Conversation; keyed migration tracked
separately once Agent/user/binding ownership settles.

Refs: archie-core-d9dv
2026-09-08 21:28:19 +10:00
sam 63a54e0850 docs(architecture): ratify UI service boundary 2026-09-07 18:29:42 +10:00
sam d64f2b12a5 ci(deploy): package archied/archie-gateway/archie-state-store/archie-playbooks as a linux/amd64 zip (archie-core-8cda.4.7.1)
New dist-zip job, independent of build-and-push (no change to the
archied/archie-agent Docker image jobs). Triggers on any archied/vX.Y.Z
tag, so a gateway-only release (RUNTIME=skip) still produces it, keyed
to the archied version alone -- these binaries aren't in
tools/release.sh's two-component split and don't need to be for this.

archie-agent is deliberately excluded: it only ever runs inside the
Docker image build-and-push already publishes, never as a host binary.

Zip contains the four linux/amd64 binaries, the new
deployments/INSTRUCTIONS.md (quickstart + deployment-profile/systemd
pointers), and CHANGELOG.md/CHANGELOG.archied.md/CHANGELOG.archie.md.
README.md is included only if present -- it doesn't exist in this repo
yet, so omitted for now rather than fabricated here.

Verified locally: all four binaries build clean under
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 (modernc.org/sqlite is pure Go,
no cgo sqlite driver to worry about), and the assembled zip's contents
match what the workflow step produces.
2026-09-07 18:10:11 +10:00
sam 89c1c53ed6 docs(agents): correct AGENTS.md's State Store section for the standalone process + task-grant reality
The in-process-multiplexed-server and daemon.StateStoreTokens bullets
described the pre-.4.6 design; neither boot.startStateStoreServer nor
daemon.StateStoreTokens exist in the codebase anymore. Replaced with
what's actually true: the standalone archie-state-store process,
staterpc.GrantIssuer/TaskGrants task-scoped credentials, and
archie-agent's single (no longer dual-path) gRPC Store.

Surfaced while reconciling beads .4.2-.4.6 against main -- all five
verified against the code and closed as genuinely done.
2026-09-07 17:34:16 +10:00
sam a9814e0910 test(state-store): restart/recovery + single-owner-SQLite checks, document task-grant scoping (archie-core-8cda.4.7)
Works through .4.7's remaining acceptance items:

- TestStateStoreDataSurvivesRestart: a task written by one
  archie-state-store process is still there for a fresh process opened
  against the same archie.db path.
- TestOpenStoresNeverOwnsTaskDB: the daemon/gateway composition path
  never populates b.st -- that field is reserved for the standalone
  binary, so archie.db always has exactly one owner.
- Audited internal/webui and friends for direct internal/store use:
  only the narrow contract interfaces (TaskStore, CaptureStore, ...)
  are imported, never a concrete *store.Store -- already satisfied,
  no code change needed.
- Updated docs/prds/state-store-contract.md's token lifecycle and
  payload-limits sections to describe what's actually implemented now
  (RegisterTaskGrant/RevokeTaskGrant, per-RPC scoping, the streaming
  capture RPCs) instead of the pre-.4.3 in-process-listener language.
- Ran the affected packages under -race; clean.

Left open, for Sam to decide rather than assume: archie-state-store has
no Dockerfile/CI build and isn't in RELEASING.md's two-component split
(tools/release.sh only tracks cmd/archied and cmd/archie-agent). Given
it's currently a native-binary/systemd deployment (deployments/*.toml),
this may be intentional -- flagging rather than inventing release
infra for a third binary unasked.
2026-09-07 16:45:30 +10:00
sam f5414a82a1 fix(state-store): task-scope container credentials, stream large capture lists, validate Update, wire dev's state-store (archie-core-8cda.4.7)
Closes six review findings against .4.6 (PR #775):

- daemon.StateStoreGrantIssuer + staterpc.TaskGrants: containers get a
  per-task credential authorized only for their own Update/Transition/
  InsertEvent, never the daemon's administrative token. Issued before
  Acquire, revoked on Release; fails closed if no issuer is wired.
- StreamCaptures/StreamUndispatchedCaptures: new streaming RPCs so a
  batch of large capture bodies can't exceed gRPC's 4MiB unary cap.
  The original unary RPCs are kept, unmodified (deprecated) rather than
  reshaped, so buf breaking stays green for an in-flight rolling deploy.
- Update rejects a nil Task with InvalidArgument instead of panicking in
  the store layer.
- RequireHostGateway only forces on for embedded NATS, so an externally
  reachable NATS/State Store no longer requires a local Docker bridge.
- Client-side token resolution now matches the server's (target_token,
  then STATE_STORE_TOKEN secret/env).
- task dev now runs archie-state-store (dev:state-store) alongside
  archied/archie-gateway/vite; deployments/dev.toml points them at it.
2026-09-07 13:18:44 +10:00
sam cead0e927d test(staterpc): verify State Store end-to-end and preserve context error identity (#775)
* test(state-store): real-process smoke + deadline/sentinel error contract

Add end-to-end verification for archie-core-8cda.4.7, the .4.7 State Store
verification stage against docs/prds/state-store-contract.md (rev. 2c):

- Real-process smoke coverage: build ./cmd/archie-state-store, run it against
  a minimal config on loopback, poll the readiness surface, and drive the
  State Store gRPC contract through staterpc.NewClient exactly as a consumer
  does. Covers the combined task/event/capture view, error-sentinel fidelity
  across the wire, the found=false-not-error convention, single-owner SQLite
  (the binary owns the one archie.db file; the consumer never opens a store
  file), per-call deadline behaviour, restart/recovery (SIGTERM + reopen on
  the same db), and the remote capture/mapping/binding surfaces.

- Fix a real deadline-identity gap in unmapError: a cancelled or
  deadline-exceeded context surfaced by gRPC-Go as a *status.Error was
  re-wrapped into a sanitised internal-style error, breaking
  errors.Is(err, context.Canceled/DeadlineExceeded). The agent's deadlineStore
  bounds each Store call with context.WithTimeout and the workflow consumer
  (workflow.go) uses errors.Is(err, context.DeadlineExceeded) to mark a stage
  interrupted rather than failed, so the re-wrap would wrongly park an
  interrupted task. Unmap Canceled/DeadlineExceeded codes back to the standard
  context sentinels.

- Pin the error-mapping contract in staterpc: context-identity preservation,
  sentinel round-trip fidelity for every store sentinel, and the sanitisation
  rule (an Internal error must not leak raw detail or map to a sentinel).

* test(state-store): assert single-owner SQLite composition (daemon/gateway require target)

The single-owner invariant (docs/prds/state-store-contract.md §12 step 7/8) is
that only archie-state-store opens archie.db; the daemon and gateway resolve
their task-store surface from openStateStoreAdapter, which requires
[services.state].target and opens no local store. This test pins that
composition root so a future change cannot silently reintroduce a local
*store.Store in the daemon or gateway path (dual-store ownership).

* test(state-store): skip real-process smoke tests under -short

The smoke tests build and exec the real archie-state-store binary, which is
slow enough that a -short run (e.g. a focused edit loop) should skip them.
task check (go test ./... -count=1) does not use -short, so they still run in
the full gate.

* test(state-store): fix subprocess log race under -race

The smoke test captured the subprocess's stdout/stderr in a strings.Builder
that the os/exec writer goroutine and the test reader goroutine touched
concurrently, racing under -race. Replace it with a mutex-guarded buffer, the
standard pattern for reading a running subprocess's output.

* test(state-store): satisfy lint (noctx, errorlint) on new tests

- exec.Command -> exec.CommandContext, http.Get -> NewRequestWithContext +
  Client.Do, so the subprocess and readiness polling honour the test context
  (noctx).
- errorlint: compare the raw non-status error via errors.Is rather than != on
  a wrapped error.

* docs(deployments): document State Store readiness verification and single-owner invariant

The deployment runbook described how to launch archie-state-store but not how
to verify it is healthy before starting archied. Document the -ready-addr
readiness surface (/healthz, /health/detailed state_db component) and the
single-owner invariant: one task SQLite file owned by archie-state-store, with
archied/archie-gateway never opening archie.db directly (state-store-contract
§12 step 7/8).

* fix(state-store): preserve context deadline/cancel identity in mapError

The symmetric server-side gap to the unmapError fix: if a store query fails
because the caller's context expires mid-call (§6 StoreTimeout), the handler's
mapError folded context.Canceled/DeadlineExceeded into codes.Internal. The
client would receive a sanitised internal error with no deadline identity, so
the agent's workflow consumer (errors.Is(err, context.DeadlineExceeded)) could
not distinguish an interrupted stage from a failed one.

Map a context cancel/deadline error to its matching gRPC code so unmapError
rehydrates it back to the standard context sentinel, completing the §6/§7
round-trip. Pin both directions with TestMapErrorPreservesContextIdentity.
2026-09-07 05:47:36 +10:00
sam 05529d48d5 refactor(state-store): cut daemon/Gateway over to the remote State Store (archie-core-8cda.4.6) (#774)
* refactor(state-store): delete NATS storerpc; agent store is gRPC-only (archie-core-8cda.4.6)

The agent is fully on the gRPC State Store contract, so the legacy NATS
storerpc path is dead (docs/prds/state-store-contract.md §12 step 4):
- delete internal/storerpc (Server, Client, subjects) and its tests
- drop the storerpc registration from registerTaskRPCServers and the
  now-unused store param
- make the agent transport's State Store URL required and Store() return
  only the gRPC staterpc.Client (no NATS fallback)
- rework agentworker/agent transport tests to back the store with a real
  gRPC State Store server instead of the deleted NATS storerpc server

The agent binary now fails fast when STATE_STORE_URL is absent. Agent store
calls (workflow.Store) go over gRPC to the archie-state-store service.

* refactor(state-store): delete daemon/gateway in-process store ownership (archie-core-8cda.4.6)

The daemon and gateway no longer open archie.db in-process: the single SQLite
file is owned by the standalone archie-state-store process, and both consumers
dial its gRPC State StateStore contract (docs/prds/state-store-contract.md
§12 step 7, no dual-store ownership).

- openStores no longer opens the production task store or seeds b.st; it only
  wires secrets/forge and the gateway conversation store
- openStateStoreAdapter now requires [services.state].target (the old local
  default is deleted) and resolves b.stateStore as the remote *staterpc.Client
- delete the in-process startStateStoreServer (the multiplexed gRPC server and
  its per-task bearer-token registry) from the daemon composition
- RunGateway now resolves the same remote State Store adapter; its session
  SQLite stays Gateway-owned and untouched
- daemon.ConnectedStateStore carries the configured target plus the resolved
  bearer token, which containerEnv injects as STATE_STORE_URL / STATE_STORE_TOKEN
  so archie-agent authenticates to the same remote service
- readiness store probe reads b.stateStore

PRD-consistent choice (noted): with the in-process server gone, per-task token
generation is removed; the daemon/gateway/agent all present the single
[services.state].target_token (or STATE_STORE_TOKEN secret) that the standalone
store validates.

* docs(webui): require [services.state].target in config and deployment templates (archie-core-8cda.4.6)

After the in-process store path is deleted, archied and archie-gateway dial the
standalone archie-state-store gRPC service via [services.state].target and never
open archie.db themselves (docs/prds/state-store-contract.md §12 step 7).

- config.example.toml documents [services.state].target as required, with the
  loopback vs Docker-bridge gateway + target_token rule
- deployment templates set [services.state] per topology: bridge gateway +
  token for container-mode agents, loopback for host-only
- deployments/README.md and systemd-user-service.md describe start commands
  for both topologies and drop the stale "not yet consumed by the daemon" note

* style: gofumpt alignment for State Store daemon literal; appease noctx/staticcheck (archie-core-8cda.4.6)

* docs(state-store): refresh stale in-process references post-cutover (archie-core-8cda.4.6)
2026-09-07 05:06:12 +10:00
sam 9ef16151f4 refactor(state-store): migrate workflow/intake/task lifecycle consumers to State Store (archie-core-8cda.4.5) (#773)
Route the daemon's task-lifecycle / work-intake / workflow consumers (the
TaskStore composite) through b.stateStore instead of reaching into b.st
directly, the same additive composition pattern .4.4 used for the
capture/mapping/binding contracts (docs/prds/state-store-contract.md
§10/§12 step 6). b.stateStore is the in-process *store.Store by default and
a remote *staterpc.Client dialed to the standalone archie-state-store gRPC
service when [services.state].target is set, so this is behaviour-neutral
until an operator flips the seam.

Consumers migrated: daemon.Daemon.Store, webui.Server.Store, the message
gateway routers, the chat task lister/log-reader adapters, the operator
task action service, event persistence (persistAndBroadcastEvents), the
manual requeue path, and the chat task creator/controller.

openStores now seeds b.stateStore = b.st so it is never nil in either the
daemon (Run) or the gateway (RunGateway) composition; the gateway never
calls openStateStoreAdapter, so it stays on the local adapter and never
dials a remote store (preserving the .4.4 fail-closed intent). The
in-process State Store gRPC server keeps fronting b.st (the agent's gRPC
and NATS storerpc handoff paths both stay on the daemon's local store) and
the in-process serving path is untouched (deletion is .4.6).

Adds focused composition-regression tests asserting the daemon, webui and
task-actions consumers resolve their store from b.stateStore, plus the
existing bufconn conformance suite (grpc adapter) already proves contract
fidelity for the TaskStore surface.
2026-09-07 04:28:10 +10:00
sam 5fe5ff6626 refactor(state-store): migrate capture/mapping/binding consumers to State Store (archie-core-8cda.4.4) (#772)
* feat(state-store): add daemon composition path for remote store adapter ($services.state)

Add the additive composition path the daemon's own capture consumers use
(archie-core-8cda.4.4, docs/prds/state-store-contract.md §10/§12 step 6).
bootstrap.go opens production store then resolves b.stateStore: the local
*store.Store by default, or a remote *staterpc.Client dialed to the
standalone archie-state-store gRPC service when [services.state].target is
set. Wire the dashboard CaptureStore surface from b.stateStore.

The in-process StateStore gRPC server keeps serving the daemon's SAME local
store (stateStoreDeps from b.st), not the consumer-facing b.d.* adapters:
task lifecycle (b.st) stays in-process until .4.5/.4.6 and there is no
dual-store ownership. Fail-closed on a non-loopback target without a bearer
token mirrors the standalone server's confinement (§9).

* feat(state-store): route daemon mapping consumers through the state store adapter

Migrate the MappingStore consumer to the State Store contract adapter
(archie-core-8cda.4.4 step 6, Mapping swap): the dashboard's mapping editor
(web.Mappings) and the daemon's dispatch loop (d.Mappings) resolve from
b.stateStore instead of reaching into b.st directly. Local by default,
remote *staterpc.Client when [services.state].target is set.

* feat(state-store): route daemon binding consumers through the state store adapter

Migrate the BindingStore/BindingDispatcher/BindingTaskCreator consumers to
the State Store contract adapter (archie-core-8cda.4.4 step 6, Binding
swap): the dashboard's binding editor (web.Bindings / web.BindingDispatcher)
and the daemon's dispatch loop (d.Bindings / d.BindingDispatcher /
d.BindingTaskCreator) resolve from b.stateStore instead of b.st. Local by
default, remote *staterpc.Client when [services.state].target is set.

* refactor(state-store): resolve daemon state store adapter only in daemon composition

openStores is shared with the standalone archie-gateway (RunGateway), which
does not consume the capture/mapping/binding surfaces. Resolve b.stateStore
from the daemon's own Run() composition instead, so a gateway config that
happens to carry [services.state].target can never dial (or fail closed
against) a state store it does not use. openStateStoreAdapter takes no ctx:
grpc.NewClient connects lazily, and the adapter resolution is synchronous.
2026-09-07 04:04:34 +10:00
sam a59505816a feat(state-store): stand up archie-state-store binary and own archie.db (archie-core-8cda.4.3) (#771) 2026-09-07 03:41:32 +10:00
sam 576e22b554 feat(state-store): generalize storerpc as the State Store gRPC contract (archie-core-8cda.4.2) (#770)
* feat(state-store): generalize storerpc as the State Store gRPC contract

Implements archie-core-8cda.4.2 against docs/prds/state-store-contract.md
rev. 2c: generate protobuf/gRPC client and server adapters for every
ratified store contract (proto/state/v1/state.proto, 40 RPCs, one
StateStoreService), add internal/infrastructure/staterpc with a bufconn
conformance suite exercising both local and remote adapters, serve the
service in-process (bridge address + mandatory per-task bearer token when
agent containers exist, loopback-only otherwise), and flip archie-agent's
workflow.Store to a long-lived gRPC client when the daemon injects
STATE_STORE_URL/STATE_STORE_TOKEN (falling back to the legacy NATS
storerpc path otherwise -- one agent process is never both).

Also drops the *sql.Tx parameter from store.BindingDispatcher.RecordDispatch
(a required wire-blocker fix: a transaction cannot cross a gRPC boundary,
and production always passed nil).

fix(store): drop RecordDispatch's unusable *sql.Tx parameter
feat(config): add services.state contract-adapter selector

* fix(state-store): apply no-mistakes review decisions

- Drop StreamTokenInterceptor (unused; the State Store service has no
  streaming RPCs and nothing calls it).
- Make StateStoreTokens.Validate compare tokens in constant time
  (crypto/subtle) per the auto-fix hardening.ing finding.
- Amend config.example.toml / config.Services.State doc text so it frames
  [services.state].target as the ratified surface consumed by the upcoming
  standalone archie-state-store binary (.4.3), not active today.
2026-09-07 03:01:33 +10:00
sam 3175526732 fix(archied): initialize conversation persistence on daemon startup 2026-09-07 00:49:54 +10:00
sam 5c0efaceaf refactor(workflow): relocate Task/Status/Source and define workflow.Store (#754)
* refactor(workflow,store): relocate Task/Status/Source and define workflow.Store

Move store.Task, store.Status*/store.Source*, and the 3-method workflow
store interface out of internal/store into internal/domain/workflow as
workflow.Task/workflow.Status/workflow.Source plus a consumer-owned
workflow.Store, superseding store.WorkflowStore. internal/domain/workflow
now imports zero internal/store symbols; internal/store persists the
domain type instead (infrastructure depends on domain), and its Store
satisfies workflow.Store.

Updates every production and test caller of the relocated symbols across
daemon, webui, gateway, agentworker, agenttransport/nats, storerpc,
taskrun, and worktreerpc, and removes TestRecordDispatchViaExplicitTx per
docs/prds/state-store-contract.md (rev. 2c) sections 4 and 12 step 2.

Two workflow-package integration tests that exercise the real *store.Store
move to an external workflow_test package (store_integration_test.go) to
avoid a workflow(test)->store->workflow(prod) import cycle now that store
depends on workflow.

* no-mistakes(document): docs(state-store): mark §4.8 relocation done in state-store-contract
2026-09-07 00:39:26 +10:00
sam a56414cf7a fix(archied): preserve channel turn ledger with remote gateway 2026-09-07 00:29:53 +10:00
sam 3b85e67cb7 docs(state-store): bump internal revision labels to rev. 2c
Update the editorial revision labels so agents reading the handoff don't think
they are on an older revision: the date and the section annotations in
docs/prds/state-store-contract.md (contract-ownership fork, transitional
internal/store, error-detail sanitisation, transport boundary, token lifecycle,
fail-closed) that describe decisions retained through rev. 2c now read
(rev. 2c), and the service-decomposition PRD's ratification pointer reads
(rev. 2c). The revision-history sentence describing what rev. 2 resolved is
kept as narrative.
2026-09-07 00:04:04 +10:00
sam 29e4ccf6eb docs(state-store): rev.2c resolves third-pass contradictions
Address the third-pass review of the State Store ratification doc:

- service-decomposition.md Q3 generic migration wording no longer contradicts
  the ratified design: the compile-time assertion example is now
  var _ workflow.Store = (*Client)(nil) (store.WorkflowStore is superseded),
  and composition selection is described as presence-based
  [services.<name>].target (empty=local, set=dial remote) instead of the dropped
  mode = "inproc" | "remote" enum. Defaults differ per service (State Store local,
  gateway remote-on-localhost). Points to state-store-contract.md section 10.
- State Store listener topology is now a single explicit rule: one in-process
  listener, bound to the host-gateway bridge address with mandatory bearer-token
  auth when agent containers consume it (the Phase 2 default), or loopback-only
  for a daemon-local-only consumer. Removed the contradictory
  "default loopback-only" wording (a container cannot reach the host's
  127.0.0.1) and the dual-listener ambiguity across sections 6, 9, and 13.
2026-09-07 00:01:38 +10:00
sam ee42cebd97 docs(state-store): rev.2b resolves second-pass review inconsistencies
Address the second-pass review of the State Store ratification doc (beads archie-core-8cda.4.1):

- State migration-decisions.md section 4 remains authoritative and open; this PRD
  does not replace it. The Task/Status/Source + interface relocation is flagged as
  a minimal, pulled-forward Phase 2 prerequisite only.
- Fix stale WorkflowStore reference in service-decomposition.md Q4: the workflow
  contract is the domain-owned workflow.Store, which supersedes internal/store's
  WorkflowStore.
- Label internal/store as a TRANSITIONAL compatibility location (target
  internal/infrastructure/store), matching dependencies-and-contracts.md.
- Call out the current-state outliers explicitly: workflow files/tests still import
  internal/store, store.WorkflowStore is still in TaskContext, and the NATS storerpc
  path is still wired -- the acceptance criterion is NOT yet met.
- Operationalize the agent gRPC handoff: who binds the in-process State Store
  listener (daemon, mirroring serveGateway), Docker-bridge host-gateway address
  selection (reuses the embedded-NATS container/network.go resolution), per-task
  injection via containerEnv beside NATS_URL, per-incumbence bearer token lifecycle
  + gRPC-interceptor validation -> codes.Unauthenticated, and fail-closed on
  non-loopback targets without TLS/token.
- Update beads archie-core-8cda.4.2/.4.5 descriptions to match the ratified scope
  (type/interface relocation, RecordDispatch *sql.Tx drop, in-process serving,
  agent gRPC handoff).
2026-09-06 23:58:17 +10:00
sam 2fe9577b0b docs(state-store): rev.2 resolves reviewer conditions on State Store contract
Address the review of the Phase 2 ratification doc (beads archie-core-8cda.4.1):

1. Domain-dependency contradiction: ownership splits by consumer layer. The
   workflow domain owns a consumer-owned workflow.Store interface plus the
   Task/Status/Source types (dependency rules #2/#7), so internal/domain/workflow
   imports no internal/store — satisfying the Phase 2 acceptance criterion. The
   daemon/webui store surfaces stay producer-owned in internal/store. The type +
   interface relocation is a Phase 2 prerequisite pulled from migration-decisions
   section 4 (store.Task -> WorkflowExecution).
2. Agent gRPC endpoint propagation: explicit boot handoff via STATE_STORE_URL /
   STATE_STORE_TOKEN (mirrors the NATS handoff), added to agentworker.Settings and
   the Transport.Store facade; per-call timeout + long-lived grpc client.
3. Transport security + error sanitisation: loopback-only + insecure default, agent
   authenticates over the container bridge with a per-task bearer token; TLS/mTLS
   is an operator choice for non-loopback/k8s. Sanitised public error details in
   gRPC; full detail logged server-side only.

Also records RecordDispatch dropping *sql.Tx as a required interface migration
before .4.2. Cross-references updated in the service-decomposition PRD.
2026-09-06 23:45:19 +10:00
sam bcde4ae69a docs(state-store): ratify Phase 2 State Store contract boundary and transport
Define the authoritative Phase 2 State Store contract boundary and
transport design (beads archie-core-8cda.4.1): one StateStore gRPC
service (40 RPCs) behind the narrow typed store contracts, producer-owned
in internal/store, with a staterpc.Client remote adapter that satisfies
the same interfaces via explicit compile-time assertions. Records method
ownership, DTO/wire representation, error semantics (structured gRPC codes
rehydrated to store sentinels so errors.Is survives), limits, mode/config
shape (presence-based [services.state].target, default local), local-
adapter compatibility, and cutover/deletion rules. Fixes the RecordDispatch
wire blocker (*sql.Tx dropped; production passes nil) and keeps Close()
off the wire. References the new doc from the service-decomposition PRD.
2026-09-06 23:36:52 +10:00
sam 703f09cf51 Merge origin/main updates 2026-09-06 22:50:45 +10:00
sam 9eba9be2e5 fix(gateway): share NATS and session ownership 2026-09-06 22:49:39 +10:00
sam 3f78066f22 fix(webui): distinguish refused vs broken task actions, handle 401 (#753)
Failed task actions in the dashboard rendered the same generic message
regardless of cause. ApiError already carried the HTTP status, but nothing
read it: a refused mutation (400/403/409 - bad input, CSRF, conflicting
state) looked identical to a broken one (5xx, network drop, timeout), and
an expired session (401, whether archied's own token cookie or an upstream
forward-auth proxy) showed a generic failure instead of prompting
re-authentication.

Add classifyActionError() in api.jsx to map a caught error to a refused /
broken / session-expired kind, and use it in tasks.jsx to render each with
distinct framing and coloring (reusing the existing warn/danger tokens
pill.css already establishes) - refused actions get "you can't do that",
broken ones get "try again, or check the daemon", and an expired session
gets a reload prompt.

Refs archie-core-h3lg.
2026-09-06 02:34:34 +10:00
sam 5bd553d8ed Merge branch 'feat/gateway-service': Phase 1 Gateway Service extraction
Stands up archie-gateway as its own binary serving ChatContract over gRPC
and makes it the daemon's only transport for webui chat (Q3 part 5's
atomic flip -- the in-process serving path is deleted, not deprecated).

- contract seams + protobuf toolchain (Phase 0)
- single taskactions.Service across webui, daemon NATS responder and the
  standalone gateway's client
- services.gateway.target is the only config surface left; mode is gone
  now that remote is the only transport
- task dev launches archie-gateway alongside archied + Vite

Verified end-to-end against live infra: real GitHub token (zero repos
configured, so no repo was polled or touched) and the real
@heyarchie_home_bot Telegram bot, both processes booted together against
shared NATS/store, chat and /status round-tripped successfully.

# Conflicts:
#	go.sum
2026-09-06 02:26:19 +10:00
sam 14e5b542bf fix(gateway): drop services.gateway.mode, remote is the only transport
Mode had exactly one legal value after the previous commit deleted the
in-process path, which made it dead config surface that could only
confuse an operator into thinking there was still a real choice.
ServiceConnection is now just a target address; composeChatContract only
checks it's non-blank, defaulting to 127.0.0.1:8585.
2026-09-06 02:13:03 +10:00
sam b319d77c56 feat(gateway): delete in-process ChatContract path, make remote the only mode
archie-gateway is now the sole owner of ChatContract; the daemon always
dials it over gRPC for webui chat, no in-process fallback left. Telegram
and email keep their own in-process routers unchanged (their streaming
responder is a callback that can't cross the wire-safe contract).

- composeChatContract only accepts mode="remote"; config defaults
  Services.Gateway to remote + 127.0.0.1:8585 when unset
- deleted taskActionsActor (dead code once the local branch is gone)
- task dev now launches archie-gateway alongside archied + Vite

Verified end-to-end: archie-gateway + archied booted against a shared
NATS/store, webui's /api/chat/sessions round-tripped through the gRPC hop.
2026-09-06 02:07:02 +10:00
sam 8e41bb9a66 feat(gateway): consolidate task actions and repair daemon default path
Phase 1 (extract the Gateway Service), first stage. Repairs the
half-migrated working tree so the default (inproc) daemon path works
again while keeping the WIP's direction:

- chatTaskActorAdapter is contract-only (no HTTP detour); nil contract
  returns ErrChatCapabilityUnavailable instead of panicking.
- openStores opens the conversation store for both processes again;
  the standalone gateway no longer owns it exclusively. Phase 1 keeps
  the daemon's Telegram/email routers and session-memory curator
  reading the same SQLite file; remote conversations arrive when a
  process that cannot open the file needs them.
- setupLLMAndChat composes the local gateway (setupGatewayChat) in
  default mode and a remote gRPC client in mode=remote; the returned
  contract is never nil, and Run handles the error.
- taskactions.Service is now the single task-mutation implementation
  shared by the dashboard (webui TaskActionService), the daemon's local
  gateway actor, and the NATS responder. The daemon registers
  taskactions.Register so the standalone Gateway's client can forward
  actions; transport follows the repo's natsrpc pattern.
- uncommitted sessions.proto / SessionStoreService gRPC contract
  deferred: it has no wired consumers (scope creep for this stage).
- task_action_adapter_test.go rewritten against the contract chain.

task check passes; archied and archie-gateway both build.
2026-09-06 01:35:15 +10:00
sam 0a6496d2cd feat(gateway): extract ChatContract service 2026-09-05 23:06:16 +10:00
sam 34cb25f130 feat(contracts): add gateway service seams and protobuf toolchain 2026-09-05 22:58:00 +10:00
sam c0a91c011c feat(contracts): add gateway service seams and protobuf toolchain 2026-09-05 22:57:45 +10:00
sam 6affe0eea3 chore(curator): add embedding client and sampler interface prerequisites (#752)
* feat(curator): add embedding client and sampler wave-1 prerequisites

Bead archie-core-1786637500752-415-434e5bb9 (GitHub #436) was closed with
no linked PR/commits, and the surprisal-sampling bead
archie-core-1786637500776-416-600a66e5 (GitHub #437) found none of its
stated prerequisites -- an embedding client, a Sampler interface, or the
four wave-1 cheap sampling strategies -- actually exist. This builds them:

- internal/domain/embedding + internal/infrastructure/embedding: a narrow
  typed embedding client, config-driven via a providers table + a role in
  [models], mirroring the existing chat model role convention. Degrades to
  capability-unavailable (never fails startup or a call) for a missing
  role, unknown provider, unsupported class, or unresolved credential.
  Wired as an optional capability in bootstrap.go's setupEmbeddings.
- internal/domain/sampling: the Sampler interface plus recency, random,
  all, and staleness-proxy strategies, per docs/prds/curator-sampler-wave1.md.
  Deterministic given fixed inputs, matching GitHub #437's stated
  acceptance criterion for the embedding-backed strategy it will add
  behind this same interface (not part of this change).

Table-driven tests throughout; httptest for the embedding client, no live
service. No curator wiring yet -- nothing consumes either capability in
this change.

* no-mistakes(document): docs(architecture): mark curator sampling prereqs as landed
2026-09-05 22:53:08 +10:00
sam cd2ab3516b feat(gateway): render workspace, managed repos and operator into chat env
The chat agent had no way to know where it lives without probing the
filesystem (pwd, find, git remote), wasting turns on every orientation.
Per docs/architecture/identity.md the <env> block is Layer 1 invariants,
so the daemon's own facts belong there, driven from live configuration.

Render into the prompt's <env> block:
- workspace path (chat.workspace)
- repositories under management with forge host and default branch
- operator identity (chat.operator, which was dead plumbing)

Unconfigured values render explicitly as unknown / none configured
rather than a plausible-looking default, and a repo with no forge
reports forge unknown, so the model never claims a fact the daemon
cannot confirm. Repos are resolved per active chat identity, mirroring
chatTaskProfiles.

gateway: add RepoEnv, SystemPromptConfig Workspace/Repos/Operator,
TurnRunnerConfig Workspace/Repos.
app: chatRepoEnv resolver wired through newChatTurnRunner.
tests: prompt rendering (incl. explicit-unknown + hostile-SOUL guard)
and chatRepoEnv identity resolution. go.sum gains module-graph
/go.mod checksum lines.
2026-09-05 22:52:03 +10:00
sam 2d8a610bab fix(deploy): pin archie-agent image away from :latest (#751)
ghcr.io/samcharles93/archie-agent:latest is re-published on every main
push that touches the runtime's package closure, so a bare :latest
reference can change under a running deployment with no warning and no
way to reproduce a prior build.

archie-agent has no published semver tag yet -- CI (deploy.yml) only
ever pushes :latest to GHCR, confirmed via the GHCR package API. Pin
docker-compose.yml and the GitHub deployment profiles to the verified
digest of :latest as of 2026-09-05
(sha256:870f1b1357979cfea23fccacb24166ebeb9ab321708ef2c2b218a763ed99b090),
and document the pinning strategy -- including the semver-tag-once-
published preference and digest pinning for production -- in
deployments/systemd-user-service.md. Update the generic
docker-nats-stack.toml example to demonstrate the same convention.

Ref archie-core-kllw (GitHub #670)
2026-09-05 22:19:05 +10:00
sam 3c304b26ce chore: cleanup outdated feature idea for remote system input 2026-09-05 22:12:32 +10:00
sam e7afa5cebe fix(releaseupdate): use context-aware database calls in tests 2026-09-05 21:14:28 +10:00
sam ecfc2c98b1 chore: update go modules
closes #shl7 / #673
2026-09-05 20:42:48 +10:00
sam 659e914cf1 chore(beads): stop tracking issues.jsonl/interactions.jsonl in git
Dolt is the source of truth and already syncs via its own git+ remote
(bd dolt push/pull). Committing the JSONL export separately produced a
steady stream of no-op "chore(beads)" commits that skewed commit counts
without adding information. Untrack the export files, disable
export.auto/git-add, and drop the beads-commit step from CLAUDE.md's
Session Completion Protocol.
2026-09-05 20:37:45 +10:00
sam b51fffe235 Merge pull request #750 from samcharles93/feat/748-skillcurator-nilerr-finding-at-skillcuratorgo110
skillcurator: nilerr finding at skillcurator.go:110 (archie)
2026-09-05 18:55:06 +10:00
samcharles93 07b4c3e3a1 skillcurator: nilerr finding at skillcurator.go:110 (archie)
Implements #748
2026-09-05 08:27:35 +00:00
samcharles93 068448bd7f fix: baseline gate repair (go test ./... -count=1) 2026-09-05 08:24:10 +00:00
sam 58956328f9 chore: beads interactions and issues 2026-09-05 17:11:43 +10:00