Commit Graph
867 Commits
Author SHA1 Message Date
sam 3ed17661b4 fix(gateway): restore the lint gate after the messaging store migration
5713231 landed with golangci-lint red: scanSearchMessages was an exact copy
of scanMessages bar two error strings (dupl), and the pre-role schema
migration test seeded its fixture with db.Exec (noctx). task check has been
failing on main since.

Collapse the two scanners into one -- SearchMessages already selects the same
column order the history reads do, which the deleted function's own comment
said -- and seed the fixture through ExecContext.

No behaviour change: the gateway suite passes unchanged.
2026-09-08 22:59:39 +10:00
sam 3d611f18ab feat(archieui): standalone UI process over remote Gateway and State Store
Add cmd/archie-ui and internal/app/archieui: a second composition root that
serves the dashboard and the compiled SPA against the two contracts the daemon
already dials, so the UI stops needing the daemon's in-process objects.

The composition root lives outside internal/app/archied deliberately. That
package pulls 88 internal packages (daemon, forge, containers, NATS, secrets,
workflow); reusing its boot the way RunGateway and RunStateStore do would make
the PRD's deletion gate unreachable. archieui links 41, and cmd/archie-ui 42.

Configuration ownership follows docs/prds/ui-service-boundary.md: the process
reads the shared file, but only [services.gateway].{target,target_token},
[services.state].{target,target_token} and [web].{listen,
trust_forwarded_headers}, with flags taking precedence. Options has no field
for anything else in that file to land in, and no config.Holder.

compose() sets only the contract-backed and process-local fields. Every daemon
runtime handle stays nil on purpose and its route degrades through the
existing 501/503/empty paths. Captures, CaptureLimiter and BindingDispatcher
are withheld for the duration of the seam: the daemon's dashboard still serves
POST /webhooks/capture/{source}, and two intakes are not permitted.

Readiness is the UI's own registry over its two dependencies via a new bounded
readiness.ContractProbe. The existing GatewayProbe was unusable here: with no
channel manager to inject it reports "no gateway channels configured", i.e.
permanently degraded.

The State Store's fail-closed non-loopback rule and the two dial paths move to
staterpc.Dial/TargetIsLoopback and gatewayrpc.Dial, so the server side and
every client share one implementation instead of three copies.

The Gateway remains loopback-only with no auth interceptor, so this process is
scoped to being colocated with it; filed as archie-core-juqx.

Refs: archie-core-8cda.5.2
2026-09-08 22:54:26 +10:00
sam 571323138d feat(messaging): migrate gateway session persistence onto domain types
SessionStore message history now stores/loads domain/messaging.Message:
sender attribution, derived role, and conversation address are canonical
record fields. SQLite gains a role column with open-time backfill for
pre-migration databases. gateway.Message remains the channel-facing
transport; conversion lives in one boundary module. ChatContract,
gatewayrpc proto, and dashboard JSON are unchanged; the RPC client
splits into ChatContract (Client) and SessionStore (StoreClient) views.

Sessions stay SessionContext: UUID keys and platform/bot/title/branch
metadata have no lossless home in Conversation; keyed migration tracked
separately once Agent/user/binding ownership settles.

Refs: archie-core-d9dv
2026-09-08 21:28:19 +10:00
sam 411f6fdf8c test(state-store): restart/recovery + single-owner-SQLite checks, document task-grant scoping (archie-core-8cda.4.7)
Works through .4.7's remaining acceptance items:

- TestStateStoreDataSurvivesRestart: a task written by one
  archie-state-store process is still there for a fresh process opened
  against the same archie.db path.
- TestOpenStoresNeverOwnsTaskDB: the daemon/gateway composition path
  never populates b.st -- that field is reserved for the standalone
  binary, so archie.db always has exactly one owner.
- Audited internal/webui and friends for direct internal/store use:
  only the narrow contract interfaces (TaskStore, CaptureStore, ...)
  are imported, never a concrete *store.Store -- already satisfied,
  no code change needed.
- Updated docs/prds/state-store-contract.md's token lifecycle and
  payload-limits sections to describe what's actually implemented now
  (RegisterTaskGrant/RevokeTaskGrant, per-RPC scoping, the streaming
  capture RPCs) instead of the pre-.4.3 in-process-listener language.
- Ran the affected packages under -race; clean.

Left open, for Sam to decide rather than assume: archie-state-store has
no Dockerfile/CI build and isn't in RELEASING.md's two-component split
(tools/release.sh only tracks cmd/archied and cmd/archie-agent). Given
it's currently a native-binary/systemd deployment (deployments/*.toml),
this may be intentional -- flagging rather than inventing release
infra for a third binary unasked.
2026-09-07 16:45:30 +10:00
sam 8fef2e7eed fix(state-store): task-scope container credentials, stream large capture lists, validate Update, wire dev's state-store (archie-core-8cda.4.7)
Closes six review findings against .4.6 (PR #775):

- daemon.StateStoreGrantIssuer + staterpc.TaskGrants: containers get a
  per-task credential authorized only for their own Update/Transition/
  InsertEvent, never the daemon's administrative token. Issued before
  Acquire, revoked on Release; fails closed if no issuer is wired.
- StreamCaptures/StreamUndispatchedCaptures: new streaming RPCs so a
  batch of large capture bodies can't exceed gRPC's 4MiB unary cap.
  The original unary RPCs are kept, unmodified (deprecated) rather than
  reshaped, so buf breaking stays green for an in-flight rolling deploy.
- Update rejects a nil Task with InvalidArgument instead of panicking in
  the store layer.
- RequireHostGateway only forces on for embedded NATS, so an externally
  reachable NATS/State Store no longer requires a local Docker bridge.
- Client-side token resolution now matches the server's (target_token,
  then STATE_STORE_TOKEN secret/env).
- task dev now runs archie-state-store (dev:state-store) alongside
  archied/archie-gateway/vite; deployments/dev.toml points them at it.
2026-09-07 13:18:44 +10:00
sam 33751abe4e test(staterpc): verify State Store end-to-end and preserve context error identity (#775)
* test(state-store): real-process smoke + deadline/sentinel error contract

Add end-to-end verification for archie-core-8cda.4.7, the .4.7 State Store
verification stage against docs/prds/state-store-contract.md (rev. 2c):

- Real-process smoke coverage: build ./cmd/archie-state-store, run it against
  a minimal config on loopback, poll the readiness surface, and drive the
  State Store gRPC contract through staterpc.NewClient exactly as a consumer
  does. Covers the combined task/event/capture view, error-sentinel fidelity
  across the wire, the found=false-not-error convention, single-owner SQLite
  (the binary owns the one archie.db file; the consumer never opens a store
  file), per-call deadline behaviour, restart/recovery (SIGTERM + reopen on
  the same db), and the remote capture/mapping/binding surfaces.

- Fix a real deadline-identity gap in unmapError: a cancelled or
  deadline-exceeded context surfaced by gRPC-Go as a *status.Error was
  re-wrapped into a sanitised internal-style error, breaking
  errors.Is(err, context.Canceled/DeadlineExceeded). The agent's deadlineStore
  bounds each Store call with context.WithTimeout and the workflow consumer
  (workflow.go) uses errors.Is(err, context.DeadlineExceeded) to mark a stage
  interrupted rather than failed, so the re-wrap would wrongly park an
  interrupted task. Unmap Canceled/DeadlineExceeded codes back to the standard
  context sentinels.

- Pin the error-mapping contract in staterpc: context-identity preservation,
  sentinel round-trip fidelity for every store sentinel, and the sanitisation
  rule (an Internal error must not leak raw detail or map to a sentinel).

* test(state-store): assert single-owner SQLite composition (daemon/gateway require target)

The single-owner invariant (docs/prds/state-store-contract.md §12 step 7/8) is
that only archie-state-store opens archie.db; the daemon and gateway resolve
their task-store surface from openStateStoreAdapter, which requires
[services.state].target and opens no local store. This test pins that
composition root so a future change cannot silently reintroduce a local
*store.Store in the daemon or gateway path (dual-store ownership).

* test(state-store): skip real-process smoke tests under -short

The smoke tests build and exec the real archie-state-store binary, which is
slow enough that a -short run (e.g. a focused edit loop) should skip them.
task check (go test ./... -count=1) does not use -short, so they still run in
the full gate.

* test(state-store): fix subprocess log race under -race

The smoke test captured the subprocess's stdout/stderr in a strings.Builder
that the os/exec writer goroutine and the test reader goroutine touched
concurrently, racing under -race. Replace it with a mutex-guarded buffer, the
standard pattern for reading a running subprocess's output.

* test(state-store): satisfy lint (noctx, errorlint) on new tests

- exec.Command -> exec.CommandContext, http.Get -> NewRequestWithContext +
  Client.Do, so the subprocess and readiness polling honour the test context
  (noctx).
- errorlint: compare the raw non-status error via errors.Is rather than != on
  a wrapped error.

* docs(deployments): document State Store readiness verification and single-owner invariant

The deployment runbook described how to launch archie-state-store but not how
to verify it is healthy before starting archied. Document the -ready-addr
readiness surface (/healthz, /health/detailed state_db component) and the
single-owner invariant: one task SQLite file owned by archie-state-store, with
archied/archie-gateway never opening archie.db directly (state-store-contract
§12 step 7/8).

* fix(state-store): preserve context deadline/cancel identity in mapError

The symmetric server-side gap to the unmapError fix: if a store query fails
because the caller's context expires mid-call (§6 StoreTimeout), the handler's
mapError folded context.Canceled/DeadlineExceeded into codes.Internal. The
client would receive a sanitised internal error with no deadline identity, so
the agent's workflow consumer (errors.Is(err, context.DeadlineExceeded)) could
not distinguish an interrupted stage from a failed one.

Map a context cancel/deadline error to its matching gRPC code so unmapError
rehydrates it back to the standard context sentinel, completing the §6/§7
round-trip. Pin both directions with TestMapErrorPreservesContextIdentity.
2026-09-07 05:47:36 +10:00
sam 3a0dfa924d refactor(state-store): cut daemon/Gateway over to the remote State Store (archie-core-8cda.4.6) (#774)
* refactor(state-store): delete NATS storerpc; agent store is gRPC-only (archie-core-8cda.4.6)

The agent is fully on the gRPC State Store contract, so the legacy NATS
storerpc path is dead (docs/prds/state-store-contract.md §12 step 4):
- delete internal/storerpc (Server, Client, subjects) and its tests
- drop the storerpc registration from registerTaskRPCServers and the
  now-unused store param
- make the agent transport's State Store URL required and Store() return
  only the gRPC staterpc.Client (no NATS fallback)
- rework agentworker/agent transport tests to back the store with a real
  gRPC State Store server instead of the deleted NATS storerpc server

The agent binary now fails fast when STATE_STORE_URL is absent. Agent store
calls (workflow.Store) go over gRPC to the archie-state-store service.

* refactor(state-store): delete daemon/gateway in-process store ownership (archie-core-8cda.4.6)

The daemon and gateway no longer open archie.db in-process: the single SQLite
file is owned by the standalone archie-state-store process, and both consumers
dial its gRPC State StateStore contract (docs/prds/state-store-contract.md
§12 step 7, no dual-store ownership).

- openStores no longer opens the production task store or seeds b.st; it only
  wires secrets/forge and the gateway conversation store
- openStateStoreAdapter now requires [services.state].target (the old local
  default is deleted) and resolves b.stateStore as the remote *staterpc.Client
- delete the in-process startStateStoreServer (the multiplexed gRPC server and
  its per-task bearer-token registry) from the daemon composition
- RunGateway now resolves the same remote State Store adapter; its session
  SQLite stays Gateway-owned and untouched
- daemon.ConnectedStateStore carries the configured target plus the resolved
  bearer token, which containerEnv injects as STATE_STORE_URL / STATE_STORE_TOKEN
  so archie-agent authenticates to the same remote service
- readiness store probe reads b.stateStore

PRD-consistent choice (noted): with the in-process server gone, per-task token
generation is removed; the daemon/gateway/agent all present the single
[services.state].target_token (or STATE_STORE_TOKEN secret) that the standalone
store validates.

* docs(webui): require [services.state].target in config and deployment templates (archie-core-8cda.4.6)

After the in-process store path is deleted, archied and archie-gateway dial the
standalone archie-state-store gRPC service via [services.state].target and never
open archie.db themselves (docs/prds/state-store-contract.md §12 step 7).

- config.example.toml documents [services.state].target as required, with the
  loopback vs Docker-bridge gateway + target_token rule
- deployment templates set [services.state] per topology: bridge gateway +
  token for container-mode agents, loopback for host-only
- deployments/README.md and systemd-user-service.md describe start commands
  for both topologies and drop the stale "not yet consumed by the daemon" note

* style: gofumpt alignment for State Store daemon literal; appease noctx/staticcheck (archie-core-8cda.4.6)

* docs(state-store): refresh stale in-process references post-cutover (archie-core-8cda.4.6)
2026-09-07 05:06:12 +10:00
sam 48033ccbf8 refactor(state-store): migrate workflow/intake/task lifecycle consumers to State Store (archie-core-8cda.4.5) (#773)
Route the daemon's task-lifecycle / work-intake / workflow consumers (the
TaskStore composite) through b.stateStore instead of reaching into b.st
directly, the same additive composition pattern .4.4 used for the
capture/mapping/binding contracts (docs/prds/state-store-contract.md
§10/§12 step 6). b.stateStore is the in-process *store.Store by default and
a remote *staterpc.Client dialed to the standalone archie-state-store gRPC
service when [services.state].target is set, so this is behaviour-neutral
until an operator flips the seam.

Consumers migrated: daemon.Daemon.Store, webui.Server.Store, the message
gateway routers, the chat task lister/log-reader adapters, the operator
task action service, event persistence (persistAndBroadcastEvents), the
manual requeue path, and the chat task creator/controller.

openStores now seeds b.stateStore = b.st so it is never nil in either the
daemon (Run) or the gateway (RunGateway) composition; the gateway never
calls openStateStoreAdapter, so it stays on the local adapter and never
dials a remote store (preserving the .4.4 fail-closed intent). The
in-process State Store gRPC server keeps fronting b.st (the agent's gRPC
and NATS storerpc handoff paths both stay on the daemon's local store) and
the in-process serving path is untouched (deletion is .4.6).

Adds focused composition-regression tests asserting the daemon, webui and
task-actions consumers resolve their store from b.stateStore, plus the
existing bufconn conformance suite (grpc adapter) already proves contract
fidelity for the TaskStore surface.
2026-09-07 04:28:10 +10:00
sam c644a70d5b refactor(state-store): migrate capture/mapping/binding consumers to State Store (archie-core-8cda.4.4) (#772)
* feat(state-store): add daemon composition path for remote store adapter ($services.state)

Add the additive composition path the daemon's own capture consumers use
(archie-core-8cda.4.4, docs/prds/state-store-contract.md §10/§12 step 6).
bootstrap.go opens production store then resolves b.stateStore: the local
*store.Store by default, or a remote *staterpc.Client dialed to the
standalone archie-state-store gRPC service when [services.state].target is
set. Wire the dashboard CaptureStore surface from b.stateStore.

The in-process StateStore gRPC server keeps serving the daemon's SAME local
store (stateStoreDeps from b.st), not the consumer-facing b.d.* adapters:
task lifecycle (b.st) stays in-process until .4.5/.4.6 and there is no
dual-store ownership. Fail-closed on a non-loopback target without a bearer
token mirrors the standalone server's confinement (§9).

* feat(state-store): route daemon mapping consumers through the state store adapter

Migrate the MappingStore consumer to the State Store contract adapter
(archie-core-8cda.4.4 step 6, Mapping swap): the dashboard's mapping editor
(web.Mappings) and the daemon's dispatch loop (d.Mappings) resolve from
b.stateStore instead of reaching into b.st directly. Local by default,
remote *staterpc.Client when [services.state].target is set.

* feat(state-store): route daemon binding consumers through the state store adapter

Migrate the BindingStore/BindingDispatcher/BindingTaskCreator consumers to
the State Store contract adapter (archie-core-8cda.4.4 step 6, Binding
swap): the dashboard's binding editor (web.Bindings / web.BindingDispatcher)
and the daemon's dispatch loop (d.Bindings / d.BindingDispatcher /
d.BindingTaskCreator) resolve from b.stateStore instead of b.st. Local by
default, remote *staterpc.Client when [services.state].target is set.

* refactor(state-store): resolve daemon state store adapter only in daemon composition

openStores is shared with the standalone archie-gateway (RunGateway), which
does not consume the capture/mapping/binding surfaces. Resolve b.stateStore
from the daemon's own Run() composition instead, so a gateway config that
happens to carry [services.state].target can never dial (or fail closed
against) a state store it does not use. openStateStoreAdapter takes no ctx:
grpc.NewClient connects lazily, and the adapter resolution is synchronous.
2026-09-07 04:04:34 +10:00
sam e6a2e5b3b4 feat(state-store): stand up archie-state-store binary and own archie.db (archie-core-8cda.4.3) (#771) 2026-09-07 03:41:32 +10:00
sam 2b102b04a2 feat(state-store): generalize storerpc as the State Store gRPC contract (archie-core-8cda.4.2) (#770)
* feat(state-store): generalize storerpc as the State Store gRPC contract

Implements archie-core-8cda.4.2 against docs/prds/state-store-contract.md
rev. 2c: generate protobuf/gRPC client and server adapters for every
ratified store contract (proto/state/v1/state.proto, 40 RPCs, one
StateStoreService), add internal/infrastructure/staterpc with a bufconn
conformance suite exercising both local and remote adapters, serve the
service in-process (bridge address + mandatory per-task bearer token when
agent containers exist, loopback-only otherwise), and flip archie-agent's
workflow.Store to a long-lived gRPC client when the daemon injects
STATE_STORE_URL/STATE_STORE_TOKEN (falling back to the legacy NATS
storerpc path otherwise -- one agent process is never both).

Also drops the *sql.Tx parameter from store.BindingDispatcher.RecordDispatch
(a required wire-blocker fix: a transaction cannot cross a gRPC boundary,
and production always passed nil).

fix(store): drop RecordDispatch's unusable *sql.Tx parameter
feat(config): add services.state contract-adapter selector

* fix(state-store): apply no-mistakes review decisions

- Drop StreamTokenInterceptor (unused; the State Store service has no
  streaming RPCs and nothing calls it).
- Make StateStoreTokens.Validate compare tokens in constant time
  (crypto/subtle) per the auto-fix hardening.ing finding.
- Amend config.example.toml / config.Services.State doc text so it frames
  [services.state].target as the ratified surface consumed by the upcoming
  standalone archie-state-store binary (.4.3), not active today.
2026-09-07 03:01:33 +10:00
sam d5e8669991 fix(archied): initialize conversation persistence on daemon startup 2026-09-07 00:49:54 +10:00
sam 5e94cef784 refactor(workflow): relocate Task/Status/Source and define workflow.Store (#754)
* refactor(workflow,store): relocate Task/Status/Source and define workflow.Store

Move store.Task, store.Status*/store.Source*, and the 3-method workflow
store interface out of internal/store into internal/domain/workflow as
workflow.Task/workflow.Status/workflow.Source plus a consumer-owned
workflow.Store, superseding store.WorkflowStore. internal/domain/workflow
now imports zero internal/store symbols; internal/store persists the
domain type instead (infrastructure depends on domain), and its Store
satisfies workflow.Store.

Updates every production and test caller of the relocated symbols across
daemon, webui, gateway, agentworker, agenttransport/nats, storerpc,
taskrun, and worktreerpc, and removes TestRecordDispatchViaExplicitTx per
docs/prds/state-store-contract.md (rev. 2c) sections 4 and 12 step 2.

Two workflow-package integration tests that exercise the real *store.Store
move to an external workflow_test package (store_integration_test.go) to
avoid a workflow(test)->store->workflow(prod) import cycle now that store
depends on workflow.

* no-mistakes(document): docs(state-store): mark §4.8 relocation done in state-store-contract
2026-09-07 00:39:26 +10:00
sam 49b0d12be0 fix(archied): preserve channel turn ledger with remote gateway 2026-09-07 00:29:53 +10:00
sam 6397188827 Merge origin/main updates 2026-09-06 22:50:45 +10:00
sam fe3d19ee89 fix(gateway): share NATS and session ownership 2026-09-06 22:49:39 +10:00
sam 51fa032feb Merge branch 'feat/gateway-service': Phase 1 Gateway Service extraction
Stands up archie-gateway as its own binary serving ChatContract over gRPC
and makes it the daemon's only transport for webui chat (Q3 part 5's
atomic flip -- the in-process serving path is deleted, not deprecated).

- contract seams + protobuf toolchain (Phase 0)
- single taskactions.Service across webui, daemon NATS responder and the
  standalone gateway's client
- services.gateway.target is the only config surface left; mode is gone
  now that remote is the only transport
- task dev launches archie-gateway alongside archied + Vite

Verified end-to-end against live infra: real GitHub token (zero repos
configured, so no repo was polled or touched) and the real
@heyarchie_home_bot Telegram bot, both processes booted together against
shared NATS/store, chat and /status round-tripped successfully.

# Conflicts:
#	go.sum
2026-09-06 02:26:19 +10:00
sam bf482f9d56 fix(gateway): drop services.gateway.mode, remote is the only transport
Mode had exactly one legal value after the previous commit deleted the
in-process path, which made it dead config surface that could only
confuse an operator into thinking there was still a real choice.
ServiceConnection is now just a target address; composeChatContract only
checks it's non-blank, defaulting to 127.0.0.1:8585.
2026-09-06 02:13:03 +10:00
sam bec9caa24e feat(gateway): delete in-process ChatContract path, make remote the only mode
archie-gateway is now the sole owner of ChatContract; the daemon always
dials it over gRPC for webui chat, no in-process fallback left. Telegram
and email keep their own in-process routers unchanged (their streaming
responder is a callback that can't cross the wire-safe contract).

- composeChatContract only accepts mode="remote"; config defaults
  Services.Gateway to remote + 127.0.0.1:8585 when unset
- deleted taskActionsActor (dead code once the local branch is gone)
- task dev now launches archie-gateway alongside archied + Vite

Verified end-to-end: archie-gateway + archied booted against a shared
NATS/store, webui's /api/chat/sessions round-tripped through the gRPC hop.
2026-09-06 02:07:02 +10:00
sam 10e76e7528 feat(gateway): consolidate task actions and repair daemon default path
Phase 1 (extract the Gateway Service), first stage. Repairs the
half-migrated working tree so the default (inproc) daemon path works
again while keeping the WIP's direction:

- chatTaskActorAdapter is contract-only (no HTTP detour); nil contract
  returns ErrChatCapabilityUnavailable instead of panicking.
- openStores opens the conversation store for both processes again;
  the standalone gateway no longer owns it exclusively. Phase 1 keeps
  the daemon's Telegram/email routers and session-memory curator
  reading the same SQLite file; remote conversations arrive when a
  process that cannot open the file needs them.
- setupLLMAndChat composes the local gateway (setupGatewayChat) in
  default mode and a remote gRPC client in mode=remote; the returned
  contract is never nil, and Run handles the error.
- taskactions.Service is now the single task-mutation implementation
  shared by the dashboard (webui TaskActionService), the daemon's local
  gateway actor, and the NATS responder. The daemon registers
  taskactions.Register so the standalone Gateway's client can forward
  actions; transport follows the repo's natsrpc pattern.
- uncommitted sessions.proto / SessionStoreService gRPC contract
  deferred: it has no wired consumers (scope creep for this stage).
- task_action_adapter_test.go rewritten against the contract chain.

task check passes; archied and archie-gateway both build.
2026-09-06 01:35:15 +10:00
sam 79375f33e9 feat(gateway): extract ChatContract service 2026-09-05 23:06:16 +10:00
sam 80eb2ccc4e feat(contracts): add gateway service seams and protobuf toolchain 2026-09-05 22:58:00 +10:00
sam 3d7666002c feat(contracts): add gateway service seams and protobuf toolchain 2026-09-05 22:57:45 +10:00
sam 268283eb16 chore(curator): add embedding client and sampler interface prerequisites (#752)
* feat(curator): add embedding client and sampler wave-1 prerequisites

Bead archie-core-1786637500752-415-434e5bb9 (GitHub #436) was closed with
no linked PR/commits, and the surprisal-sampling bead
archie-core-1786637500776-416-600a66e5 (GitHub #437) found none of its
stated prerequisites -- an embedding client, a Sampler interface, or the
four wave-1 cheap sampling strategies -- actually exist. This builds them:

- internal/domain/embedding + internal/infrastructure/embedding: a narrow
  typed embedding client, config-driven via a providers table + a role in
  [models], mirroring the existing chat model role convention. Degrades to
  capability-unavailable (never fails startup or a call) for a missing
  role, unknown provider, unsupported class, or unresolved credential.
  Wired as an optional capability in bootstrap.go's setupEmbeddings.
- internal/domain/sampling: the Sampler interface plus recency, random,
  all, and staleness-proxy strategies, per docs/prds/curator-sampler-wave1.md.
  Deterministic given fixed inputs, matching GitHub #437's stated
  acceptance criterion for the embedding-backed strategy it will add
  behind this same interface (not part of this change).

Table-driven tests throughout; httptest for the embedding client, no live
service. No curator wiring yet -- nothing consumes either capability in
this change.

* no-mistakes(document): docs(architecture): mark curator sampling prereqs as landed
2026-09-05 22:53:08 +10:00
sam d0079271d5 feat(gateway): render workspace, managed repos and operator into chat env
The chat agent had no way to know where it lives without probing the
filesystem (pwd, find, git remote), wasting turns on every orientation.
Per docs/architecture/identity.md the <env> block is Layer 1 invariants,
so the daemon's own facts belong there, driven from live configuration.

Render into the prompt's <env> block:
- workspace path (chat.workspace)
- repositories under management with forge host and default branch
- operator identity (chat.operator, which was dead plumbing)

Unconfigured values render explicitly as unknown / none configured
rather than a plausible-looking default, and a repo with no forge
reports forge unknown, so the model never claims a fact the daemon
cannot confirm. Repos are resolved per active chat identity, mirroring
chatTaskProfiles.

gateway: add RepoEnv, SystemPromptConfig Workspace/Repos/Operator,
TurnRunnerConfig Workspace/Repos.
app: chatRepoEnv resolver wired through newChatTurnRunner.
tests: prompt rendering (incl. explicit-unknown + hostile-SOUL guard)
and chatRepoEnv identity resolution. go.sum gains module-graph
/go.mod checksum lines.
2026-09-05 22:52:03 +10:00
sam 9310fa5898 fix(releaseupdate): use context-aware database calls in tests 2026-09-05 21:14:28 +10:00
samcharles93 52941d152a skillcurator: nilerr finding at skillcurator.go:110 (archie)
Implements #748
2026-09-05 08:27:35 +00:00
samcharles93 d6cec97de1 fix: baseline gate repair (go test ./... -count=1) 2026-09-05 08:24:10 +00:00
Samuel Catlow afba5800b2 feat(gateway): wire progressive tool disclosure (bridge tools)
Compose the base registry with per-turn extras into a turn-local registry and
let ContextPressureGate decide full-vs-bridge disclosure per chat turn. The
three bridge tools (tool_search, tool_describe, tool_call) are now wired into
the chat toolset and only disclosed under context pressure, keeping the LLM's
schema footprint small while preserving full arsenal access on demand.

Per-turn extras are identity-bound per channel and cannot live in the
process-wide registry (agentexec.BuildToolSetFrom invariant), so the bridge is
composed turn-locally rather than registered globally. Thread the model's
context window through TurnModel.Prepare (TurnPrepareContext) so the gate
activates with a real window.

Refs #167
2026-09-05 16:02:53 +10:00
sam e39467b83e feat(drain): honour external drain marker with instantiation epoch (#718)
Implement drain control via an external marker file. The daemon polls
$XDG_CONFIG_HOME/archie/.drain_request.json (the repo's config-home
convention) for a drain request, validates it against the current
instantiation epoch (kernel boot id + PID 1 start time) and, only when
the marker is live, cancels the boot context to trigger the same
graceful shutdown an operator SIGTERM does.

A marker written against a prior instantiation (reboot or PID 1/container
restart) is stale and ignored, so a leftover request never drains a
daemon that came up after it was written. An unreadable marker or epoch
fails closed to no drain. The daemon never invents a new shutdown path.

domain/drain owns the pure decision (Epoch, Marker, Decide);
infrastructure/drain reads the marker file and /proc boot facts;
app/archied wires a poller that cancels the root context on a valid
request.

Path note: the epic named ~/.archie/.drain_request.json, but archied
resolves every per-user path under ${XDG_CONFIG_HOME:-~/.config}/archie
(see AGENTS.md 'Config path'), so the marker lives there too rather than
at a path this daemon never uses.

Refs bead archie-core-1786637496817-266-06b64a72 (drain control: marker
file and instantiation epoch), part of the operations epic
archie-core-1786637496703-262-1e49aef1.

Tests: table-driven Decide (epoch mismatch -> stale, matching -> valid,
nil -> none, unreadable -> stale), marker round-trip from operator JSON,
and monitor (valid -> shutdown, stale/none/error -> no shutdown).
2026-09-05 13:38:02 +10:00
sam 2dd095c8a7 fix(build): clear pre-existing task check gate blockers (#719)
Baseline `task check` failed on a clean main checkout, blocking unrelated
in-flight work. Fixes only those pre-existing failures:

- bootstrap.go: extract wireWebStoreSurfaces from setupObservability
  (cyclop 16 > 15) and startEmbeddedNATS from connectNATS (nestif).
- telegram_setup_test.go: os.Setenv -> t.Setenv (usetesting).
- install_script_test.go: exec.Command -> exec.CommandContext with
  t.Context() (noctx).
- Taskfile: add a ui:deps install step shared by `ui` and `test:ui`.
  ui/node_modules is gitignored and nothing in the gate installed it, so
  `task test:ui` died on a missing jsdom rather than a real test failure.
- shutdown_watchdog_test.go: `go fix` output adopted verbatim.

Refs bead archie-core-14cs.
2026-09-05 13:37:34 +10:00
sam bf21033208 feat(webui): add /health and /health/detailed readiness endpoints (#717)
* feat(webui): add /health and /health/detailed readiness endpoints (archie-core-1786637496789-265-b052b772)

Implements the Operations: readiness probes bead from the hermes gap analysis
(docs/prds/hermes-gap-analysis.md sections 9-10).

- GET /health: liveness, no auth, always 200 ok.
- GET /health/detailed: authenticated (dashboard token), JSON report with
  overall ok/degraded status plus per-subsystem probe results.

New readiness probes, each reporting real current state:
- state_db: read-only StatusCounts against the store.
- config: configuration.Validate against the running config.
- disk: statfs, degraded above 90% used.
- model: active model configured + provider endpoint reachable.
- gateway: channel running/failed state + connected session count.

Layering: probe contract in internal/domain/health, implementations in
internal/infrastructure/readiness, wiring in internal/app/archied,
HTTP surface in internal/webui.

* no-mistakes(document): docs(archied): fix stale doc pointer in readiness probe comment
2026-09-05 13:25:04 +10:00
sam ff740689cd feat(archied): add shutdown watchdog that force-exits a hung graceful shutdown (#716)
Once graceful shutdown begins (SIGINT/SIGTERM cancels the daemon's
signal context), start an out-of-loop watchdog timer for the drain
window plus a 60s grace period. If the shutdown chain has not completed
by the time that leash expires, force-exit the process (os.Exit) as a
backstop against a hung shutdown.

The watchdog runs in its own goroutine outside the daemon's event loop
and graceful-shutdown machinery, so it fires even when that machinery is
itself the thing that is stuck. It is disarmed via the FIRST registered
cleanup so it runs LAST in the LIFO shutdown chain: it stands down only
after every subsystem has shut down cleanly, and a hung subsystem never
reaches the disarm, letting the watchdog force-exit.

A shutdown that completes within the leash does not trigger the
watchdog; a shutdown that hangs past the leash results in a forced
process exit. The leash uses a test-only short timeout in tests.

Reference bead archie-core-1786637496842-267-8b2984f9.
2026-09-05 13:24:40 +10:00
sam 3e442c569d fix(servicediscovery): check lister/watcher Stop() errors
golangci-lint (errcheck) flagged both deferred Stop() calls in the NATS
KV registry as unchecked. Explicit discard, no behavior change.
2026-09-05 03:13:52 +10:00
sam fda9a48d64 Merge crew/servicereg-impl: scaffold ServiceRegistry contract
Adds internal/servicediscovery (broker-neutral ServiceRegistry contract,
zero internal imports) and internal/infrastructure/servicediscovery/nats
(NATS JetStream KV implementation: installed-marker bucket + heartbeat
bucket to express NotInstalled distinctly from Down/Unhealthy, per
docs/prds/service-decomposition.md). Additive and unwired -- no daemon
bootstrap path touched. Verified independently: go build ./... clean,
go vet clean, gofmt clean, full package tests pass with -race.
2026-09-05 03:12:30 +10:00
sam 0bc65b680d feat(servicediscovery): scaffold ServiceRegistry contract and NATS KV implementation
Add a broker-neutral service registry contract (Endpoint, EventKind,
Event, ServiceRegistry, ErrNotInstalled) and a NATS JetStream KV
implementation, plus the registration side a service uses to announce itself.

The contract's governing semantic -- NotInstalled is not Down -- is modeled
by two buckets: a durable installed marker (never expires) distinct from
expiring heartbeat keys. Instances heartbeat their key on an interval; a
stopped process leaves via TTL expiry. Register writes the marker once then
heartbeats; Resolve returns ErrNotInstalled only when the marker is absent,
and an empty roster for installed-but-down. Watch dedupes heartbeat refreshes
so it emits membership changes, not liveness pings.

Unwired: no daemon/bootstrap path consumes it yet. Deliberately uses a dot
key separator instead of the research sketch's slash so per-service watch
wildcards match.
2026-09-05 03:11:43 +10:00
sam a4a8ae7e92 feat(eda): thread Playbook ID/Version and DispatchInput task identity (gap-2 prereqs)
Playbook now carries ID (path-relative to the configured root) and Version
(sha256 content hash recomputed on load), both populated in loadOne, mirroring
Binding.Version's provenance pin without a DB row. DispatchInput gains TaskID
carrying TaskEnvelope.IdempotencyKey(), the identity available at the
discovery point (pollNATS / webhook receiver compute kind+labels from a
TaskEnvelope before any store.Task row exists), NOT a store.Task.ID int64.

These are the two implementation prerequisites named by the t2db.17
idempotency resolution; the playbook_dispatches ledger is built on top.

Note: task check fails on 5 lint findings PRE-EXISTING at HEAD in unrelated
files (bootstrap.go cyclop/nestif, install_script_test.go noctx,
telegram_setup_test.go usetesting), reproduced in a clean worktree and not
caused by this change; the eda/playbook package is gofumpt/vet/lint clean.
2026-09-05 03:10:58 +10:00
sam 7a9122b50e fix(store): migrate binding owner and repo columns 2026-09-03 06:43:03 +00:00
sam bcb78797ab Merge pull request #706 from samcharles93/alert-autofix-6
Potential fix for code scanning alert no. 6: Slice memory allocation with excessive size value
2026-09-03 16:16:04 +10:00
sam 6aa53684ad test(webui): cover log limit validation 2026-09-03 05:36:17 +00:00
sam e077b13640 fix(archied): preserve ephemeral gateway ports (#707) 2026-09-03 15:24:36 +10:00
samandCopilot Autofix powered by AI 50b41e32e0 Potential fix for code scanning alert no. 6: Slice memory allocation with excessive size value
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-09-03 15:13:35 +10:00
sam febc9f075a fix(update): make daemon rollback schema-safe (#704) 2026-09-03 14:53:03 +10:00
sam 7cdd7adb4e feat(telegram): support secret refs for bot tokens (#705) 2026-09-03 14:52:31 +10:00
sam b698322505 fix(gateway): split /status (health) and /tasks (work) apart
/status conflated two unrelated concerns: a thin per-status task
count that duplicated information belonging to a task view, and
runtime config already visible via /model. Worse, a bare status
count actively misled -- "Running: 1" read the same whether that
task was making progress or stuck in a failure loop, because a count
carries no age or identity.

/status now reports one aggregate queue-depth line (in-flight work:
running/waiting/parked, health signal only) instead of an itemized
breakdown -- queued and terminal states are excluded as neither
actionable nor a health concern.

/tasks is new: the live, per-task view with enough identity and
state to act on it (id, title, status, workflow/stage, age since
last transition, attempt once a retry has happened, and a parked
task's reason). Age comes from Task.UpdatedAt, already persisted and
now threaded through ChatTaskSummary and chatTaskListerAdapter --
no new instrumentation needed. Tasks sort actionable-first (running,
waiting, parked, ...) ahead of terminal states, stable within a
status group so the lister's own recency order survives.

Wired via the existing configureTaskCommands call sites (web,
telegram, email, webhook) using the same chatTaskListerAdapter
already built for the task_list LLM tool -- no parallel data path.

Note: internal/gateway's separate /agents command (AgentReader) is
NOT touched here and remains dead in production (Router.Agents is
never wired by any composition-root call site) -- tracked as its own
finding, not folded into this fix.
2026-09-03 07:15:59 +10:00
sam 6c1caf2f5c fix(eda): make playbook key collision reporting deterministic
loadPlaybookFile ranged over the raw YAML map directly, so which key
"won" when a file declared two colliding bindings depended on Go's
randomized map iteration order -- contradicting the function's own
doc comment promise of a reproducible, deterministic collision error.
Surfaced as a flaky failure in TestLintReportsLoadErrorAsFinding
(internal/app/archieplaybooks), which has two independent collisions
in its fixture and asserted on whichever one used to win by chance.

Sort the file's keys before checking them, matching the existing
sorted-filenames convention for cross-file order. Updated the test's
assertion to the now-deterministic outcome ("bug", alphabetically
first) with a comment explaining why.
2026-09-03 07:15:36 +10:00
sam 5aaa6fd432 refactor(eda): extract helpers to satisfy cyclop/gocognit gates
loadWorkflows (bootstrap.go) split into loadWorkflowRouting,
loadModules, loadEDAPlaybooks -- cyclop 17>15.

LoadPlaybookDirs (routing.go) split into sortedPlaybookFilenames,
loadPlaybookFile, bindPlaybookKey -- gocognit 52>30.

Pure extraction: identical log messages, error strings, and
control flow. golangci-lint run ./... now reports 0 issues.

Closes archie-core-t2db.16.
2026-09-03 06:42:27 +10:00
sam 912db8f590 fix(binding): split ValidateForUpdate so edits can preserve the secret
Real bug found via browser-testing the new bindings UI (t2db.8):
UpdateBinding's own store-layer contract documents that an empty
Secret means "preserve the existing one" (COALESCE against NULLIF),
but handleBindingUpdate validated with the create-time Validate(),
which unconditionally requires a full-length secret -- rejecting the
exact "leave blank to keep it" edit flow the store was built to allow.

Root-cause fix, not a UI workaround: Binding.Validate() (create,
unchanged, still requires a real secret -- there's nothing to fall
back to on insert) now delegates shared field checks to a private
validateCommon(); a new ValidateForUpdate() reuses the same checks but
only enforces the secret-length floor when the caller actually
supplied a new one. handleBindingUpdate now calls ValidateForUpdate.

3 new domain tests + 1 new API-layer test pin: empty secret accepted
on update, a genuinely-short new secret still rejected on update,
empty secret still rejected on create.
2026-09-03 06:17:43 +10:00
sam 7705fda777 fix(binding): add Owner/Repo pin, fixing multi-repo dispatch
Root-cause fix for the correctness gap named in archie-core-t2db.8:
resolveBindingRepo refused to dispatch with zero or multiple configured
repos, meaning a multi-repo deployment could never use bindings at all.

- binding.Binding gains optional Owner/Repo fields; Validate rejects
  setting only one (a pin is a complete pair or not a pin at all).
- Persisted as new owner/repo columns (NOT NULL DEFAULT '', additive --
  no migration needed since bindings is a CREATE TABLE IF NOT EXISTS
  with no existing deployment yet).
- resolveBindingRepo now takes the binding: a pinned binding dispatches
  to its own owner/repo unconditionally; an unpinned binding keeps
  today's single-configured-repo fallback (0 or 2+ repos still refuses,
  unchanged -- verified by a dedicated regression test, not just
  assumed preserved).
- Wired through the API request/response shape (create/update accept
  and return owner/repo); no repo-existence validation against config
  added -- smallest workable change, not the named bug.

Red-green throughout: 8 new tests across domain/store/daemon/webui
layers, each proven to fail before the corresponding implementation
step. Zero regressions in existing binding/dispatch test suites.
Dashboard UI to expose this (the other half of t2db.8) is unstarted --
this closes the backend correctness gap only.
2026-09-03 04:38:45 +10:00
sam 28d6e15a4c fix(skillcurator): document and scope the intentional nilerr in reviewOne
golangci-lint's nilerr finding was syntactically correct but not a bug:
a per-skill parse failure is deliberately converted into a reportable
curator.Action instead of propagated as an execution error, because
Pass()'s loop aborts the whole batch on a non-nil error -- which would
stop reviewing every other skill over one bad file. Confirmed against
skillcurator_test.go's existing "must not abort review of the others"
coverage before touching anything. Scoped //nolint with the reasoning
inline, not a blind suppression.
2026-09-03 04:17:05 +10:00