Every service connected to PostgreSQL as the database owner, which is a
superuser, so anything that compromised an app process owned the cluster, and
every connection reached the server with an empty application_name, so
pg_stat_activity could not say which service a session belonged to.
Split the DB credential into two login roles and name every pool:
- archie_migrator owns public and its objects and runs goose. Migrations run
on a short-lived pool opened from the new migration_database_url, before the
serving pool opens. It is not a superuser.
- archie_runtime serves requests with SELECT, INSERT, UPDATE, DELETE, sequence
USAGE/SELECT and function EXECUTE only. It has no CREATE privilege, so it
cannot change the schema.
- Every pool sets application_name: archie-state-store, archie-gateway, and a
-migrate / -recovery / -schema / -provision suffix on the transient
administrative connections. Existing one-URL installs keep working: an
absent migration_database_url falls back to database_url and the migration
runs on the serving pool exactly as before.
Provisioning is a new idempotent `archie-state-store provision` subcommand: it
needs an admin URL the services never hold (database owner or superuser),
creates or reconciles both roles, hands every public-schema object to the
migration role, revokes PUBLIC's CREATE on public, sets row-level grants, and
sets the migration role's default privileges so tables future migrations
create are usable without a hand-run grant. It stores SCRAM verifiers, refuses
superuser roles, and verifies the boundary before committing.
install.sh now provisions the bundled path and separates the credentials: the
superuser password the Compose database is created with lives only in a
compose-only postgres.env, while the env file the units load holds the
non-superuser PGPASSWORD. A bundled install that predates the split is
upgraded in place: the roles are created, the superuser password is rotated
out of the env file, and config.toml is moved to the runtime and migration
URLs. External databases keep their operator-managed credential; the
deployments README documents the one-shot provision command.
setup writes both bundled URLs, dev:postgres provisions the dev database, the
e2e stack provisions its throwaway server, and the deployment templates point
at the two roles.
Replace the service bearer tokens with [tls] ca_file and cert_dir. Caller
identity comes from the verified certificate, not metadata. Without TLS,
services listen and dial on loopback only. Task containers get only the CA
and keep their grant.
The dashboard serves plain HTTP, so a non-loopback bind carries the
session cookie and provider token in clear unless TLS terminates in front.
Warn at startup unless the operator declares a proxy with
[web].trust_forwarded_headers, and record the choice in deployments/README
where the dashboard is deployed. Point the non-loopback token error at
that surviving doc, not the deleted UI-boundary PRD.
Architecture, deployment, installer, skill and PRD text now name the Postgres
State Store, the offline recovery commands on database_url, state_dir for host
state, and db_path as the legacy-file locator for archie-state-store import.
Finalised PRDs, CHANGELOG, docs/news and docs/archive are unchanged records.
stateadmin served read-only PocketBase views over the SQLite task file on -admin-addr. PocketBase is SQLite-only and the State Store moves to Postgres (archie-core-7r0x), so the view goes; operators use a Postgres tool of their choice.
The index built google/codesearch over the chat workspace on every daemon
start, for the process lifetime, through a child-process helper and a
SQLite metadata DB -- and grep consulted it only for searches confined to
the workspace root, which the analysed sessions showed was a small
fraction of searches. The compute was continuous; the benefit was not.
Removes internal/indexing, the workspace-codesearch helper subcommand,
the [indexing] config section, and the GrepIndex accelerator seam from
the grep tool, which returns to its direct walk. google/codesearch
leaves go.mod with it.
The v1.30.0 extraction moved Telegram, email and webhook out of archied and into
cmd/archie-messaging, and nothing packaged it. dist-zip built and copied only the
five older binaries, and scripts/archie-update-install omitted it from both its
service and binary lists. A host upgraded across v1.30.0 therefore ran on with no
messaging process at all: archied logged that it stopped serving the telegram
gateway at the extraction, nothing replaced it, and the bot was dead for two days
with no error anywhere -- the daemon looked healthy throughout (archie-core-1c01).
Fixed in all four places the host component set was enumerated: the zip's build
loop, the zip's copy list, the installer's GATEWAY_SERVICES (messaging is a
service, so it is installed AND restarted) and its GATEWAY_BINARIES. The comment
claiming every process in the documented deployment ships in the zip is now true,
and says what makes it true.
The duplication was the actual defect, so the sets are pinned to cmd/ instead:
TestHostCommandListsCannotDrift derives the host-run commands from cmd/ and
asserts all four lists, plus the reference runbook's units, agree with it. Adding
a command or extracting one fails the test until it is packaged -- the check that
would have caught this on the day. The installer's own test grew the same
coverage, and the reason it passed while the bug was live is worth recording: its
fixture created only the stale cmd/ set, so the script's `[ -d ./cmd/$cmd ]` guard
silently skipped messaging.
Documentation, because the installer now refuses an update when a service unit is
missing and sends the operator to the runbook -- so a service the runbook omits is
a dead end: a messaging unit in deployments/systemd-user-service.md, the component
list and start sequence in deployments/INSTRUCTIONS.md (which ships inside the
zip), and the packaged-binary list in RELEASING.md.
Verified: task check exit 0, including test:tools and test:ui. Both new guards
were made to fail on purpose: reintroducing each omission produces "missing
archie-messaging" and "no call with fragments [./cmd/archie-messaging]".
Bead archie-core-8cda.5.5:
- task dev now runs archie-ui (dev:dashboard) on :8484 -- after the
cutover archied binds no dashboard listener, and Vite's /api,/events
proxy needs a live receiver there. dev:mobile starts archie-ui with a
LAN bind and -token-file (reusing the pre-cutover token path) instead
of a mobile archied overlay; dev:ui's summary points at archie-ui.
- deployments/dev.toml documents itself as the archie-ui overlay and
states the [services.gateway] target explicitly.
- systemd-user-service.md: the auth gap is now the documented posture --
archie-ui has no secret registry, resolves GATEWAY_TOKEN/
STATE_STORE_TOKEN from its process environment only, so the starter
(EnvironmentFile) exports them; worked env-file example added, plus
/healthz as the token-free liveness probe.
- INSTRUCTIONS.md ships five binaries with archie-ui in the quickstart
and the token story; README.md carries the auth note; the
docker-nats-stack profile notes archie-ui binds [web] and dials the
same [services.*] targets.
- release tooling: archied's component closure extends with
cmd/archie-ui and internal/app/archieui, so a UI-process-only commit
can no longer fall between the two changelogs; RELEASING.md documents
that the UI Service ships inside the archied release.
- ui-service-boundary.md: the /api/captures|/mappings|/bindings row is
resolved by the cutover; archie-ui composes the store contracts and
mounts the captureintake receiver.
- event-capture-storage.md HTTP surface: the receiver mounts on the one
process serving the dashboard; work intake stays with the daemon.
- config.example.toml: [web].listen is the archie-ui process's bind
address; [capture] is projected by archie-ui.
- deployments: document running archie-ui alongside the other services
and add its systemd unit; capabilities comments updated for the
single-serving composition.
The update watchdog verifies archied's restart, so archied has to be what
answers. It was probing the dashboard's /healthz, which belongs to the
dashboard: switch [web] off and nothing answers, and in the extracted UI
topology that endpoint reports a different process entirely, which would
confirm a release while the daemon it was meant to verify is down.
[health].listen (127.0.0.1:8485, defaulted, always on) serves /healthz,
/health and /health/detailed for the life of the process. Liveness answers
503 until the daemon reaches its run loop: the listener binds before the
subsystems do, so an unconditional 200 would tell the watchdog a half-booted
release came up. Failing to bind is fatal, because a daemon nothing can probe
gets every update rolled back. The State Store's readiness surface now shares
the listener plumbing, keeping its log and error text verbatim.
Closes archie-core-exbz, which asked which of two bad options to take when
the dashboard is off. Neither: the daemon owns its own liveness.
Also deletes TestInstallUpdateDetachesInstallerFromCallbackContext. It has
failed intermittently under load more than once, racing its fake bot server's
multipart parse against the cancellation it induces.
* test(state-store): real-process smoke + deadline/sentinel error contract
Add end-to-end verification for archie-core-8cda.4.7, the .4.7 State Store
verification stage against docs/prds/state-store-contract.md (rev. 2c):
- Real-process smoke coverage: build ./cmd/archie-state-store, run it against
a minimal config on loopback, poll the readiness surface, and drive the
State Store gRPC contract through staterpc.NewClient exactly as a consumer
does. Covers the combined task/event/capture view, error-sentinel fidelity
across the wire, the found=false-not-error convention, single-owner SQLite
(the binary owns the one archie.db file; the consumer never opens a store
file), per-call deadline behaviour, restart/recovery (SIGTERM + reopen on
the same db), and the remote capture/mapping/binding surfaces.
- Fix a real deadline-identity gap in unmapError: a cancelled or
deadline-exceeded context surfaced by gRPC-Go as a *status.Error was
re-wrapped into a sanitised internal-style error, breaking
errors.Is(err, context.Canceled/DeadlineExceeded). The agent's deadlineStore
bounds each Store call with context.WithTimeout and the workflow consumer
(workflow.go) uses errors.Is(err, context.DeadlineExceeded) to mark a stage
interrupted rather than failed, so the re-wrap would wrongly park an
interrupted task. Unmap Canceled/DeadlineExceeded codes back to the standard
context sentinels.
- Pin the error-mapping contract in staterpc: context-identity preservation,
sentinel round-trip fidelity for every store sentinel, and the sanitisation
rule (an Internal error must not leak raw detail or map to a sentinel).
* test(state-store): assert single-owner SQLite composition (daemon/gateway require target)
The single-owner invariant (docs/prds/state-store-contract.md §12 step 7/8) is
that only archie-state-store opens archie.db; the daemon and gateway resolve
their task-store surface from openStateStoreAdapter, which requires
[services.state].target and opens no local store. This test pins that
composition root so a future change cannot silently reintroduce a local
*store.Store in the daemon or gateway path (dual-store ownership).
* test(state-store): skip real-process smoke tests under -short
The smoke tests build and exec the real archie-state-store binary, which is
slow enough that a -short run (e.g. a focused edit loop) should skip them.
task check (go test ./... -count=1) does not use -short, so they still run in
the full gate.
* test(state-store): fix subprocess log race under -race
The smoke test captured the subprocess's stdout/stderr in a strings.Builder
that the os/exec writer goroutine and the test reader goroutine touched
concurrently, racing under -race. Replace it with a mutex-guarded buffer, the
standard pattern for reading a running subprocess's output.
* test(state-store): satisfy lint (noctx, errorlint) on new tests
- exec.Command -> exec.CommandContext, http.Get -> NewRequestWithContext +
Client.Do, so the subprocess and readiness polling honour the test context
(noctx).
- errorlint: compare the raw non-status error via errors.Is rather than != on
a wrapped error.
* docs(deployments): document State Store readiness verification and single-owner invariant
The deployment runbook described how to launch archie-state-store but not how
to verify it is healthy before starting archied. Document the -ready-addr
readiness surface (/healthz, /health/detailed state_db component) and the
single-owner invariant: one task SQLite file owned by archie-state-store, with
archied/archie-gateway never opening archie.db directly (state-store-contract
§12 step 7/8).
* fix(state-store): preserve context deadline/cancel identity in mapError
The symmetric server-side gap to the unmapError fix: if a store query fails
because the caller's context expires mid-call (§6 StoreTimeout), the handler's
mapError folded context.Canceled/DeadlineExceeded into codes.Internal. The
client would receive a sanitised internal error with no deadline identity, so
the agent's workflow consumer (errors.Is(err, context.DeadlineExceeded)) could
not distinguish an interrupted stage from a failed one.
Map a context cancel/deadline error to its matching gRPC code so unmapError
rehydrates it back to the standard context sentinel, completing the §6/§7
round-trip. Pin both directions with TestMapErrorPreservesContextIdentity.
* refactor(state-store): delete NATS storerpc; agent store is gRPC-only (archie-core-8cda.4.6)
The agent is fully on the gRPC State Store contract, so the legacy NATS
storerpc path is dead (docs/prds/state-store-contract.md §12 step 4):
- delete internal/storerpc (Server, Client, subjects) and its tests
- drop the storerpc registration from registerTaskRPCServers and the
now-unused store param
- make the agent transport's State Store URL required and Store() return
only the gRPC staterpc.Client (no NATS fallback)
- rework agentworker/agent transport tests to back the store with a real
gRPC State Store server instead of the deleted NATS storerpc server
The agent binary now fails fast when STATE_STORE_URL is absent. Agent store
calls (workflow.Store) go over gRPC to the archie-state-store service.
* refactor(state-store): delete daemon/gateway in-process store ownership (archie-core-8cda.4.6)
The daemon and gateway no longer open archie.db in-process: the single SQLite
file is owned by the standalone archie-state-store process, and both consumers
dial its gRPC State StateStore contract (docs/prds/state-store-contract.md
§12 step 7, no dual-store ownership).
- openStores no longer opens the production task store or seeds b.st; it only
wires secrets/forge and the gateway conversation store
- openStateStoreAdapter now requires [services.state].target (the old local
default is deleted) and resolves b.stateStore as the remote *staterpc.Client
- delete the in-process startStateStoreServer (the multiplexed gRPC server and
its per-task bearer-token registry) from the daemon composition
- RunGateway now resolves the same remote State Store adapter; its session
SQLite stays Gateway-owned and untouched
- daemon.ConnectedStateStore carries the configured target plus the resolved
bearer token, which containerEnv injects as STATE_STORE_URL / STATE_STORE_TOKEN
so archie-agent authenticates to the same remote service
- readiness store probe reads b.stateStore
PRD-consistent choice (noted): with the in-process server gone, per-task token
generation is removed; the daemon/gateway/agent all present the single
[services.state].target_token (or STATE_STORE_TOKEN secret) that the standalone
store validates.
* docs(webui): require [services.state].target in config and deployment templates (archie-core-8cda.4.6)
After the in-process store path is deleted, archied and archie-gateway dial the
standalone archie-state-store gRPC service via [services.state].target and never
open archie.db themselves (docs/prds/state-store-contract.md §12 step 7).
- config.example.toml documents [services.state].target as required, with the
loopback vs Docker-bridge gateway + target_token rule
- deployment templates set [services.state] per topology: bridge gateway +
token for container-mode agents, loopback for host-only
- deployments/README.md and systemd-user-service.md describe start commands
for both topologies and drop the stale "not yet consumed by the daemon" note
* style: gofumpt alignment for State Store daemon literal; appease noctx/staticcheck (archie-core-8cda.4.6)
* docs(state-store): refresh stale in-process references post-cutover (archie-core-8cda.4.6)
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.
Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
docker-compose.yml was never committed -- it only existed on the
deployment host. It bind-mounted ./config.docker.toml and passed
-config-overlay unconditionally, which made an untracked repo-root
file load-bearing: Docker creates a directory when a bind source is
missing, and the loader rejects a directory, so a fresh clone took the
daemon down at boot.
The stack now passes a single -config pointing at a path outside the
repo. Nothing in the working tree is required at runtime. The
docker-specific values move to deployments/docker-nats-stack.toml as a
reference config, alongside the other deployment guides.
Also drops the ./:/workspace mount, which existed only for
desktop-commander's work_dir.