21 Commits
Author SHA1 Message Date
sam c54d328f0d feat(store): run services as a non-superuser role and name every connection (#1857)
Every service connected to PostgreSQL as the database owner, which is a
superuser, so anything that compromised an app process owned the cluster, and
every connection reached the server with an empty application_name, so
pg_stat_activity could not say which service a session belonged to.

Split the DB credential into two login roles and name every pool:

- archie_migrator owns public and its objects and runs goose. Migrations run
  on a short-lived pool opened from the new migration_database_url, before the
  serving pool opens. It is not a superuser.
- archie_runtime serves requests with SELECT, INSERT, UPDATE, DELETE, sequence
  USAGE/SELECT and function EXECUTE only. It has no CREATE privilege, so it
  cannot change the schema.
- Every pool sets application_name: archie-state-store, archie-gateway, and a
  -migrate / -recovery / -schema / -provision suffix on the transient
  administrative connections. Existing one-URL installs keep working: an
  absent migration_database_url falls back to database_url and the migration
  runs on the serving pool exactly as before.

Provisioning is a new idempotent `archie-state-store provision` subcommand: it
needs an admin URL the services never hold (database owner or superuser),
creates or reconciles both roles, hands every public-schema object to the
migration role, revokes PUBLIC's CREATE on public, sets row-level grants, and
sets the migration role's default privileges so tables future migrations
create are usable without a hand-run grant. It stores SCRAM verifiers, refuses
superuser roles, and verifies the boundary before committing.

install.sh now provisions the bundled path and separates the credentials: the
superuser password the Compose database is created with lives only in a
compose-only postgres.env, while the env file the units load holds the
non-superuser PGPASSWORD. A bundled install that predates the split is
upgraded in place: the roles are created, the superuser password is rotated
out of the env file, and config.toml is moved to the runtime and migration
URLs. External databases keep their operator-managed credential; the
deployments README documents the one-shot provision command.

setup writes both bundled URLs, dev:postgres provisions the dev database, the
e2e stack provisions its throwaway server, and the deployment templates point
at the two roles.
2026-11-11 03:12:54 +11:00
sam 9d8f364af1 feat(releaseupdate): signed OCI distribution sources for releases and the catalogue (#1715)
Quality / check (push) Successful in 5m53s
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-31 18:18:43 +11:00
sam 3b6de96782 docs(deploy): name the registry each agent-image path publishes to
The pinning section said CI republished ghcr.io on every push to main. CI
publishes git.catlow.cloud, and only for a release tag; ghcr is where the
hand-run tools/publish.sh writes, which is the reference the default names.

A release zip pins a digest on a registry an anonymous pull cannot reach, and
[containers].registry_auth -- the setting that fixes it -- was documented
nowhere. Documented there, beside the pin.

Also corrected: every shipped profile does not pin the image, and the section
link omitted its number so it matched no heading.
2026-10-31 00:35:49 +11:00
sam 3b7193c207 refactor!: remove the diff-size cap and its gate.diff-size step 2026-10-24 22:19:23 +11:00
sam 9cdae5642e feat(rpc): mTLS between services from operator-issued certificates
Replace the service bearer tokens with [tls] ca_file and cert_dir. Caller
identity comes from the verified certificate, not metadata. Without TLS,
services listen and dial on loopback only. Task containers get only the CA
and keep their grant.
2026-10-16 02:37:51 +11:00
sam 209f2b4adb docs: server quickstart, standalone template first, install starts PostgreSQL 2026-10-06 01:43:16 +11:00
sam c73ff313ea refactor(config): drop unused dispatch state labels
The daemon stopped setting forge state labels when label-driven retry was removed; the config field, helpers and defaults had no consumer. Remove them and the Scheduling policy page's State labels section.
2026-10-05 15:05:06 +11:00
sam 75988204d5 docs: strip doc and issue references from config, deployments and examples 2026-10-03 22:27:08 +10:00
sam c1cd5c8f5b chore(state-store): remove the one-time SQLite import and db_path
Every install serves from Postgres, so the legacyread/legacyimport packages,
the archie-state-store import command, the boot gate that refused to serve
over unimported SQLite files, and db_path (which only located those files)
go. Migration 0010 drops the import_completion table.
2026-09-25 00:40:48 +10:00
sam e5105280ad chore(config): remove token_env aliases for forge, artifacts and telegram
Each credential is now only a {engine, key} secret reference. A stale
token_env in a file is reported at startup as an unrecognised key.
nats.token_env stays (no secret-ref form), and providers.api_key_env stays:
it is the carrier the resolver writes each resolved key's env name into.
2026-09-24 23:58:26 +10:00
sam 9864766804 docs: describe the PostgreSQL-only stores, state_dir and the import-only db_path
Architecture, deployment, installer, skill and PRD text now name the Postgres
State Store, the offline recovery commands on database_url, state_dir for host
state, and db_path as the legacy-file locator for archie-state-store import.
Finalised PRDs, CHANGELOG, docs/news and docs/archive are unchanged records.
2026-09-24 14:51:48 +10:00
sam feeaa16c7b feat(postgres): wire the State Store to Postgres — DB URL, fail-closed pool, composition seam
Wave 1 of the Postgres migration (archie-core-7r0x.9). This is the shared shape
every port lane builds on; nothing here switches a live path to Postgres yet.

- config: add `database_url` beside `db_path` (config.go), mark it bootstrap-only
  in denied_keys, expose it in the webui StorageView/config schema, and add it to
  config.example.toml plus all deployments/*.toml.
- docker-compose: add a postgres:18 service; `task dev` (dev:postgres) brings it
  up and waits for it before the State Store boots.
- archied: `openStateStore` now opens and migrates the process-scoped Postgres
  pool via `openStateStorePool`. The State Store fails closed (D1.2) when
  database_url is absent or invalid rather than fall back to SQLite; serving
  against Postgres later requires an import-completion record (D1.3), which is a
  later lane's gate, not this one.
- composition seam: task-store construction (state_store_task.go) and EDA-store
  construction (state_store_eda.go) are extracted from openStateStore, with the
  pool open in state_store_pool.go. This is a behaviour-preserving refactor only
  — no store, interface, error, ordering or logging change — so L1 and L2 build
  against separate files instead of sharing state_store.go.

Tests: fail-closed pool behaviour is pinned (absent/invalid URL); the State Store
boot/restart tests now run against a throwaway Postgres 18 container (pgtest
TestMain). Scoped test command and module-wide build pass; see lane report.
2026-09-24 02:18:56 +10:00
sam 851c46d8ed docs(deploy): UI Service deployment, auth, and operations profiles
Bead archie-core-8cda.5.5:

- task dev now runs archie-ui (dev:dashboard) on :8484 -- after the
  cutover archied binds no dashboard listener, and Vite's /api,/events
  proxy needs a live receiver there. dev:mobile starts archie-ui with a
  LAN bind and -token-file (reusing the pre-cutover token path) instead
  of a mobile archied overlay; dev:ui's summary points at archie-ui.
- deployments/dev.toml documents itself as the archie-ui overlay and
  states the [services.gateway] target explicitly.
- systemd-user-service.md: the auth gap is now the documented posture --
  archie-ui has no secret registry, resolves GATEWAY_TOKEN/
  STATE_STORE_TOKEN from its process environment only, so the starter
  (EnvironmentFile) exports them; worked env-file example added, plus
  /healthz as the token-free liveness probe.
- INSTRUCTIONS.md ships five binaries with archie-ui in the quickstart
  and the token story; README.md carries the auth note; the
  docker-nats-stack profile notes archie-ui binds [web] and dials the
  same [services.*] targets.
- release tooling: archied's component closure extends with
  cmd/archie-ui and internal/app/archieui, so a UI-process-only commit
  can no longer fall between the two changelogs; RELEASING.md documents
  that the UI Service ships inside the archied release.
2026-09-10 01:14:14 +10:00
sam 05529d48d5 refactor(state-store): cut daemon/Gateway over to the remote State Store (archie-core-8cda.4.6) (#774)
* refactor(state-store): delete NATS storerpc; agent store is gRPC-only (archie-core-8cda.4.6)

The agent is fully on the gRPC State Store contract, so the legacy NATS
storerpc path is dead (docs/prds/state-store-contract.md §12 step 4):
- delete internal/storerpc (Server, Client, subjects) and its tests
- drop the storerpc registration from registerTaskRPCServers and the
  now-unused store param
- make the agent transport's State Store URL required and Store() return
  only the gRPC staterpc.Client (no NATS fallback)
- rework agentworker/agent transport tests to back the store with a real
  gRPC State Store server instead of the deleted NATS storerpc server

The agent binary now fails fast when STATE_STORE_URL is absent. Agent store
calls (workflow.Store) go over gRPC to the archie-state-store service.

* refactor(state-store): delete daemon/gateway in-process store ownership (archie-core-8cda.4.6)

The daemon and gateway no longer open archie.db in-process: the single SQLite
file is owned by the standalone archie-state-store process, and both consumers
dial its gRPC State StateStore contract (docs/prds/state-store-contract.md
§12 step 7, no dual-store ownership).

- openStores no longer opens the production task store or seeds b.st; it only
  wires secrets/forge and the gateway conversation store
- openStateStoreAdapter now requires [services.state].target (the old local
  default is deleted) and resolves b.stateStore as the remote *staterpc.Client
- delete the in-process startStateStoreServer (the multiplexed gRPC server and
  its per-task bearer-token registry) from the daemon composition
- RunGateway now resolves the same remote State Store adapter; its session
  SQLite stays Gateway-owned and untouched
- daemon.ConnectedStateStore carries the configured target plus the resolved
  bearer token, which containerEnv injects as STATE_STORE_URL / STATE_STORE_TOKEN
  so archie-agent authenticates to the same remote service
- readiness store probe reads b.stateStore

PRD-consistent choice (noted): with the in-process server gone, per-task token
generation is removed; the daemon/gateway/agent all present the single
[services.state].target_token (or STATE_STORE_TOKEN secret) that the standalone
store validates.

* docs(webui): require [services.state].target in config and deployment templates (archie-core-8cda.4.6)

After the in-process store path is deleted, archied and archie-gateway dial the
standalone archie-state-store gRPC service via [services.state].target and never
open archie.db themselves (docs/prds/state-store-contract.md §12 step 7).

- config.example.toml documents [services.state].target as required, with the
  loopback vs Docker-bridge gateway + target_token rule
- deployment templates set [services.state] per topology: bridge gateway +
  token for container-mode agents, loopback for host-only
- deployments/README.md and systemd-user-service.md describe start commands
  for both topologies and drop the stale "not yet consumed by the daemon" note

* style: gofumpt alignment for State Store daemon literal; appease noctx/staticcheck (archie-core-8cda.4.6)

* docs(state-store): refresh stale in-process references post-cutover (archie-core-8cda.4.6)
2026-09-07 05:06:12 +10:00
sam 5bd553d8ed Merge branch 'feat/gateway-service': Phase 1 Gateway Service extraction
Stands up archie-gateway as its own binary serving ChatContract over gRPC
and makes it the daemon's only transport for webui chat (Q3 part 5's
atomic flip -- the in-process serving path is deleted, not deprecated).

- contract seams + protobuf toolchain (Phase 0)
- single taskactions.Service across webui, daemon NATS responder and the
  standalone gateway's client
- services.gateway.target is the only config surface left; mode is gone
  now that remote is the only transport
- task dev launches archie-gateway alongside archied + Vite

Verified end-to-end against live infra: real GitHub token (zero repos
configured, so no repo was polled or touched) and the real
@heyarchie_home_bot Telegram bot, both processes booted together against
shared NATS/store, chat and /status round-tripped successfully.

# Conflicts:
#	go.sum
2026-09-06 02:26:19 +10:00
sam 14e5b542bf fix(gateway): drop services.gateway.mode, remote is the only transport
Mode had exactly one legal value after the previous commit deleted the
in-process path, which made it dead config surface that could only
confuse an operator into thinking there was still a real choice.
ServiceConnection is now just a target address; composeChatContract only
checks it's non-blank, defaulting to 127.0.0.1:8585.
2026-09-06 02:13:03 +10:00
sam 0a6496d2cd feat(gateway): extract ChatContract service 2026-09-05 23:06:16 +10:00
sam 2d8a610bab fix(deploy): pin archie-agent image away from :latest (#751)
ghcr.io/samcharles93/archie-agent:latest is re-published on every main
push that touches the runtime's package closure, so a bare :latest
reference can change under a running deployment with no warning and no
way to reproduce a prior build.

archie-agent has no published semver tag yet -- CI (deploy.yml) only
ever pushes :latest to GHCR, confirmed via the GHCR package API. Pin
docker-compose.yml and the GitHub deployment profiles to the verified
digest of :latest as of 2026-09-05
(sha256:870f1b1357979cfea23fccacb24166ebeb9ab321708ef2c2b218a763ed99b090),
and document the pinning strategy -- including the semver-tag-once-
published preference and digest pinning for production -- in
deployments/systemd-user-service.md. Update the generic
docker-nats-stack.toml example to demonstrate the same convention.

Ref archie-core-kllw (GitHub #670)
2026-09-05 22:19:05 +10:00
sam 972fb5b8e1 feat(deployment): default to managed NATS workers
Autonomous workflows now always hand a complete task to a scoped archie-agent container. Embedded NATS is the default and external NATS changes only broker placement; legacy agent and containers.enabled inputs decode for migration but cannot select an execution path. The daemon flushes task-event subscriptions before a worker can publish, and runtime updates snapshot and restore the configured worker image.
2026-08-24 14:56:57 +10:00
sam 691e10c251 fix(telegram): compact tool output and drop turn caps
Render completed ToolCallEvents as Telegram-safe streaming blocks
with compact name/status and a short cleaned preview. Summarize
file contents and listings instead of dumping source or quoted
JSON. Collapse repeated legacy turn-budget failures.

Remove max_tokens turn gating and the 200000 turn_budget_chars
default so absent or commented config cannot restore an aggregate
cap. Keep per-result max_result_chars and spill.

Confine find/grep discovery to the active workspace/worktree.

Refs #604
2026-08-23 16:40:16 +00:00
sam 6be754787d polish: add install.sh, XDG setup, deployments/ examples, and github-token docs\n\n- Add install.sh following XDG Base Directory specification (~/.config/archie, ~/.local/share/archie, ~/.local/bin)\n- Add interactive setup for code forge (GitHub / Gitea / Standalone) and LLM provider selection (Ollama model picker / Cloud keys)\n- Implement NoopForge in internal/forge/noop.go for running Archie in standalone mode without a forge\n- Enable loginctl user linger and systemd user unit setup in install.sh\n- Add docs/github-token.md with step-by-step guidance for Classic PAT generation\n- Add top-level deployments/ directory with scenario templates (single-forge GitHub, multi-forge GitHub+Gitea, local Ollama standalone, systemd runbook) 2026-07-30 01:25:46 +10:00