77 Commits
Author SHA1 Message Date
sam 0346bfef77 fix(configuration): require containers.image instead of a private-registry default (#2054)
applyContainerDefaults no longer fills containers.image with a hard-coded
private-registry reference. An unset image now fails configuration
validation with 'containers.image is required for autonomous workflow
workers', so a fresh instance is told at startup (and via the readiness
probe at runtime) instead of at the first failed container pull.

Configs that relied on the default now set it explicitly: dev.toml (the
local task image:agent build, which e2e also overlays), plus commented
[containers] blocks in config.example.toml and standalone.toml. The
deploy docs no longer describe a code-level default.
2026-11-21 01:16:41 +11:00
sam deb3865d75 chore: use generic host names in a deploy comment and test labels 2026-11-16 00:13:04 +11:00
sam 93c9eb51f3 fix(deploy): let the staging container confine extensions in user and network namespaces
Docker's default seccomp profile refuses the clone the extension host
confines with, so a connector that declares egress couldn't start. The profile
is Docker's default plus that clone, limited to the user and network
namespace flags.
2026-11-15 22:21:03 +11:00
sam e787a79af4 feat(connectors): credentials sealed with age, and secret engines that resolve them (#1977, #1982)
Credentials are typed (machine, age), write-only through the API and sealed
to the age identity in the bootstrap config. A field holds a value or names a
secret engine key; connectors and engines are accepted with credential slots
that resolve into their Configure secrets. The secret engine contract gains
secrets so an engine can receive its own key.
2026-11-15 22:14:26 +11:00
sam b89d9e0f05 feat(events): hear Alertmanager through its connector: verified intake, dedupe, links, triggers that start runs (#1980, #1978) 2026-11-15 18:05:38 +11:00
sam 99a3d675a8 chore(deploy): staging stack for the single binary on nebula (#1998) 2026-11-15 17:05:53 +11:00
sam 3a965e2a4f feat(nats): an external broker defers logins to archie through auth callout (#1866)
Task containers log in with their run credential on an external broker and
reach only their own subjects, decided by the embedded broker's authenticator.
The broker token is never put in a container environment.
2026-11-14 04:04:15 +11:00
sam 55e477794e fix(dev): task dev uses embedded NATS regardless of the installed config 2026-11-13 12:10:46 +11:00
sam f92c5b4536 chore(runtime): delete Kit and external agent harness support (#1877)
ci/woodpecker/pr/quality Pipeline was successful
Quality / check (pull_request) Canceled after 7m39s
Archie runs its own agent loop only. Removes Kit resolution and launch,
the harness runner and adapters, the setup terminal and sandbox view,
harness secrets and OAuth capture (table dropped in migration 0075), the
task-tools contract that only Kit containers reached, and the
sandbox-kit-spec dependency. Agent profiles carry image and tools only.
The egress proxy keeps the model-session path; the sandbox relay code
stays for #1867.
2026-11-12 21:29:05 +11:00
sam c54d328f0d feat(store): run services as a non-superuser role and name every connection (#1857)
Every service connected to PostgreSQL as the database owner, which is a
superuser, so anything that compromised an app process owned the cluster, and
every connection reached the server with an empty application_name, so
pg_stat_activity could not say which service a session belonged to.

Split the DB credential into two login roles and name every pool:

- archie_migrator owns public and its objects and runs goose. Migrations run
  on a short-lived pool opened from the new migration_database_url, before the
  serving pool opens. It is not a superuser.
- archie_runtime serves requests with SELECT, INSERT, UPDATE, DELETE, sequence
  USAGE/SELECT and function EXECUTE only. It has no CREATE privilege, so it
  cannot change the schema.
- Every pool sets application_name: archie-state-store, archie-gateway, and a
  -migrate / -recovery / -schema / -provision suffix on the transient
  administrative connections. Existing one-URL installs keep working: an
  absent migration_database_url falls back to database_url and the migration
  runs on the serving pool exactly as before.

Provisioning is a new idempotent `archie-state-store provision` subcommand: it
needs an admin URL the services never hold (database owner or superuser),
creates or reconciles both roles, hands every public-schema object to the
migration role, revokes PUBLIC's CREATE on public, sets row-level grants, and
sets the migration role's default privileges so tables future migrations
create are usable without a hand-run grant. It stores SCRAM verifiers, refuses
superuser roles, and verifies the boundary before committing.

install.sh now provisions the bundled path and separates the credentials: the
superuser password the Compose database is created with lives only in a
compose-only postgres.env, while the env file the units load holds the
non-superuser PGPASSWORD. A bundled install that predates the split is
upgraded in place: the roles are created, the superuser password is rotated
out of the env file, and config.toml is moved to the runtime and migration
URLs. External databases keep their operator-managed credential; the
deployments README documents the one-shot provision command.

setup writes both bundled URLs, dev:postgres provisions the dev database, the
e2e stack provisions its throwaway server, and the deployment templates point
at the two roles.
2026-11-11 03:12:54 +11:00
sam fc50b026cf docs(deployments): name the release pipeline, not one CI system's file
deploy / plan (push) Successful in 6s
deploy / gate (push) Successful in 7m38s
deploy / dist-zip (push) Skipped
deploy / agent-image (push) Successful in 22s
deploy / archied-image (push) Successful in 2m28s
2026-11-09 08:39:08 +11:00
sam 5539835dd6 docs(deployments): agent image tags match the Woodpecker pipeline; drop the missing publish.sh
deploy / gate (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / plan (push) Canceled after 8s
2026-11-09 08:35:25 +11:00
sam 9616dd72ce refactor(playbook): drop the directory loader and docs left by the playbooks command
deploy / gate (push) Successful in 14m32s
deploy / plan (push) Successful in 7s
deploy / agent-image (push) Successful in 12s
deploy / dist-zip (push) Skipped
deploy / archied-image (push) Successful in 3m2s
2026-11-09 07:53:02 +11:00
sam 6e718e165a feat(curator): issue_list and issue_create tools via the daemon (#1770)
Quality / check (push) Successful in 4m6s
deploy / gate (push) Successful in 4m30s
deploy / archied-image (push) Successful in 2m50s
deploy / agent-image (push) Successful in 12s
deploy / dist-zip (push) Skipped
2026-11-03 08:12:05 +11:00
sam abcb72a76a feat(curator): dry-run self-intake pass with run inspection tools (#1770)
Curators could only reach the workflow/integration tools; there was no way
to survey task runs. Adds run_list/run_events to the shared registry
(guarded as read on task kind), attaches the instance's acting principal
to curator model calls so guarded tools authorize, and records a pass's
final text as a pass.summary action so the report is inspectable. Ships a
dry-run self-intake definition in dev.toml/config.example.toml declaring
only those two tools, so it cannot file issues yet.
2026-11-03 02:35:06 +11:00
sam 9d8f364af1 feat(releaseupdate): signed OCI distribution sources for releases and the catalogue (#1715)
Quality / check (push) Successful in 5m53s
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-31 18:18:43 +11:00
sam 3b6de96782 docs(deploy): name the registry each agent-image path publishes to
The pinning section said CI republished ghcr.io on every push to main. CI
publishes git.catlow.cloud, and only for a release tag; ghcr is where the
hand-run tools/publish.sh writes, which is the reference the default names.

A release zip pins a digest on a registry an anonymous pull cannot reach, and
[containers].registry_auth -- the setting that fixes it -- was documented
nowhere. Documented there, beside the pin.

Also corrected: every shipped profile does not pin the image, and the section
link omitted its number so it matched no heading.
2026-10-31 00:35:49 +11:00
sam 6ca5c7f851 fix(release): ship archie-agent beside archied so Kit runs have one to mount 2026-10-27 02:49:31 +11:00
sam 66daa071a5 fix(configuration): raise the default max uptime above the task limit 2026-10-26 18:32:25 +11:00
sam 3b7193c207 refactor!: remove the diff-size cap and its gate.diff-size step 2026-10-24 22:19:23 +11:00
sam 2cdf3b2c85 build(dev): serve the dev State Store over TLS on the Docker bridge so task containers reach it 2026-10-17 17:19:08 +11:00
sam 7b504f9ed6 refactor(archied): discover issues through organisation connections 2026-10-17 13:38:46 +11:00
sam 05987f04e2 feat(daemon): execute and verify organisation repository connections 2026-10-17 09:32:39 +11:00
sam 939b912889 refactor(ui): unify account administration under Access 2026-10-16 02:57:13 +11:00
sam 83aa17cf8b feat(identity): add browser setup and user accounts 2026-10-16 02:41:24 +11:00
sam 9cdae5642e feat(rpc): mTLS between services from operator-issued certificates
Replace the service bearer tokens with [tls] ca_file and cert_dir. Caller
identity comes from the verified certificate, not metadata. Without TLS,
services listen and dial on loopback only. Task containers get only the CA
and keep their grant.
2026-10-16 02:37:51 +11:00
sam 8540c9f7c8 feat(releaseupdate): transactional self-update through archied (#1633)
archied update install verifies the release zip, checks every binary
reports the release, pulls the agent image by digest, backs up the
database in-process and stages the binaries before handing off to
archied update watchdog in its own process. The watchdog restarts units
in release order, requires archied healthy for 120s and otherwise rolls
back binaries and image policy, restoring the database only when a
migration ran. Boot recovery resumes the frozen watchdog.

Deletes scripts/archie-update-install, scripts/archie-update-watchdog,
the updater's --from-source and --auto, CommandInstaller,
ScriptInstaller and the update-image and update-topology subcommands.
2026-10-12 03:21:02 +11:00
sam c097e50405 fix(archieui): warn when the dashboard is reachable off-host over plain HTTP
The dashboard serves plain HTTP, so a non-loopback bind carries the
session cookie and provider token in clear unless TLS terminates in front.
Warn at startup unless the operator declares a proxy with
[web].trust_forwarded_headers, and record the choice in deployments/README
where the dashboard is deployed. Point the non-loopback token error at
that surviving doc, not the deleted UI-boundary PRD.
2026-10-10 22:09:05 +11:00
sam 74a3ec925c feat(releaseupdate): archied update check finds releases through go-selfupdate; archie-update-check is deleted 2026-10-07 01:12:11 +11:00
sam 204b65a0e3 fix(controlplane): update-settings selects a release channel; services run only the installed update scripts 2026-10-07 01:00:28 +11:00
sam a8dbab0cff feat(controlplane): update commands are an update-settings kind edited on Status, defaulting to the installed scripts 2026-10-07 00:57:01 +11:00
sam 209f2b4adb docs: server quickstart, standalone template first, install starts PostgreSQL 2026-10-06 01:43:16 +11:00
sam 9e4ec516c9 feat(releaseupdate): install from release artifacts with journaled rollback, topology check and channels 2026-10-05 17:02:29 +11:00
sam c73ff313ea refactor(config): drop unused dispatch state labels
The daemon stopped setting forge state labels when label-driven retry was removed; the config field, helpers and defaults had no consumer. Remove them and the Scheduling policy page's State labels section.
2026-10-05 15:05:06 +11:00
sam 3a8879e6e9 fix(configuration): an overlay stands in for a missing config file; dev overlay is complete 2026-10-04 03:47:17 +11:00
sam c486fe291c refactor(archieplaybooks): fold archie-playbooks into archied playbooks lint|serve 2026-10-04 01:20:33 +10:00
sam 7d795fdd4f docs(deployments): service data ownership; start order no longer matters 2026-10-04 01:17:48 +10:00
sam 75988204d5 docs: strip doc and issue references from config, deployments and examples 2026-10-03 22:27:08 +10:00
sam 8f8e732b3a fix(configuration): make bot_user required, defaulted, and named
bot_user names the agent that chat turns and the session curator attribute memory
to. The loader requires it of a single-identity deployment, but nothing covered
the two paths between that requirement and the consumers.

local-ollama-standalone.toml set no bot_user and defined no [[identities]], so a
profile AGENTS.md lists as supported could not load at all (archie-core-cx0d).
Give the profile the field, and make the regression guard enumerate
deployments/*.toml through the production loader instead of naming two files, so
a profile added later is covered the moment it is committed. dev.toml is
excluded because it is a development overlay layered over an installed config
rather than a standalone profile, and the exclusion is itself checked so a
renamed overlay is noticed instead of silently widening the rule.

The root BotUser was validated only when no [[identities]] block was present, so
a config with exactly one identity left it empty: the curator then skipped every
session with "no agent id" while the chat turn path ran with the same empty agent
(archie-core-4slp). applyIdentityDefaults now adopts a sole identity as the root
single-identity values -- which is what those fields mean -- and leaves the root
alone when two or more identities leave no single answer. The skip reason now
names bot_user, so the remaining case reads as configuration rather than as a
data problem.

Both regressions were red-first: the enumerated profile test failed on
local-ollama-standalone.toml before the profile was fixed, and the sole-identity
test failed before applyIdentityDefaults adopted it.
2026-10-03 01:48:28 +10:00
sam 0d35e753b5 refactor: format markdown files using prettier, fix spelling concerns, reduce Taskfile bloat 2026-09-25 01:10:28 +10:00
sam c1cd5c8f5b chore(state-store): remove the one-time SQLite import and db_path
Every install serves from Postgres, so the legacyread/legacyimport packages,
the archie-state-store import command, the boot gate that refused to serve
over unimported SQLite files, and db_path (which only located those files)
go. Migration 0010 drops the import_completion table.
2026-09-25 00:40:48 +10:00
sam e5105280ad chore(config): remove token_env aliases for forge, artifacts and telegram
Each credential is now only a {engine, key} secret reference. A stale
token_env in a file is reported at startup as an unrecognised key.
nats.token_env stays (no secret-ref form), and providers.api_key_env stays:
it is the carrier the resolver writes each resolved key's env name into.
2026-09-24 23:58:26 +10:00
sam 9864766804 docs: describe the PostgreSQL-only stores, state_dir and the import-only db_path
Architecture, deployment, installer, skill and PRD text now name the Postgres
State Store, the offline recovery commands on database_url, state_dir for host
state, and db_path as the legacy-file locator for archie-state-store import.
Finalised PRDs, CHANGELOG, docs/news and docs/archive are unchanged records.
2026-09-24 14:51:48 +10:00
sam feeaa16c7b feat(postgres): wire the State Store to Postgres — DB URL, fail-closed pool, composition seam
Wave 1 of the Postgres migration (archie-core-7r0x.9). This is the shared shape
every port lane builds on; nothing here switches a live path to Postgres yet.

- config: add `database_url` beside `db_path` (config.go), mark it bootstrap-only
  in denied_keys, expose it in the webui StorageView/config schema, and add it to
  config.example.toml plus all deployments/*.toml.
- docker-compose: add a postgres:18 service; `task dev` (dev:postgres) brings it
  up and waits for it before the State Store boots.
- archied: `openStateStore` now opens and migrates the process-scoped Postgres
  pool via `openStateStorePool`. The State Store fails closed (D1.2) when
  database_url is absent or invalid rather than fall back to SQLite; serving
  against Postgres later requires an import-completion record (D1.3), which is a
  later lane's gate, not this one.
- composition seam: task-store construction (state_store_task.go) and EDA-store
  construction (state_store_eda.go) are extracted from openStateStore, with the
  pool open in state_store_pool.go. This is a behaviour-preserving refactor only
  — no store, interface, error, ordering or logging change — so L1 and L2 build
  against separate files instead of sharing state_store.go.

Tests: fail-closed pool behaviour is pinned (absent/invalid URL); the State Store
boot/restart tests now run against a throwaway Postgres 18 container (pgtest
TestMain). Scoped test command and module-wide build pass; see lane report.
2026-09-24 02:18:56 +10:00
sam c6df4b4a54 refactor(state-store): remove the PocketBase admin view
stateadmin served read-only PocketBase views over the SQLite task file on -admin-addr. PocketBase is SQLite-only and the State Store moves to Postgres (archie-core-7r0x), so the view goes; operators use a Postgres tool of their choice.
2026-09-24 01:09:42 +10:00
sam 6703cc77cf refactor(indexing): delete the workspace codesearch indexer
The index built google/codesearch over the chat workspace on every daemon
start, for the process lifetime, through a child-process helper and a
SQLite metadata DB -- and grep consulted it only for searches confined to
the workspace root, which the analysed sessions showed was a small
fraction of searches. The compute was continuous; the benefit was not.

Removes internal/indexing, the workspace-codesearch helper subcommand,
the [indexing] config section, and the GrepIndex accelerator seam from
the grep tool, which returns to its direct walk. google/codesearch
leaves go.mod with it.
2026-09-23 00:36:38 +10:00
sam faf32d44b5 feat(install): write a unit for every host service
install.sh built and installed five host binaries but wrote only
archied.service, so a fresh install ran one of five services and the updater
refused its own first update for units the installer never created
(archie-core-enow). The runbook stood in as the mechanism, which is the
hand-written step that let a messaging binary and unit drift apart
(archie-core-1c01).

- install.sh writes archie-state-store.service, archie-gateway.service,
  archie-ui.service and archie-messaging.service alongside archied.service,
  with the contents the runbook used to tell an operator to paste, and starts
  all five with the store first.
- deployments/systemd-user-service.md no longer carries the unit contents; it
  names the units, explains why each exists, and points at
  systemctl --user cat and drop-ins for inspection and overrides.
- The updater's unmanaged-unit refusal now reads as a safety net for a host
  that drifted rather than the normal path.
2026-09-22 14:41:38 +10:00
sam d8d7662c13 fix(release): package and update archie-messaging on a host install
The v1.30.0 extraction moved Telegram, email and webhook out of archied and into
cmd/archie-messaging, and nothing packaged it. dist-zip built and copied only the
five older binaries, and scripts/archie-update-install omitted it from both its
service and binary lists. A host upgraded across v1.30.0 therefore ran on with no
messaging process at all: archied logged that it stopped serving the telegram
gateway at the extraction, nothing replaced it, and the bot was dead for two days
with no error anywhere -- the daemon looked healthy throughout (archie-core-1c01).

Fixed in all four places the host component set was enumerated: the zip's build
loop, the zip's copy list, the installer's GATEWAY_SERVICES (messaging is a
service, so it is installed AND restarted) and its GATEWAY_BINARIES. The comment
claiming every process in the documented deployment ships in the zip is now true,
and says what makes it true.

The duplication was the actual defect, so the sets are pinned to cmd/ instead:
TestHostCommandListsCannotDrift derives the host-run commands from cmd/ and
asserts all four lists, plus the reference runbook's units, agree with it. Adding
a command or extracting one fails the test until it is packaged -- the check that
would have caught this on the day. The installer's own test grew the same
coverage, and the reason it passed while the bug was live is worth recording: its
fixture created only the stale cmd/ set, so the script's `[ -d ./cmd/$cmd ]` guard
silently skipped messaging.

Documentation, because the installer now refuses an update when a service unit is
missing and sends the operator to the runbook -- so a service the runbook omits is
a dead end: a messaging unit in deployments/systemd-user-service.md, the component
list and start sequence in deployments/INSTRUCTIONS.md (which ships inside the
zip), and the packaged-binary list in RELEASING.md.

Verified: task check exit 0, including test:tools and test:ui. Both new guards
were made to fail on purpose: reintroducing each omission produces "missing
archie-messaging" and "no call with fragments [./cmd/archie-messaging]".
2026-09-22 05:42:17 +10:00
sam 3b5994acef docs: drop one host's service names from port guidance
Five files named a specific host's service and the port it holds as the
reason a deployment might move the state store's listen address, including
config.example.toml, which ships to every operator. The guidance keeps its
point -- move the listen/target pair together when a port is already taken
on the host -- without naming a service the reader may not run, or have
heard of.

Also corrects the Taskfile's dev-stack summary, which still described the
state store as binding 127.0.0.1:9090. deployments/dev.toml pairs it with
9091, and has since the pair became configurable, so the comment described
a stack that does not start.
2026-09-22 04:09:16 +10:00
sam 33939b541a chore(dev): move the dev State Store off the contested 9090
cockpit binds 9090 by default on this host, so task dev could not start the
State Store and every service came up without one. dev.toml's own comment
says to move the listen/target pair when that happens.
2026-09-20 16:00:14 +10:00