72 Commits
Author SHA1 Message Date
sam 073959b4f9 feat(memory): scope chat notes and preserve deletions on restore (#2069)
Keep personal notes private by default and require review before publishing shared notes. Restore backups and replay forgotten notes in one transaction, and retain the installed release when an unattended rollback could bring back forgotten notes.
2026-11-21 13:20:31 +11:00
sam c5f24ec914 chore: delete code no binary reaches
Removes the Kit task-tool result shim (and with it the MCP go-sdk
dependency), the messaging enable/disable/restart helpers and channel
rebuild closures nothing calls since channels became extensions, the
unused messaging capability, ephemeral and text-fallback types, and
single stragglers in telemetry, releaseupdate, skill, storepkg and
webui. The egress sandbox relay stays for #1867.
2026-11-13 03:33:32 +11:00
sam db346b159d fix(webui): mark cookies Secure behind a TLS proxy; run gosec in the gate (#1919)
Cookies took Secure from r.TLS, which is nil behind a proxy that terminates
TLS, so sign-in cookies went out without it. Every dashboard cookie now goes
through one builder that reads the scheme the way the origin check does.

The worktree reset and the agent worker's chown walk use os.Root, so a
symlink swapped in mid-walk can't point them outside the worktree.

gosec runs in golangci-lint, tests included. G104, G115, G204 and G304 are
excluded in config; each remaining finding is fixed or suppressed at the site
with its reason.
2026-11-13 03:16:15 +11:00
sam 061227708d chore(release): remove the canary, release news, changelog and Woodpecker pipelines (#1917) 2026-11-13 02:35:59 +11:00
sam f92c5b4536 chore(runtime): delete Kit and external agent harness support (#1877)
ci/woodpecker/pr/quality Pipeline was successful
Quality / check (pull_request) Canceled after 7m39s
Archie runs its own agent loop only. Removes Kit resolution and launch,
the harness runner and adapters, the setup terminal and sandbox view,
harness secrets and OAuth capture (table dropped in migration 0075), the
task-tools contract that only Kit containers reached, and the
sandbox-kit-spec dependency. Agent profiles carry image and tools only.
The egress proxy keeps the model-session path; the sandbox relay code
stays for #1867.
2026-11-12 21:29:05 +11:00
sam c9248a251a fix(releaseupdate): label the daemon release row Services, not Gateway
deploy / plan (push) Failing after 2s
deploy / gate (push) Skipped
deploy / archied-image (push) Skipped
deploy / agent-image (push) Skipped
deploy / dist-zip (push) Skipped
2026-11-09 09:08:39 +11:00
sam 618b22db15 fix(releaseupdate): canary ignores task alerts, which the park rate already judges
deploy / plan (push) Successful in 6s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
deploy / gate (push) Canceled after 4m28s
2026-11-09 08:56:22 +11:00
sam 5356745968 fix(releaseupdate): address review findings on the #1725 canary change
Quality / check (push) Successful in 3m39s
deploy / gate (push) Successful in 4m28s
deploy / archied-image (push) Successful in 3m11s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
- one shared UpdateStateDir() resolver for daemon and dashboard, instead of
  two copies that could drift
- the recovery path for a committed/rolled-back journal writes
  installed.json too, so a crash after the outcome phase cannot lose the
  install record
- JudgeCanaryNow is the explicit signals-only entry point for promoting
  without the window, replacing the zero-install-time sentinel
- canary version comparisons use IssuedAt, matching Promote's replay rule,
  so semver/issued-at divergence cannot wedge check red; queuedBehind
  reuses the judgeable predicate instead of a looser check
- promote prints the channel/pin; promote.yml cancels superseded runs
- canary card: no duplicated elapsed rendering, channel row waits for the
  settings resource instead of showing a default, agent fallback '?'
- rehearsal script bounds its docker build/push/run calls

task check passes on this tree.
2026-11-02 02:45:26 +11:00
sam 5b5059615b feat(releaseupdate): judge the installed release, window from install (#1725)
Quality / check (push) Successful in 4m2s
deploy / gate (push) Successful in 4m31s
deploy / archied-image (push) Successful in 3m14s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
The canary window now counts from when a release is installed on the host
(installed.json / installed_at in the update snapshot), not from the
statement's issued_at, and 'release canary check' judges the release the
canary actually runs via its exact-pin tag instead of the moving next tag.
A newer release on next queues behind the one being judged instead of
restarting the clock, so stable moves once per release.

New subcommands: canary status (read-only state, signals, verdict), canary
promote (promote now when hard checks pass), canary unpin (back to next),
and start -force. A held/pinned canary is verified installable every check
and a stranded pin fails the run red instead of looking green. Rollback
pre-flight checks stable's zip and agent image digests before pinning.
Relative signals are judged whenever both windows carry minimum data, so a
breach rolls back mid-window; promote still needs the full window unless
the operator promotes early (new promote.yml workflow_dispatch). The window
is configurable via -window / CANARY_WINDOW.

Also: dashboard canary status card (ui), and task canary:rehearse, a local
end-to-end rehearsal of publish -> canary start -> status -> check ->
promote against a throwaway registry and host.

task check passes; rehearsal run green (REHEARSE_EXIT=0).
2026-11-02 01:15:29 +11:00
sam efd3eb2854 fix(releaseupdate): wait for the canary to offer a release before installing (#1716)
Quality / check (push) Successful in 3m14s
deploy / gate (push) Successful in 6m48s
deploy / archied-image (push) Successful in 3m3s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-01 02:55:31 +11:00
sam 0e931485a0 docs(releaseupdate): use contractions in the release and canary text (#1716)
Quality / check (push) Successful in 7m33s
deploy / gate (push) Successful in 6m27s
deploy / archied-image (push) Successful in 2m57s
deploy / agent-image (push) Successful in 12s
deploy / dist-zip (push) Skipped
2026-10-31 22:18:57 +11:00
sam 1bf536d4ab docs(releaseupdate): rewrite the canary comments in plain English (#1716)
deploy / gate (push) Successful in 6m20s
Quality / check (push) Canceled after 8m7s
deploy / archied-image (push) Successful in 2m57s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-10-31 22:11:04 +11:00
sam 2181b5fd08 ci(deploy): move the images' :latest only when a release is promoted (#1716)
Quality / check (push) Successful in 3m47s
deploy / gate (push) Successful in 6m29s
deploy / archied-image (push) Successful in 2m59s
deploy / agent-image (push) Successful in 10s
deploy / dist-zip (push) Skipped
2026-10-31 21:48:10 +11:00
sam a4a7c96da6 feat(releaseupdate): publish to next and promote to stable only after a canary evaluation (#1716)
Quality / check (push) Successful in 3m49s
deploy / gate (push) Failing after 16m14s
deploy / archied-image (push) Skipped
deploy / agent-image (push) Skipped
deploy / dist-zip (push) Skipped
2026-10-31 21:39:32 +11:00
sam 9d8f364af1 feat(releaseupdate): signed OCI distribution sources for releases and the catalogue (#1715)
Quality / check (push) Successful in 5m53s
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-31 18:18:43 +11:00
sam c5802cf86d fix(releaseupdate): load the env file beside the config so update install gets the database password from a shell (#1431) 2026-10-31 14:40:41 +11:00
sam 0c137d9319 refactor(fsutil): consolidate nine copies of the atomic write helper
Nine packages each had their own write-temp-then-rename helper with
slightly different behaviour (some synced, some did not; some created
the parent directory, some did not; modes and temp-file patterns
varied). They are replaced by one fsutil.WriteFileAtomic that always
syncs before rename, creates the parent directory, and takes the file
mode as a parameter.
2026-10-28 00:48:59 +11:00
sam 6ca5c7f851 fix(release): ship archie-agent beside archied so Kit runs have one to mount 2026-10-27 02:49:31 +11:00
sam 6ff09f92a5 fix(releaseupdate): a development build names no release to compare 2026-10-24 02:25:25 +11:00
sam d915a24f4d fix(webui): compare a describe build's release, not its raw version, for drift 2026-10-24 02:17:53 +11:00
sam ab16bc673d fix(releaseupdate): run the release's watchdog and bound control-plane calls 2026-10-17 17:00:04 +11:00
sam cb29967e7b fix(releaseupdate): find releases in the public releases repository
The source repository is private, so the updater and install.sh got 404.
Releases are now published to samcharles93/archie-releases by
task release:publish.
2026-10-16 03:11:14 +11:00
sam 8540c9f7c8 feat(releaseupdate): transactional self-update through archied (#1633)
archied update install verifies the release zip, checks every binary
reports the release, pulls the agent image by digest, backs up the
database in-process and stages the binaries before handing off to
archied update watchdog in its own process. The watchdog restarts units
in release order, requires archied healthy for 120s and otherwise rolls
back binaries and image policy, restoring the database only when a
migration ran. Boot recovery resumes the frozen watchdog.

Deletes scripts/archie-update-install, scripts/archie-update-watchdog,
the updater's --from-source and --auto, CommandInstaller,
ScriptInstaller and the update-image and update-topology subcommands.
2026-10-12 03:21:02 +11:00
sam fff67838e0 fix(releaseupdate): read GitHub releases directly; drop go-selfupdate and its openpgp dependency 2026-10-07 23:18:03 +11:00
sam 1e062fd7f0 fix(releaseupdate): reuse a found release for ten minutes to stay under GitHub's unauthenticated rate limit 2026-10-07 01:17:21 +11:00
sam 74a3ec925c feat(releaseupdate): archied update check finds releases through go-selfupdate; archie-update-check is deleted 2026-10-07 01:12:11 +11:00
sam 204b65a0e3 fix(controlplane): update-settings selects a release channel; services run only the installed update scripts 2026-10-07 01:00:28 +11:00
sam a8dbab0cff feat(controlplane): update commands are an update-settings kind edited on Status, defaulting to the installed scripts 2026-10-07 00:57:01 +11:00
sam 9e4ec516c9 feat(releaseupdate): install from release artifacts with journaled rollback, topology check and channels 2026-10-05 17:02:29 +11:00
sam 3360cddb5a refactor(staterpc,releaseupdate,agentexec,storecontract,worktree): cut comments down to what the code does 2026-10-03 22:00:43 +10:00
sam 030b4ed176 refactor: strip doc and issue narration from code comments
Remove PRD/doc paths, bead IDs, phase notes and deleted-test references
from comments; drop test-only hooks left by the test deletion. Includes
in-progress archied bootstrap, gateway and query changes.
2026-10-03 21:47:46 +10:00
sam 780bc44d82 refactor: testing from the beginning. a journey in 150k lines of tests to none 2026-10-03 21:16:25 +10:00
sam aa24551d98 fix(update): treat the runtime as the release version; stamp it in dist-zip
The update check still read the retired archie/v* tag stream, so a host on
1.41.1 was offered runtime 1.35.0, and the installer built the agent image
from that stale tag. Both now use the release's archied/v* tag. The dist-zip
build also stamps runtimeVersion, so /version stops reporting 'dev'.
2026-09-25 01:58:20 +10:00
sam dda12dcdba refactor(scripts): drop the SQLite task-database backup from the installer and watchdog
The pre-update PostgreSQL snapshot is the only backup; the watchdog restores
binaries and never the database.
2026-09-24 14:43:25 +10:00
sam 1b5ba30113 feat(archied): recovery commands on PostgreSQL; fail-closed pre-update snapshot, no automatic schema rollback
Without -db, backup/restore/validate/rollback resolve database_url from the
config and work on the whole PostgreSQL database. Restore claims every service
role; rollback claims the State Store role; validate refuses a schema newer
than the binary. The installer refuses a daemon update it cannot snapshot
first; the watchdog never restores the database and prints the explicit
offline restore instead.
2026-09-24 09:50:10 +10:00
sam 59614cc63f style(update): adopt the formatter's alignment in the watchdog test
task check's fmt pass rewrites this literal; the version committed with
the readiness-wait change predated that pass.
2026-09-23 12:30:10 +10:00
sam a65eae2458 docs(agents): a bead never carries production instance detail
Beads sync to a Dolt remote, so anything written into one is published.
Two beads filed earlier today named an operator host, its internal
address and its filesystem paths -- the tracker was treated as a private
store, which it is not.

The rule is stated where the tracker is described, next to the existing
one about .beads/ never reaching a commit, because it is the same class:
the tracker is not a private store. Host-specific facts belong in the
operator's memos instance, which a bead may point at.
2026-09-23 12:29:29 +10:00
sam 6703cc77cf refactor(indexing): delete the workspace codesearch indexer
The index built google/codesearch over the chat workspace on every daemon
start, for the process lifetime, through a child-process helper and a
SQLite metadata DB -- and grep consulted it only for searches confined to
the workspace root, which the analysed sessions showed was a small
fraction of searches. The compute was continuous; the benefit was not.

Removes internal/indexing, the workspace-codesearch helper subcommand,
the [indexing] config section, and the GrepIndex accelerator seam from
the grep tool, which returns to its direct walk. google/codesearch
leaves go.mod with it.
2026-09-23 00:36:38 +10:00
sam 70ea9b2a44 feat(update): let every host binary report its version, and refuse skew
The updater is component-driven: it compares the daemon and agent versions and
acts only when they differ, so a host whose individual binaries disagree looks
healthy. Carina ran a hand-installed archie-messaging 1.35.0 against a 1.30.0
state store and crash-looped on an RPC the older store does not serve, while
the updater reported "no approved component changes; nothing to install".

The reason a comparison was impossible: three of the six host binaries carried
no version at all. archie-ui, archie-messaging and archie-playbooks did not link
internal/app/archied, so the release ldflags never reached them; the gateway and
state store linked the variable but exposed no flag. Only archied could say what
it was.

- internal/buildinfo carries Version, stamped by all three build sites: the
  zip, install.sh, and the updater itself.
- Every host binary answers -version ("<name> <version>"). archied's existing
  two-line output is unchanged, because something parses it.
- archie-update-install reads each installed binary before it can report
  "nothing to install" and refuses, naming every binary that disagrees with the
  daemon's version; after an install it verifies the same set against the
  release it just installed, so a binary the release did not rebuild is skew
  rather than success.
- TestDistZipShipsEveryHostCommand pins stamping across the three build sites
  and requires each cmd/ to report, so a seventh command cannot arrive unstamped.
2026-09-22 14:52:18 +10:00
sam d41aacfb3d fix(release): build and guard every host service the installers manage
install.sh built five of the six host commands, omitting archie-messaging, and
scripts/archie-update-install's guard only checked that every listed service had
a unit -- never that every host unit was listed. A v1.30.0 updater with the old
list therefore neither backed up nor restarted the messaging service, leaving a
hand-installed binary running under an otherwise updated host (archie-core-1c01,
archie-core-1faq).

- install.sh builds archie-messaging, and its summary names it.
- The updater refuses when a host unit this release ships is not in
  GATEWAY_SERVICES, naming it, alongside the existing refusal for a listed
  service with no unit.
- TestDistZipShipsEveryHostCommand -- the name deploy.yml:255 already promised --
  now pins all four lists to cmd/: the zip's build loop, the zip's copy list,
  install.sh's build loop, and the updater's GATEWAY_BINARIES/GATEWAY_SERVICES.

A fresh install still seeds only archied.service; that gap is tracked separately
as archie-core-enow.
2026-09-22 14:38:10 +10:00
sam d8d7662c13 fix(release): package and update archie-messaging on a host install
The v1.30.0 extraction moved Telegram, email and webhook out of archied and into
cmd/archie-messaging, and nothing packaged it. dist-zip built and copied only the
five older binaries, and scripts/archie-update-install omitted it from both its
service and binary lists. A host upgraded across v1.30.0 therefore ran on with no
messaging process at all: archied logged that it stopped serving the telegram
gateway at the extraction, nothing replaced it, and the bot was dead for two days
with no error anywhere -- the daemon looked healthy throughout (archie-core-1c01).

Fixed in all four places the host component set was enumerated: the zip's build
loop, the zip's copy list, the installer's GATEWAY_SERVICES (messaging is a
service, so it is installed AND restarted) and its GATEWAY_BINARIES. The comment
claiming every process in the documented deployment ships in the zip is now true,
and says what makes it true.

The duplication was the actual defect, so the sets are pinned to cmd/ instead:
TestHostCommandListsCannotDrift derives the host-run commands from cmd/ and
asserts all four lists, plus the reference runbook's units, agree with it. Adding
a command or extracting one fails the test until it is packaged -- the check that
would have caught this on the day. The installer's own test grew the same
coverage, and the reason it passed while the bug was live is worth recording: its
fixture created only the stale cmd/ set, so the script's `[ -d ./cmd/$cmd ]` guard
silently skipped messaging.

Documentation, because the installer now refuses an update when a service unit is
missing and sends the operator to the runbook -- so a service the runbook omits is
a dead end: a messaging unit in deployments/systemd-user-service.md, the component
list and start sequence in deployments/INSTRUCTIONS.md (which ships inside the
zip), and the packaged-binary list in RELEASING.md.

Verified: task check exit 0, including test:tools and test:ui. Both new guards
were made to fail on purpose: reintroducing each omission produces "missing
archie-messaging" and "no call with fragments [./cmd/archie-messaging]".
2026-09-22 05:42:17 +10:00
sam e94b1473e1 feat(statestore): offline backup, restore, validate and rollback
archie-state-store gains the offline recovery commands
docs/prds/runtime-control-plane.md ("Bootstrap, migration, and recovery")
asks for. They work on the task database file directly -- <db_path>-tasks.sqlite,
not db_path -- and need neither archied nor the Web UI, which matters because
the failure they exist for is the one that stops archied starting:

  backup   -db F -out F     consistent snapshot (VACUUM INTO), verified as a
                            usable store before it is published; safe against
                            a serving store
  restore  -db F -from F    replace the database with a snapshot, dropping the
                            replaced file's -wal/-shm
  validate -db F            readable, not corrupt, schema no newer than this
                            binary, every stored resource decoding under the
                            definition that owns it
  rollback -db F -kind K    replay an earlier revision of a stored resource
           [-revision N]    through the ordinary replace, recorded as a new
                            revision

No rollback RPC is added and none is needed: a revision carries its own value,
so restoring one is the same replace the dashboard and the channels already
use, by hand, with the store stopped.

"The State Store is stopped" is checked rather than assumed. The serving
process now takes an exclusive flock on <db>.lock for its whole life
(store.AcquireOwnership), and restore/rollback refuse with
`task database is owned by another process` while it holds it. backup is the
deliberate exception: the update installer runs inside the process it is
replacing and cannot stop the State Store.

scripts/archie-update-install takes its pre-flight snapshot through
`go run ./cmd/archie-state-store backup` from the release being installed
instead of its own sqlite3 shell, so the installer and an operator run one
implementation. The watchdog's refusal semantics are unchanged: an update that
cannot back up still must not start.

Docs: safe-change-and-recovery.md's "Recovery: not yet built" now describes
what exists.
2026-09-21 22:04:41 +10:00
sam 00eb1f3786 fix(update): write the agent version sidecar with a real newline
printf '%s\\n' emitted the literal characters backslash-n, which
archie-update-check's release-version check rejects, so every managed-agent
update left the installed agent version reported as unknown.
2026-09-18 23:14:34 +10:00
sam 72d80f1dfc fix(releaseupdate): don't treat empty available as deferred
Service.Check matched deferred[component.ID] == component.Available, and a
never-deferred recipient's absent entry is the empty string -- equal to an
empty Available emitted for a component with no update. Guard on a real
(non-empty) available version before applying the deferral, so an
up-to-date recipient is not told their update is deferred. No WIRE/DELETE
decision applies: this is a correctness fix on a live consumer, not
unreachable code.
2026-09-15 12:53:25 +10:00
sam 3a33caf45f fix(update): build each component from its own approved release tag
Components are independently versioned (RELEASING.md), so the newest
archied/v* and archie/v* tags usually sit on different commits --
gateway-only releases are the norm. The installer required both tags to
resolve to the same commit whenever both components changed, which
refused every update once the two newest releases diverged
(archied/v1.24.0 at 037326a, archie/v1.23.0 at 43bc29f).

Build each changed component from its own tag's commit instead: daemon
binaries and the adapter refresh from the daemon release, the managed
worker image from the agent release.
2026-09-14 23:25:06 +10:00
sam b11086e926 test(releaseupdate): fix a data race and a sleep in the install-cancellation test
Found by a full-repo 'go test -race ./...' sweep. internal/releaseupdate was
the only package that failed it.

The race: blockingInstallerStub.Install assigned s.gotCancelled lazily --

    if s.gotCancelled == nil { s.gotCancelled = make(chan struct{}) }

Install runs on the caller's goroutine (Service.Install calls it), while the
test goroutine reads s.gotCancelled. Two goroutines, one unsynchronised field.
The channel is now built by a newBlockingInstallerStub constructor, so no
goroutine ever writes it after the stub is handed to the test. Fixing it at
the producer rather than guarding the read.

The sleep: the test asserted 'the installer did not observe cancellation' with

    select { case <-s.gotCancelled: ... case <-time.After(50*time.Millisecond): }

which is a race against the clock to prove a negative, and is weaker than the
thing it stands in for -- a channel nothing has closed is simply not ready.
It is now a select/default, asserted synchronously.

The assertion was also unfalsifiable in one direction: if the channel were
never closed at all, the old test passed for the wrong reason. It now also
asserts the positive half -- after Install returns, the detached context is
released, so the observer MUST have fired -- which is what makes the negative
check meaningful.

A second constructor of the same stub is updated too; it relied on the lazy
assignment and would have closed a nil channel.

Reproduced before the fix (race detector, same test):
  WARNING: DATA RACE ... service_test.go:201
  --- FAIL: TestServiceInstallSurvivesCallerContextCancellation
After: 3/3 clean runs under -race.
2026-09-14 10:14:27 +00:00
sam cc0dc3915a chore(releaseupdate): adopt go fix errors.AsType rewrite 2026-09-13 15:00:36 +10:00
sam dad9389530 fix(update): back up the task store the daemon actually uses, or refuse
configured_task_db_path() read the top-level db_path and handed that literal
file to the watchdog as ARCHIE_TASK_DB_PATH (scripts/archie-update-install).
The daemon's task store is not that file: taskDBPath() in
internal/app/archied/main.go derives `<db_path>-tasks.sqlite`, and the
configured path is commonly a zero-byte placeholder beside it. The updater
therefore backed up an empty file, and a rollback would have restored that
empty file over the real database and deleted its -wal/-shm -- destroying the
task store the backup exists to protect.

That awk also matched db_path in any section. [indexing] owns its own
database, so a config listing it before the top-level key handed the watchdog
an unrelated database as ARCHIE_TASK_DB_PATH and backed that up instead.

Resolve the top-level key only, append the sibling suffix the daemon uses, and
refuse the update when the backup cannot be taken. The backup is the only
route back from a forward schema migration, so an update that cannot take one
must not start; it now runs as pre-flight, ahead of the adapter refresh, so a
refusal changes nothing on the host.

The tests drive the real script with a config file the harness controls; all
three fail against the previous adapter.
2026-09-12 03:01:58 +00:00
sam 176533d181 test(update): use errors.As in the check adapter harness 2026-09-12 02:59:52 +00:00
sam 1bf95f45c0 fix(update): resolve releases without a checkout, and fail loudly when it cannot
The check adapter resolved available releases with
`git -C "$REPO" ls-remote`, where REPO defaulted to $HOME/projects/archie-core.
With that checkout absent, moved or renamed, the swallowed error left Available
empty for every component -- which FormatSnapshot renders as "Archie is up to
date." A failed lookup therefore read as parity: the exact inverse of what an
operator needs to know.

Resolve releases from the forge API and `git ls-remote` against the remote
directly, with no working-tree state and no dependence on a clone existing. When
neither source answers, exit non-zero so the /update command reports that it
could not check instead of fabricating an up-to-date result.

A tagged release is announced whether or not it has a published artifact, and
Reference now records which source answered and the artifact state, so an
operator is not sent to a tag with nothing to download without being told.
This is what hid archied/v1.24.0 from the live host: tagged, never published.

The failure tests run the real script through a fake git and curl, so they
assert the source-selection logic rather than a stub. Against the previous
script all four fail, including the checkout dependency itself.
2026-09-12 02:59:23 +00:00