The update check still read the retired archie/v* tag stream, so a host on
1.41.1 was offered runtime 1.35.0, and the installer built the agent image
from that stale tag. Both now use the release's archied/v* tag. The dist-zip
build also stamps runtimeVersion, so /version stops reporting 'dev'.
Without -db, backup/restore/validate/rollback resolve database_url from the
config and work on the whole PostgreSQL database. Restore claims every service
role; rollback claims the State Store role; validate refuses a schema newer
than the binary. The installer refuses a daemon update it cannot snapshot
first; the watchdog never restores the database and prints the explicit
offline restore instead.
Beads sync to a Dolt remote, so anything written into one is published.
Two beads filed earlier today named an operator host, its internal
address and its filesystem paths -- the tracker was treated as a private
store, which it is not.
The rule is stated where the tracker is described, next to the existing
one about .beads/ never reaching a commit, because it is the same class:
the tracker is not a private store. Host-specific facts belong in the
operator's memos instance, which a bead may point at.
The index built google/codesearch over the chat workspace on every daemon
start, for the process lifetime, through a child-process helper and a
SQLite metadata DB -- and grep consulted it only for searches confined to
the workspace root, which the analysed sessions showed was a small
fraction of searches. The compute was continuous; the benefit was not.
Removes internal/indexing, the workspace-codesearch helper subcommand,
the [indexing] config section, and the GrepIndex accelerator seam from
the grep tool, which returns to its direct walk. google/codesearch
leaves go.mod with it.
CHANGELOG.archied.md and CHANGELOG.archie.md were two version streams that
drifted: a release that moved both components had two numbers, two pages and
two tags, and a reader had to visit two files to learn what shipped.
The merged history holds 65 versions from 64 archied and 34 archie sections --
33 that both components released, 31 only archied, one only archie -- so 98
published pages become 65. A version's date is the newest of its sections'
dates, and a component whose own date differs carries it on its label, because
the history has two dates and the merge must not invent one. `## [Unreleased]`
is not a release: it gets no page, and it stays at the top of the file.
A level-two heading is where the merge could silently drop a section, so the
merge is verified by content rather than by the generator: section counts per
source, the merged version set against the exact union, and every source `###`
title present both as a substring and, after stripping the component prefix, as
an identical heading. Counting `##` headings rather than releases overcounts by
one here, which is how the empty `[Unreleased]` block was caught.
- newsgen reads one file and writes `docs/news/<version>.md`, the index, the
canonical `releases.json`, and `docs/news/redirects.json`. The redirect table
is what keeps the retired `/news/<component>/<version>/` URLs alive; it is
generated beside the pages so it cannot drift from them.
- release.sh takes one `--version`, keeps commit and tag (the Woodpecker
pipeline that will own them does not exist yet), and exits 3 when the commit
set justifies no release, so a pipeline can finish green without tagging.
- The consumers of the two files follow: the deploy workflow's tag gate and
release-notes extraction, the zip contents, the update check's changelog
links, the packaged-path note, the docs skill, and newsgen's tests.
The per-component detail survives as labelled sections inside each release, so
a reader still learns whether archied or the agent changed; what goes away is
the separate numbers and the separate pages.
The updater is component-driven: it compares the daemon and agent versions and
acts only when they differ, so a host whose individual binaries disagree looks
healthy. Carina ran a hand-installed archie-messaging 1.35.0 against a 1.30.0
state store and crash-looped on an RPC the older store does not serve, while
the updater reported "no approved component changes; nothing to install".
The reason a comparison was impossible: three of the six host binaries carried
no version at all. archie-ui, archie-messaging and archie-playbooks did not link
internal/app/archied, so the release ldflags never reached them; the gateway and
state store linked the variable but exposed no flag. Only archied could say what
it was.
- internal/buildinfo carries Version, stamped by all three build sites: the
zip, install.sh, and the updater itself.
- Every host binary answers -version ("<name> <version>"). archied's existing
two-line output is unchanged, because something parses it.
- archie-update-install reads each installed binary before it can report
"nothing to install" and refuses, naming every binary that disagrees with the
daemon's version; after an install it verifies the same set against the
release it just installed, so a binary the release did not rebuild is skew
rather than success.
- TestDistZipShipsEveryHostCommand pins stamping across the three build sites
and requires each cmd/ to report, so a seventh command cannot arrive unstamped.
install.sh built and installed five host binaries but wrote only
archied.service, so a fresh install ran one of five services and the updater
refused its own first update for units the installer never created
(archie-core-enow). The runbook stood in as the mechanism, which is the
hand-written step that let a messaging binary and unit drift apart
(archie-core-1c01).
- install.sh writes archie-state-store.service, archie-gateway.service,
archie-ui.service and archie-messaging.service alongside archied.service,
with the contents the runbook used to tell an operator to paste, and starts
all five with the store first.
- deployments/systemd-user-service.md no longer carries the unit contents; it
names the units, explains why each exists, and points at
systemctl --user cat and drop-ins for inspection and overrides.
- The updater's unmanaged-unit refusal now reads as a safety net for a host
that drifted rather than the normal path.
install.sh built five of the six host commands, omitting archie-messaging, and
scripts/archie-update-install's guard only checked that every listed service had
a unit -- never that every host unit was listed. A v1.30.0 updater with the old
list therefore neither backed up nor restarted the messaging service, leaving a
hand-installed binary running under an otherwise updated host (archie-core-1c01,
archie-core-1faq).
- install.sh builds archie-messaging, and its summary names it.
- The updater refuses when a host unit this release ships is not in
GATEWAY_SERVICES, naming it, alongside the existing refusal for a listed
service with no unit.
- TestDistZipShipsEveryHostCommand -- the name deploy.yml:255 already promised --
now pins all four lists to cmd/: the zip's build loop, the zip's copy list,
install.sh's build loop, and the updater's GATEWAY_BINARIES/GATEWAY_SERVICES.
A fresh install still seeds only archied.service; that gap is tracked separately
as archie-core-enow.
The v1.30.0 extraction moved Telegram, email and webhook out of archied and into
cmd/archie-messaging, and nothing packaged it. dist-zip built and copied only the
five older binaries, and scripts/archie-update-install omitted it from both its
service and binary lists. A host upgraded across v1.30.0 therefore ran on with no
messaging process at all: archied logged that it stopped serving the telegram
gateway at the extraction, nothing replaced it, and the bot was dead for two days
with no error anywhere -- the daemon looked healthy throughout (archie-core-1c01).
Fixed in all four places the host component set was enumerated: the zip's build
loop, the zip's copy list, the installer's GATEWAY_SERVICES (messaging is a
service, so it is installed AND restarted) and its GATEWAY_BINARIES. The comment
claiming every process in the documented deployment ships in the zip is now true,
and says what makes it true.
The duplication was the actual defect, so the sets are pinned to cmd/ instead:
TestHostCommandListsCannotDrift derives the host-run commands from cmd/ and
asserts all four lists, plus the reference runbook's units, agree with it. Adding
a command or extracting one fails the test until it is packaged -- the check that
would have caught this on the day. The installer's own test grew the same
coverage, and the reason it passed while the bug was live is worth recording: its
fixture created only the stale cmd/ set, so the script's `[ -d ./cmd/$cmd ]` guard
silently skipped messaging.
Documentation, because the installer now refuses an update when a service unit is
missing and sends the operator to the runbook -- so a service the runbook omits is
a dead end: a messaging unit in deployments/systemd-user-service.md, the component
list and start sequence in deployments/INSTRUCTIONS.md (which ships inside the
zip), and the packaged-binary list in RELEASING.md.
Verified: task check exit 0, including test:tools and test:ui. Both new guards
were made to fail on purpose: reintroducing each omission produces "missing
archie-messaging" and "no call with fragments [./cmd/archie-messaging]".
archie-state-store gains the offline recovery commands
docs/prds/runtime-control-plane.md ("Bootstrap, migration, and recovery")
asks for. They work on the task database file directly -- <db_path>-tasks.sqlite,
not db_path -- and need neither archied nor the Web UI, which matters because
the failure they exist for is the one that stops archied starting:
backup -db F -out F consistent snapshot (VACUUM INTO), verified as a
usable store before it is published; safe against
a serving store
restore -db F -from F replace the database with a snapshot, dropping the
replaced file's -wal/-shm
validate -db F readable, not corrupt, schema no newer than this
binary, every stored resource decoding under the
definition that owns it
rollback -db F -kind K replay an earlier revision of a stored resource
[-revision N] through the ordinary replace, recorded as a new
revision
No rollback RPC is added and none is needed: a revision carries its own value,
so restoring one is the same replace the dashboard and the channels already
use, by hand, with the store stopped.
"The State Store is stopped" is checked rather than assumed. The serving
process now takes an exclusive flock on <db>.lock for its whole life
(store.AcquireOwnership), and restore/rollback refuse with
`task database is owned by another process` while it holds it. backup is the
deliberate exception: the update installer runs inside the process it is
replacing and cannot stop the State Store.
scripts/archie-update-install takes its pre-flight snapshot through
`go run ./cmd/archie-state-store backup` from the release being installed
instead of its own sqlite3 shell, so the installer and an operator run one
implementation. The watchdog's refusal semantics are unchanged: an update that
cannot back up still must not start.
Docs: safe-change-and-recovery.md's "Recovery: not yet built" now describes
what exists.
printf '%s\\n' emitted the literal characters backslash-n, which
archie-update-check's release-version check rejects, so every managed-agent
update left the installed agent version reported as unknown.
Components are independently versioned (RELEASING.md), so the newest
archied/v* and archie/v* tags usually sit on different commits --
gateway-only releases are the norm. The installer required both tags to
resolve to the same commit whenever both components changed, which
refused every update once the two newest releases diverged
(archied/v1.24.0 at 037326a, archie/v1.23.0 at 43bc29f).
Build each changed component from its own tag's commit instead: daemon
binaries and the adapter refresh from the daemon release, the managed
worker image from the agent release.
configured_task_db_path() read the top-level db_path and handed that literal
file to the watchdog as ARCHIE_TASK_DB_PATH (scripts/archie-update-install).
The daemon's task store is not that file: taskDBPath() in
internal/app/archied/main.go derives `<db_path>-tasks.sqlite`, and the
configured path is commonly a zero-byte placeholder beside it. The updater
therefore backed up an empty file, and a rollback would have restored that
empty file over the real database and deleted its -wal/-shm -- destroying the
task store the backup exists to protect.
That awk also matched db_path in any section. [indexing] owns its own
database, so a config listing it before the top-level key handed the watchdog
an unrelated database as ARCHIE_TASK_DB_PATH and backed that up instead.
Resolve the top-level key only, append the sibling suffix the daemon uses, and
refuse the update when the backup cannot be taken. The backup is the only
route back from a forward schema migration, so an update that cannot take one
must not start; it now runs as pre-flight, ahead of the adapter refresh, so a
refusal changes nothing on the host.
The tests drive the real script with a config file the harness controls; all
three fail against the previous adapter.
The check adapter resolved available releases with
`git -C "$REPO" ls-remote`, where REPO defaulted to $HOME/projects/archie-core.
With that checkout absent, moved or renamed, the swallowed error left Available
empty for every component -- which FormatSnapshot renders as "Archie is up to
date." A failed lookup therefore read as parity: the exact inverse of what an
operator needs to know.
Resolve releases from the forge API and `git ls-remote` against the remote
directly, with no working-tree state and no dependence on a clone existing. When
neither source answers, exit non-zero so the /update command reports that it
could not check instead of fabricating an up-to-date result.
A tagged release is announced whether or not it has a published artifact, and
Reference now records which source answered and the artifact state, so an
operator is not sent to a tag with nothing to download without being told.
This is what hid archied/v1.24.0 from the live host: tagged, never published.
The failure tests run the real script through a fake git and curl, so they
assert the source-selection logic rather than a stub. Against the previous
script all four fail, including the checkout dependency itself.
A self-update to 1.23.0 failed its health check and rolled back. The
adapter built and installed ./cmd/archied and nothing else, and the
watchdog restarted archied.service and nothing else, so the new daemon
came up against a State Store, Gateway and UI still on the previous
release. Since 1.23.0 archied cannot boot without a reachable State
Store (openStateStoreAdapter), which on a host that had never run one is
an immediate exit, a failed health check, and a rollback that looks like
a bad release rather than an incomplete deployment.
Install every binary the reference deployment runs, versioned off the
archied tag, and back each up so the rollback restores all of them
together. A binary with no .prev was introduced by the failed release, so
rollback removes it instead of leaving one behind that points at
processes the restored release does not run.
Restart in dependency order: the State Store owns archie.db and every
other process dials it, the Gateway serves the contract archied and the
UI dial, so each comes up before its consumers and stops after them.
archied itself is cycled unconditionally and a failed stop still aborts
the rollback, because a presence probe that guesses wrong must never
skip the stop that keeps a live process off the binary and the database
being restored. The siblings are best-effort, which is safe because the
adapter now refuses the update outright, before touching the host, when
a unit for one of them is missing, naming what to create.
install.sh had the same single-binary assumption, so a fresh install
produced a host that could not start either.
Closes archie-core-hbqk.
The update watchdog verifies archied's restart, so archied has to be what
answers. It was probing the dashboard's /healthz, which belongs to the
dashboard: switch [web] off and nothing answers, and in the extracted UI
topology that endpoint reports a different process entirely, which would
confirm a release while the daemon it was meant to verify is down.
[health].listen (127.0.0.1:8485, defaulted, always on) serves /healthz,
/health and /health/detailed for the life of the process. Liveness answers
503 until the daemon reaches its run loop: the listener binds before the
subsystems do, so an unconditional 200 would tell the watchdog a half-booted
release came up. Failing to bind is fatal, because a daemon nothing can probe
gets every update rolled back. The State Store's readiness surface now shares
the listener plumbing, keeping its log and error text verbatim.
Closes archie-core-exbz, which asked which of two bad options to take when
the dashboard is off. Neither: the daemon owns its own liveness.
Also deletes TestInstallUpdateDetachesInstallerFromCallbackContext. It has
failed intermittently under load more than once, racing its fake bot server's
multipart parse against the cancellation it induces.
The watchdog polled http://127.0.0.1:8420/healthz. Nothing binds 8420: the
daemon serves /healthz on [web].listen (127.0.0.1:8484 by default), so every
dashboard- or Telegram-initiated update waited out the 60s health timeout and
rolled back a release that had come up fine.
The daemon is the only process that knows which address it binds, so it now
derives the probe URL from its own config and hands it to the installer;
CommandInstaller exports ARCHIE_HEALTH_URL and the install script forwards it
into the watchdog unit, whose remaining default matches the dashboard's.
The suite never dialled the URL: every watchdog test faked curl as an
immediate failure, so only the unhealthy branch existed. One test now runs
the real curl against a real listener, and another pins the script default.
-X main.gatewayVersion/main.runtimeVersion targeted package main, but
those vars live in internal/app/archied (cmd/archied is a thin
wrapper). An -X target that doesn't exist is silently ignored by the
linker, so every archied binary this shipped -- task build, install.sh,
Dockerfile.archied, and the carina self-update script -- reported
"dev" instead of its real version. Point every one of those ldflags at
the actual package.
Also, while chasing the carina incident this surfaced:
- restart.go: /restart didn't check for an in-flight /update install.
The install runs as archied's own child process, so a restart killed
it mid-build (observed: "signal: terminated") instead of letting it
finish or fail cleanly. Refuse the restart while updateInProgress.
- update.go / update_report.go: the "no version could be confirmed
running" branch still opened with "Update complete", contradicting
its own next clause. Reworded so an unverified outcome isn't
reported as completion.
Dashboard-initiated updates also retain a configured Telegram restart-report route, while falling back to the web UI report when Telegram is unavailable.
Autonomous workflows now always hand a complete task to a scoped archie-agent container. Embedded NATS is the default and external NATS changes only broker placement; legacy agent and containers.enabled inputs decode for migration but cannot select an execution path. The daemon flushes task-event subscriptions before a worker can publish, and runtime updates snapshot and restore the configured worker image.
Build selected components from a temporary worktree at the exact approved tag commit instead of the moving branch tip. Validate release version inputs and require combined component tags to resolve to the same source commit.\n\nRefs #606
Carry the revalidated update snapshot through the installer contract so the shell adapter receives an explicit per-component plan. Build, back up, install, and roll back only selected components, and pass the runtime release version into agent image builds.\n\nCovers gateway-only, runtime-only, combined, and no-op updates.\n\nRefs #606\nBead: archie-core-462t
The version committed in the previous commit was authored blind,
without knowing a real archie-update-check already existed and has
been running on carina (production) since Aug 9 -- just never
committed to the repo. That live version is better designed than what
I wrote: it introspects the actually-running binary via `archied
-version` rather than trusting git/filesystem state that could have
silently diverged from what's really installed, and uses `git
ls-remote` for the available version instead of a local fetch, so the
script has no side effects and no dependency on the checkout being a
buildable Go workspace.
Adopt it as the canonical version, with one real fix: it used
Component IDs "archied"/"archie-agent", which don't match
releaseupdate.ComponentDaemon="daemon"/ComponentAgent="agent" -- the
same convention archie-update-watchdog's report and this session's
Service.Enrich already use. Under the old IDs, neither ever actually
applied to carina's real update checks; the mismatch was silent since
an unrecognised component ID just leaves InstallType/Reference/
Verify() unset rather than erroring.
Install only refreshed itself and the watchdog from the freshly
pulled checkout, so a bootstrapped archie-update-check would never
pick up a later fix to itself the way the other two adapters do.
The check-side of the reference update adapter never existed:
scripts/ had archie-update-install and archie-update-watchdog but no
archie-update-check, so any deployment that configured
chat.telegram.update_check_command pointing at the documented
"/usr/local/bin/archie-update-check" path had nothing there to run --
/update would report "not configured" regardless of what was actually
set. Add it: read-only, reports the daemon/agent components via git
describe against the local checkout and origin/main (no fetch-and-pull
the way install does), using the same "daemon"/"agent" component IDs
archie-update-watchdog's report and releaseupdate.ComponentDaemon/
ComponentAgent already use, so Service.Enrich and Report.Verify (this
session's earlier work) recognise it automatically.
Also fix archie-update-install's archie-agent build: it stamped
archied's ldflags but built archie-agent with none at all, so even a
native (non-container) archie-agent install reported an unstamped
"dev" build with installtype.Unknown.
config.example.toml gained every section that existed in the Go schema
but wasn't documented anywhere, not even commented: skills_dir,
plugin_dir, [chat.email], [chat].webhook_addr, [log], [memory] (noted
as currently unwired -- RegisterExternal is declared but never called
from cmd/archied, so the section has no effect yet), [[identities]]
(pointer to docs/architecture/identity.md rather than duplicating it),
and [[repos]].max_retries. tools.tool_policy.parallel_execution was
deliberately left undocumented: it's declared on ToolPolicy but never
read anywhere, genuinely dead rather than just underdocumented.
The /update command went silent between "starting" and one final message,
gave no version summary, and required manual intervention on failure.
CommandInstaller now streams the installer script's stdout live instead of
buffering it, and parses a structured result (previous/installed versions,
per-component status) from a sentinel line. Because the reference install
script hands the daemon restart off to a detached systemd unit specifically
so restarting archied.service doesn't kill the process that requested the
update, the outcome is reported in two phases: the synchronous build/install
result comes back through Install() itself, and the post-restart health
check/rollback outcome is written by a new watchdog script and relayed by
whichever archied process boots next, via a small JSON handoff file cleared
once relayed.
Also fixes three defects an adversarial review pass found in the first
draft: the caller's request/handler context was passed straight into the
install child process, so a concurrent /restart or a browser disconnect
could SIGKILL it mid binary-copy; Telegram and the web dashboard each
tracked "install in progress" with their own private flag despite sharing
one Service, allowing two installs to race the same .prev backups; and the
watchdog reported rolled_back:true even when no backup existed to restore,
which would have told an operator to stand down while the daemon was still
broken.
internal/webui gains an unauthenticated /healthz the watchdog polls after
restarting archied.