Embed install-type at build time; verify updates against actual running state, not installer exit code #982

Closed
opened 2026-10-27 13:38:33 +00:00 by sam · 0 comments
Owner

Problem

/update and /version currently trust whatever the operator's external update_check_command/update_install_command reports (internal/releaseupdate/command.go). Nothing verifies that the reported version actually matches what's running. This already produced a false success: /update reported archied/archie-agent both updated to v1.9.0 (the release containing the #468 fix), and a task run immediately after still hit the exact pre-fix failure — see #468 and #511 for the full evidence trail.

Separately, Archie ships in multiple install shapes (systemd-managed binary, Docker container via compose, possibly others). Nothing today tells the update mechanism which shape a given running component actually is, so a wrong-path update (e.g. attempting a container-style update against a systemd-managed process) is possible and would currently fail silently or claim success incorrectly.

Design (researched against prior art: restic, Tailscale, Docker/kubectl version output)

1. Stamp install type at build/release time — never probe for it at runtime.
Add a small cross-cutting internal/installtype package: var channel = "unknown", read via Channel() string. Stamp the real value per release artifact with -ldflags "-X .../installtype.channel=systemd" (native binary build) or =container (Docker image build) — one line per build target in the existing release pipeline, since archied and archie-agent are already built as separate artifacts for their respective channels.

Precedent: restic strips its self-update subcommand entirely via a build tag for distro-packaged builds — the wrong path isn't just mislabeled, it's absent. Tailscale, despite runtime-detecting almost everything else, treats Docker as an unconditional "no self-update, manage via image tag" — even the most sophisticated self-updater in this survey doesn't try to be clever about containers.

2. Fail closed, never guess. The /update dispatcher must refuse to proceed if installtype.Channel() == "unknown" or doesn't match what the install/check command expects, with a clear operator-facing message. No fallback path.

3. Verify after, don't trust the exit code. After Installer.Install returns success, re-check the actual running reference (binary path + hash, or container image digest) against the target version, and only then report success to the user. This is the concrete fix for the false-success bug. Tailscale's confirm() does the equivalent version comparison before/after in its own updater.

4. Extend releaseupdate.Component (internal/releaseupdate/service.go) with InstallType string and Reference string fields (e.g. "container", "ghcr.io/samcharles93/archie-agent:1.9.0@sha256:..."), populated by each component: archied reads its own compiled-in value; archie-agent reports its own the same way (already has a channel back to the daemon over NATS for logs — extend for a status/version response); third-party dependencies Archie doesn't build (e.g. NATS) can't self-report, so their InstallType/Reference comes from the deployment profile (deployments/*.toml) rather than detection.

5. Surface per-component status on the Configuration page and in /version, structured as one block per component: {install_type, running_version, reference, latest_available, status}, where status is an enum (ok / update_available / drift / unknown) — not a bare color, with the actual comparison (e.g. running digest vs. latest digest) visible on hover/expand. Modeled on docker version's Client/Server nested-block convention, which kubectl version independently converged on too.

Relates to

  • #468 (the fix this update-verification gap let slip through undetected)
  • #511 (update command UX — this issue is the underlying mechanism that makes accurate progress/success reporting possible; #511 shouldn't report a nicer-looking version of a still-unverified claim)

Notes

2026-08-21: design point 3 committed as c25799e on branch fix/522-verify-update-took-effect (not pushed).

Report.Verify(running) partitions every claimed component into Confirmed/Drift/Unverified. Ground truth is the relaying process's own compiled-in gatewayVersion. Telegram now has three wordings: confirmed, 'could not confirm which version is actually running', and 'Update did NOT take effect' with per-component detail. Placeholders 'unknown' (watchdog default) and 'dev' (unstamped build) are unverifiable, never versions. Agent stays Unverified by design and is named as unchecked in the success message.

Gate: task check exit 0, golangci-lint 0 issues, gofumpt clean. Three adversarial review passes; all findings fixed. Regression tests proven to fail against pre-fix code; every new guard mutation-tested.

STILL OPEN on this issue:

  • point 4: Component.InstallType + Reference fields on releaseupdate.Component
  • point 5: per-component status block {install_type, running_version, reference, latest_available, status} on the Configuration page and in /version

Split out: archie-core-nln7 (dashboard updates get no phase-2 report, so the new verification never runs for them).
--notes=2026-08-21: design point 3 MERGED to main as 3fb4857 (PR #530, squash of c25799e). Verified merged main passes task check exit 0.

Report.Verify(running) partitions each claimed component into Confirmed/Drift/Unverified against the relaying process's own compiled-in gatewayVersion. Telegram has three wordings: confirmed, 'could not confirm which version is actually running', and 'Update did NOT take effect' with per-component detail. 'unknown' (watchdog default) and 'dev' (unstamped build) are unverifiable, never versions. Agent is Unverified by design and named as unchecked in the success message.

STILL OPEN, this issue stays in_progress:

  • point 4: Component.InstallType + Reference fields on releaseupdate.Component
  • point 5: per-component status block {install_type, running_version, reference, latest_available, status} on the Configuration page and in /version

Dependency note: archie-core-nln7 no longer depends on this issue. Its real prerequisite was the verification machinery, which landed in 3fb4857; it does not need points 4 or 5, so the blocking edge was removed to stop it sitting out of bd ready for no reason.

Close reason

All 5 design points landed: installtype package + fail-closed dispatcher + post-install Verify (3fb4857/c25799e, prior work), archie-agent build stamping + NATS status self-report (.1), Component.InstallType/Reference enrichment (.2), and GET /api/version + Configuration status block (.3). Each sub-issue passed an independent adversarial review pass.


Imported from beads archie-core-1786751948782-3-837a8fd0 · GitHub #522 · type feature · priority P1 · created 2026-08-14 · closed 2026-08-23

## Problem `/update` and `/version` currently trust whatever the operator's external `update_check_command`/`update_install_command` reports (`internal/releaseupdate/command.go`). Nothing verifies that the reported version actually matches what's running. This already produced a false success: `/update` reported `archied`/`archie-agent` both updated to `v1.9.0` (the release containing the #468 fix), and a task run immediately after still hit the exact pre-fix failure — see #468 and #511 for the full evidence trail. Separately, Archie ships in multiple install shapes (systemd-managed binary, Docker container via compose, possibly others). Nothing today tells the update mechanism which shape a given running component actually is, so a wrong-path update (e.g. attempting a container-style update against a systemd-managed process) is possible and would currently fail silently or claim success incorrectly. ## Design (researched against prior art: restic, Tailscale, Docker/kubectl version output) **1. Stamp install type at build/release time — never probe for it at runtime.** Add a small cross-cutting `internal/installtype` package: `var channel = "unknown"`, read via `Channel() string`. Stamp the real value per release artifact with `-ldflags "-X .../installtype.channel=systemd"` (native binary build) or `=container` (Docker image build) — one line per build target in the existing release pipeline, since `archied` and `archie-agent` are already built as separate artifacts for their respective channels. Precedent: restic strips its `self-update` subcommand entirely via a build tag for distro-packaged builds — the wrong path isn't just mislabeled, it's absent. Tailscale, despite runtime-detecting almost everything else, treats Docker as an unconditional "no self-update, manage via image tag" — even the most sophisticated self-updater in this survey doesn't try to be clever about containers. **2. Fail closed, never guess.** The `/update` dispatcher must refuse to proceed if `installtype.Channel() == "unknown"` or doesn't match what the install/check command expects, with a clear operator-facing message. No fallback path. **3. Verify after, don't trust the exit code.** After `Installer.Install` returns success, re-check the *actual* running reference (binary path + hash, or container image digest) against the target version, and only then report success to the user. This is the concrete fix for the false-success bug. Tailscale's `confirm()` does the equivalent version comparison before/after in its own updater. **4. Extend `releaseupdate.Component`** (`internal/releaseupdate/service.go`) with `InstallType string` and `Reference string` fields (e.g. `"container"`, `"ghcr.io/samcharles93/archie-agent:1.9.0@sha256:..."`), populated by each component: `archied` reads its own compiled-in value; `archie-agent` reports its own the same way (already has a channel back to the daemon over NATS for logs — extend for a status/version response); third-party dependencies Archie doesn't build (e.g. NATS) can't self-report, so their `InstallType`/`Reference` comes from the deployment profile (`deployments/*.toml`) rather than detection. **5. Surface per-component status** on the Configuration page and in `/version`, structured as one block per component: `{install_type, running_version, reference, latest_available, status}`, where `status` is an enum (`ok` / `update_available` / `drift` / `unknown`) — not a bare color, with the actual comparison (e.g. running digest vs. latest digest) visible on hover/expand. Modeled on `docker version`'s Client/Server nested-block convention, which `kubectl version` independently converged on too. ## Relates to - #468 (the fix this update-verification gap let slip through undetected) - #511 (update command UX — this issue is the underlying mechanism that makes accurate progress/success reporting possible; #511 shouldn't report a nicer-looking version of a still-unverified claim) ### Notes 2026-08-21: design point 3 committed as c25799e on branch fix/522-verify-update-took-effect (not pushed). Report.Verify(running) partitions every claimed component into Confirmed/Drift/Unverified. Ground truth is the relaying process's own compiled-in gatewayVersion. Telegram now has three wordings: confirmed, 'could not confirm which version is actually running', and 'Update did NOT take effect' with per-component detail. Placeholders 'unknown' (watchdog default) and 'dev' (unstamped build) are unverifiable, never versions. Agent stays Unverified by design and is named as unchecked in the success message. Gate: task check exit 0, golangci-lint 0 issues, gofumpt clean. Three adversarial review passes; all findings fixed. Regression tests proven to fail against pre-fix code; every new guard mutation-tested. STILL OPEN on this issue: - point 4: Component.InstallType + Reference fields on releaseupdate.Component - point 5: per-component status block {install_type, running_version, reference, latest_available, status} on the Configuration page and in /version Split out: archie-core-nln7 (dashboard updates get no phase-2 report, so the new verification never runs for them). --notes=2026-08-21: design point 3 MERGED to main as 3fb4857 (PR #530, squash of c25799e). Verified merged main passes task check exit 0. Report.Verify(running) partitions each claimed component into Confirmed/Drift/Unverified against the relaying process's own compiled-in gatewayVersion. Telegram has three wordings: confirmed, 'could not confirm which version is actually running', and 'Update did NOT take effect' with per-component detail. 'unknown' (watchdog default) and 'dev' (unstamped build) are unverifiable, never versions. Agent is Unverified by design and named as unchecked in the success message. STILL OPEN, this issue stays in_progress: - point 4: Component.InstallType + Reference fields on releaseupdate.Component - point 5: per-component status block {install_type, running_version, reference, latest_available, status} on the Configuration page and in /version Dependency note: archie-core-nln7 no longer depends on this issue. Its real prerequisite was the verification machinery, which landed in 3fb4857; it does not need points 4 or 5, so the blocking edge was removed to stop it sitting out of bd ready for no reason. ### Close reason All 5 design points landed: installtype package + fail-closed dispatcher + post-install Verify (3fb4857/c25799e, prior work), archie-agent build stamping + NATS status self-report (.1), Component.InstallType/Reference enrichment (.2), and GET /api/version + Configuration status block (.3). Each sub-issue passed an independent adversarial review pass. --- _Imported from beads `archie-core-1786751948782-3-837a8fd0` · GitHub #522 · type feature · priority P1 · created 2026-08-14 · closed 2026-08-23_
sam added the highfeature labels 2026-10-27 13:38:33 +00:00
sam self-assigned this 2026-10-27 13:38:33 +00:00
sam closed this issue 2026-10-27 13:38:33 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#982