Commit Graph
32 Commits
Author SHA1 Message Date
sam e2e24f9d56 ci(deploy): pull the container image the uncached test suite needs
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-11 04:26:08 +11:00
sam c4a441d1c3 chore(task): install the pinned tooling with task setup
task setup installs the golangci-lint CI pins, the protobuf generators and
the dashboard dependencies, and warns by name about the tools it cannot
install (Go, Task, Node, Docker).

The linter version now lives in one file read by both the setup task and
both workflows, so a local checkout and the gate cannot run different
linters.
2026-10-09 12:27:45 +11:00
sam 5176938e95 ci(deploy): build the archied and agent images in parallel so the release zip waits only on the agent image 2026-10-07 00:35:13 +11:00
sam ab9db7cfe3 feat(install): bundled PostgreSQL has no network and is reached by Unix socket; print an AppArmor profile instead of disabling the userns restriction 2026-10-06 03:04:12 +11:00
sam 1d02c7e3e6 ci: allow unprivileged user namespaces for extension confinement 2026-10-06 01:55:37 +11:00
sam cf34d03476 fix(release): push the release commit and its tag atomically 2026-10-06 00:50:12 +11:00
sam 9e4ec516c9 feat(releaseupdate): install from release artifacts with journaled rollback, topology check and channels 2026-10-05 17:02:29 +11:00
sam c486fe291c refactor(archieplaybooks): fold archie-playbooks into archied playbooks lint|serve 2026-10-04 01:20:33 +10:00
sam e0701b18a3 refactor(archiegateway): own package for the Gateway service
archie-gateway no longer links the daemon, the web UI or archied. Shared
runtime layering, watches, catalog, NATS discovery and the State Store client
live in servicekit; chat task profiles in chattask. The Gateway waits for the
daemon's NATS endpoint instead of starting a second broker, and chat
approvals go through the task-action client instead of a local service with
no daemon behind it. Version stamping uses buildinfo only.
2026-10-04 00:59:02 +10:00
sam f0fe550a35 fix(build): restore the go fix fixpoint, and give the formatter one owner
A "reduce Taskfile bloat" refactor (0d35e753) removed the fixpoint loop from
task fmt and deleted the guard test that proved it, putting the tree back to a
single go fix pass with nothing left to notice. One pass is not a fixpoint:
stditerators rewrites `for i := range t.NumFields()` into a field iterator that
copies its loop variable, and forvar only deletes that copy on the following
pass, leaving a tree copyloopvar rejects. archie-core-wwho regressed silently.

Restore the loop and the guard. The deploy workflow has always inlined its own
single `go fix ./...`, which is how the same hazard reached CI as a spurious
git-diff failure rather than a lint error (archie-core-6kda); it now calls task
fmt, the one owner of the formatter sequence, and its guard rejects an inline
copy so the split cannot come back.
2026-10-03 02:28:33 +10:00
sam aa24551d98 fix(update): treat the runtime as the release version; stamp it in dist-zip
The update check still read the retired archie/v* tag stream, so a host on
1.41.1 was offered runtime 1.35.0, and the installer built the agent image
from that stale tag. Both now use the release's archied/v* tag. The dist-zip
build also stamps runtimeVersion, so /version stops reporting 'dev'.
2026-09-25 01:58:20 +10:00
sam 065606b2a5 fix(ci): make the gate tolerate formatting and give deploy a PG 18 client
The page-registry test parsed the route table one line per route and the
task-meta fixture was byte-compared, so the prettier reformat broke both.
The registry test now splits route objects and ignores comments; the
fixture compares compacted JSON. Also drops a stale router comment that
described the removed redirects, rebuilds ui/dist, regenerates the
contracts artifact, keeps prettier off generated output, and installs
postgresql-client-18 in deploy.yml's gate as quality.yml already does.
2026-09-25 01:29:23 +10:00
sam 5d6fbd5c30 fix(release): CI owns the release stamp; :latest moves only from a release
Two beads, one root: four sites stamped a version and only two of them were CI.

`Taskfile.yml`'s build and `install.sh` derived GATEWAY_VERSION/RUNTIME_VERSION
from `git describe`, so a from-source build claimed a release it was not
running, and the stamp became a function of the commit graph -- the one input a
content-checksum build cache cannot see. An empty commit moved `git describe`
while `task build` reported "up to date" and shipped the previous stamp
(archie-core-pdq2). Both sites now stamp `dev`: a source build says so, and the
version a release carries is written into the artifact CI publishes. That
removes the non-file input rather than caching it, and traceability for a local
build is the checkout it came from.

`deploy.yml` published `:latest` from any push to main, so a release's own
stamping survived only until the next ordinary commit -- minutes, in an active
repo -- and a host deploying :latest reported dev, leaving the update path
unable to tell current from stale (archie-core-i5nb). `:latest` now moves only
from a commit that IS a release; every other build publishes `:edge` (mutable,
newest main) and `sha-<commit>` (immutable). A host that runs releases follows
`:latest`; a host that tracks main follows `:edge`, which RELEASING.md now says.

Verified: `task build` reports `archied dev`; the new workflow test fails
against the old workflow on all five guards and passes after; deploy.yml
parses; install.sh passes `bash -n`.

Gated with `task check` (exit 0) at 8faa0f6a -- the v1.39.0 base, CI-green --
with these five files applied. It is NOT gated at HEAD: HEAD carries an
unrelated red, `TestTaskProtoCarriesEveryDomainField`, because d6f61c2c added
ParkClass and RemediationRounds without the proto mapping. This change neither
causes nor fixes that.
2026-09-23 13:09:01 +10:00
sam 3fdffa7203 fix(release): teach the image job the one-stream tag
The one-stream change updated the changelog gate, release.sh and the docs,
but missed the image job. Measured on the v1.37.1 run: `gate` resolved
1.37.1 from the `v*` tag and then warned "No release tags at HEAD", while
`build-and-push` read only `archied/v*`/`archie/v*`, found neither, and fell
back to `dev+<sha>` for both components. It also skipped the archie-agent
image entirely, because that step was gated on an `archie/v*` tag being
present -- so a one-stream release would have published a dev-stamped
gateway and never rebuilt the runtime.

Three changes:
- version resolution falls back to the one-stream `v*` tag, so both
  components are stamped with the release version;
- the runtime image builds on a `v*` release as well, since one version
  covers the release; a legacy gateway-only release still skips it, which
  is what release_tag exists to distinguish;
- the "no release tags at HEAD" warning recognises `v*`, so a real release
  stops being told it is not one.
2026-09-23 03:51:36 +10:00
sam 83c06b6aff feat(release): one release stream, one version, one page per version
CHANGELOG.archied.md and CHANGELOG.archie.md were two version streams that
drifted: a release that moved both components had two numbers, two pages and
two tags, and a reader had to visit two files to learn what shipped.

The merged history holds 65 versions from 64 archied and 34 archie sections --
33 that both components released, 31 only archied, one only archie -- so 98
published pages become 65. A version's date is the newest of its sections'
dates, and a component whose own date differs carries it on its label, because
the history has two dates and the merge must not invent one. `## [Unreleased]`
is not a release: it gets no page, and it stays at the top of the file.

A level-two heading is where the merge could silently drop a section, so the
merge is verified by content rather than by the generator: section counts per
source, the merged version set against the exact union, and every source `###`
title present both as a substring and, after stripping the component prefix, as
an identical heading. Counting `##` headings rather than releases overcounts by
one here, which is how the empty `[Unreleased]` block was caught.

- newsgen reads one file and writes `docs/news/<version>.md`, the index, the
  canonical `releases.json`, and `docs/news/redirects.json`. The redirect table
  is what keeps the retired `/news/<component>/<version>/` URLs alive; it is
  generated beside the pages so it cannot drift from them.
- release.sh takes one `--version`, keeps commit and tag (the Woodpecker
  pipeline that will own them does not exist yet), and exits 3 when the commit
  set justifies no release, so a pipeline can finish green without tagging.
- The consumers of the two files follow: the deploy workflow's tag gate and
  release-notes extraction, the zip contents, the update check's changelog
  links, the packaged-path note, the docs skill, and newsgen's tests.

The per-component detail survives as labelled sections inside each release, so
a reader still learns whether archied or the agent changed; what goes away is
the separate numbers and the separate pages.
2026-09-22 17:18:06 +10:00
sam 2ba4b1f848 ci(deploy): derive the commit by dereferencing the ref, and stop stamping bare dev
Two defects, both from consuming a ref's object id as if it were a commit.

1. IMMUTABLE IDENTITY. Both images published only `:latest`, so a deployed
   artifact could not be traced to a commit or restored. Each build now also
   publishes `sha-<commit>`. The commit is NOT `github.sha`: on a tag push that
   is the ref's object id, and every release tag in this repo is annotated —
   measured, 80 of 80 archied/v* and archie/v* tags are tag objects (archied/
   v1.36.0 points at 6720bc42 and dereferences to 576718c3). So the new step
   resolves `git rev-parse "${GITHUB_REF}^{commit}"` and the tags use that.
   Annotated tags are the right thing for the producer to create; the consumer
   stops guessing rather than the tag type being downgraded.

2. NO BARE dev STAMP. `[ -n "$GATEWAY_VERSION" ] || GATEWAY_VERSION=dev` made a
   main build indistinguishable from any other, and is why a release's stamping
   looked transient: the next commit to main republished :latest as plain `dev`.
   The fallback is now `dev+<short sha>`, so an unreleased build is identifiable
   and visibly not a release. The warning at the gate now states the consequence
   it previously only implied.

Scope: this file only. Nothing else in the checkout was touched.
2026-09-22 14:53:04 +10:00
sam 70ea9b2a44 feat(update): let every host binary report its version, and refuse skew
The updater is component-driven: it compares the daemon and agent versions and
acts only when they differ, so a host whose individual binaries disagree looks
healthy. Carina ran a hand-installed archie-messaging 1.35.0 against a 1.30.0
state store and crash-looped on an RPC the older store does not serve, while
the updater reported "no approved component changes; nothing to install".

The reason a comparison was impossible: three of the six host binaries carried
no version at all. archie-ui, archie-messaging and archie-playbooks did not link
internal/app/archied, so the release ldflags never reached them; the gateway and
state store linked the variable but exposed no flag. Only archied could say what
it was.

- internal/buildinfo carries Version, stamped by all three build sites: the
  zip, install.sh, and the updater itself.
- Every host binary answers -version ("<name> <version>"). archied's existing
  two-line output is unchanged, because something parses it.
- archie-update-install reads each installed binary before it can report
  "nothing to install" and refuses, naming every binary that disagrees with the
  daemon's version; after an install it verifies the same set against the
  release it just installed, so a binary the release did not rebuild is skew
  rather than success.
- TestDistZipShipsEveryHostCommand pins stamping across the three build sites
  and requires each cmd/ to report, so a seventh command cannot arrive unstamped.
2026-09-22 14:52:18 +10:00
sam d8d7662c13 fix(release): package and update archie-messaging on a host install
The v1.30.0 extraction moved Telegram, email and webhook out of archied and into
cmd/archie-messaging, and nothing packaged it. dist-zip built and copied only the
five older binaries, and scripts/archie-update-install omitted it from both its
service and binary lists. A host upgraded across v1.30.0 therefore ran on with no
messaging process at all: archied logged that it stopped serving the telegram
gateway at the extraction, nothing replaced it, and the bot was dead for two days
with no error anywhere -- the daemon looked healthy throughout (archie-core-1c01).

Fixed in all four places the host component set was enumerated: the zip's build
loop, the zip's copy list, the installer's GATEWAY_SERVICES (messaging is a
service, so it is installed AND restarted) and its GATEWAY_BINARIES. The comment
claiming every process in the documented deployment ships in the zip is now true,
and says what makes it true.

The duplication was the actual defect, so the sets are pinned to cmd/ instead:
TestHostCommandListsCannotDrift derives the host-run commands from cmd/ and
asserts all four lists, plus the reference runbook's units, agree with it. Adding
a command or extracting one fails the test until it is packaged -- the check that
would have caught this on the day. The installer's own test grew the same
coverage, and the reason it passed while the bug was live is worth recording: its
fixture created only the stale cmd/ set, so the script's `[ -d ./cmd/$cmd ]` guard
silently skipped messaging.

Documentation, because the installer now refuses an update when a service unit is
missing and sends the operator to the runbook -- so a service the runbook omits is
a dead end: a messaging unit in deployments/systemd-user-service.md, the component
list and start sequence in deployments/INSTRUCTIONS.md (which ships inside the
zip), and the packaged-binary list in RELEASING.md.

Verified: task check exit 0, including test:tools and test:ui. Both new guards
were made to fail on purpose: reintroducing each omission produces "missing
archie-messaging" and "no call with fragments [./cmd/archie-messaging]".
2026-09-22 05:42:17 +10:00
sam 0efd812387 fix(build): make formatting and CI converge (#901) 2026-09-20 00:38:51 +10:00
sam 4b51722440 fix(build): pin gofumpt and golangci-lint as one matched pair
`task fmt` formats with the standalone gofumpt and `task lint` re-checks that
formatting with the gofumpt golangci-lint vendors. `task check` runs `fmt`
then `lint` without diffing the tree, so `lint` is the only hard failure for
formatting -- and when the two versions differ, any construct they disagree on
makes the gate unsatisfiable: the standalone formatter rewrites a file one way
and lint rejects it, with nothing a contributor can run to satisfy both.

The surfaces had drifted onto three different pairs. This machine ran gofumpt
v0.7.0 against golangci-lint 2.13.2 (vendors v0.11.0), the image pinned
golangci-lint 2.12.2 (vendors v0.9.2) next to gofumpt v0.11.0, and CI
installed gofumpt@latest. `any((Trees)(nil))` in changes_test.go is the
construct that surfaced it: gofumpt <= v0.9.2 keeps the redundant parens,
>= v0.10.0 removes them.

- Dockerfile: golangci-lint 2.13.2, whose vendored gofumpt v0.11.0 now equals
  the pinned GOFUMPT_VERSION, so the image's two formatters agree.
- Dockerfile: fail the build if the pins ever diverge, rather than ship an
  image whose gate cannot pass.
- deploy.yml: gofumpt@v0.11.0 instead of @latest, which drifts ahead of
  whatever golangci-lint vendors.
- Record the invariant where an agent will meet it: AGENTS.md (CLAUDE.md) and
  the build-and-env skill, whose snapshots now name the pinned pair instead of
  calling both unpinned.
2026-09-19 18:52:10 +10:00
sam 3f99bdc96a ci(deploy): publish a GitHub Release with the distribution zip
The dist-zip job built the linux/amd64 zip and uploaded it as an Actions
artifact, then stopped. An artifact is reachable only from the run's page,
behind an Actions login, and expires after 90 days -- so every release since
the zip job landed produced a download an operator could not find. The one
Release on this repo (archied/v1.22.0) was created by hand and carries no
asset at all: nothing in CI has ever created one.

Publish the Release from this job and attach the zip:

- Job-level `permissions: contents: write`. The workflow default is
  contents:read, so a publish step would fail without it; scoping it to this
  job keeps the build-and-push jobs read-only.
- The body is this version's changelog section, extracted from the "## [x.y.z]"
  heading to the next "## [" heading rather than pasting the whole file. The
  changelog stays the single source of truth, and the tag/changelog gate in
  `gate` has already proven the section exists.
- Attached to the archied tag, because the zip is named and versioned for
  archied; `--follow-tags` pushes both tags at one commit, so the runtime
  release is the same commit.
- Idempotent: `gh release view` decides between `create` and
  `upload --clobber`, so a retried or superseded deploy run cannot fail on an
  existing Release or duplicate the asset.
- Fails loudly when the section extracts empty, rather than publishing a
  Release with no notes.
2026-09-17 07:17:27 +00:00
sam 2d228a00b4 ci(deploy): produce a complete distribution zip on every release
Two independent faults meant archied v1.23.0 shipped no zip at all.

dist-zip was gated on `startsWith(github.ref, 'refs/tags/archied/v')`. A
release is pushed as main plus both tags at one SHA, the SHA-keyed
concurrency group collapses those three runs into one, and the survivor
is whichever ref started last. For v1.23.0 that was refs/heads/main, so
the job was skipped and both tag runs that would have matched had already
been cancelled: 0 artifacts across all three runs. Resolve the archied
tag from the commit, the way build-and-push already does, and gate on
that instead of on the ref that happened to win the race.

The zip was also incomplete. It carried archied, archie-gateway,
archie-state-store and archie-playbooks, but not archie-ui, which since
1.23.0 is the process that serves the dashboard: a deployment cannot be
stood up from those four. Build every process the reference deployment
runs, and stamp each with the archied version rather than leaving the
state store and playbooks unversioned.

Closes archie-core-803r, archie-core-xi02.
2026-09-12 09:51:44 +10:00
sam d64f2b12a5 ci(deploy): package archied/archie-gateway/archie-state-store/archie-playbooks as a linux/amd64 zip (archie-core-8cda.4.7.1)
New dist-zip job, independent of build-and-push (no change to the
archied/archie-agent Docker image jobs). Triggers on any archied/vX.Y.Z
tag, so a gateway-only release (RUNTIME=skip) still produces it, keyed
to the archied version alone -- these binaries aren't in
tools/release.sh's two-component split and don't need to be for this.

archie-agent is deliberately excluded: it only ever runs inside the
Docker image build-and-push already publishes, never as a host binary.

Zip contains the four linux/amd64 binaries, the new
deployments/INSTRUCTIONS.md (quickstart + deployment-profile/systemd
pointers), and CHANGELOG.md/CHANGELOG.archied.md/CHANGELOG.archie.md.
README.md is included only if present -- it doesn't exist in this repo
yet, so omitted for now rather than fabricated here.

Verified locally: all four binaries build clean under
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 (modernc.org/sqlite is pure Go,
no cgo sqlite driver to worry about), and the assembled zip's contents
match what the workflow step produces.
2026-09-07 18:10:11 +10:00
sam c0a91c011c feat(contracts): add gateway service seams and protobuf toolchain 2026-09-05 22:57:45 +10:00
sam 6bf6e5e591 fix(ci): skip runtime image on gateway-only releases 2026-09-03 06:43:44 +00:00
sam 155e20af39 ci(deploy): key concurrency group on commit SHA, not ref
git push --follow-tags pushes main plus both archied/v* and archie/v*
tags in one operation, all pointing at the same commit. on.push
matches branches and tags, so every release push fired three identical
deploy runs -- observed just now: archied/v1.19.7, archie/v1.19.7, and
main all running in parallel from the same commit. The ref-keyed
concurrency group added earlier couldn't see these as the same work
since github.ref differs per trigger. Key on github.sha instead so all
three collapse into one.
2026-09-02 00:14:57 +10:00
sam b0cde5c1ed ci(deploy): cache image layers and cancel superseded deploy runs
Both images built via plain docker build/push with no cache, so every
push to main rebuilt archie-agent's full toolchain (apt packages,
golangci-lint, gofumpt, buf, uv+python, npm globals) from scratch --
several minutes, even when none of those inputs changed. Switch to
docker/build-push-action with GHA-backed layer caching per image.

Also add a concurrency group so a burst of pushes (e.g. a fix commit
immediately followed by its release commit) cancels the superseded
deploy rather than running three in parallel for capacity none of them
can use -- only the last one's :latest push matters anyway.
2026-09-02 00:06:52 +10:00
sam d96f403d93 chore(ci): pin Go version from go.mod and add a govulncheck step
CI's go-version was a floating "1.26" independent of go.mod, so it
could silently drift from the toolchain the project actually builds
with. Switched to go-version-file: go.mod as the single source of
truth, and added `task vuln` (govulncheck) plus a CI scan step -- kept
out of `task check` since it needs network access to the vuln DB.
2026-08-31 21:25:54 +10:00
sam f03e927fe4 ci: strip the v prefix when matching release tags to changelog sections
deploy / gate (push) Failing after 2m15s
deploy / build-and-push (push) Skipped
The gate derived the version with ${tag#archied/}, which leaves "v1.7.0",
then looked for a "## [v1.7.0]" heading. Changelog sections carry no v --
tools/release.sh writes and validates "## [1.7.0]" -- so the gate could
never match and every tagged release failed before the images were built.

This is the gate's first tagged run: it landed in dd9bddc, after the
archied/v1.6.0 tag, so nothing had exercised this branch until v1.7.0.
2026-08-10 02:22:50 +10:00
sam 8b21418e38 fix(build): install ripgrep where grep.go actually needs it on PATH
grepBinary() resolves rg from PATH since the embedded binaries were
dropped, but nothing installed it: not the GitHub Actions runner (CI
grep tests fell back to direct search and failed), and not the
archie-agent image (Dockerfile) that runs grep tool calls for sandboxed
tasks. Production was masked by carina's host already having ripgrep
installed system-wide for archied's own tool execution.

Dockerfile.archied is unused by the current deployment (docker-compose
builds only Dockerfile; archied runs on the host under systemd) --
corrected its stale "ripgrep is embedded" comment rather than adding
the package to an image nothing builds from.
2026-08-09 02:06:20 +10:00
sam 87c4fb91f7 ci: install gofumpt before the quality gate
GitHub's hosted runner has no gofumpt preinstalled, unlike the
self-hosted Gitea runner this workflow was copied from -- gofumpt -w .
failed with "command not found" on the first run.
2026-08-09 02:00:38 +10:00
sam dd9bddc245 ci: build and push images to GHCR, drop the Gitea workflow
Mirrors the existing Gitea deploy workflow's gate (fmt/vet/build/test,
dashboard tests, tag<->changelog check) but publishes to
ghcr.io/samcharles93/{archied,archie-agent} using the built-in
GITHUB_TOKEN. Additive: docker-compose.yml and the carina watchtower
setup are untouched, so production still runs off the Gitea-built
images until that's explicitly cut over.

.gitea/workflows/deploy.yml is removed from this checkout now that the
Gitea remote is gone; the git.catlow.cloud-hosted copy is unaffected
until that repo is pushed to again.
2026-08-09 01:47:57 +10:00