Every service connected to PostgreSQL as the database owner, which is a
superuser, so anything that compromised an app process owned the cluster, and
every connection reached the server with an empty application_name, so
pg_stat_activity could not say which service a session belonged to.
Split the DB credential into two login roles and name every pool:
- archie_migrator owns public and its objects and runs goose. Migrations run
on a short-lived pool opened from the new migration_database_url, before the
serving pool opens. It is not a superuser.
- archie_runtime serves requests with SELECT, INSERT, UPDATE, DELETE, sequence
USAGE/SELECT and function EXECUTE only. It has no CREATE privilege, so it
cannot change the schema.
- Every pool sets application_name: archie-state-store, archie-gateway, and a
-migrate / -recovery / -schema / -provision suffix on the transient
administrative connections. Existing one-URL installs keep working: an
absent migration_database_url falls back to database_url and the migration
runs on the serving pool exactly as before.
Provisioning is a new idempotent `archie-state-store provision` subcommand: it
needs an admin URL the services never hold (database owner or superuser),
creates or reconciles both roles, hands every public-schema object to the
migration role, revokes PUBLIC's CREATE on public, sets row-level grants, and
sets the migration role's default privileges so tables future migrations
create are usable without a hand-run grant. It stores SCRAM verifiers, refuses
superuser roles, and verifies the boundary before committing.
install.sh now provisions the bundled path and separates the credentials: the
superuser password the Compose database is created with lives only in a
compose-only postgres.env, while the env file the units load holds the
non-superuser PGPASSWORD. A bundled install that predates the split is
upgraded in place: the roles are created, the superuser password is rotated
out of the env file, and config.toml is moved to the runtime and migration
URLs. External databases keep their operator-managed credential; the
deployments README documents the one-shot provision command.
setup writes both bundled URLs, dev:postgres provisions the dev database, the
e2e stack provisions its throwaway server, and the deployment templates point
at the two roles.
`task dist` builds the binaries release.json names into dist/ with the
compiler flags `build` uses, so after the gate has built bin/ it only links.
`build` is now static (CGO_ENABLED=0), like what ships.
`ui` runs once per task invocation: check built the dashboard, then build's
dependency built it again under the Go steps that embed it.
Both Dockerfiles compile with cache mounts for Go's module and build caches.
The canary window now counts from when a release is installed on the host
(installed.json / installed_at in the update snapshot), not from the
statement's issued_at, and 'release canary check' judges the release the
canary actually runs via its exact-pin tag instead of the moving next tag.
A newer release on next queues behind the one being judged instead of
restarting the clock, so stable moves once per release.
New subcommands: canary status (read-only state, signals, verdict), canary
promote (promote now when hard checks pass), canary unpin (back to next),
and start -force. A held/pinned canary is verified installable every check
and a stranded pin fails the run red instead of looking green. Rollback
pre-flight checks stable's zip and agent image digests before pinning.
Relative signals are judged whenever both windows carry minimum data, so a
breach rolls back mid-window; promote still needs the full window unless
the operator promotes early (new promote.yml workflow_dispatch). The window
is configurable via -window / CANARY_WINDOW.
Also: dashboard canary status card (ui), and task canary:rehearse, a local
end-to-end rehearsal of publish -> canary start -> status -> check ->
promote against a throwaway registry and host.
task check passes; rehearsal run green (REHEARSE_EXIT=0).
deploy.yml inlined its own sequence, which drifted weaker than `task check`:
no db:check, news:check or ui:check, so a release could ship stale sqlc
output, a stale ui/dist, or release notes that disagree with CHANGELOG.md,
and the dashboard suite ran outside the gate without its type check.
The release now runs `task check TEST_FLAGS=-count=1`: same definition, the
full uncached suite including the Docker-backed tests -short skips. quality.yml
already invoked the Taskfile, so both pipelines now share one gate.
The container build was the one build path that could not learn a version: the
build context excludes .git, so it has to be passed in, and nothing passed one
-- Compose handed the Dockerfile no build args, so every locally built agent
answered "dev" while the daemon reported the tag. The Taskfile now derives the
version from git describe once, for both the host binaries and the image, and
task image:agent passes it through.
docker compose build agent also failed outright: the service named the image by
digest, and Compose refuses to tag a digest. The build-only service names it by
tag now; what a task container runs is still pinned in the daemon's own
[containers] image.
The prd step returns the PRD as structured fields and human.approve takes
them as its document. The plan text every surface reads and the approval
card's layout are both rendered from that one document, so what the human
approves is what the implement workflow is handed. Replaces the separately
generated OpenUI view, which could drift from the plan.
agent.run gains views: the agent may return an OpenUI Lang view of its
answer through an optional view tool, read as steps.<id>.view. human.approve
stores it as the task's plan_view, and the approval card renders it with
the dashboard's own components, falling back to the plan text when there
is no view or it does not parse. The feasibility prd step uses it.
The source repository is private, so the updater and install.sh got 404.
Releases are now published to samcharles93/archie-releases by
task release:publish.
archied update install verifies the release zip, checks every binary
reports the release, pulls the agent image by digest, backs up the
database in-process and stages the binaries before handing off to
archied update watchdog in its own process. The watchdog restarts units
in release order, requires archied healthy for 120s and otherwise rolls
back binaries and image policy, restoring the database only when a
migration ran. Boot recovery resumes the frozen watchdog.
Deletes scripts/archie-update-install, scripts/archie-update-watchdog,
the updater's --from-source and --auto, CommandInstaller,
ScriptInstaller and the update-image and update-topology subcommands.
task setup installs the golangci-lint CI pins, the protobuf generators and
the dashboard dependencies, and warns by name about the tools it cannot
install (Go, Task, Node, Docker).
The linter version now lives in one file read by both the setup task and
both workflows, so a local checkout and the gate cannot run different
linters.
archie-gateway no longer links the daemon, the web UI or archied. Shared
runtime layering, watches, catalog, NATS discovery and the State Store client
live in servicekit; chat task profiles in chattask. The Gateway waits for the
daemon's NATS endpoint instead of starting a second broker, and chat
approvals go through the task-action client instead of a local service with
no daemon behind it. Version stamping uses buildinfo only.
Replace CLAUDE.md with a short decision-rule constitution; move PRDs,
architecture, development and design docs and repo skills out of the tree;
delete the docs tooling (docsgen, prdlint, docdrift, docsite, contractaudit,
reachaudit) and its gate steps; stop install.sh copying repo skills into
runtime skills.
A "reduce Taskfile bloat" refactor (0d35e753) removed the fixpoint loop from
task fmt and deleted the guard test that proved it, putting the tree back to a
single go fix pass with nothing left to notice. One pass is not a fixpoint:
stditerators rewrites `for i := range t.NumFields()` into a field iterator that
copies its loop variable, and forvar only deletes that copy on the following
pass, leaving a tree copyloopvar rejects. archie-core-wwho regressed silently.
Restore the loop and the guard. The deploy workflow has always inlined its own
single `go fix ./...`, which is how the same hazard reached CI as a spurious
git-diff failure rather than a lint error (archie-core-6kda); it now calls task
fmt, the one owner of the formatter sequence, and its guard rejects an inline
copy so the split cannot come back.
docs/architecture is declared authoritative for settled design, so a page that
cites code which has moved actively misleads every session and every subagent
brief that reads it. Nothing in the gate compared a prose citation against the
tree: docsgen checks the contract schema against Go types, docsite checks
generated artifacts against their Markdown sources, prdlint checks PRD writing
style. All three verify internal consistency, so a page can describe a package
deleted months ago and every check stays green -- which is how a page describing
a removed system survives behind a hand-written "removed" banner.
Add tools/docdrift. It resolves every internal/... citation under docs/ against
the tree and fails on one that dangles, handling the shapes prose actually uses:
a package with a symbol (internal/config.Config), a test with its package
(internal/app/archieui/TestFoo), a file-shape template
(internal/webui/api_<concern>.go), and a Go package pattern (internal/...).
Generated artifacts are skipped so embedded page bodies are not double-counted.
The baseline is a burn-down list of the 48 citations already broken -- packages
that moved (internal/store, internal/memory, internal/nats) and ones never built
(internal/policy, the image and research capability docs). It is not a licence:
the check also fails when an allowlisted citation starts resolving again, so an
exemption cannot outlive its reason. Regenerate with `docdrift list`.
Wire it into `task check` as docs:drift.
Also correct AGENTS.md, which listed `policy` among the cross-cutting packages
sitting under internal/. No such package has ever existed in this tree; the
design is approved and unbuilt (docs/architecture/policy.md). That is exactly
the class of stale claim this check now catches.
Scores a reviewer's findings on the 38 runnable Martian Code-Review-Bench
PRs: the deterministic core turns findings into posted comments, an
independent judge model matches them to the goldens, and the run reports
golden micro-recall and golden-only precision. The reviewer is a findings
fixture directory until the pr-review workflow lands.
Run with: task bench:prreview -- -findings DIR
Task templates every command, so the PostgreSQL health check's
'{{.State.Health.Status}}' rendered as an empty format string: dev:postgres
always reported unhealthy after 60s, the State Store never started, and the
Gateway and daemon exited on boot or retried against a store that was never
coming. Escape the template, and have dev:daemon and dev:gateway wait for
the State Store's port before starting.
Wave 1 of the Postgres migration (archie-core-7r0x.9). This is the shared shape
every port lane builds on; nothing here switches a live path to Postgres yet.
- config: add `database_url` beside `db_path` (config.go), mark it bootstrap-only
in denied_keys, expose it in the webui StorageView/config schema, and add it to
config.example.toml plus all deployments/*.toml.
- docker-compose: add a postgres:18 service; `task dev` (dev:postgres) brings it
up and waits for it before the State Store boots.
- archied: `openStateStore` now opens and migrates the process-scoped Postgres
pool via `openStateStorePool`. The State Store fails closed (D1.2) when
database_url is absent or invalid rather than fall back to SQLite; serving
against Postgres later requires an import-completion record (D1.3), which is a
later lane's gate, not this one.
- composition seam: task-store construction (state_store_task.go) and EDA-store
construction (state_store_eda.go) are extracted from openStateStore, with the
pool open in state_store_pool.go. This is a behaviour-preserving refactor only
— no store, interface, error, ordering or logging change — so L1 and L2 build
against separate files instead of sharing state_store.go.
Tests: fail-closed pool behaviour is pinned (absent/invalid URL); the State Store
boot/restart tests now run against a throwaway Postgres 18 container (pgtest
TestMain). Scoped test command and module-wide build pass; see lane report.
goose migration 0001 lands the twelve State Store tables on Postgres, and sqlc generates the task queries from it. Two behaviours that genuinely differ from SQLite are pinned by tests against a real PG 18: claiming takes the oldest queued row with FOR UPDATE SKIP LOCKED (without the clause a second claimer blocks, which the test shows by timing out), and every column behind a Go int is bigint -- SQLite's INTEGER is 64-bit, and the chat issue-number allocator seeds at 10^15, which does not fit an int4.
task db:generate writes the committed postgresdb package; task db:check regenerates it in the gate, as proto:check and ui:check do.
The remaining store methods (events, resources, identities, the status tables) still run on SQLite: porting them is the rest of archie-core-7r0x.2.
stateadmin served read-only PocketBase views over the SQLite task file on -admin-addr. PocketBase is SQLite-only and the State Store moves to Postgres (archie-core-7r0x), so the view goes; operators use a Postgres tool of their choice.
task test runs -short with the test cache on; task test:full keeps the
uncached run. The 50s keepalive proof skips under -short (the deploy
workflow's full go test still runs it). Sleep-bound archied watch tests
run in parallel, and the read-only gate steps run concurrently after fmt
and proto:check. ui:deps is run: once so the parallel UI steps install once.
Two beads, one root: four sites stamped a version and only two of them were CI.
`Taskfile.yml`'s build and `install.sh` derived GATEWAY_VERSION/RUNTIME_VERSION
from `git describe`, so a from-source build claimed a release it was not
running, and the stamp became a function of the commit graph -- the one input a
content-checksum build cache cannot see. An empty commit moved `git describe`
while `task build` reported "up to date" and shipped the previous stamp
(archie-core-pdq2). Both sites now stamp `dev`: a source build says so, and the
version a release carries is written into the artifact CI publishes. That
removes the non-file input rather than caching it, and traceability for a local
build is the checkout it came from.
`deploy.yml` published `:latest` from any push to main, so a release's own
stamping survived only until the next ordinary commit -- minutes, in an active
repo -- and a host deploying :latest reported dev, leaving the update path
unable to tell current from stale (archie-core-i5nb). `:latest` now moves only
from a commit that IS a release; every other build publishes `:edge` (mutable,
newest main) and `sha-<commit>` (immutable). A host that runs releases follows
`:latest`; a host that tracks main follows `:edge`, which RELEASING.md now says.
Verified: `task build` reports `archied dev`; the new workflow test fails
against the old workflow on all five guards and passes after; deploy.yml
parses; install.sh passes `bash -n`.
Gated with `task check` (exit 0) at 8faa0f6a -- the v1.39.0 base, CI-green --
with these five files applied. It is NOT gated at HEAD: HEAD carries an
unrelated red, `TestTaskProtoCarriesEveryDomainField`, because d6f61c2c added
ParkClass and RemediationRounds without the proto mapping. This change neither
causes nor fixes that.
The index built google/codesearch over the chat workspace on every daemon
start, for the process lifetime, through a child-process helper and a
SQLite metadata DB -- and grep consulted it only for searches confined to
the workspace root, which the analysed sessions showed was a small
fraction of searches. The compute was continuous; the benefit was not.
Removes internal/indexing, the workspace-codesearch helper subcommand,
the [indexing] config section, and the GrepIndex accelerator seam from
the grep tool, which returns to its direct walk. google/codesearch
leaves go.mod with it.
CHANGELOG.archied.md and CHANGELOG.archie.md were two version streams that
drifted: a release that moved both components had two numbers, two pages and
two tags, and a reader had to visit two files to learn what shipped.
The merged history holds 65 versions from 64 archied and 34 archie sections --
33 that both components released, 31 only archied, one only archie -- so 98
published pages become 65. A version's date is the newest of its sections'
dates, and a component whose own date differs carries it on its label, because
the history has two dates and the merge must not invent one. `## [Unreleased]`
is not a release: it gets no page, and it stays at the top of the file.
A level-two heading is where the merge could silently drop a section, so the
merge is verified by content rather than by the generator: section counts per
source, the merged version set against the exact union, and every source `###`
title present both as a substring and, after stripping the component prefix, as
an identical heading. Counting `##` headings rather than releases overcounts by
one here, which is how the empty `[Unreleased]` block was caught.
- newsgen reads one file and writes `docs/news/<version>.md`, the index, the
canonical `releases.json`, and `docs/news/redirects.json`. The redirect table
is what keeps the retired `/news/<component>/<version>/` URLs alive; it is
generated beside the pages so it cannot drift from them.
- release.sh takes one `--version`, keeps commit and tag (the Woodpecker
pipeline that will own them does not exist yet), and exits 3 when the commit
set justifies no release, so a pipeline can finish green without tagging.
- The consumers of the two files follow: the deploy workflow's tag gate and
release-notes extraction, the zip contents, the update check's changelog
links, the packaged-path note, the docs skill, and newsgen's tests.
The per-component detail survives as labelled sections inside each release, so
a reader still learns whether archied or the agent changed; what goes away is
the separate numbers and the separate pages.
The site's sidebar held the landing page and one guide next to seventeen
architecture records, the migration register, the review procedure and five
PRDs. Nothing there is secret -- every development page may be read by anyone --
but a reader who wants to know what Archie is and how to point an event at it
was meeting contributor material with no signal that it was not for them.
One source tree now yields two sets: public (index, guides/) under /docs/, and
development (architecture/, prds/) under /dev/, each with its own artifact and
its own derived sidebar. archive/, inspiration/, news/ and the credential notes
stay out of both, unchanged.
Selective PRD publication is gone rather than carried over. It existed only
because settled architecture cites PRDs and a link to a held-back target fails
the generator, so five PRDs published to keep those citations alive; with both
halves of every such citation in the development set, the exception has no work
left to do. Both sets are generated in one pass over one url map, so a citation
that crosses between them resolves to the base its target is served under, and
the refusal now fires only for a target genuinely in neither set.
Widening the set exposed a scanner bug worth its own note: the link regex
matched `Resolve[T](i, src, exportPath)` inside a code span and refused the page
for a link to `i,`. Code spans and fenced blocks are now skipped, because what a
renderer draws as a link is the only thing this tool may rewrite or refuse a
page for.
docs/prds/public-and-dev-docs.md records the decision. Draft: the three choices
in it are the maintainer's, the write-up is not yet reviewed.
task build already stamped archied, archie-gateway and archie-agent with
installtype.buildType=binary, but archie-state-store, archie-ui and
archie-messaging were built unstamped. A binary built by task build is
then permanently un-updatable: releaseupdate.Service.Install refuses on
installtype.Unknown.
Add the same stamp to the three missing go build commands and pin the
contract with a root-level deployment test that fails when any go build
line under the build task's cmds targets ./cmd/<name> without the stamp,
naming the offending command.
MkDocs retires and the Astro app becomes the only public site, so the artifact is
the product and the Markdown stays the source -- the split tools/newsgen already
states for releases. tools/docsite walks the published set and writes
docs/data/generated/docs.json: every page's directory-style url, title, section and
raw Markdown body, plus the sidebar tree derived from the layout.
Three things the artifact has to get right, each of which was wrong in a draft:
- LINKS. The published pages carry relative .md links that MkDocs rewrote at build
time and nothing else will. Each is rewritten to the target page's url using the
same function that builds urls, so a link and its target cannot disagree; anchors
and scheme-qualified links pass through; assets resolve by the same rule and none
are referenced.
- FRAGMENTS. A link can resolve to a page and still be broken, because a fragment
matching no heading lands the reader at the top of a long document. Each is
checked against the target's headings, by the slug rule a table of contents uses.
- REFUSAL. A link whose target the set holds back, or whose fragment matches no
heading, fails the generator naming the file, the line and the target. This found
three citations from settled architecture pointing at two PRDs the blanket prds/*
exclusion withheld -- links that were already 404s on the published site, so the
exclusion broke citations rather than withholding drafts. Both targets are now
published, for a reason distinct from the three ratified PRDs already published.
The published set moves out of mkdocs.yml into the generator with its reasons: a
renderer reading the artifact cannot see why a page is absent. docs/data/generated
is accounted for by docsgen's generatedArtifacts, which now names this file beside
contracts.json -- one ownership registry for the directory rather than one per
writer, so neither tool reports the other's artifact as unaccounted.
docs:artifact regenerates; docs:artifact:check reproduces into memory and fails on
drift without touching the tree, wired into task check beside news:check. The
existing docs:site target is MkDocs and was left alone -- a first attempt at that
name silently ran MkDocs and produced nothing.
Verified: task check exit 0, docs:artifact:check reporting 23 pages. The artifact
carries 23 pages, no archive/ inspiration/ or news/ entry, and the five PRDs the
set declares. Coverage: the published set and its count, url derivation, the nav
tree, both refusal paths, and check's no-write guarantee.
Derive a public news page and one release-notes page per component version
from CHANGELOG.archied.md and CHANGELOG.archie.md, the notes the release
process already freezes.
- tools/newsgen writes docs/news/releases.json, the canonical artifact a host
renders from, plus the MkDocs pages. Check mode fails when the committed
files lag a changelog edit, and runs in task check; tools/release.sh --tag
regenerates and stages them in the release commit, skipping files already
current.
- mkdocs.yml keeps the version pages out of the derived navigation without
excluding them from the site.
Five files named a specific host's service and the port it holds as the
reason a deployment might move the state store's listen address, including
config.example.toml, which ships to every operator. The guidance keeps its
point -- move the listen/target pair together when a port is already taken
on the host -- without naming a service the reader may not run, or have
heard of.
Also corrects the Taskfile's dev-stack summary, which still described the
state store as binding 127.0.0.1:9090. deployments/dev.toml pairs it with
9091, and has since the pair became configurable, so the comment described
a stack that does not start.