Every service connected to PostgreSQL as the database owner, which is a superuser, so anything that compromised an app process owned the cluster, and every connection reached the server with an empty application_name, so pg_stat_activity could not say which service a session belonged to. Split the DB credential into two login roles and name every pool: - archie_migrator owns public and its objects and runs goose. Migrations run on a short-lived pool opened from the new migration_database_url, before the serving pool opens. It is not a superuser. - archie_runtime serves requests with SELECT, INSERT, UPDATE, DELETE, sequence USAGE/SELECT and function EXECUTE only. It has no CREATE privilege, so it cannot change the schema. - Every pool sets application_name: archie-state-store, archie-gateway, and a -migrate / -recovery / -schema / -provision suffix on the transient administrative connections. Existing one-URL installs keep working: an absent migration_database_url falls back to database_url and the migration runs on the serving pool exactly as before. Provisioning is a new idempotent `archie-state-store provision` subcommand: it needs an admin URL the services never hold (database owner or superuser), creates or reconciles both roles, hands every public-schema object to the migration role, revokes PUBLIC's CREATE on public, sets row-level grants, and sets the migration role's default privileges so tables future migrations create are usable without a hand-run grant. It stores SCRAM verifiers, refuses superuser roles, and verifies the boundary before committing. install.sh now provisions the bundled path and separates the credentials: the superuser password the Compose database is created with lives only in a compose-only postgres.env, while the env file the units load holds the non-superuser PGPASSWORD. A bundled install that predates the split is upgraded in place: the roles are created, the superuser password is rotated out of the env file, and config.toml is moved to the runtime and migration URLs. External databases keep their operator-managed credential; the deployments README documents the one-shot provision command. setup writes both bundled URLs, dev:postgres provisions the dev database, the e2e stack provisions its throwaway server, and the deployment templates point at the two roles.
373 lines
14 KiB
YAML
373 lines
14 KiB
YAML
version: "3"
|
|
|
|
vars:
|
|
BIN_DIR: "bin"
|
|
# The version a build reports: the nearest tag with the distance and commit,
|
|
# or a bare commit when the checkout has no tags (a shallow clone). A build
|
|
# should not answer "dev" only because the caller did not name a version.
|
|
VERSION:
|
|
sh: git describe --tags --always --dirty 2>/dev/null | sed 's/^v//' || echo dev
|
|
|
|
tasks:
|
|
fmt:
|
|
desc: Fix and format Go source with the gate's configured formatters
|
|
cmds:
|
|
# One go fix pass is not a fixpoint: it applies each fixer once, to the
|
|
# parse it started with, so a rewrite made by one fixer is only fixed up
|
|
# by the next pass. stditerators turns `for i := range t.NumField()` into
|
|
# a field iterator that copies its loop variable, and forvar deletes that
|
|
# copy on the following pass -- leaving the copy behind for copyloopvar
|
|
# to reject. Apply passes until a dry run reports nothing left to fix,
|
|
# and fail rather than hand the lint step a tree it must reject.
|
|
- |
|
|
set -eu
|
|
converged=false
|
|
for pass in 1 2 3 4 5 6 7 8 9 10; do
|
|
go fix ./...
|
|
if go fix -diff ./... >/dev/null; then
|
|
converged=true
|
|
break
|
|
fi
|
|
done
|
|
if [ "$converged" != true ]; then
|
|
echo "go fix still rewrites the tree after 10 passes" >&2
|
|
exit 1
|
|
fi
|
|
# This must use the same engine and configuration as task lint. Running
|
|
# standalone gofumpt cannot reproduce gci's custom import grouping, so it
|
|
# can leave a tree that the gate rejects.
|
|
- golangci-lint fmt
|
|
|
|
proto:install:
|
|
desc: Install the pinned Buf and protobuf Go generators
|
|
cmds:
|
|
- bash tools/proto.sh install
|
|
|
|
proto:generate:
|
|
desc: Generate and format committed grpc-go service contracts
|
|
cmds:
|
|
- bash tools/proto.sh generate
|
|
|
|
proto:lint:
|
|
desc: Lint protobuf contracts and check compatibility with main
|
|
cmds:
|
|
- bash tools/proto.sh lint
|
|
|
|
db:generate:
|
|
desc: "Regenerate the sqlc query code from the schema and query files"
|
|
cmds:
|
|
- go -C tools tool sqlc generate -f {{.ROOT_DIR}}/internal/infrastructure/postgres/sqlc.yaml
|
|
|
|
db:check:
|
|
desc: "Regenerate the sqlc code and reject uncommitted generated differences"
|
|
cmds:
|
|
- task: db:generate
|
|
- git diff --exit-code -- internal/infrastructure/postgres/postgresdb
|
|
|
|
proto:check:
|
|
desc: Regenerate contracts and reject uncommitted generated differences
|
|
cmds:
|
|
- bash tools/proto.sh check
|
|
|
|
news:
|
|
desc: Regenerate the release data artifacts from the changelog
|
|
cmds:
|
|
- go -C tools run -mod=readonly ./newsgen --repo-root ..
|
|
|
|
news:check:
|
|
desc: Verify the committed release notes match the changelog
|
|
cmds:
|
|
- go -C tools run -mod=readonly ./newsgen check --repo-root ..
|
|
|
|
vet:
|
|
desc: Run go vet
|
|
cmds:
|
|
- go vet ./...
|
|
|
|
lint:
|
|
desc: Run golangci-lint
|
|
cmds:
|
|
- golangci-lint run ./...
|
|
- cd tools && golangci-lint run ./...
|
|
|
|
ui:deps:
|
|
desc: Install dashboard dependencies
|
|
internal: true
|
|
# Parallel UI checks must share one dependency install.
|
|
run: once
|
|
dir: ui
|
|
cmds:
|
|
- pnpm install --frozen-lockfile
|
|
|
|
ui:
|
|
desc: Build the dashboard into ui/dist, which the Go build embeds
|
|
# check builds the dashboard and then build depends on it again; a second
|
|
# build empties ui/dist under the Go steps that embed it.
|
|
run: once
|
|
dir: ui
|
|
deps: [ui:deps]
|
|
cmds:
|
|
- pnpm run build
|
|
|
|
test:ui:
|
|
desc: Test and type-check the dashboard
|
|
dir: ui
|
|
deps: [ui:deps]
|
|
cmds:
|
|
- pnpm test
|
|
- pnpm run typecheck
|
|
|
|
bench:prreview:
|
|
desc: Run the PR review benchmark on the configured models and write docs/benchmarks/pr-review-*.md (-- -model ALIAS -judge-model REF; --help for flags)
|
|
cmds:
|
|
- go run ./cmd/archied prbench {{.CLI_ARGS}}
|
|
|
|
dev:
|
|
desc: Run development services and Vite at http://localhost:5173
|
|
deps: [dev:daemon, dev:gateway, dev:messaging, dev:state-store, dev:dashboard, dev:ui]
|
|
|
|
dev:tls:
|
|
desc: Issue the development CA and service certificates deployments/dev.toml names
|
|
status:
|
|
- test -f tmp/dev-tls/archie-ui.pem
|
|
cmds:
|
|
- |
|
|
# Task containers reach the State Store over the Docker bridge, which
|
|
# services may only do over TLS.
|
|
set -eu
|
|
d=tmp/dev-tls
|
|
mkdir -p "$d"
|
|
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 -nodes -days 3650 \
|
|
-subj /CN=archie-dev-ca -keyout "$d/ca-key.pem" -out "$d/ca.pem" 2>/dev/null
|
|
for s in archied archie-gateway archie-messaging archie-state-store archie-ui; do
|
|
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 -nodes -days 3650 \
|
|
-subj "/CN=$s" -addext "subjectAltName=DNS:$s" -addext extendedKeyUsage=serverAuth,clientAuth \
|
|
-CA "$d/ca.pem" -CAkey "$d/ca-key.pem" -keyout "$d/$s-key.pem" -out "$d/$s.pem" 2>/dev/null
|
|
done
|
|
|
|
dev:postgres:
|
|
desc: Start PostgreSQL, wait for readiness, provision the database roles
|
|
cmds:
|
|
- docker compose up -d postgres
|
|
- |
|
|
set -eu
|
|
for _ in $(seq 1 60); do
|
|
status=$(docker inspect --format '{{`{{.State.Health.Status}}`}}' "$(docker compose ps -q postgres)" 2>/dev/null || true)
|
|
if [ "$status" = "healthy" ]; then
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
echo "postgres did not become healthy within 60s" >&2
|
|
exit 1
|
|
- |
|
|
# docker-compose.yml creates the dev database with a superuser named
|
|
# archie; provision the least-privilege roles deployments/dev.toml
|
|
# points at. Idempotent, so this also upgrades a volume created before
|
|
# the role split.
|
|
go run ./cmd/archie-state-store provision \
|
|
-admin-url "postgres://archie:archie@127.0.0.1:5432/archie?sslmode=disable" \
|
|
-runtime-url "postgres://archie_runtime:archie@127.0.0.1:5432/archie?sslmode=disable" \
|
|
-migration-url "postgres://archie_migrator:archie@127.0.0.1:5432/archie?sslmode=disable"
|
|
|
|
dev:wait-state:
|
|
internal: true
|
|
cmds:
|
|
- |
|
|
# Services read their settings from the State Store at boot and exit
|
|
# if it is not up yet, so wait for its port ([services.state] in
|
|
# deployments/dev.toml).
|
|
for _ in $(seq 1 120); do nc -z 127.0.0.1 9091 && exit 0; sleep 1; done
|
|
echo "state store did not come up on 127.0.0.1:9091 within 120s" >&2
|
|
exit 1
|
|
|
|
dev:daemon:
|
|
desc: Run archied with live reload
|
|
cmds:
|
|
- task: dev:wait-state
|
|
- air -build.args_bin="-config-overlay,deployments/dev.toml"
|
|
|
|
dev:dashboard:
|
|
desc: Run the dashboard backend
|
|
cmds:
|
|
- go run ./cmd/archie-ui -config-overlay deployments/dev.toml
|
|
|
|
dev:state-store:
|
|
desc: Run the State Store with development PostgreSQL
|
|
deps: [dev:postgres, dev:tls]
|
|
cmds:
|
|
- go run ./cmd/archie-state-store -config-overlay deployments/dev.toml
|
|
|
|
dev:gateway:
|
|
desc: Run the development Gateway
|
|
cmds:
|
|
- task: dev:wait-state
|
|
- go run ./cmd/archie-gateway -config-overlay deployments/dev.toml
|
|
|
|
dev:messaging:
|
|
desc: Run the development Messaging Service
|
|
cmds:
|
|
- task: dev:wait-state
|
|
- go run ./cmd/archie-messaging -config-overlay deployments/dev.toml
|
|
|
|
dev:ui:
|
|
desc: Run Vite (requires the dashboard backend)
|
|
dir: ui
|
|
cmds:
|
|
- pnpm install
|
|
- pnpm run dev
|
|
|
|
e2e:up:
|
|
desc: Start the local stack without Docker (PostgreSQL 18, services, Vite) and seed it
|
|
cmds:
|
|
- bash e2e/stack.sh start
|
|
- bash e2e/seed.sh
|
|
|
|
e2e:down:
|
|
desc: Stop the local e2e stack
|
|
cmds:
|
|
- bash e2e/stack.sh stop
|
|
|
|
e2e:check:
|
|
desc: List browser checks; use -- all or -- CHECK to run, add --headed to watch
|
|
cmds:
|
|
- node e2e/run.cjs {{.CLI_ARGS}}
|
|
|
|
canary:rehearse:
|
|
# Local rehearsal only: spins up its own registry, database and host, and
|
|
# is deliberately absent from check and CI.
|
|
desc: Rehearse the release-canary loop on a throwaway local host (needs docker; never part of the gate)
|
|
cmds:
|
|
- sh tools/canary-rehearse.sh
|
|
|
|
build:
|
|
desc: Build archie services and agent binaries
|
|
deps: [ui]
|
|
# Static, like the binaries a release ships, so `dist` reuses these
|
|
# compiled packages and only links again.
|
|
env:
|
|
CGO_ENABLED: "0"
|
|
vars:
|
|
GATEWAY_VERSION: '{{.GATEWAY_VERSION | default .VERSION}}'
|
|
RUNTIME_VERSION: '{{.RUNTIME_VERSION | default .VERSION}}'
|
|
SERVICE_LDFLAGS: >-
|
|
-X github.com/samcharles93/archie-core/internal/installtype.buildType=binary
|
|
-X github.com/samcharles93/archie-core/internal/buildinfo.Version={{.GATEWAY_VERSION}}
|
|
-X github.com/samcharles93/archie-core/internal/buildinfo.Runtime={{.RUNTIME_VERSION}}
|
|
sources:
|
|
- Taskfile.yml
|
|
- "**/*.go"
|
|
# Files the binaries embed.
|
|
- "internal/**/*.sql"
|
|
- "internal/**/*.yaml"
|
|
- "internal/**/*.json"
|
|
- "internal/**/*.tpl"
|
|
- "internal/**/*.md"
|
|
- go.mod
|
|
- go.sum
|
|
- "ui/dist/**"
|
|
generates:
|
|
- "{{.BIN_DIR}}/archied"
|
|
- "{{.BIN_DIR}}/archie-gateway"
|
|
- "{{.BIN_DIR}}/archie-state-store"
|
|
- "{{.BIN_DIR}}/archie-ui"
|
|
- "{{.BIN_DIR}}/archie-messaging"
|
|
- "{{.BIN_DIR}}/archie-agent"
|
|
cmds:
|
|
# One flag set for every service, matching the release workflow, so each
|
|
# binary reports the same version.
|
|
- for: [archied, archie-gateway, archie-state-store, archie-ui, archie-messaging]
|
|
cmd: go build -trimpath -ldflags "{{.SERVICE_LDFLAGS}}" -o {{.BIN_DIR}}/{{.ITEM}} ./cmd/{{.ITEM}}
|
|
- go build -trimpath -ldflags "-X github.com/samcharles93/archie-core/internal/app/agentworker.version={{.RUNTIME_VERSION}} -X github.com/samcharles93/archie-core/internal/installtype.buildType=binary" -o {{.BIN_DIR}}/archie-agent ./cmd/archie-agent
|
|
|
|
dist:
|
|
desc: Build the linux/amd64 binaries a release ships into dist/ (VERSION=1.2.3; needs jq)
|
|
deps: [ui]
|
|
env:
|
|
GOOS: linux
|
|
GOARCH: amd64
|
|
CGO_ENABLED: "0"
|
|
vars:
|
|
DIST_LDFLAGS: >-
|
|
-X github.com/samcharles93/archie-core/internal/installtype.buildType=binary
|
|
-X github.com/samcharles93/archie-core/internal/buildinfo.Version={{.VERSION}}
|
|
-X github.com/samcharles93/archie-core/internal/buildinfo.Runtime={{.VERSION}}
|
|
-X github.com/samcharles93/archie-core/internal/app/agentworker.version={{.VERSION}}
|
|
cmds:
|
|
# release.json names what a release ships, so the zip and this build
|
|
# cannot disagree. The compiler flags match `build`, so after the gate
|
|
# has built bin/ this only links.
|
|
- |
|
|
set -eu
|
|
mkdir -p dist
|
|
for cmd in $(jq -r '.required_topology.units[], .required_topology.binaries[]?' release.json); do
|
|
go build -trimpath -ldflags "{{.DIST_LDFLAGS}}" -o "dist/$cmd" "./cmd/$cmd"
|
|
done
|
|
|
|
test:
|
|
desc: Run Go tests; the release gate passes TEST_FLAGS=-count=1
|
|
cmds:
|
|
- go test {{.TEST_FLAGS | default "-short"}} ./...
|
|
|
|
image:agent:
|
|
desc: Build the agent image stamped with this checkout's version
|
|
# Compose owns the image build; this only supplies the version, which a
|
|
# plain `docker build` has no way to learn from a context that excludes .git.
|
|
cmds:
|
|
- RUNTIME_VERSION={{.VERSION}} docker compose build agent
|
|
|
|
check:
|
|
desc: Run the full quality gate
|
|
# Finish mutations before starting parallel checks.
|
|
cmds:
|
|
# Node packages land under ui/, which `go test ./...` walks: installing
|
|
# them while the parallel checks scan the tree makes a lockfile update
|
|
# delete Go sources mid-walk. Install first, then fan out. Every Go step
|
|
# embeds ui/dist, so the dashboard is built before any of them.
|
|
- task: ui
|
|
- task: fmt
|
|
- task: proto:check
|
|
- task: db:check
|
|
- task: check:readonly
|
|
|
|
check:readonly:
|
|
internal: true
|
|
deps:
|
|
- task: proto:lint
|
|
- task: news:check
|
|
- task: vet
|
|
- task: lint
|
|
- task: build
|
|
- task: test
|
|
- task: test:ui
|
|
|
|
release:
|
|
desc: "Turn [Unreleased] into a release, then commit and tag it (RELEASE=major|minor|patch|1.2.3)"
|
|
requires:
|
|
vars: [RELEASE]
|
|
cmds:
|
|
- |
|
|
set -euo pipefail
|
|
die() { echo "release: $*" >&2; exit 1; }
|
|
[ "$(git rev-parse --abbrev-ref HEAD)" = main ] || die "releases are cut from main"
|
|
[ -z "$(git status --porcelain)" ] || die "the working tree is dirty"
|
|
last="$(git tag --list 'v*' --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -1 || true)"
|
|
IFS=. read -r major minor patch <<<"${last#v}"
|
|
case "{{.RELEASE}}" in
|
|
major) version="$((major + 1)).0.0" ;;
|
|
minor) version="$major.$((minor + 1)).0" ;;
|
|
patch) version="$major.$minor.$((patch + 1))" ;;
|
|
*) version="$(echo "{{.RELEASE}}" | sed 's/^v//')" ;;
|
|
esac
|
|
echo "$version" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.]+)?$' || die "$version is not a version"
|
|
! git rev-parse -q --verify "refs/tags/v$version" >/dev/null || die "v$version already exists"
|
|
awk '/^## \[Unreleased\]/ { u = 1; next } u && /^## \[/ { exit } u && /^- / { found = 1 } END { exit !found }' CHANGELOG.md ||
|
|
die "[Unreleased] in CHANGELOG.md has no entries"
|
|
awk -v hdr="## [$version] - $(date -u +%F)" '{ print } /^## \[Unreleased\]/ { print ""; print hdr }' CHANGELOG.md >CHANGELOG.md.tmp
|
|
mv CHANGELOG.md.tmp CHANGELOG.md
|
|
go -C tools run -mod=readonly ./newsgen --repo-root ..
|
|
git add CHANGELOG.md docs/news
|
|
git commit -q -m "chore(release): v$version"
|
|
git tag -a "v$version" -m "v$version"
|
|
echo "tagged v$version"
|