The canary window now counts from when a release is installed on the host (installed.json / installed_at in the update snapshot), not from the statement's issued_at, and 'release canary check' judges the release the canary actually runs via its exact-pin tag instead of the moving next tag. A newer release on next queues behind the one being judged instead of restarting the clock, so stable moves once per release. New subcommands: canary status (read-only state, signals, verdict), canary promote (promote now when hard checks pass), canary unpin (back to next), and start -force. A held/pinned canary is verified installable every check and a stranded pin fails the run red instead of looking green. Rollback pre-flight checks stable's zip and agent image digests before pinning. Relative signals are judged whenever both windows carry minimum data, so a breach rolls back mid-window; promote still needs the full window unless the operator promotes early (new promote.yml workflow_dispatch). The window is configurable via -window / CANARY_WINDOW. Also: dashboard canary status card (ui), and task canary:rehearse, a local end-to-end rehearsal of publish -> canary start -> status -> check -> promote against a throwaway registry and host. task check passes; rehearsal run green (REHEARSE_EXIT=0).
62 lines
2.0 KiB
YAML
62 lines
2.0 KiB
YAML
# Every 15 minutes, checks how the release installed on the canary host is
|
|
# running, judged over the window counted from its install (-window, default
|
|
# 24h; CANARY_WINDOW overrides it). If a hard check fails, the canary goes
|
|
# back to the stable release straight away. If the release stays healthy for
|
|
# the window, it's promoted to stable. A rollback fails the run, so it shows
|
|
# up red.
|
|
name: canary
|
|
|
|
on:
|
|
schedule:
|
|
- cron: "*/15 * * * *"
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
group: canary
|
|
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
|
|
jobs:
|
|
judge:
|
|
if: vars.CANARY_URL != ''
|
|
runs-on: self-hosted
|
|
steps:
|
|
# Full history with tags: a rollback compares the migrations between
|
|
# the stable and failed release tags.
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
with:
|
|
ref: main
|
|
fetch-depth: 0
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v5
|
|
with:
|
|
go-version-file: go.mod
|
|
# The self-hosted runner keeps the Go caches on disk between jobs, and
|
|
# archiving them for the Actions cache takes minutes.
|
|
cache: false
|
|
|
|
- name: Build archied
|
|
# The check never serves the dashboard, so the build leaves it out.
|
|
run: go build -tags no_ui -o "$RUNNER_TEMP/archied" ./cmd/archied
|
|
|
|
# Promotion retags stable and the images' latest in the registry.
|
|
- name: Log in to the Gitea registry
|
|
run: echo "${{ secrets.DEPLOY_TOKEN }}" | docker login git.catlow.cloud -u "${{ github.actor }}" --password-stdin
|
|
|
|
- name: Judge the canary
|
|
env:
|
|
ARCHIE_CANARY_TOKEN: ${{ secrets.CANARY_TOKEN }}
|
|
CANARY_WINDOW: ${{ vars.CANARY_WINDOW }}
|
|
run: |
|
|
window_args=()
|
|
if [ -n "$CANARY_WINDOW" ]; then
|
|
window_args=(-window "$CANARY_WINDOW")
|
|
fi
|
|
"$RUNNER_TEMP/archied" release canary check -dashboard "${{ vars.CANARY_URL }}" \
|
|
"${window_args[@]}" \
|
|
-latest git.catlow.cloud/sam/archied,git.catlow.cloud/sam/archie-agent
|