Compare commits

...
36 Commits
Author SHA1 Message Date
sam 56044476fc chore(release): v1.59.0
Quality / check (push) Successful in 3m53s
deploy / gate (push) Successful in 4m21s
deploy / archied-image (push) Successful in 2m45s
deploy / agent-image (push) Successful in 3m15s
deploy / dist-zip (push) Successful in 2m35s
2026-11-03 06:15:09 +11:00
sam 796bfd0045 docs(changelog): forge extensions gain CreateIssue
Quality / check (push) Successful in 7m12s
deploy / gate (push) Successful in 5m16s
deploy / archied-image (push) Successful in 3m2s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-03 02:43:25 +11:00
sam abcb72a76a feat(curator): dry-run self-intake pass with run inspection tools (#1770)
Curators could only reach the workflow/integration tools; there was no way
to survey task runs. Adds run_list/run_events to the shared registry
(guarded as read on task kind), attaches the instance's acting principal
to curator model calls so guarded tools authorize, and records a pass's
final text as a pass.summary action so the report is inspectable. Ships a
dry-run self-intake definition in dev.toml/config.example.toml declaring
only those two tools, so it cannot file issues yet.
2026-11-03 02:35:06 +11:00
sam 443a7a501b refactor(storecontract): split the store contract into topical files (#1781)
Pure move: one 624-line storecontract.go becomes storecontract.go
(package doc + composed TaskStore), task.go, capture.go, binding.go,
mapping.go, source.go, status.go, resource.go, credentials.go and
errors.go. The package remains the single seam; no declaration changed
(verified line-identical against HEAD). task check green.
2026-11-03 02:27:57 +11:00
sam 88e86776e8 feat(forge): add CreateIssue to the forge contract (#1770)
Quality / check (push) Successful in 4m10s
deploy / gate (push) Successful in 4m43s
deploy / archied-image (push) Successful in 2m54s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
Issue-level writes could comment and close but not open an issue, so no
curator or workflow stage can file its own work yet. Adds the rpc to
proto/forge/v1, regenerates the contracts, and threads it through the
domain interface, forgerpc client/server (comment permission, like
CloseIssue), forgeext, and the archipelago SDK + gitea/github plugins.
2026-11-03 02:04:56 +11:00
sam e10bb161fe docs(prds): mark the self-development ADR as decided, not built (#1769) 2026-11-03 01:52:06 +11:00
sam 3b525b7482 docs(prds): mark the self-development ADR as record, not mechanism (#1769) 2026-11-03 01:50:54 +11:00
sam c2cc10ebfa docs(prds): record the self-development loop decision (#1769) 2026-11-03 01:36:31 +11:00
sam 2f8076d7f8 fix(access): forbid clause scopes the canary grant against its developer role (#1727)
Quality / check (push) Successful in 3m37s
deploy / gate (push) Successful in 4m45s
deploy / archied-image (push) Successful in 2m53s
deploy / agent-image (push) Successful in 10s
deploy / dist-zip (push) Skipped
Service accounts created through the identities API are assigned the
developer role in the request's org, whose shipped policy permits read and
write across the org -- the permit alone did not scope the canary. Install a
matching org-canary-grant-forbid policy (access.CanaryGrantForbidPolicy) that
negates exactly the permit's surface for the canary identity; a forbid wins
within the level, so the role can no longer widen access. Engine tests pin
the surface with the principal carrying the developer role, and the
rehearsal now proves it live: the scoped token gets 403 on /api/access/policies
and 200 on /api/chat/update, and the full canary loop runs on it.

task check passes; task canary:rehearse green (REHEARSE_EXIT=0).
2026-11-02 06:23:38 +11:00
sam c180a2debb feat(access): scoped canary grant replaces instance-admin CANARY_TOKEN (#1727)
Quality / check (push) Successful in 3m49s
deploy / gate (push) Successful in 4m48s
deploy / archied-image (push) Successful in 2m55s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
The canary now authenticates as a dedicated service-account identity in the
system org whose only grant is access.CalaryGrantPolicy: read the signal
endpoints (chat update, alerts, logs, tasks), read and write the
update-settings resource, and trigger an update install. CANARY_TOKEN no
longer carries instance admin; DEPLOY_TOKEN stays CI-only.

- authorize: POST /api/chat/update/install and /defer map to
  update on Object::"update" (dashboard) instead of run on any workflow,
  so the grant can cover installs without granting workflow runs
- accounts: active service-account identities may authenticate with bearer
  tokens (CanAct), not just users
- new POST /api/identities/{id}/tokens (behind manage_identities) mints a
  token for a named non-user identity; new AccountService RPC
  BindIdentityPersonalToken backs it remotely, refusing user-kind targets,
  with the caller recorded as audit actor
- engine tests pin the grant's exact allow/deny surface

RELEASING.md documents the one-time setup. task check passes on this tree.
2026-11-02 06:03:41 +11:00
sam 5356745968 fix(releaseupdate): address review findings on the #1725 canary change
Quality / check (push) Successful in 3m39s
deploy / gate (push) Successful in 4m28s
deploy / archied-image (push) Successful in 3m11s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
- one shared UpdateStateDir() resolver for daemon and dashboard, instead of
  two copies that could drift
- the recovery path for a committed/rolled-back journal writes
  installed.json too, so a crash after the outcome phase cannot lose the
  install record
- JudgeCanaryNow is the explicit signals-only entry point for promoting
  without the window, replacing the zero-install-time sentinel
- canary version comparisons use IssuedAt, matching Promote's replay rule,
  so semver/issued-at divergence cannot wedge check red; queuedBehind
  reuses the judgeable predicate instead of a looser check
- promote prints the channel/pin; promote.yml cancels superseded runs
- canary card: no duplicated elapsed rendering, channel row waits for the
  settings resource instead of showing a default, agent fallback '?'
- rehearsal script bounds its docker build/push/run calls

task check passes on this tree.
2026-11-02 02:45:26 +11:00
sam 5b5059615b feat(releaseupdate): judge the installed release, window from install (#1725)
Quality / check (push) Successful in 4m2s
deploy / gate (push) Successful in 4m31s
deploy / archied-image (push) Successful in 3m14s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
The canary window now counts from when a release is installed on the host
(installed.json / installed_at in the update snapshot), not from the
statement's issued_at, and 'release canary check' judges the release the
canary actually runs via its exact-pin tag instead of the moving next tag.
A newer release on next queues behind the one being judged instead of
restarting the clock, so stable moves once per release.

New subcommands: canary status (read-only state, signals, verdict), canary
promote (promote now when hard checks pass), canary unpin (back to next),
and start -force. A held/pinned canary is verified installable every check
and a stranded pin fails the run red instead of looking green. Rollback
pre-flight checks stable's zip and agent image digests before pinning.
Relative signals are judged whenever both windows carry minimum data, so a
breach rolls back mid-window; promote still needs the full window unless
the operator promotes early (new promote.yml workflow_dispatch). The window
is configurable via -window / CANARY_WINDOW.

Also: dashboard canary status card (ui), and task canary:rehearse, a local
end-to-end rehearsal of publish -> canary start -> status -> check ->
promote against a throwaway registry and host.

task check passes; rehearsal run green (REHEARSE_EXIT=0).
2026-11-02 01:15:29 +11:00
sam bca1f3b412 chore(release): v1.58.1
Quality / check (push) Successful in 3m43s
deploy / gate (push) Successful in 4m36s
deploy / archied-image (push) Successful in 2m52s
deploy / agent-image (push) Successful in 3m18s
deploy / dist-zip (push) Successful in 2m37s
2026-11-02 00:27:14 +11:00
sam 3af96fd818 fix(daemon): watch the task intake loop too and name the stalled loop
Quality / check (push) Canceled after 26s
deploy / gate (push) Successful in 4m30s
deploy / archied-image (push) Successful in 2m50s
deploy / agent-image (push) Successful in 12s
deploy / dist-zip (push) Skipped
2026-11-02 00:26:50 +11:00
sam 1043982529 fix(daemon): stamp each run-loop cycle so the systemd watchdog keeps beating
Quality / check (push) Successful in 3m37s
deploy / gate (push) Successful in 4m27s
deploy / archied-image (push) Successful in 2m54s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-02 00:22:24 +11:00
sam 6abfe32540 chore(release): v1.58.0
Quality / check (push) Successful in 3m39s
deploy / gate (push) Successful in 4m19s
deploy / archied-image (push) Successful in 2m49s
deploy / agent-image (push) Successful in 3m9s
deploy / dist-zip (push) Successful in 4m52s
2026-11-01 23:55:55 +11:00
sam 1f9807a99c fix(ui/chat): give the collapsed composer its own card and reserve its height (#1719)
Quality / check (push) Successful in 3m39s
deploy / gate (push) Successful in 4m33s
deploy / archied-image (push) Successful in 3m16s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-01 23:11:01 +11:00
sam b2edf1fa8b build(ui): reinstall Trace and Collapsible with the shadcn-vue CLI
Quality / check (push) Successful in 3m34s
deploy / gate (push) Successful in 4m32s
deploy / archied-image (push) Successful in 3m26s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-01 23:05:18 +11:00
sam bb897e76b8 Merge pull request 'feat(agentexec): record each agent run's transcript and show it per step (#1718)' (#1724) from claude/intelligent-tereshkova-5422da into main
deploy / gate (push) Successful in 4m37s
Quality / check (push) Successful in 3m41s
deploy / archied-image (push) Successful in 3m13s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
Reviewed-on: #1724
2026-11-01 11:43:32 +00:00
sam 07062352b4 Merge branch 'main' into claude/intelligent-tereshkova-5422da
Quality / check (pull_request) Successful in 3m35s
2026-11-01 11:43:20 +00:00
sam 856c15858b feat(ui): replace the top bar navigation with the sidebar-07 block (#1723)
Quality / check (push) Successful in 3m24s
deploy / gate (push) Successful in 4m32s
deploy / archied-image (push) Successful in 3m18s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-01 22:09:54 +11:00
sam 69a46249cc feat(webui): draw each transcript entry with its Trace pattern (#1718)
Quality / check (pull_request) Successful in 3m38s
Record the model's reasoning parts. The step run view shows reasoning as a
Thought row, shell calls as a terminal, the gate as a test run, web_fetch as a
source, finish as the verdict, and each fan-out branch as a subagent.
2026-11-01 22:09:53 +11:00
sam 6f10ecc04f feat(agentexec): record each agent run's transcript and show it per step (#1718)
The loop runner hooks every model call and tool call and reports ordered,
redacted, clipped entries. Each is persisted as an agent_transcript task
event through InsertEvent, left out of the timeline read, and served per
attempt at /api/tasks/{id}/transcript. The step run view shows it with Trace.
2026-11-01 22:09:53 +11:00
sam d6ca6a72a5 build(ui): build the dashboard in CI and the Taskfile instead of committing ui/dist
deploy / gate (push) Successful in 4m28s
Quality / check (push) Canceled after 1m27s
deploy / archied-image (push) Successful in 3m29s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-01 22:07:36 +11:00
sam 243832fee6 docs(ui): add components with the shadcn-vue CLI, not by hand
Quality / check (push) Successful in 3m7s
deploy / gate (push) Successful in 3m51s
deploy / archied-image (push) Successful in 2m47s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-01 21:47:54 +11:00
sam e8c2c844a8 ci: skip setup-go caching on the self-hosted runner (#1716)
Quality / check (push) Successful in 3m1s
deploy / gate (push) Successful in 4m6s
deploy / archied-image (push) Successful in 2m54s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-01 21:21:09 +11:00
sam 6220387ed6 fix(install): restart running services on reinstall (#1716)
Quality / check (push) Successful in 3m32s
deploy / gate (push) Successful in 7m18s
deploy / archied-image (push) Successful in 3m4s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-01 17:59:05 +11:00
sam efd3eb2854 fix(releaseupdate): wait for the canary to offer a release before installing (#1716)
Quality / check (push) Successful in 3m14s
deploy / gate (push) Successful in 6m48s
deploy / archied-image (push) Successful in 3m3s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-11-01 02:55:31 +11:00
sam c78063eba0 chore(release): v1.57.0
Quality / check (push) Successful in 3m20s
deploy / gate (push) Successful in 6m29s
deploy / archied-image (push) Successful in 2m51s
deploy / agent-image (push) Successful in 3m25s
deploy / dist-zip (push) Failing after 1m56s
2026-11-01 02:26:01 +11:00
sam 0e931485a0 docs(releaseupdate): use contractions in the release and canary text (#1716)
Quality / check (push) Successful in 7m33s
deploy / gate (push) Successful in 6m27s
deploy / archied-image (push) Successful in 2m57s
deploy / agent-image (push) Successful in 12s
deploy / dist-zip (push) Skipped
2026-10-31 22:18:57 +11:00
sam 1bf536d4ab docs(releaseupdate): rewrite the canary comments in plain English (#1716)
deploy / gate (push) Successful in 6m20s
Quality / check (push) Canceled after 8m7s
deploy / archied-image (push) Successful in 2m57s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-10-31 22:11:04 +11:00
sam 2181b5fd08 ci(deploy): move the images' :latest only when a release is promoted (#1716)
Quality / check (push) Successful in 3m47s
deploy / gate (push) Successful in 6m29s
deploy / archied-image (push) Successful in 2m59s
deploy / agent-image (push) Successful in 10s
deploy / dist-zip (push) Skipped
2026-10-31 21:48:10 +11:00
sam a4a7c96da6 feat(releaseupdate): publish to next and promote to stable only after a canary evaluation (#1716)
Quality / check (push) Successful in 3m49s
deploy / gate (push) Failing after 16m14s
deploy / archied-image (push) Skipped
deploy / agent-image (push) Skipped
deploy / dist-zip (push) Skipped
2026-10-31 21:39:32 +11:00
sam a7f6e517ad ci(release): cut releases from [Unreleased] by workflow dispatch (#1716)
deploy / gate (push) Successful in 7m17s
Quality / check (push) Canceled after 9m13s
deploy / archied-image (push) Successful in 2m56s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-10-31 21:30:33 +11:00
sam 812d9004a7 feat(webui): keep the Model setup step open until the provider credential resolves (#1045)
Quality / check (push) Successful in 4m40s
deploy / gate (push) Successful in 6m15s
deploy / archied-image (push) Successful in 2m57s
deploy / agent-image (push) Successful in 11s
deploy / dist-zip (push) Skipped
2026-10-31 20:04:06 +11:00
sam ebc6dd67e6 fix(setup): offer the bundled database as an explicit choice on a re-run (#1388)
With an existing database_url, setup prefilled the URL, so a blank answer
kept it and the bundled socket database was unreachable without typing its
URL. The re-run database step now presents a choice menu: keep the existing
URL (still the default), switch to the bundled database (socket URL plus a
generated PGPASSWORD written to the env file), or enter a different URL.
Fresh installs and -database-url are unchanged; a blank answer anywhere
still keeps the existing URL. Pinned at the setup API level with fake
prompter/sink: TestRunParamsDatabaseChoice.
2026-10-31 20:03:33 +11:00
238 changed files with 8082 additions and 2397 deletions

No files matched your search

+61
View File
@@ -0,0 +1,61 @@
# Every 15 minutes, checks how the release installed on the canary host is
# running, judged over the window counted from its install (-window, default
# 24h; CANARY_WINDOW overrides it). If a hard check fails, the canary goes
# back to the stable release straight away. If the release stays healthy for
# the window, it's promoted to stable. A rollback fails the run, so it shows
# up red.
name: canary
on:
schedule:
- cron: "*/15 * * * *"
workflow_dispatch:
concurrency:
group: canary
permissions:
contents: read
packages: write
jobs:
judge:
if: vars.CANARY_URL != ''
runs-on: self-hosted
steps:
# Full history with tags: a rollback compares the migrations between
# the stable and failed release tags.
- name: Checkout
uses: actions/checkout@v4
with:
ref: main
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
# The self-hosted runner keeps the Go caches on disk between jobs, and
# archiving them for the Actions cache takes minutes.
cache: false
- name: Build archied
# The check never serves the dashboard, so the build leaves it out.
run: go build -tags no_ui -o "$RUNNER_TEMP/archied" ./cmd/archied
# Promotion retags stable and the images' latest in the registry.
- name: Log in to the Gitea registry
run: echo "${{ secrets.DEPLOY_TOKEN }}" | docker login git.catlow.cloud -u "${{ github.actor }}" --password-stdin
- name: Judge the canary
env:
ARCHIE_CANARY_TOKEN: ${{ secrets.CANARY_TOKEN }}
CANARY_WINDOW: ${{ vars.CANARY_WINDOW }}
run: |
window_args=()
if [ -n "$CANARY_WINDOW" ]; then
window_args=(-window "$CANARY_WINDOW")
fi
"$RUNNER_TEMP/archied" release canary check -dashboard "${{ vars.CANARY_URL }}" \
"${window_args[@]}" \
-latest git.catlow.cloud/sam/archied,git.catlow.cloud/sam/archie-agent
+47 -20
View File
@@ -59,6 +59,9 @@ jobs:
uses: actions/setup-go@v5
with:
go-version-file: go.mod
# The self-hosted runner keeps the Go caches on disk between jobs, and
# archiving them for the Actions cache takes minutes.
cache: false
- uses: pnpm/action-setup@v4
with:
@@ -96,13 +99,8 @@ jobs:
- name: Pull test container images
run: docker pull alpine:3
# One definition of the gate, in the Taskfile. This step used to inline
# its own list, which drifted weaker than `task check`: it ran no
# db:check, news:check or ui:check, so a release could ship stale
# generated SQL, a stale ui/dist, or release notes that disagree with
# CHANGELOG.md. It also ran the dashboard suite here rather than in the
# gate, where test:ui runs it with the type check. TEST_FLAGS is the one
# difference -- the release runs the whole suite, uncached and including
# One definition of the gate, in the Taskfile. TEST_FLAGS is the one
# difference: the release runs the whole suite, uncached and including
# the Docker-backed tests -short skips. The diff after it asserts
# `task fmt` changed nothing.
- name: Quality gate
@@ -134,12 +132,11 @@ jobs:
if [ $fail -ne 0 ]; then
echo
echo "A release tag points at this commit but its changelog does not"
echo "describe that version. Run: task release:prepare VERSION=<ver>"
echo "edit the generated notes, then: task release VERSION=<ver>"
echo "describe that version. Cut releases with the release workflow."
exit 1
fi
if ! git tag --points-at HEAD | grep -qE '^v[0-9]' ; then
echo "This commit is not a release, so the images are published as :edge and sha-<sha>; :latest keeps naming the newest release."
echo "This commit isn't a release, so the images are published as :edge and sha-<sha>; :latest keeps naming the newest promoted release."
fi
archied-image:
@@ -186,19 +183,18 @@ jobs:
echo "release_tag=$RELEASE_TAG" >> "$GITHUB_OUTPUT"
echo "gateway=$GATEWAY_VERSION" >> "$GITHUB_OUTPUT"
echo "runtime=$RUNTIME_VERSION" >> "$GITHUB_OUTPUT"
# :latest names the newest RELEASE, so only a commit that IS one
# publishes it. A push to main publishes :edge (mutable, newest main)
# and sha-<sha> (immutable) instead. That is what stops a release's
# :latest stamping from being overwritten minutes later by the next
# ordinary commit, which left a host reporting dev and made the update
# path unable to tell current from stale (archie-core-i5nb).
# :latest names the newest PROMOTED release, so a release publishes
# :v<version> and canary promotion moves :latest to it. A push to
# main publishes :edge (mutable, newest main) and sha-<sha>
# (immutable), so an ordinary commit never overwrites a release's
# stamping.
#
# `if` blocks rather than `[ -n "$X" ] && echo`: the latter returns
# non-zero when the test fails, and these steps run under `set -e`.
{
echo "archied_tags<<EOF"
if [ -n "$RELEASE_TAG" ]; then
echo "git.catlow.cloud/sam/archied:latest"
echo "git.catlow.cloud/sam/archied:$RELEASE_TAG"
fi
echo "git.catlow.cloud/sam/archied:edge"
echo "git.catlow.cloud/sam/archied:sha-${{ steps.commit.outputs.sha }}"
@@ -261,7 +257,7 @@ jobs:
file: Dockerfile
push: true
tags: |
git.catlow.cloud/sam/archie-agent:latest
git.catlow.cloud/sam/archie-agent:${{ steps.versions.outputs.release_tag }}
git.catlow.cloud/sam/archie-agent:sha-${{ steps.commit.outputs.sha }}
build-args: |
RUNTIME_VERSION=${{ steps.versions.outputs.runtime }}
@@ -294,6 +290,25 @@ jobs:
uses: actions/setup-go@v5
with:
go-version-file: go.mod
# The self-hosted runner keeps the Go caches on disk between jobs, and
# archiving them for the Actions cache takes minutes.
cache: false
- uses: pnpm/action-setup@v4
with:
package_json_file: ui/package.json
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "24"
# archied and archie-ui embed the dashboard.
- name: Build the dashboard
working-directory: ui
run: |
pnpm install --frozen-lockfile
pnpm run build
- name: Resolve archied version
id: version
@@ -354,8 +369,8 @@ jobs:
# Installs and updates take a release from the signed artifact in the
# registry, so this is what publishes it: the statement signed with
# RELEASE_SIGNING_KEY and the zip, tagged v<version>, next, and stable
# unless it is a prerelease.
# RELEASE_SIGNING_KEY and the zip, tagged v<version> and next. stable
# moves only when canary.yml promotes it.
- name: Log in to the Gitea registry
run: echo "${{ secrets.DEPLOY_TOKEN }}" | docker login git.catlow.cloud -u "${{ github.actor }}" --password-stdin
@@ -403,6 +418,18 @@ jobs:
"$api/repos/$repo/releases" >/dev/null
echo "published release notes for $tag"
# The canary host follows the `next` channel. canary.yml then judges the
# release installed on the canary and decides whether it is promoted to
# stable or the canary is rolled back. The start is queue-safe: a release
# published while the canary still judges a newer one is queued behind
# it and picked up by the next check, so the running window isn't
# restarted under a different release.
- name: Install the release on the canary
if: vars.CANARY_URL != ''
env:
ARCHIE_CANARY_TOKEN: ${{ secrets.CANARY_TOKEN }}
run: ./dist/archied release canary start -dashboard "${{ vars.CANARY_URL }}" -version "${{ steps.version.outputs.version }}"
# A Gitea package belongs to its owner, not a repository, so the ones this
# workflow publishes are linked here. Linking is per package and a no-op
# once done, so an already linked package is not an error.
+55
View File
@@ -0,0 +1,55 @@
# Promotes the release the canary is running right now, without waiting out
# the judging window: the canary's signals must show no hard failure and no
# relative breach, otherwise the run refuses and fails. Dispatch it with the
# version the canary runs (`archied release canary status` shows it).
name: promote
on:
workflow_dispatch:
inputs:
version:
description: "release the canary is expected to run; must match what it runs"
required: true
type: string
concurrency:
group: promote
cancel-in-progress: true
permissions:
contents: read
packages: write
jobs:
promote:
if: vars.CANARY_URL != ''
runs-on: self-hosted
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: main
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
# The self-hosted runner keeps the Go caches on disk between jobs, and
# archiving them for the Actions cache takes minutes.
cache: false
- name: Build archied
# The promote never serves the dashboard, so the build leaves it out.
run: go build -tags no_ui -o "$RUNNER_TEMP/archied" ./cmd/archied
# Promotion retags stable and the images' latest in the registry.
- name: Log in to the Gitea registry
run: echo "${{ secrets.DEPLOY_TOKEN }}" | docker login git.catlow.cloud -u "${{ github.actor }}" --password-stdin
- name: Promote the canary's release
env:
ARCHIE_CANARY_TOKEN: ${{ secrets.CANARY_TOKEN }}
run: |
"$RUNNER_TEMP/archied" release canary promote -dashboard "${{ vars.CANARY_URL }}" \
-version "${{ inputs.version }}" \
-latest git.catlow.cloud/sam/archied,git.catlow.cloud/sam/archie-agent
+54
View File
@@ -0,0 +1,54 @@
# Cuts a release from main: [Unreleased] in CHANGELOG.md becomes the version's
# section, then the commit and its tag are pushed together. The tag push starts
# deploy.yml, which publishes the release to `next` for the canary.
name: release
on:
workflow_dispatch:
inputs:
release:
description: "major, minor, patch, or an exact version such as 1.57.0"
required: true
default: minor
concurrency:
group: release
permissions:
contents: write
jobs:
cut:
runs-on: self-hosted
steps:
# DEPLOY_TOKEN rather than the job token: a push made with the job token
# doesn't start workflows, and the tag push must start deploy.yml.
- name: Checkout
uses: actions/checkout@v4
with:
ref: main
fetch-depth: 0
token: ${{ secrets.DEPLOY_TOKEN }}
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
# The self-hosted runner keeps the Go caches on disk between jobs, and
# archiving them for the Actions cache takes minutes.
cache: false
- name: Install Task
run: go install github.com/go-task/task/v3/cmd/task@v3.48.0
- name: Cut the release
env:
RELEASE: ${{ inputs.release }}
run: |
git config user.name "Sam Catlow"
git config user.email "sam@catlow.tech"
task release RELEASE="$RELEASE"
# Atomic: a commit without its tag would build images stamped dev.
- name: Push the release
run: git push --atomic origin main "$(git describe --exact-match --match 'v*' HEAD)"
+1 -1
View File
@@ -25,7 +25,7 @@ config.production.toml
/ui/node_modules/
/ui/.vite/
/ui/*.tsbuildinfo
# ui/dist IS committed: the Go build must not need a Node toolchain.
/ui/dist/
# air live-reload
/.air/
+25
View File
@@ -5,6 +5,31 @@ release's per-component sections are labelled beneath its heading.
## [Unreleased]
## [1.59.0] - 2026-11-02
### archied
- Forge extensions can open issues: the forge surface gains `CreateIssue` alongside list, comment and close, and the bundled Gitea and GitHub extensions implement it.
## [1.58.1] - 2026-11-01
### archied
- The systemd watchdog now also watches the task intake loop, and its warning names the loop that stalled.
- archied no longer restarts about a minute after it starts. Under systemd, the watchdog never saw a poll pass and killed it on every start since 1.56.0.
## [1.58.0] - 2026-11-01
### archied
- Running `install.sh` over an install whose services are already running now restarts them, so they run the new binaries.
## [1.57.0] - 2026-10-31
### archied
- A release reaches `stable` only after it has run on a canary for a day and passed: `archied release publish` now publishes to `next` alone, `archied release canary start|check` installs a release on a canary and judges it from the canary's dashboard, promoting it or rolling the canary back, and `archied release promote` moves `stable` by hand. Promotion refuses a prerelease and a release older than the one `stable` names. The `archied` and `archie-agent` images are published as `:v<version>`, and `:latest` moves only when a release is promoted.
## [1.56.0] - 2026-10-31
### archied
+1 -1
View File
@@ -105,7 +105,7 @@ When sources disagree, the higher one wins:
- If a fix fails twice, throw the block away and rewrite it simply.
- Comments say why, briefly. No migration history, phase names or issue
narration in code.
- Generated files (`ui/dist/`, protobuf, sqlc) are regenerated and committed
- Generated files (protobuf, sqlc) are regenerated and committed
with the change that caused them, never hand-edited.
## Tests
+9
View File
@@ -1,11 +1,20 @@
# archied — the archie daemon
# Run via docker compose alongside NATS.
FROM node:24-trixie-slim AS dashboard
WORKDIR /src/ui
RUN corepack enable
COPY ui/package.json ui/pnpm-lock.yaml ui/pnpm-workspace.yaml ./
RUN pnpm install --frozen-lockfile
COPY ui/ ./
RUN pnpm run build
FROM golang:1.27-trixie AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
COPY --from=dashboard /src/ui/dist ./ui/dist
ARG GATEWAY_VERSION=dev
ARG RUNTIME_VERSION=dev
RUN CGO_ENABLED=0 go build \
+215 -113
View File
@@ -1,141 +1,243 @@
# Releasing
One release stream. `CHANGELOG.md` carries every release section, one generated
page per version describes it, and one tag names the release. A release is one
version covering whatever changed; the components' entries are labelled sections
inside it (`### archied`, `### archie-agent`), not separate numbers or separate
pages.
page per version describes it, and one `v*` tag names the release. One version
covers every process in the host bundle (`archied`, `archie-gateway`,
`archie-state-store`, `archie-ui`, `archie-messaging`) and the `archie-agent`
image; per-component detail is a labelled section inside the version
(`### archied`, `### archie-agent`).
The host bundle ships five processes (`archied`, `archie-gateway`,
`archie-state-store`, `archie-ui`, `archie-messaging`), and
one version covers all of them. `archie-agent`, the per-task sandboxed runtime,
ships as a container image and its changes are the `## [version]` release's
`### archie-agent` section.
A release is cut by CI, published to `next`, run on the canary for a day, and
promoted to `stable` only if the canary's signals pass. Nothing is run locally.
`tools/release.sh` decides which commits a release carries from the package
closure of **every shipped binary**, not from commit-message guessing: a change
to a package only the messaging service, state store, playbook runner or UI
links still lands in the changelog. Read its header comment for how it
partitions commits and enforces `main`-only, clean-tree releases.
## Notes are written with the change
## What justifies a version
`task release:preview VERSION=X.Y.Z` shows exactly what would land, before
anything is written. A commit set that justifies no version produces no version:
`release.sh` exits **3** with "nothing to release" rather than inventing one, so
a pipeline can finish green without tagging.
Most sessions touch the gateway side (daemon, gateway, web UI, chat, messaging)
and leave the agent runtime alone. That is no longer a per-component `skip`
decision — one version covers the release — but a runtime code change being
merged to `main` is still not the same as it being in production: the
`archie-agent` image is rebuilt by the release, and a host that runs the agent
must pull the new image (`docker compose pull agent`) to get it.
## What the notes are for
A commit that changes what a reader experiences adds its bullet under
`## [Unreleased]` in `CHANGELOG.md`, in the same commit. The release takes that
section as it stands, so write the bullet as the release note, not as a commit
subject.
**Release notes describe the reader's experience of what shipped.** The test is
the reader, not whether a sentence is negative:
- A capability that **never existed** leaves no expectation to correct, so a
note about it reads as a feature to someone who never had the thing — out.
- **The implementation's journey** — what was rebuilt, what a guard finally
covers, what a browser API refuses to surface — is not the reader's experience
either — out.
- A capability that **shipped but is not yet wired** _is_ the reader's
note about it reads as a feature to someone who never had the thing: out.
- **The implementation's journey** (what was rebuilt, what a guard finally
covers, what a browser API refuses to surface) isn't the reader's experience
either: out.
- A capability that **shipped but isn't yet wired** _is_ the reader's
experience, so it belongs, worded as a caveat on the capability and paired
with the behaviour that compensates for it.
Design constraints and withheld capabilities are not release news.
Do not apply this mechanically: deleting a caveat a reader needs misleads them,
which is the failure this rule exists to prevent.
Design constraints and withheld capabilities are not release news. Deleting a
caveat a reader needs misleads them, which is the failure this rule exists to
prevent.
## Cutting a release
```bash
task release:preview VERSION=1.3.0 # preview what would land
task release:prepare VERSION=1.3.0 # write the section into CHANGELOG.md, uncommitted
# edit CHANGELOG.md -- generated notes are a starting point, not the release
task release VERSION=1.3.0 # commit + annotated tag
git push --atomic origin main v1.3.0 # both refs or neither
```
Dispatch the **release** workflow (`.github/workflows/release.yml`) with
`major`, `minor`, `patch` or an exact version. It runs `task release` on `main`:
Pushing the tag is the release. The deploy workflow builds the images and the
zip, then `archied release publish` signs the release with the
`RELEASE_SIGNING_KEY` Actions secret and pushes it to
`git.catlow.cloud/sam/archie` as `v1.3.0`, `next`, and `stable` unless the
version is a prerelease. `install.sh` and the updater install only what that
key signed.
1. refuses a dirty tree, an existing tag, or an `[Unreleased]` with no entries;
2. turns `[Unreleased]` into `## [x.y.z] - date` under a new empty
`[Unreleased]`, and runs `task news`;
3. commits `chore(release): vx.y.z` and tags `vx.y.z`;
`release:prepare` emits bare commit subjects; they are a starting point. Rewrite
them to say what a reader gets, per "What the notes are for" above.
then pushes `main` and the tag atomically with `DEPLOY_TOKEN`. A push made with
the job token wouldn't start other workflows, so the user token is what lets
the tag start the deploy.
Pushing is a separate, explicit step — never push without confirming first (see
the repo's general "check before doing anything hard to reverse" rule). The
deploy workflow reads tags pointing at `HEAD`; pushing the commit without its
tag gets images stamped `dev`, which is why the push names the tag and is
`--atomic`.
### After a release, `main` and the working branch have diverged
The release commit lands on `main` — `tools/release.sh` refuses to run anywhere
else — while the working branch keeps advancing. So after every release, `main`
is ahead by exactly that commit and the branch is ahead by however far it has
moved: **neither tip is an ancestor of the other**, and `--ff-only` fails in
both directions. That refusal reads as "the tree is not ready" when in fact the
tree is fine.
Merge the release commit into the branch first, so any conflict is resolved on
the branch, where it is reversible and nobody is reading the tip:
```bash
git merge main # on the branch
git merge --ff-only <branch> # then, in whichever worktree holds main
```
Confirm which worktree holds `main` with `git worktree list` before the
`--ff-only`. A detached worktree sitting at the same commit will accept it and
move nothing, which looks like success.
After a release, pull `main` before committing on it again: CI has added the
release commit.
## What a release publishes
Tagging one version produces, from a single `deploy` run:
The tag push runs `deploy.yml`:
- **Container images** — `git.catlow.cloud/sam/archied:latest` and, on a
release tag, `git.catlow.cloud/sam/archie-agent:latest`. The runtime image is
rebuilt by the release, so runtime commits merged to `main` do not reach a
running sandbox until the host pulls the new image. The hand-run
`tools/publish.sh` publishes the same two images to
`git.catlow.cloud/sam` instead, which is the registry the binary's default
`[containers].image` names.
- **`:latest` names the newest release, not the newest commit.** A push to
`main` that is not a release publishes `archied:edge` (mutable, tracking
`main`) and `archied:sha-<commit>` (immutable) instead, so a release's
`:latest` stamping is not overwritten minutes later by the next ordinary
commit. A host that runs releases follows `:latest`; a host that tracks `main`
follows `:edge`.
- **A GitHub Release with a distribution zip** — `dist-zip` builds every process
the reference deployment runs (`archied`, `archie-gateway`,
`archie-state-store`, `archie-ui`, `archie-messaging`) for
linux/amd64, packs them with `deployments/INSTRUCTIONS.md` and `CHANGELOG.md`,
and attaches the zip to the Release. The Release body is that version's
`## [version]` section of `CHANGELOG.md`.
- **Gate.** `task check TEST_FLAGS=-count=1` (uncached, including the
Docker-backed tests `-short` skips) and `govulncheck`, then a check that the
tag has a `[version]` section in `CHANGELOG.md`.
- **Images.** `archied:v<version>` and `archie-agent:v<version>` on a release
tag; `archied:edge` and `archied:sha-<commit>` on any other push to `main`.
`:latest` moves only on promotion, so it names the newest promoted release.
- **The signed release.** `archied release publish` signs a statement
(version, issued_at, zip digest, agent image by digest) with
`RELEASE_SIGNING_KEY` and pushes it with the distribution zip to
`git.catlow.cloud/sam/archie` as `v<version>` and `next`. It never moves
`stable`.
- **The release-notes page** on Gitea, from that version's `CHANGELOG.md`
section, which the statement links to.
- **The canary install**: `archied release canary start` points the canary at
`next` and installs the release through its dashboard.
The zip is also uploaded as an Actions artifact, but that is a backup copy:
artifacts expire after 90 days and need an Actions login to download. The
Release asset is the durable, operator-reachable one.
GitHub-style Actions run the workflow from the tagged commit, so a fix to
`deploy.yml` reaches a release only if it is an ancestor of the tag. Re-cut the
release rather than committing the fix on top of a tagged one.
## Gate
## The canary
CI (`deploy.yml`) runs `task check TEST_FLAGS=-count=1` before building images,
then verifies any release tag at `HEAD` has a matching `[version]` section in
`CHANGELOG.md`. The gate has one definition, in the Taskfile: the release runs
it with the uncached suite, including the Docker-backed tests `-short` skips.
A tag with no changelog entry is a hard failure. No tags at `HEAD` is a
warning, not a failure -- images get stamped `dev`.
The canary is the production host, following `next`. Every quarter hour
`canary.yml` runs `archied release canary check`, which reads the canary
through its dashboard API and judges **the release the canary actually runs**,
not the moving `next` tag:
One consequence worth knowing when a release commit needs amending: GitHub
Actions runs the workflow **from the commit that triggered it**, so a fix to
`deploy.yml` only takes effect for a release if it is an ancestor of the tagged
commit. Delete and re-cut the tag (`git tag -d`, `task release`) rather than
committing the fix on top of an already-tagged release.
| Signal | Source | Fails when |
| --- | --- | --- |
| Install outcome | `GET /api/chat/update` | the canary doesn't run the judged release (the watchdog rolled it back) |
| Readiness | `GET /health/detailed` | the rollup isn't `ok` |
| Alerts | `GET /api/alerts` | an open `action` alert was first raised after the release was installed |
| Error log | `GET /api/logs?level=error` | error lines per window exceed `2 × baseline + 10` |
| Task outcomes | `GET /api/tasks` | the share of finished tasks that parked or died rose more than 10 points |
- **Control.** The canary's own window before the release was installed. With
one production host there's no simultaneous control.
- **Window.** 24 hours counted from the **install** on the canary, not from
the release's issue time: the host records it in `installed.json`
(`committed_at`) and reports it as `installed_at` in `GET /api/chat/update`.
One cycle of schedules and traffic. The first 30 minutes are for the install
and restart. `canary.yml` passes `-window` when the `CANARY_WINDOW` Actions
variable is set; the flag defaults to 24h. A host that doesn't report an
install time degrades to the release statement's issue time.
- **Hard signals** (install, readiness, alerts) roll back at any check after
that. **Relative signals** (errors, tasks) are judged against the baseline
whenever both windows carry enough data, so a breach rolls the release back
even mid-window; a full window with no failure promotes it.
- **Too little data skips a signal** rather than failing it: fewer than 5
finished tasks in the window, or an error log that reaches back less than an
hour into the baseline. A log read that stops short of a window is scaled to
a full window from what it covers.
### Queueing
`deploy.yml` installs each new release on the canary with
`archied release canary start -version V`, which first points the canary back
at `next` (clearing any rollback pin) and then installs `V` — unless the
canary is already judging a release newer than stable that isn't `V`, in which
case `V` is **queued** behind it and the canary keeps running what it judges.
`-force` installs regardless. A `check` that promotes (or finds nothing to
judge) installs the newer `next` in the same run, so a queue drains one
release per promotion.
### Subcommands
```
archied release canary status read-only: channel/pin, installed release and
installed_at, stable/next, window progress, and
the verdict the current signals would give
archied release canary start point the canary at next and install -version,
queued behind a release being judged (-force to
override)
archied release canary check judge the installed release over -window
(default 24h): promote, pick up a newer next,
or roll back; a held canary is reported, and a
held pin that is no longer installable fails
the run red
archied release canary promote promote the installed release now if its
signals pass; refuses with exit 1 otherwise
archied release canary unpin return the canary to next from any pin
```
### Promote
A release that passes the window is promoted: `stable` is retagged to the
`v<version>` manifest. The signature covers the statement, not the tag, so the
retag needs no signing key. The `archied` and `archie-agent` images' `:latest`
move to their `:v<version>` first, and the agent image must be the digest the
statement pins. Promotion refuses a prerelease and a release issued before the
one `stable` names, the same replay rule hosts enforce. The decision and its
reasons are in the `canary` run log.
Early promotion is a dispatch away: the **promote** workflow
(`.github/workflows/promote.yml`) takes a `version` input and runs
`archied release canary promote -version V` on the canary. It judges the
release the canary runs — the window doesn't matter for the decision — and
refuses (red run) on any hard failure or relative breach with enough data.
### Roll back
`stable` stays put, and the run turns red. Before the canary is pinned to
`stable`, `check` verifies `stable` is actually installable — its statement's
zip blob must still be in the release repository and its pinned agent image
must still resolve (`releaseupdate.Installable`). If it isn't, the run fails
loudly and the canary stays on `next` rather than being stranded on a pin.
Otherwise the canary is pinned to the `stable` version through its update
settings and reinstalled through its dashboard.
A release whose migrations differ from `stable`'s can't go back without
restoring the pre-update database snapshot, which loses every write since. The
canary is held on the failed release instead (pinned to it) and the fix goes
forward in the next release. The next release's deploy points the canary back
at `next`.
A pinned canary isn't judged, so a rollback is decided once — but `check`
verifies the pin is still installable every run, so a stranded pin shows red
instead of green. `archied release canary unpin` returns the canary to `next`
by hand.
### Configuration
| Name | Kind | Holds |
| --- | --- | --- |
| `CANARY_URL` | Actions variable | the canary's dashboard URL; unset disables the canary steps |
| `CANARY_TOKEN` | Actions secret | an API token bound to a dedicated `service_account` identity on the canary, carrying only the canary grant policy (see below) — never instance admin |
| `CANARY_WINDOW` | Actions variable | optional `-window` override for `canary check`; unset is 24h |
| `DEPLOY_TOKEN` | Actions secret | registry and repository write, for publishing, promoting and the release push; stays CI-only, promotion runs in CI and never on the host |
| `RELEASE_SIGNING_KEY` | Actions secret | the release signing seed |
#### Setting up `CANARY_TOKEN` once
`CANARY_TOKEN` is a bearer token for a dedicated service-account identity in
the system org whose only grant is the canary policy
(`access.CanaryGrantPolicy`): read the signal endpoints (`GET /api/chat/update`,
`GET /api/alerts`, `GET /api/logs`, `GET /api/tasks`), read and write the
`update-settings` resource, and trigger an update install
(`POST /api/chat/update/install`). It carries no other rights, and instance
admin is never needed. With an instance-admin token (for example the operator
session), run against the canary host:
```bash
# 1. Create the service-account identity in the system org.
curl -X POST "$CANARY_URL/api/identities" -H "Authorization: Bearer $ADMIN" \
-H 'Content-Type: application/json' \
-d '{"kind":"service_account","display_name":"ci-canary"}'
# -> {"id":"<canary-identity-id>", ...}
# 2. Issue its token (shown exactly once).
curl -X POST "$CANARY_URL/api/identities/<canary-identity-id>/tokens" \
-H "Authorization: Bearer $ADMIN" -H 'Content-Type: application/json' \
-d '{"name":"ci-canary","days":90}'
# -> {"id":"<token-hash>","token":"archie_pat_..."} -- store the token value as the CANARY_TOKEN secret
# 3. Install the grant pair, scoped to that identity. The policies' org is
# pinned to the caller's org (org-sys for the instance admin), so the
# bodies need no org field. The permit alone is not enough: service
# accounts created through the API are assigned the developer role, so a
# matching forbid (installed second) negates everything outside the
# permit's surface for that identity.
curl -X PUT "$CANARY_URL/api/access/policies" -H "Authorization: Bearer $ADMIN" \
-H 'Content-Type: application/json' \
-d '{"id":"org-canary-grant","level":"org","text":"permit(principal, action, resource) when {\n resource.org == principal.org &&\n principal == Archie::Identity::\"<canary-identity-id>\" &&\n (\n (action == Archie::Action::\"read\" && [\"dashboard\", \"task\"].contains(resource.kind)) ||\n (action == Archie::Action::\"read\" && resource == Archie::Object::\"update-settings\") ||\n (action == Archie::Action::\"update\" && resource == Archie::Object::\"update\") ||\n (action == Archie::Action::\"manage_identities\" && resource == Archie::Object::\"update-settings\")\n )\n};"}'
```
The second policy forbids the canary identity from everything the permit does
not name, so its developer role cannot widen the grant:
```bash
curl -X PUT "$CANARY_URL/api/access/policies" -H "Authorization: Bearer $ADMIN" \
-H 'Content-Type: application/json' \
-d '{"id":"org-canary-grant-forbid","level":"org","text":"forbid(principal, action, resource) when {\\n resource.org == principal.org &&\\n principal == Archie::Identity::\\\"<canary-identity-id>\\\" &&\\n !(\\n (action == Archie::Action::\\\"read\\\" && [\\\"dashboard\\\", \\\"task\\\"].contains(resource.kind)) ||\\n (action == Archie::Action::\\\"read\\\" && resource == Archie::Object::\\\"update-settings\\\") ||\\n (action == Archie::Action::\\\"update\\\" && resource == Archie::Object::\\\"update\\\") ||\\n (action == Archie::Action::\\\"manage_identities\\\" && resource == Archie::Object::\\\"update-settings\\\")\\n )\\n};"}'
```
The policy texts are exactly `access.CanaryGrantPolicy` and
`access.CanaryGrantForbidPolicy` from `internal/domain/access/canary.go` with
the identity id filled in. The canary needs no other rights: `GET
/health/detailed` is unauthenticated, and every other call it makes is covered
by the grant pair.
The runner must reach `CANARY_URL`. Without a canary a release stays on `next`
until promoted by hand, under the same refusals:
```bash
archied release promote -version 1.57.0 \
-latest git.catlow.cloud/sam/archied,git.catlow.cloud/sam/archie-agent
```
+41 -30
View File
@@ -100,18 +100,12 @@ tasks:
- pnpm install --frozen-lockfile
ui:
desc: Build the dashboard into ui/dist (committed; the Go build embeds it)
desc: Build the dashboard into ui/dist, which the Go build embeds
dir: ui
deps: [ui:deps]
cmds:
- pnpm run build
ui:check:
desc: Rebuild the dashboard into a temporary directory and reject a stale ui/dist
deps: [ui:deps]
cmds:
- bash tools/uidist.sh check
test:ui:
desc: Test and type-check the dashboard
dir: ui
@@ -227,8 +221,16 @@ tasks:
cmds:
- node e2e/run.cjs {{.CLI_ARGS}}
canary:rehearse:
# Local rehearsal only: spins up its own registry, database and host, and
# is deliberately absent from check and CI.
desc: Rehearse the release-canary loop on a throwaway local host (needs docker; never part of the gate)
cmds:
- sh tools/canary-rehearse.sh
build:
desc: Build archie services and agent binaries
deps: [ui]
vars:
GATEWAY_VERSION: '{{.GATEWAY_VERSION | default .VERSION}}'
RUNTIME_VERSION: '{{.RUNTIME_VERSION | default .VERSION}}'
@@ -278,13 +280,14 @@ tasks:
desc: Run the full quality gate
# Finish mutations before starting parallel checks.
cmds:
# Node packages land under ui/, which `go test ./...` walks: installing
# them while the parallel checks scan the tree makes a lockfile update
# delete Go sources mid-walk. Install first, then fan out. Every Go step
# embeds ui/dist, so the dashboard is built before any of them.
- task: ui
- task: fmt
- task: proto:check
- task: db:check
# Node packages land under ui/, which `go test ./...` walks: installing
# them while the parallel checks scan the tree makes a lockfile update
# delete Go sources mid-walk. Install first, then fan out.
- task: ui:deps
- task: check:readonly
check:readonly:
@@ -297,25 +300,33 @@ tasks:
- task: build
- task: test
- task: test:ui
- task: ui:check
release:preview:
desc: "Show the release notes that would be generated (VERSION=1.2.0)"
cmds:
- tools/release.sh --version {{.VERSION}} --dry-run
requires:
vars: [VERSION]
release:prepare:
desc: Prepare changelog notes for editing (VERSION=1.2.0)
cmds:
- tools/release.sh --version {{.VERSION}} --prepare
requires:
vars: [VERSION]
release:
desc: Commit and tag the prepared release (VERSION=1.2.0)
cmds:
- tools/release.sh --version {{.VERSION}} --tag
desc: "Turn [Unreleased] into a release, then commit and tag it (RELEASE=major|minor|patch|1.2.3)"
requires:
vars: [VERSION]
vars: [RELEASE]
cmds:
- |
set -euo pipefail
die() { echo "release: $*" >&2; exit 1; }
[ "$(git rev-parse --abbrev-ref HEAD)" = main ] || die "releases are cut from main"
[ -z "$(git status --porcelain)" ] || die "the working tree is dirty"
last="$(git tag --list 'v*' --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -1 || true)"
IFS=. read -r major minor patch <<<"${last#v}"
case "{{.RELEASE}}" in
major) version="$((major + 1)).0.0" ;;
minor) version="$major.$((minor + 1)).0" ;;
patch) version="$major.$minor.$((patch + 1))" ;;
*) version="$(echo "{{.RELEASE}}" | sed 's/^v//')" ;;
esac
echo "$version" | grep -Eq '^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.]+)?$' || die "$version is not a version"
! git rev-parse -q --verify "refs/tags/v$version" >/dev/null || die "v$version already exists"
awk '/^## \[Unreleased\]/ { u = 1; next } u && /^## \[/ { exit } u && /^- / { found = 1 } END { exit !found }' CHANGELOG.md ||
die "[Unreleased] in CHANGELOG.md has no entries"
awk -v hdr="## [$version] - $(date -u +%F)" '{ print } /^## \[Unreleased\]/ { print ""; print hdr }' CHANGELOG.md >CHANGELOG.md.tmp
mv CHANGELOG.md.tmp CHANGELOG.md
go -C tools run -mod=readonly ./newsgen --repo-root ..
git add CHANGELOG.md docs/news
git commit -q -m "chore(release): v$version"
git tag -a "v$version" -m "v$version"
echo "tagged v$version"
+15
View File
@@ -94,3 +94,18 @@ target = "127.0.0.1:9090"
# releases_key = "base64 public key"
# catalogue = "registry.example/owner/archipelago/catalogue"
# catalogue_key = "base64 public key"
# Long-running agent loops that maintain memory, skills and — in dry run —
# file follow-up issues from recent runs. Definitions are data: the server
# validates them and the gateway applies them live.
# [[curators]]
# name = "self-intake"
# interval = "1h"
# tools = ["run_list", "run_events"]
# instructions = """
# You are archie's self-intake curator, in dry run. Each pass: list recent
# task runs with run_list, inspect anything parked or failed with run_events,
# then report candidate follow-up issues — concrete, each tied to the run
# that produced it. Do not create issues; this pass reports only (no
# issue-creating tool is declared, so it cannot).
# """
+11
View File
@@ -45,3 +45,14 @@ listen = "127.0.0.1:8585"
[services.daemon]
target = "127.0.0.1:8587"
listen = "127.0.0.1:8587"
# Dry-run self-intake curator (#1770): surveys recent runs and reports
# candidate follow-up issues as pass summaries; declares no issue-creating
# tool, so it cannot file. Needs a chat model configured to run at all.
[[curators]]
name = "self-intake"
interval = "1h"
tools = ["run_list", "run_events"]
instructions = """
You are archie's self-intake curator, in dry run. Each pass: list recent task runs with run_list, inspect anything parked or failed with run_events, then report candidate follow-up issues — concrete, each tied to the run that produced it. Do not create issues; this pass reports only (no issue-creating tool is declared, so it cannot). End your reply with the report.
"""
+8
View File
@@ -85,6 +85,10 @@
"/news/archie/1.54.3/": "/news/1.54.3/",
"/news/archie/1.55.0/": "/news/1.55.0/",
"/news/archie/1.56.0/": "/news/1.56.0/",
"/news/archie/1.57.0/": "/news/1.57.0/",
"/news/archie/1.58.0/": "/news/1.58.0/",
"/news/archie/1.58.1/": "/news/1.58.1/",
"/news/archie/1.59.0/": "/news/1.59.0/",
"/news/archie/1.6.0/": "/news/1.6.0/",
"/news/archie/1.7.0/": "/news/1.7.0/",
"/news/archie/1.8.0/": "/news/1.8.0/",
@@ -186,6 +190,10 @@
"/news/archied/1.54.3/": "/news/1.54.3/",
"/news/archied/1.55.0/": "/news/1.55.0/",
"/news/archied/1.56.0/": "/news/1.56.0/",
"/news/archied/1.57.0/": "/news/1.57.0/",
"/news/archied/1.58.0/": "/news/1.58.0/",
"/news/archied/1.58.1/": "/news/1.58.1/",
"/news/archied/1.59.0/": "/news/1.59.0/",
"/news/archied/1.6.0/": "/news/1.6.0/",
"/news/archied/1.7.0/": "/news/1.7.0/",
"/news/archied/1.8.0/": "/news/1.8.0/",
+20
View File
@@ -1,4 +1,24 @@
[
{
"version": "1.59.0",
"date": "2026-11-02",
"body": "### archied\n\n- Forge extensions can open issues: the forge surface gains `CreateIssue` alongside list, comment and close, and the bundled Gitea and GitHub extensions implement it."
},
{
"version": "1.58.1",
"date": "2026-11-01",
"body": "### archied\n\n- The systemd watchdog now also watches the task intake loop, and its warning names the loop that stalled.\n- archied no longer restarts about a minute after it starts. Under systemd, the watchdog never saw a poll pass and killed it on every start since 1.56.0."
},
{
"version": "1.58.0",
"date": "2026-11-01",
"body": "### archied\n\n- Running `install.sh` over an install whose services are already running now restarts them, so they run the new binaries."
},
{
"version": "1.57.0",
"date": "2026-10-31",
"body": "### archied\n\n- A release reaches `stable` only after it has run on a canary for a day and passed: `archied release publish` now publishes to `next` alone, `archied release canary start|check` installs a release on a canary and judges it from the canary's dashboard, promoting it or rolling the canary back, and `archied release promote` moves `stable` by hand. Promotion refuses a prerelease and a release older than the one `stable` names. The `archied` and `archie-agent` images are published as `:v<version>`, and `:latest` moves only when a release is promoted."
},
{
"version": "1.56.0",
"date": "2026-10-31",
+68
View File
@@ -0,0 +1,68 @@
# Self-development loop: archie develops its own codebase
**Status:** Decided, not yet built (2026-11-03, sam) **Scope:** how archie
persistently takes on development of archie-core itself. This file records a
desired state, not an implementation or a product: nothing below exists unless
the "Built today" section says so. The plan is tracked as epic #1769 with child
issues #1770–#1775. This file is a record, not a mechanism: agents only see
what is in their context, so verify every claim here against the code before
acting on it — where they disagree, the code wins. The boundary below is
load-bearing only where it is enforced (the gate, access control, the daemon's
git monopoly) and stated in `AGENTS.md`; everywhere else this file is a pointer.
## Built today
The machinery the loop will reuse exists: shipped triage/implement/pr-review
workflows, the adversarial self-review stage before a PR opens (epic #1348,
closed), the prbench judge for review quality with its acceptance bar, canary
releases judged on the installed release (releaseupdate), and the curator
engine. What does not exist is the loop itself: no self-intake curator, no
approved-issue auto-run, no judge for diffs, no outcome distillation, no
canary routing specific to self-directed changes.
## Decision
Archie improves its own codebase through a closed loop, built in stages:
1. **Self-intake** (#1770): a curator pass reads run outcomes, step transcripts,
gate failures and alerts, dedupes against open issues, and files new issues.
Archie files its own work; the maintainer approves priority.
2. **Approved-issue loop** (#1771): an approved issue runs the shipped
triage/implement workflows end to end — read-only plan, gated build,
adversarial self-review, daemon commits and opens a green PR. The maintainer
still merges at this stage.
3. **Implementation judge** (#1772): the prbench judge pattern scores diffs,
calibrated against the maintainer's merge decisions; auto-merge applies only
when the gate is green, the review stage is clean, and the judge is above the
calibrated bar.
4. **Outcome distillation** (#1773): a curator turns merged-run postmortems into
proposals for the repo's agent instructions, each added only because it
changes a decision; rules that stop changing decisions are removed.
5. **Canary** (#1774): self-directed changes ship through the canary release
window and are judged on the installed release before promotion. Rollback is
a plain revert, because every self-directed change is one gated commit.
## Boundary
Enforcement lives in the harness, never in the prompt (#1775). Changes to the
commit gate, the invariants in `AGENTS.md`, access control and grant scoping,
the daemon-only git path, and webhook HMAC verification may be proposed by
archie but are never merged by archie — they require a maintainer merge.
Everything else (workflows, curators, memory, docs, the dashboard) is fair game
for the loop.
## Why
Self-improvement degrades when it relies on the model choosing to follow rules.
Gates, scoped grants and the daemon's monopoly on git cannot be argued with; a
prompt can. Keeping quality measurable — a judge bar and a canary verdict as
numbers, tests guarding the protected seams — is what lets the loop widen
without turning the codebase to garbage.
## Consequences
- One issue → one small PR applies doubly to self-directed work: past about 100
lines or any new mechanism, the change stops and reassesses.
- Every self-directed commit cites the issue that authorized it.
- Rule proposals from the distillation curator must name the run that triggered
them, and `AGENTS.md` stays short enough to read in full every session.
+4 -2
View File
@@ -475,10 +475,12 @@ EOF
fi
# Auto-enable and start the services, the store first so a fresh boot does
# not race a dial it depends on.
# not race a dial it depends on. Restart rather than start, because a
# reinstall over running services would otherwise leave the old binaries
# running.
if [ "${AUTO_START}" = true ]; then
echo "==> Enabling and starting Archie services..."
systemctl --user enable --now $INSTANCE_UNITS ||
{ systemctl --user enable $INSTANCE_UNITS && systemctl --user restart $INSTANCE_UNITS; } ||
echo " Notice: Could not start every service automatically. Check: systemctl --user status ${INSTANCE_UNITS}"
fi
fi
+1 -1
View File
@@ -52,7 +52,7 @@ type HarnessStages struct {
Spec agentrun.HarnessSpec
}
func (r HarnessStages) Run(ctx context.Context, workspace string, req agentrun.Request, report agentrun.ToolCallReporter) (agentrun.Result, error) {
func (r HarnessStages) Run(ctx context.Context, workspace string, req agentrun.Request, report agentrun.Reporter) (agentrun.Result, error) {
spec := r.Spec
req.Harness = &spec
return r.Runner.Run(ctx, workspace, req, report)
+1 -1
View File
@@ -34,7 +34,7 @@ func TestHarnessRequiresMCPInitialization(t *testing.T) {
if !central {
req.CaptureTools = []agentrun.CaptureTool{{Name: "capture_result", Parameters: json.RawMessage(`{"type":"object"}`)}}
}
result, err := NewHarnessRunner(nil).Run(t.Context(), workspace, req, nil)
result, err := NewHarnessRunner(nil).Run(t.Context(), workspace, req, agentrun.Reporter{})
if err != nil {
t.Fatal(err)
}
+7 -9
View File
@@ -28,7 +28,7 @@ func newClaudeCodeHarnessOutput() *claudeCodeHarnessOutput {
}
}
func (o *claudeCodeHarnessOutput) Line(line []byte, report agentrun.ToolCallReporter) {
func (o *claudeCodeHarnessOutput) Line(line []byte, report agentrun.Reporter) {
var event struct {
Type string `json:"type"`
SessionID string `json:"session_id"`
@@ -55,7 +55,7 @@ func (o *claudeCodeHarnessOutput) Line(line []byte, report agentrun.ToolCallRepo
}
}
func (o *claudeCodeHarnessOutput) assistant(raw json.RawMessage, _ agentrun.ToolCallReporter) {
func (o *claudeCodeHarnessOutput) assistant(raw json.RawMessage, _ agentrun.Reporter) {
var message struct {
ID string `json:"id"`
Usage json.RawMessage `json:"usage"`
@@ -79,7 +79,7 @@ func (o *claudeCodeHarnessOutput) assistant(raw json.RawMessage, _ agentrun.Tool
}
}
func (o *claudeCodeHarnessOutput) user(raw json.RawMessage, report agentrun.ToolCallReporter) {
func (o *claudeCodeHarnessOutput) user(raw json.RawMessage, report agentrun.Reporter) {
var message struct {
Content []struct {
Type string `json:"type"`
@@ -100,12 +100,10 @@ func (o *claudeCodeHarnessOutput) user(raw json.RawMessage, report agentrun.Tool
continue
}
delete(o.pending, block.ToolUseID)
if report != nil {
report(agentrun.ToolCallReport{
Tool: tool, Detail: ClipToolCallDetail(claudeResultContent(block.Content)),
Failed: block.IsError,
})
}
report.ReportToolCall(agentrun.ToolCallReport{
Tool: tool, Detail: ClipToolCallDetail(claudeResultContent(block.Content)),
Failed: block.IsError,
})
}
}
+3 -3
View File
@@ -15,7 +15,7 @@ type codexHarnessOutput struct {
usage agentrun.Usage
}
func (o *codexHarnessOutput) Line(line []byte, report agentrun.ToolCallReporter) {
func (o *codexHarnessOutput) Line(line []byte, report agentrun.Reporter) {
var event struct {
Type string `json:"type"`
ThreadID string `json:"thread_id"`
@@ -43,8 +43,8 @@ func (o *codexHarnessOutput) Line(line []byte, report agentrun.ToolCallReporter)
})
}
case "item.completed":
if call, ok := event.Item.report(); ok && report != nil {
report(call)
if call, ok := event.Item.report(); ok {
report.ReportToolCall(call)
}
}
}
+2 -4
View File
@@ -18,7 +18,7 @@ func newCopilotHarnessOutput() *copilotHarnessOutput {
return &copilotHarnessOutput{pending: make(map[string]string)}
}
func (o *copilotHarnessOutput) Line(line []byte, report agentrun.ToolCallReporter) {
func (o *copilotHarnessOutput) Line(line []byte, report agentrun.Reporter) {
var event struct {
Type string `json:"type"`
SessionID string `json:"sessionId"`
@@ -52,9 +52,7 @@ func (o *copilotHarnessOutput) Line(line []byte, report agentrun.ToolCallReporte
return
}
delete(o.pending, event.Data.ToolCallID)
if report != nil {
report(agentrun.ToolCallReport{Tool: tool, Detail: ClipToolCallDetail(event.Data.Result.Content), Failed: !event.Data.Success})
}
report.ReportToolCall(agentrun.ToolCallReport{Tool: tool, Detail: ClipToolCallDetail(event.Data.Result.Content), Failed: !event.Data.Success})
}
}
+3 -3
View File
@@ -15,7 +15,7 @@ type piHarnessOutput struct {
usage agentrun.Usage
}
func (o *piHarnessOutput) Line(line []byte, report agentrun.ToolCallReporter) {
func (o *piHarnessOutput) Line(line []byte, report agentrun.Reporter) {
var event struct {
Type string `json:"type"`
ID string `json:"id"`
@@ -49,7 +49,7 @@ func (o *piHarnessOutput) Line(line []byte, report agentrun.ToolCallReporter) {
o.usage = addUsage(o.usage, usageFromFields(u.Input, u.Output, u.CacheRead, u.CacheWrite))
}
case "tool_execution_end":
if event.ToolName == "" || report == nil {
if event.ToolName == "" {
return
}
texts := make([]string, 0, len(event.Result.Content))
@@ -58,7 +58,7 @@ func (o *piHarnessOutput) Line(line []byte, report agentrun.ToolCallReporter) {
texts = append(texts, block.Text)
}
}
report(agentrun.ToolCallReport{Tool: event.ToolName, Detail: ClipToolCallDetail(strings.Join(texts, "\n")), Failed: event.IsError})
report.ReportToolCall(agentrun.ToolCallReport{Tool: event.ToolName, Detail: ClipToolCallDetail(strings.Join(texts, "\n")), Failed: event.IsError})
}
}
+7 -7
View File
@@ -45,7 +45,7 @@ const (
// knows its CLI's stream format; it reports tool calls as they complete and
// the session to resume.
type HarnessOutput interface {
Line(line []byte, report agentrun.ToolCallReporter)
Line(line []byte, report agentrun.Reporter)
SessionID() string
Usage() agentrun.Usage
}
@@ -67,7 +67,7 @@ func NewHarnessRunner(newOutput func() HarnessOutput) *HarnessRunner {
return &HarnessRunner{newOutput: newOutput}
}
func (r *HarnessRunner) Run(ctx context.Context, workspace string, req agentrun.Request, report agentrun.ToolCallReporter) (agentrun.Result, error) {
func (r *HarnessRunner) Run(ctx context.Context, workspace string, req agentrun.Request, report agentrun.Reporter) (agentrun.Result, error) {
if err := validateHarness(req); err != nil {
return agentrun.Result{}, err
}
@@ -95,7 +95,7 @@ func (r *HarnessRunner) Run(ctx context.Context, workspace string, req agentrun.
return res, err
}
func (r *HarnessRunner) run(ctx context.Context, workspace string, req agentrun.Request, mcpConfig string, report agentrun.ToolCallReporter) (agentrun.Result, error) {
func (r *HarnessRunner) run(ctx context.Context, workspace string, req agentrun.Request, mcpConfig string, report agentrun.Reporter) (agentrun.Result, error) {
res := agentrun.Result{Version: agentrun.ProtocolVersion, TaskID: req.TaskID, Attempt: req.Attempt, Stage: req.Stage}
before, err := snapshotRepo(workspace)
if err != nil {
@@ -244,7 +244,7 @@ func resumeVerb(h *agentrun.HarnessSpec, sessionID string) []string {
// invoke runs one harness invocation in its own process group, feeding
// stdout to out line by line. Cancelling ctx kills the whole group, so a
// CLI's children cannot outlive the step.
func invoke(ctx context.Context, workspace string, h *agentrun.HarnessSpec, argv []string, out HarnessOutput, report agentrun.ToolCallReporter) error {
func invoke(ctx context.Context, workspace string, h *agentrun.HarnessSpec, argv []string, out HarnessOutput, report agentrun.Reporter) error {
cmd := exec.CommandContext(ctx, argv[0], argv[1:]...)
cmd.Dir = workspace
cmd.WaitDelay = killGrace
@@ -467,9 +467,9 @@ func policyViolation(before, after repoState, changes []string, req agentrun.Req
type silentOutput struct{}
func (silentOutput) Line([]byte, agentrun.ToolCallReporter) {}
func (silentOutput) SessionID() string { return "" }
func (silentOutput) Usage() agentrun.Usage { return agentrun.Usage{} }
func (silentOutput) Line([]byte, agentrun.Reporter) {}
func (silentOutput) SessionID() string { return "" }
func (silentOutput) Usage() agentrun.Usage { return agentrun.Usage{} }
// tailBuffer keeps the last stderrTailBytes written to it.
type tailBuffer struct{ buf bytes.Buffer }
+1 -1
View File
@@ -18,7 +18,7 @@ func TestHarnessLiveOutput(t *testing.T) {
if os.Geteuid() == 0 {
user = "65534"
}
err := invoke(t.Context(), "/tmp", &agentrun.HarnessSpec{User: user}, []string{"sh", "-c", `printf '%s\n' '{"type":"thread.started","thread_id":"live-thread"}'; printf '%s\n' 'live diagnostic' >&2`}, adapter, nil)
err := invoke(t.Context(), "/tmp", &agentrun.HarnessSpec{User: user}, []string{"sh", "-c", `printf '%s\n' '{"type":"thread.started","thread_id":"live-thread"}'; printf '%s\n' 'live diagnostic' >&2`}, adapter, agentrun.Reporter{})
if err != nil {
t.Fatal(err)
}
+11 -3
View File
@@ -53,7 +53,7 @@ func NewLoopRunner(
// work on the chat workspace and bypass ReadOnly and ProtectPaths.
const workspaceToolset = "workspace"
func (r *LoopRunner) Run(ctx context.Context, workspace string, req agentrun.Request, report agentrun.ToolCallReporter) (agentrun.Result, error) {
func (r *LoopRunner) Run(ctx context.Context, workspace string, req agentrun.Request, report agentrun.Reporter) (agentrun.Result, error) {
if err := req.Validate(); err != nil {
return agentrun.Result{}, err
}
@@ -67,12 +67,18 @@ func (r *LoopRunner) Run(ctx context.Context, workspace string, req agentrun.Req
notes := &memoryNotes{initial: req.Notes}
captures := make(map[string][]json.RawMessage)
toolOpts := r.Limits.Options(workspaceToolset)
toolOpts.OnToolCall = report
toolOpts.OnToolCall = report.ToolCall
centralTools, err := BuildToolSet(r.tools, toolOpts)
if err != nil {
return agentrun.Result{}, fmt.Errorf("build central tool set: %w", err)
}
captureTools := captureToolSet(req.CaptureTools, captures)
var modelHooks []core.ModelHook
var toolHooks []core.ToolHook
if report.Transcript != nil {
recorder := newTranscript(report.Transcript)
modelHooks, toolHooks = []core.ModelHook{recorder}, []core.ToolHook{recorder}
}
res, err := r.run(ctx, agentloop.Config{
Runtime: r.runtime,
ModelRef: req.Model,
@@ -101,7 +107,9 @@ func (r *LoopRunner) Run(ctx context.Context, workspace string, req agentrun.Req
allowedTools(centralTools, r.AllowTools),
captureTools,
),
Logger: r.logger(req),
ModelHooks: modelHooks,
ToolHooks: toolHooks,
Logger: r.logger(req),
})
result := resultFromRun(req, res, notes.appended, captures)
if cause := context.Cause(ctx); cause != nil {
+1 -1
View File
@@ -74,7 +74,7 @@ type ToolSetOptions struct {
// OnToolCall, when non-nil, is notified once per completed tool call.
// LoopRunner populates it so a workflow stage can surface tool activity on
// the task timeline. Nil means no one is listening.
OnToolCall agentrun.ToolCallReporter
OnToolCall func(agentrun.ToolCallReport)
}
// reportToolCallCompletion notifies opts.OnToolCall, if set, of one
+187
View File
@@ -0,0 +1,187 @@
package modelloop
import (
"context"
"encoding/json"
"strings"
"sync"
"time"
"unicode/utf8"
"github.com/samcharles93/ai-sdk/chat"
"github.com/samcharles93/ai-sdk/core"
"github.com/samcharles93/archie-core/internal/domain/agentrun"
"github.com/samcharles93/archie-core/internal/webhookguard"
)
// transcriptFieldLimit caps each recorded text field, in bytes.
const transcriptFieldLimit = 32 << 10
// compactionPrefix starts the instruction agentloop sends to summarise the
// history, which marks a model call as a compaction rather than a step.
const compactionPrefix = "Compact the conversation history above"
// transcript records one agent run's conversation through agentloop's model
// and tool hooks. Tool hooks run concurrently, so entries are numbered and
// reported under one lock to keep their order.
type transcript struct {
report func(agentrun.TranscriptEntry)
mu sync.Mutex
seq int
prompted bool
lastUser string
started map[string]time.Time
}
func newTranscript(report func(agentrun.TranscriptEntry)) *transcript {
return &transcript{report: report, started: map[string]time.Time{}}
}
func (t *transcript) add(entry agentrun.TranscriptEntry) {
t.seq++
entry.Seq = t.seq
t.report(entry)
}
func (t *transcript) ModelCallStarted(context.Context, chat.Request) {}
func (t *transcript) ModelCallFinished(_ context.Context, req chat.Request, resp *chat.Response, err error, latency time.Duration, usage *chat.Usage) {
t.mu.Lock()
defer t.mu.Unlock()
entry := agentrun.TranscriptEntry{Kind: agentrun.TranscriptModel, LatencyMS: latency.Milliseconds(), Usage: transcriptUsage(usage)}
if isCompaction(req) {
entry.Kind = agentrun.TranscriptCompaction
} else {
t.recordInput(req.Messages)
}
if err != nil {
entry.Error = clip(err.Error(), nil)
}
if resp != nil {
entry.Text = clip(resp.Content, &entry.TextBytes)
entry.Reasoning = clip(reasoning(resp.Parts), &entry.ReasoningBytes)
entry.FinishReason = resp.FinishReason
for _, call := range resp.ToolCalls {
recorded := agentrun.TranscriptToolCall{ID: call.ID, Name: call.Name}
recorded.Args = clipArgs(call.Arguments, &recorded.ArgsBytes)
entry.ToolCalls = append(entry.ToolCalls, recorded)
}
}
t.add(entry)
}
// recordInput records the prompt on the first call, and after that each user
// message the loop appended since the model last answered: gate feedback,
// recovery nudges. Assistant and tool messages are recorded from the response
// and the tool hooks instead.
func (t *transcript) recordInput(messages []chat.Message) {
if !t.prompted {
t.prompted = true
for _, message := range messages {
t.recordMessage(message)
}
return
}
start := len(messages)
for start > 0 && messages[start-1].Role == chat.RoleUser {
start--
}
for _, message := range messages[start:] {
if message.Content != t.lastUser {
t.recordMessage(message)
}
}
}
func (t *transcript) recordMessage(message chat.Message) {
kind := agentrun.TranscriptUser
switch message.Role {
case chat.RoleSystem:
kind = agentrun.TranscriptSystem
case chat.RoleUser:
t.lastUser = message.Content
default:
return
}
entry := agentrun.TranscriptEntry{Kind: kind}
entry.Text = clip(message.Content, &entry.TextBytes)
t.add(entry)
}
func (t *transcript) BeforeToolExecute(_ context.Context, call *core.ToolCall) (*core.Skip, error) {
t.mu.Lock()
t.started[call.ToolCallID] = time.Now()
t.mu.Unlock()
return nil, nil
}
func (t *transcript) AfterToolExecute(_ context.Context, call core.ToolCall, result *core.ToolResult) {
t.mu.Lock()
defer t.mu.Unlock()
entry := agentrun.TranscriptEntry{Kind: agentrun.TranscriptTool, CallID: call.ToolCallID, Tool: call.ToolName}
if started, ok := t.started[call.ToolCallID]; ok {
entry.LatencyMS = time.Since(started).Milliseconds()
delete(t.started, call.ToolCallID)
}
entry.Args = clipArgs(call.Input, &entry.ArgsBytes)
entry.Output = clip(result.Output, &entry.OutputBytes)
entry.Error = clip(result.Error, nil)
t.add(entry)
}
func reasoning(parts chat.Parts) string {
var texts []string
for _, part := range parts {
if thought, ok := part.(chat.ReasoningPart); ok && thought.Text != "" {
texts = append(texts, thought.Text)
}
}
return strings.Join(texts, "\n\n")
}
func isCompaction(req chat.Request) bool {
n := len(req.Messages)
return n > 0 && req.Messages[n-1].Role == chat.RoleUser && strings.HasPrefix(req.Messages[n-1].Content, compactionPrefix)
}
func transcriptUsage(usage *chat.Usage) *agentrun.Usage {
if usage == nil {
return nil
}
return &agentrun.Usage{
PromptTokens: usage.PromptTokens,
CompletionTokens: usage.CompletionTokens,
TotalTokens: usage.TotalTokens,
CachedTokens: usage.CachedTokens,
CacheCreationTokens: usage.CacheCreationTokens,
}
}
// clipArgs redacts tool arguments by key when they are JSON, then clips them.
func clipArgs(args string, full *int) string {
if json.Valid([]byte(args)) {
if redacted, err := webhookguard.RedactPayload([]byte(args)); err == nil {
args = string(redacted)
}
}
return clip(args, full)
}
// clip redacts secret-shaped spans and cuts text to transcriptFieldLimit on a
// rune boundary, storing the original size in full when it cuts.
func clip(text string, full *int) string {
text = webhookguard.RedactText(text)
if len(text) <= transcriptFieldLimit {
return text
}
if full != nil {
*full = len(text)
}
cut := transcriptFieldLimit
for cut > 0 && !utf8.RuneStart(text[cut]) {
cut--
}
return text[:cut]
}
@@ -0,0 +1,71 @@
package modelloop
import (
"strings"
"testing"
"github.com/samcharles93/ai-sdk/core"
"github.com/samcharles93/archie-core/internal/domain/agentrun"
)
func TestTranscriptToolEntryRedactsAndClips(t *testing.T) {
const jwt = "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhcmNoaWUifQ.c2lnbmF0dXJlc2lnbmF0dXJl"
const pem = "-----BEGIN RSA PRIVATE KEY-----\nMIIEow\n-----END RSA PRIVATE KEY-----"
long := strings.Repeat("x", transcriptFieldLimit+10)
tests := []struct {
name string
input string
output string
wantArgs string
wantOutput string
wantBytes int
}{
{
name: "secret key in arguments, token inside output",
input: `{"path":"a.txt","api_key":"plain-value"}`,
output: "curl -H 'Authorization: Bearer " + jwt + "' done",
wantArgs: `{"api_key":"[redacted]","path":"a.txt"}`,
wantOutput: "curl -H 'Authorization: Bearer [redacted]' done",
},
{
name: "private key block inside output",
input: `{}`,
output: "before\n" + pem + "\nafter",
wantArgs: `{}`,
wantOutput: "before\n[redacted]\nafter",
},
{
name: "oversized output keeps its full size",
input: `{}`,
output: long,
wantArgs: `{}`,
wantOutput: long[:transcriptFieldLimit],
wantBytes: len(long),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var got []agentrun.TranscriptEntry
recorder := newTranscript(func(entry agentrun.TranscriptEntry) { got = append(got, entry) })
call := core.ToolCall{ToolCallID: "c1", ToolName: "shell", Input: tt.input}
if _, err := recorder.BeforeToolExecute(t.Context(), &call); err != nil {
t.Fatal(err)
}
recorder.AfterToolExecute(t.Context(), call, &core.ToolResult{Output: tt.output})
if len(got) != 1 {
t.Fatalf("entries = %d, want 1", len(got))
}
entry := got[0]
if entry.Args != tt.wantArgs {
t.Errorf("args = %q, want %q", entry.Args, tt.wantArgs)
}
if entry.Output != tt.wantOutput {
t.Errorf("output = %q, want %q", entry.Output, tt.wantOutput)
}
if entry.OutputBytes != tt.wantBytes {
t.Errorf("output bytes = %d, want %d", entry.OutputBytes, tt.wantBytes)
}
})
}
}
@@ -44,7 +44,7 @@ func readProjectMemory(path string) (string, error) {
return "Project memory from earlier sessions:\n" + strings.Join(entries, "\n"), nil
}
func (r persistentRunner) Run(ctx context.Context, workspace string, request agentrun.Request, report agentrun.ToolCallReporter) (agentrun.Result, error) {
func (r persistentRunner) Run(ctx context.Context, workspace string, request agentrun.Request, report agentrun.Reporter) (agentrun.Result, error) {
if !r.enabled {
return r.Runner.Run(ctx, workspace, request, report)
}
+49 -5
View File
@@ -17,7 +17,13 @@ import (
const releaseUsage = `usage: archied release <command>
keygen -seed FILE write a new signing key's seed to FILE and print its public key
publish sign a release and push it to a release source`
publish sign a release and push it to a release source's next channel
promote -version V point stable at a published release by hand
canary status show the canary's channel, installed release, window and verdict (read-only)
canary start point the canary at next and install a release on it, queued behind one being judged
canary check judge the release the canary runs: promote it, pick up a newer next, or roll back
canary promote promote the release the canary runs now, if its signals pass
canary unpin return the canary to the next channel from any pin`
// RunRelease is the publisher's side of a release source: it makes the key
// a source signs with and pushes signed releases to it.
@@ -31,6 +37,10 @@ func RunRelease(args []string, out, stderr io.Writer) int {
return runReleaseKeygen(args[1:], out, stderr)
case "publish":
return runReleasePublish(args[1:], out, stderr)
case "promote":
return runReleasePromote(args[1:], out, stderr)
case "canary":
return runReleaseCanary(args[1:], out, stderr)
default:
fmt.Fprintln(stderr, releaseUsage)
return 2
@@ -79,7 +89,7 @@ func runReleasePublish(args []string, out, stderr io.Writer) int {
flags.StringVar(&req.Repository, "source", "", "OCI repository to publish to")
seedEnv := flags.String("seed-env", "ARCHIE_RELEASE_SEED", "environment variable holding the base64 private key seed")
flags.StringVar(&req.Version, "version", "", "release version")
flags.BoolVar(&req.Prerelease, "prerelease", false, "publish to next only")
flags.BoolVar(&req.Prerelease, "prerelease", false, "mark the release a prerelease, which never reaches stable")
flags.StringVar(&req.ZipPath, "zip", "", "distribution zip")
flags.StringVar(&req.AgentImage, "agent-image", "", "agent image pinned by digest")
flags.StringVar(&req.Changelog, "changelog", "", "URL of the release notes")
@@ -97,10 +107,8 @@ func runReleasePublish(args []string, out, stderr io.Writer) int {
}
req.Signer = ed25519.NewKeyFromSeed(seed)
public := distribution.EncodeKey(ed25519.PublicKey(req.Signer[ed25519.SeedSize:]))
// stable moves only when the canary promotes the release.
req.Channels = []string{"next"}
if !req.Prerelease {
req.Channels = append(req.Channels, "stable")
}
statement, err := releaseupdate.Publish(context.Background(), req)
if err != nil {
fmt.Fprintln(stderr, err)
@@ -110,3 +118,39 @@ func runReleasePublish(args []string, out, stderr io.Writer) int {
strings.Join(req.Channels, ", "), public)
return 0
}
// runReleasePromote moves stable without a canary verdict, under the same
// rules the canary promotes by.
func runReleasePromote(args []string, out, stderr io.Writer) int {
flags := flag.NewFlagSet("release promote", flag.ContinueOnError)
flags.SetOutput(stderr)
reference := flags.String("source", "", "OCI repository releases are published to; blank is the official releases")
key := flags.String("key", "", "base64 ed25519 public key the source signs with")
version := flags.String("version", "", "release to promote")
latest := flags.String("latest", "", "comma-separated image repositories whose v<version> tag becomes latest")
if err := flags.Parse(args); err != nil {
return 2
}
source, err := distribution.Parse(*reference, *key, distribution.DefaultReleases())
if err != nil || *version == "" {
fmt.Fprintln(stderr, "release promote: -version is required, and -source with -key name another source")
return 2
}
statement, err := releaseupdate.Promote(context.Background(), source.Reference, source.Key, *version, splitList(*latest))
if err != nil {
fmt.Fprintln(stderr, err)
return 1
}
fmt.Fprintf(out, "stable is %s\n", statement.Version)
return 0
}
func splitList(value string) []string {
var items []string
for item := range strings.SplitSeq(value, ",") {
if item = strings.TrimSpace(item); item != "" {
items = append(items, item)
}
}
return items
}
+622
View File
@@ -0,0 +1,622 @@
package archied
import (
"context"
"errors"
"flag"
"fmt"
"io"
"os"
"os/exec"
"time"
"oras.land/oras-go/v2/errdef"
"github.com/samcharles93/archie-core/internal/domain/distribution"
"github.com/samcharles93/archie-core/internal/releaseupdate"
)
// canaryInstallWait bounds how long an install on the canary may take,
// including the watchdog's health wait and a rollback.
const canaryInstallWait = 20 * time.Minute
// migrationsDir holds the schema migrations; a release that changes it can't
// be rolled back without restoring the database snapshot.
const migrationsDir = "internal/infrastructure/postgres/migrations"
func runReleaseCanary(args []string, out, stderr io.Writer) int {
if len(args) == 0 {
fmt.Fprintln(stderr, releaseUsage)
return 2
}
switch args[0] {
case "status":
return runCanaryStatus(args[1:], out, stderr)
case "start":
return runCanaryStart(args[1:], out, stderr)
case "check":
return runCanaryCheck(args[1:], out, stderr)
case "promote":
return runCanaryPromote(args[1:], out, stderr)
case "unpin":
return runCanaryUnpin(args[1:], out, stderr)
default:
fmt.Fprintln(stderr, releaseUsage)
return 2
}
}
// addCanaryFlags registers the flags every canary subcommand takes.
func addCanaryFlags(flags *flag.FlagSet) (dashboardURL, tokenEnv *string) {
dashboardURL = flags.String("dashboard", "", "canary dashboard URL")
tokenEnv = flags.String("token-env", "ARCHIE_CANARY_TOKEN", "environment variable holding a dashboard API token for the canary")
return dashboardURL, tokenEnv
}
// addSourceFlags registers the flags naming the release source.
func addSourceFlags(flags *flag.FlagSet) (reference, key *string) {
reference = flags.String("source", "", "OCI repository releases are published to; blank is the official releases")
key = flags.String("key", "", "base64 ed25519 public key the source signs with")
return reference, key
}
func canaryDashboard(url, tokenEnv string) (releaseupdate.CanaryDashboard, error) {
if url == "" {
return releaseupdate.CanaryDashboard{}, errors.New("-dashboard is required")
}
return releaseupdate.CanaryDashboard{URL: url, Token: os.Getenv(tokenEnv)}, nil
}
func canarySource(reference, key string) (distribution.Source, error) {
return distribution.Parse(reference, key, distribution.DefaultReleases())
}
// fetchChannel fetches a channel tag; found is false when the channel has no
// release yet.
func fetchChannel(ctx context.Context, source distribution.Source, tag string) (release releaseupdate.Release, found bool, err error) {
release, err = releaseupdate.Fetch(ctx, source, tag)
if errors.Is(err, errdef.ErrNotFound) {
return releaseupdate.Release{}, false, nil
}
return release, err == nil, err
}
// publishedChannels are the releases the source currently publishes.
type publishedChannels struct {
next, stable releaseupdate.Release
hasNext, hasStable bool
}
func fetchChannels(ctx context.Context, source distribution.Source) (publishedChannels, error) {
var channels publishedChannels
var err error
if channels.next, channels.hasNext, err = fetchChannel(ctx, source, "next"); err != nil {
return channels, err
}
if channels.stable, channels.hasStable, err = fetchChannel(ctx, source, "stable"); err != nil {
return channels, err
}
return channels, nil
}
// canaryRun is the release the canary actually runs: its version, when it was
// installed and its fetched release statement.
type canaryRun struct {
version string
installedAt time.Time
release releaseupdate.Release
found bool
}
// readCanaryRun reads what the canary runs. A host that doesn't record install
// times (it updated before they were tracked) degrades to the release
// statement's issue time.
func readCanaryRun(ctx context.Context, dashboard releaseupdate.CanaryDashboard, source distribution.Source) (canaryRun, error) {
var run canaryRun
version, installedAt, _, err := dashboard.Installed(ctx)
if err != nil {
return run, err
}
run.version, run.installedAt = version, installedAt
release, found, err := fetchChannel(ctx, source, releaseupdate.Tag("exact-pin", version))
if err != nil {
return run, err
}
run.release, run.found = release, found
if run.installedAt.IsZero() && found {
run.installedAt = release.Statement.IssuedAt
}
return run, nil
}
// newerIssued reports whether release a was issued after b. IssuedAt, not
// semver order: Promote refuses a release issued before the one stable names,
// so the judge must agree with that replay rule.
func newerIssued(a, b releaseupdate.Release) bool {
return a.Statement.IssuedAt.After(b.Statement.IssuedAt)
}
func displayVersion(version string) string {
if version == "" {
return "unknown"
}
return version
}
func displayRelease(release releaseupdate.Release, found bool) string {
if !found {
return "none"
}
return release.Statement.Version
}
func printVerdict(out io.Writer, version string, verdict releaseupdate.CanaryVerdict) {
fmt.Fprintf(out, "%s: %s\n", displayVersion(version), verdict.Decision)
for _, reason := range verdict.Reasons {
fmt.Fprintf(out, " %s\n", reason)
}
}
func runCanaryStatus(args []string, out, stderr io.Writer) int {
flags := flag.NewFlagSet("release canary status", flag.ContinueOnError)
flags.SetOutput(stderr)
dashboardURL, tokenEnv := addCanaryFlags(flags)
reference, key := addSourceFlags(flags)
window := flags.Duration("window", releaseupdate.CanaryWindow, "window a release is judged over, counted from its install")
if err := flags.Parse(args); err != nil {
return 2
}
dashboard, err := canaryDashboard(*dashboardURL, *tokenEnv)
if err != nil {
fmt.Fprintln(stderr, "release canary status:", err)
return 2
}
source, err := canarySource(*reference, *key)
if err != nil {
fmt.Fprintln(stderr, "release canary status:", err)
return 2
}
if err := printCanaryStatus(context.Background(), dashboard, source, *window, out); err != nil {
fmt.Fprintln(stderr, "release canary status:", err)
return 1
}
return 0
}
// printCanaryStatus reports the canary's channel, installed release, the
// published channels, window progress and the verdict the current signals
// would give, without acting on anything.
func printCanaryStatus(ctx context.Context, dashboard releaseupdate.CanaryDashboard, source distribution.Source, window time.Duration, out io.Writer) error {
channel, pin, err := dashboard.Channel(ctx)
if err != nil {
return err
}
channels, err := fetchChannels(ctx, source)
if err != nil {
return err
}
run, err := readCanaryRun(ctx, dashboard, source)
if err != nil {
return err
}
printCanaryState(out, channel, pin, run, channels, window)
return printCanaryVerdict(ctx, dashboard, run, window, out)
}
func printCanaryState(out io.Writer, channel, pin string, run canaryRun, channels publishedChannels, window time.Duration) {
fmt.Fprintf(out, "channel: %s\n", channelDisplay(channel, pin))
fmt.Fprintf(out, "installed: %s", displayVersion(run.version))
if run.installedAt.IsZero() {
fmt.Fprintln(out, " (installed_at not reported)")
} else {
fmt.Fprintf(out, " since %s\n", run.installedAt.Format(time.RFC3339))
}
fmt.Fprintf(out, "stable: %s\n", displayRelease(channels.stable, channels.hasStable))
fmt.Fprintf(out, "next: %s\n", displayRelease(channels.next, channels.hasNext))
if run.installedAt.IsZero() {
fmt.Fprintln(out, "window: unknown, no install time to count from")
} else {
elapsed := time.Since(run.installedAt)
fmt.Fprintf(out, "window: %s of %s elapsed since the install\n", elapsed.Round(time.Minute), window)
}
}
func printCanaryVerdict(ctx context.Context, dashboard releaseupdate.CanaryDashboard, run canaryRun, window time.Duration, out io.Writer) error {
if run.installedAt.IsZero() {
fmt.Fprintln(out, "verdict: unknown, no install time to judge from")
return nil
}
now := time.Now().UTC()
signals, err := dashboard.Signals(ctx, run.installedAt, now, window)
if err != nil {
return fmt.Errorf("read canary signals: %w", err)
}
printSignals(out, signals)
verdict := releaseupdate.JudgeCanary(run.version, run.installedAt, now, window, signals)
fmt.Fprintf(out, "verdict on %s: %s (read-only)\n", displayVersion(run.version), verdict.Decision)
for _, reason := range verdict.Reasons {
fmt.Fprintf(out, " %s\n", reason)
}
return nil
}
func printSignals(out io.Writer, s releaseupdate.CanarySignals) {
fmt.Fprintf(out, "signals: runs %q, ready=%t, new action alerts=%d\n", s.Installed, s.Ready, len(s.NewActionAlerts))
fmt.Fprintf(out, " tasks finished %d, parked or died %d (baseline %d, %d)\n",
s.Canary.Finished, s.Canary.Bad, s.Baseline.Finished, s.Baseline.Bad)
fmt.Fprintf(out, " error lines %d, log covers %s (baseline %d, covers %s)\n",
s.Canary.Errors, s.Canary.Covered.Round(time.Minute), s.Baseline.Errors, s.Baseline.Covered.Round(time.Minute))
}
func channelDisplay(channel, pin string) string {
if pin != "" {
return channel + " (pinned to " + pin + ")"
}
return channel
}
func runCanaryStart(args []string, out, stderr io.Writer) int {
flags := flag.NewFlagSet("release canary start", flag.ContinueOnError)
flags.SetOutput(stderr)
dashboardURL, tokenEnv := addCanaryFlags(flags)
reference, key := addSourceFlags(flags)
version := flags.String("version", "", "release to install on the canary")
force := flags.Bool("force", false, "install even when the canary is still judging a newer release")
if err := flags.Parse(args); err != nil {
return 2
}
dashboard, err := canaryDashboard(*dashboardURL, *tokenEnv)
if err != nil {
fmt.Fprintln(stderr, "release canary start:", err)
return 2
}
if *version == "" {
fmt.Fprintln(stderr, "release canary start: -version is required")
return 2
}
source, err := canarySource(*reference, *key)
if err != nil {
fmt.Fprintln(stderr, "release canary start:", err)
return 2
}
if err := startCanary(context.Background(), dashboard, source, *version, *force, out); err != nil {
fmt.Fprintln(stderr, "release canary start:", err)
return 1
}
return 0
}
// startCanary points the canary at next — clearing any rollback pin — and
// installs version on it. A release published while the canary still judges a
// newer one queues behind it, so the running window isn't restarted under a
// different release.
func startCanary(ctx context.Context, dashboard releaseupdate.CanaryDashboard, source distribution.Source, version string, force bool, out io.Writer) error {
// The channel is read before following next: a canary already on next is
// judging the release it runs, and a new release queues behind that one.
channel, _, err := dashboard.Channel(ctx)
if err != nil {
return err
}
if err := dashboard.Follow(ctx, "next", ""); err != nil {
return fmt.Errorf("point the canary at next: %w", err)
}
if !force {
queued, err := queuedBehind(ctx, dashboard, source, channel, version, out)
if err != nil {
return err
}
if queued {
return nil
}
}
if err := dashboard.Install(ctx, version, canaryInstallWait); err != nil {
return err
}
fmt.Fprintf(out, "the canary follows next and runs %s\n", version)
return nil
}
// queuedBehind reports whether version must wait: the canary already follows
// next and is still judging the release it runs (a judgeable one), which is a
// different release from version.
func queuedBehind(ctx context.Context, dashboard releaseupdate.CanaryDashboard, source distribution.Source, channel, version string, out io.Writer) (bool, error) {
if channel != "next" {
return false, nil
}
run, err := readCanaryRun(ctx, dashboard, source)
if err != nil {
return false, err
}
if run.version == version {
return false, nil
}
channels, err := fetchChannels(ctx, source)
if err != nil {
return false, err
}
if judgeable(run, channels) {
fmt.Fprintf(out, "%s is queued behind %s, which the canary is still judging\n", version, run.version)
return true, nil
}
return false, nil
}
func runCanaryCheck(args []string, out, stderr io.Writer) int {
flags := flag.NewFlagSet("release canary check", flag.ContinueOnError)
flags.SetOutput(stderr)
dashboardURL, tokenEnv := addCanaryFlags(flags)
reference, key := addSourceFlags(flags)
latest := flags.String("latest", "", "comma-separated image repositories whose v<version> tag becomes latest on promotion")
checkout := flags.String("checkout", ".", "repository checkout with release tags, to see whether a release migrated the database")
window := flags.Duration("window", releaseupdate.CanaryWindow, "window the installed release is judged over, counted from its install")
if err := flags.Parse(args); err != nil {
return 2
}
dashboard, err := canaryDashboard(*dashboardURL, *tokenEnv)
if err != nil {
fmt.Fprintln(stderr, "release canary check:", err)
return 2
}
source, err := canarySource(*reference, *key)
if err != nil {
fmt.Fprintln(stderr, "release canary check:", err)
return 2
}
code, err := checkCanary(context.Background(), dashboard, source, splitList(*latest), *checkout, *window, out)
if err != nil {
fmt.Fprintln(stderr, "release canary check:", err)
return 1
}
return code
}
// checkCanary judges the release the canary actually runs — not the moving
// next tag — and acts on the verdict. The exit code is 1 when the run should
// show red: a rollback happened, or the canary is held on a pin that can no
// longer be installed.
func checkCanary(ctx context.Context, dashboard releaseupdate.CanaryDashboard, source distribution.Source, latest []string, checkout string, window time.Duration, out io.Writer) (int, error) {
channels, err := fetchChannels(ctx, source)
if err != nil {
return 1, err
}
channel, pin, err := dashboard.Channel(ctx)
if err != nil {
return 1, err
}
if channel != "next" {
return heldCanary(ctx, source, channel, pin, out)
}
run, err := readCanaryRun(ctx, dashboard, source)
if err != nil {
return 1, err
}
if !judgeable(run, channels) {
fmt.Fprintf(out, "nothing to judge: the canary runs %s\n", displayVersion(run.version))
return pickupNext(ctx, dashboard, channels, run.version, out)
}
return judgeInstalled(ctx, dashboard, source, run, channels, latest, checkout, window, out)
}
// judgeable reports whether the installed release is one the canary judges: a
// published, non-prerelease release issued after stable's.
func judgeable(run canaryRun, channels publishedChannels) bool {
if !run.found || run.release.Statement.Prerelease {
return false
}
return !channels.hasStable || newerIssued(run.release, channels.stable)
}
func judgeInstalled(ctx context.Context, dashboard releaseupdate.CanaryDashboard, source distribution.Source, run canaryRun, channels publishedChannels, latest []string, checkout string, window time.Duration, out io.Writer) (int, error) {
now := time.Now().UTC()
signals, err := dashboard.Signals(ctx, run.installedAt, now, window)
if err != nil {
return 1, fmt.Errorf("read canary signals: %w", err)
}
verdict := releaseupdate.JudgeCanary(run.version, run.installedAt, now, window, signals)
printVerdict(out, run.version, verdict)
switch verdict.Decision {
case releaseupdate.CanaryPromote:
return 0, promoteJudged(ctx, dashboard, source, run, channels, latest, out)
case releaseupdate.CanaryRollBack:
return 1, rollBackCanary(ctx, dashboard, source, run, channels, checkout, out)
default:
return 0, nil
}
}
// promoteJudged moves stable to the judged release, then picks up a newer next
// queued behind it.
func promoteJudged(ctx context.Context, dashboard releaseupdate.CanaryDashboard, source distribution.Source, run canaryRun, channels publishedChannels, latest []string, out io.Writer) error {
if _, err := releaseupdate.Promote(ctx, source.Reference, source.Key, run.version, latest); err != nil {
return fmt.Errorf("promote %s: %w", run.version, err)
}
fmt.Fprintf(out, "stable is %s\n", run.version)
fresh := channels
fresh.stable, fresh.hasStable = run.release, true
_, err := pickupNext(ctx, dashboard, fresh, run.version, out)
return err
}
// pickupNext installs the next release when it is newer than stable and the
// canary doesn't already run it.
func pickupNext(ctx context.Context, dashboard releaseupdate.CanaryDashboard, channels publishedChannels, installed string, out io.Writer) (int, error) {
if !channels.hasNext || channels.next.Statement.Version == installed {
return 0, nil
}
if channels.hasStable && !newerIssued(channels.next, channels.stable) {
return 0, nil
}
if err := dashboard.Install(ctx, channels.next.Statement.Version, canaryInstallWait); err != nil {
return 1, err
}
fmt.Fprintf(out, "the canary picks up %s from next\n", channels.next.Statement.Version)
return 0, nil
}
// heldCanary reports a canary that isn't following next. Its pin must still be
// installable: a stranded pin fails the run instead of looking green.
func heldCanary(ctx context.Context, source distribution.Source, channel, pin string, out io.Writer) (int, error) {
if pin != "" {
err := checkPin(ctx, source, pin)
if errors.Is(err, errStrandedPin) {
fmt.Fprintf(out, "ERROR: the canary is held on %s %s, which is stranded: %v; unpin it or republish the release\n", channel, pin, err)
return 1, nil
}
if err != nil {
return 1, err
}
fmt.Fprintf(out, "held on %s %s, nothing to judge\n", channel, pin)
return 0, nil
}
fmt.Fprintf(out, "held on %s, nothing to judge\n", channel)
return 0, nil
}
// errStrandedPin marks a held pin that can no longer be installed.
var errStrandedPin = errors.New("pin is stranded")
// checkPin fails when a held pin can no longer be installed: the release is
// gone from the source, or its artifacts are.
func checkPin(ctx context.Context, source distribution.Source, pin string) error {
release, found, err := fetchChannel(ctx, source, releaseupdate.Tag("exact-pin", pin))
if err != nil {
return err
}
if !found {
return fmt.Errorf("%w: no release %s in the source", errStrandedPin, releaseupdate.Tag("exact-pin", pin))
}
if err := releaseupdate.Installable(ctx, source, release); err != nil {
return fmt.Errorf("%w: %w", errStrandedPin, err)
}
return nil
}
// rollBackCanary returns the canary to stable after a failed verdict. stable
// is checked installable before the canary is pinned to it: a pin whose
// artifacts are gone would strand the canary, so the run fails loudly and the
// canary stays on next. A release that changed the migrations is held on the
// failed version instead, because going back past a migration needs the
// pre-update database snapshot.
func rollBackCanary(ctx context.Context, dashboard releaseupdate.CanaryDashboard, source distribution.Source, run canaryRun, channels publishedChannels, checkout string, out io.Writer) error {
failed := run.version
if !channels.hasStable {
return holdCanary(ctx, dashboard, failed, "there's no stable release to return to", out)
}
stable := channels.stable
if err := releaseupdate.Installable(ctx, source, stable); err != nil {
return fmt.Errorf("stable %s is not installable, the canary stays on next: %w", stable.Statement.Version, err)
}
if migrated(ctx, checkout, stable.Statement.Version, failed) {
return holdCanary(ctx, dashboard, failed, failed+" changed the database schema", out)
}
if err := dashboard.Follow(ctx, "exact-pin", stable.Statement.Version); err != nil {
return fmt.Errorf("pin the canary to %s: %w", stable.Statement.Version, err)
}
if err := dashboard.Install(ctx, stable.Statement.Version, canaryInstallWait); err != nil {
return fmt.Errorf("return the canary to %s: %w", stable.Statement.Version, err)
}
fmt.Fprintf(out, "the canary is back on %s and stable is unchanged\n", stable.Statement.Version)
return nil
}
func holdCanary(ctx context.Context, dashboard releaseupdate.CanaryDashboard, version, why string, out io.Writer) error {
if err := dashboard.Follow(ctx, "exact-pin", version); err != nil {
return fmt.Errorf("hold the canary on %s: %w", version, err)
}
fmt.Fprintf(out, "%s: the canary is held on %s and stable is unchanged; fix forward\n", why, version)
return nil
}
// migrated reports whether the migrations differ between two release tags.
// If it can't tell, it says they differ, so the canary is held rather than
// rolled back.
func migrated(ctx context.Context, checkout, from, to string) bool {
err := exec.CommandContext(ctx, "git", "-C", checkout, "diff", "--quiet", "v"+from, "v"+to, "--", migrationsDir).Run()
return err != nil
}
func runCanaryPromote(args []string, out, stderr io.Writer) int {
flags := flag.NewFlagSet("release canary promote", flag.ContinueOnError)
flags.SetOutput(stderr)
dashboardURL, tokenEnv := addCanaryFlags(flags)
reference, key := addSourceFlags(flags)
latest := flags.String("latest", "", "comma-separated image repositories whose v<version> tag becomes latest on promotion")
version := flags.String("version", "", "release expected on the canary; must match what it runs")
window := flags.Duration("window", releaseupdate.CanaryWindow, "window the signals are read over, counted from the install")
if err := flags.Parse(args); err != nil {
return 2
}
dashboard, err := canaryDashboard(*dashboardURL, *tokenEnv)
if err != nil {
fmt.Fprintln(stderr, "release canary promote:", err)
return 2
}
source, err := canarySource(*reference, *key)
if err != nil {
fmt.Fprintln(stderr, "release canary promote:", err)
return 2
}
code, err := promoteCanaryNow(context.Background(), dashboard, source, *version, splitList(*latest), *window, out)
if err != nil {
fmt.Fprintln(stderr, "release canary promote:", err)
return 1
}
return code
}
// promoteCanaryNow promotes the release the canary runs right now, without
// waiting out the window: the signals must show no hard failure and no
// relative breach. It refuses, with exit 1, when anything fails.
func promoteCanaryNow(ctx context.Context, dashboard releaseupdate.CanaryDashboard, source distribution.Source, expectVersion string, latest []string, window time.Duration, out io.Writer) (int, error) {
run, err := readCanaryRun(ctx, dashboard, source)
if err != nil {
return 1, err
}
channel, pin, err := dashboard.Channel(ctx)
if err != nil {
return 1, err
}
fmt.Fprintf(out, "channel: %s\n", channelDisplay(channel, pin))
if expectVersion != "" && expectVersion != run.version {
return 1, fmt.Errorf("the canary runs %s, not %s", displayVersion(run.version), expectVersion)
}
if !run.found {
return 1, fmt.Errorf("no release %s in the source to promote", displayVersion(run.version))
}
now := time.Now().UTC()
signals, err := dashboard.Signals(ctx, run.installedAt, now, window)
if err != nil {
return 1, fmt.Errorf("read canary signals: %w", err)
}
verdict := releaseupdate.JudgeCanaryNow(run.version, now, window, signals)
printVerdict(out, run.version, verdict)
if verdict.Decision == releaseupdate.CanaryRollBack {
fmt.Fprintln(out, "refusing to promote: the canary's signals fail")
return 1, nil
}
if _, err := releaseupdate.Promote(ctx, source.Reference, source.Key, run.version, latest); err != nil {
return 1, fmt.Errorf("promote %s: %w", run.version, err)
}
fmt.Fprintf(out, "stable is %s\n", run.version)
return 0, nil
}
func runCanaryUnpin(args []string, out, stderr io.Writer) int {
flags := flag.NewFlagSet("release canary unpin", flag.ContinueOnError)
flags.SetOutput(stderr)
dashboardURL, tokenEnv := addCanaryFlags(flags)
if err := flags.Parse(args); err != nil {
return 2
}
dashboard, err := canaryDashboard(*dashboardURL, *tokenEnv)
if err != nil {
fmt.Fprintln(stderr, "release canary unpin:", err)
return 2
}
if err := dashboard.Follow(context.Background(), "next", ""); err != nil {
fmt.Fprintln(stderr, "release canary unpin:", err)
return 1
}
fmt.Fprintln(out, "the canary follows next again, with no pin")
return 0
}
+2 -2
View File
@@ -7,7 +7,7 @@
// because archie-messaging, a separate process that cannot import this
// package, is the next process that must be able to announce itself.
// What stays here is what is the daemon's own: which
// marker counts as progress (daemon.LastPollAt) and how long the loop may go
// marker counts as progress (daemon.StalestLoop) and how long a loop may go
// without beginning a poll pass.
//
// Nothing here is fatal -- a process with no NOTIFY_SOCKET (a plain terminal,
@@ -53,7 +53,7 @@ func (b *boot) startWatchdog(ctx context.Context) {
sampler := sdnotify.Watchdog{
Interval: interval,
Cadence: b.loopCadence,
Progress: b.d.LastPollAt,
Progress: b.d.StalestLoop,
Beat: func() { n.Send(sdnotify.WatchdogState) },
Log: b.log,
}
+1 -4
View File
@@ -295,10 +295,7 @@ func hasConfigSection(document map[string]any, section string) bool {
}
func updateStateDir() string {
if dir := os.Getenv("ARCHIE_UPDATE_STATE_DIR"); dir != "" {
return dir
}
return configuration.DefaultUpdateStateDir()
return configuration.UpdateStateDir()
}
// resumeUpdate hands an interrupted transaction to the frozen watchdog and
+39 -1
View File
@@ -12,7 +12,9 @@ import (
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/curator"
"github.com/samcharles93/archie-core/internal/domain/org"
"github.com/samcharles93/archie-core/internal/domain/usage"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanestore"
@@ -40,7 +42,7 @@ func (b *server) setupCurators(ctx context.Context) bool {
MemoryEngines: b.memEngines,
Skills: skillcurator.NewStore(skillsRoot),
Conversations: sessioncurator.NewAdapter(b.chatSessionStore, b.cfg.BotUser),
LLM: curatorLLMRunner{llm: b.chatLLM, outcomes: b.providerOutcomes, usage: b.usage},
LLM: actingPrincipalRunner{inner: curatorLLMRunner{llm: b.chatLLM, outcomes: b.providerOutcomes, usage: b.usage}, resolve: b.actingPrincipal},
Model: model,
})
@@ -95,6 +97,42 @@ func (s curatorEventSink) Emit(kind, detail string, data map[string]any) {
s.b.Publish(context.Background(), events.Event{Kind: kind, Detail: detail, Data: data})
}
// actingPrincipal resolves the identity the instance acts as for automated
// work (the default chat identity, falling back to the bot user), so curator
// passes authorize guarded tools exactly as that identity would.
func (b *server) actingPrincipal(ctx context.Context) (access.Principal, error) {
name := b.defaultChatIdentity
if name == "" {
name = b.cfg.BotUser
}
if name == "" {
return access.Principal{}, fmt.Errorf("no acting identity configured (chat identity or bot user)")
}
idn, err := b.stateStore.ResolveLegacyName(ctx, name)
if err != nil {
return access.Principal{}, fmt.Errorf("resolve acting identity %q: %w", name, err)
}
return b.stateStore.PrincipalFor(ctx, idn.ID)
}
// actingPrincipalRunner attaches the acting principal to every curator model
// call. Tool handlers run in the call's context, so guarded tools see the
// curator as the instance's identity, never as an anonymous or caller-supplied
// one; a pass that cannot resolve the principal fails closed.
type actingPrincipalRunner struct {
inner curator.LLMRunner
resolve func(context.Context) (access.Principal, error)
}
func (r actingPrincipalRunner) Chat(ctx context.Context, req curator.ChatRequest) (curator.ChatResult, error) {
p, err := r.resolve(ctx)
if err != nil {
return curator.ChatResult{}, fmt.Errorf("curator pass: %w", err)
}
ctx = org.WithScope(org.WithOrg(access.WithPrincipal(ctx, p), p.Org), p.Org)
return r.inner.Chat(ctx, req)
}
// curatorLLMRunner adapts the shared ai-sdk runtime to the curator
// family's narrow LLMRunner contract: one model reference, plain
// messages, and the declared tool set -- no streaming. Tools are built
+3
View File
@@ -109,6 +109,9 @@ func (b *server) registerWorkflowTools(ctx context.Context) {
authz := infraaccess.StartLive(ctx, b.stateStore.ListPolicies, b.stateStore.PoliciesVersion, b.log)
workflows := controlplanerpc.NewWorkflows(b.stateStore.ControlPlane())
entries := append(gateway.WorkflowTools(workflows, authz), gateway.IntegrationTools(integrationSource{Workflows: workflows, store: b.stateStore}, authz)...)
// Run-inspection tools back long-running curators (self-intake first);
// the State Store client satisfies the source directly.
entries = append(entries, gateway.RunTools(b.stateStore, authz)...)
for _, entry := range entries {
if err := b.toolReg.Register(entry); err != nil {
b.log.WarnContext(ctx, "workflow tool registration failed", "tool", entry.Name, "err", err)
File diff suppressed because it is too large. Load diff
@@ -23,6 +23,7 @@ const (
ForgeService_AssignedIssues_FullMethodName = "/forge.v1.ForgeService/AssignedIssues"
ForgeService_IssuesWithLabel_FullMethodName = "/forge.v1.ForgeService/IssuesWithLabel"
ForgeService_Comment_FullMethodName = "/forge.v1.ForgeService/Comment"
ForgeService_CreateIssue_FullMethodName = "/forge.v1.ForgeService/CreateIssue"
ForgeService_CloseIssue_FullMethodName = "/forge.v1.ForgeService/CloseIssue"
ForgeService_React_FullMethodName = "/forge.v1.ForgeService/React"
ForgeService_SetStateLabel_FullMethodName = "/forge.v1.ForgeService/SetStateLabel"
@@ -60,6 +61,7 @@ type ForgeServiceClient interface {
AssignedIssues(ctx context.Context, in *AssignedIssuesRequest, opts ...grpc.CallOption) (*AssignedIssuesResponse, error)
IssuesWithLabel(ctx context.Context, in *IssuesWithLabelRequest, opts ...grpc.CallOption) (*IssuesWithLabelResponse, error)
Comment(ctx context.Context, in *CommentRequest, opts ...grpc.CallOption) (*CommentResponse, error)
CreateIssue(ctx context.Context, in *CreateIssueRequest, opts ...grpc.CallOption) (*CreateIssueResponse, error)
CloseIssue(ctx context.Context, in *CloseIssueRequest, opts ...grpc.CallOption) (*CloseIssueResponse, error)
React(ctx context.Context, in *ReactRequest, opts ...grpc.CallOption) (*ReactResponse, error)
SetStateLabel(ctx context.Context, in *SetStateLabelRequest, opts ...grpc.CallOption) (*SetStateLabelResponse, error)
@@ -132,6 +134,16 @@ func (c *forgeServiceClient) Comment(ctx context.Context, in *CommentRequest, op
return out, nil
}
func (c *forgeServiceClient) CreateIssue(ctx context.Context, in *CreateIssueRequest, opts ...grpc.CallOption) (*CreateIssueResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(CreateIssueResponse)
err := c.cc.Invoke(ctx, ForgeService_CreateIssue_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *forgeServiceClient) CloseIssue(ctx context.Context, in *CloseIssueRequest, opts ...grpc.CallOption) (*CloseIssueResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(CloseIssueResponse)
@@ -338,6 +350,7 @@ type ForgeServiceServer interface {
AssignedIssues(context.Context, *AssignedIssuesRequest) (*AssignedIssuesResponse, error)
IssuesWithLabel(context.Context, *IssuesWithLabelRequest) (*IssuesWithLabelResponse, error)
Comment(context.Context, *CommentRequest) (*CommentResponse, error)
CreateIssue(context.Context, *CreateIssueRequest) (*CreateIssueResponse, error)
CloseIssue(context.Context, *CloseIssueRequest) (*CloseIssueResponse, error)
React(context.Context, *ReactRequest) (*ReactResponse, error)
SetStateLabel(context.Context, *SetStateLabelRequest) (*SetStateLabelResponse, error)
@@ -382,6 +395,9 @@ func (UnimplementedForgeServiceServer) IssuesWithLabel(context.Context, *IssuesW
func (UnimplementedForgeServiceServer) Comment(context.Context, *CommentRequest) (*CommentResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Comment not implemented")
}
func (UnimplementedForgeServiceServer) CreateIssue(context.Context, *CreateIssueRequest) (*CreateIssueResponse, error) {
return nil, status.Error(codes.Unimplemented, "method CreateIssue not implemented")
}
func (UnimplementedForgeServiceServer) CloseIssue(context.Context, *CloseIssueRequest) (*CloseIssueResponse, error) {
return nil, status.Error(codes.Unimplemented, "method CloseIssue not implemented")
}
@@ -529,6 +545,24 @@ func _ForgeService_Comment_Handler(srv interface{}, ctx context.Context, dec fun
return interceptor(ctx, in, info, handler)
}
func _ForgeService_CreateIssue_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(CreateIssueRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ForgeServiceServer).CreateIssue(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: ForgeService_CreateIssue_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ForgeServiceServer).CreateIssue(ctx, req.(*CreateIssueRequest))
}
return interceptor(ctx, in, info, handler)
}
func _ForgeService_CloseIssue_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(CloseIssueRequest)
if err := dec(in); err != nil {
@@ -869,6 +903,10 @@ var ForgeService_ServiceDesc = grpc.ServiceDesc{
MethodName: "Comment",
Handler: _ForgeService_Comment_Handler,
},
{
MethodName: "CreateIssue",
Handler: _ForgeService_CreateIssue_Handler,
},
{
MethodName: "CloseIssue",
Handler: _ForgeService_CloseIssue_Handler,
File diff suppressed because it is too large. Load diff
@@ -38,6 +38,7 @@ const (
AccountService_AddPersonalToken_FullMethodName = "/state.v1.AccountService/AddPersonalToken"
AccountService_ListPersonalTokens_FullMethodName = "/state.v1.AccountService/ListPersonalTokens"
AccountService_RevokePersonalToken_FullMethodName = "/state.v1.AccountService/RevokePersonalToken"
AccountService_BindIdentityPersonalToken_FullMethodName = "/state.v1.AccountService/BindIdentityPersonalToken"
)
// AccountServiceClient is the client API for AccountService service.
@@ -65,6 +66,12 @@ type AccountServiceClient interface {
AddPersonalToken(ctx context.Context, in *AddPersonalTokenRequest, opts ...grpc.CallOption) (*AddPersonalTokenResponse, error)
ListPersonalTokens(ctx context.Context, in *ListPersonalTokensRequest, opts ...grpc.CallOption) (*ListPersonalTokensResponse, error)
RevokePersonalToken(ctx context.Context, in *RevokePersonalTokenRequest, opts ...grpc.CallOption) (*RevokePersonalTokenResponse, error)
// BindIdentityPersonalToken binds a personal token to a NAMED identity
// instead of the caller's principal, which is how a service account gets
// its bearer credential. Only non-user targets are accepted: a person's
// token comes from AddPersonalToken. The audit actor is the caller's
// principal, never a request field.
BindIdentityPersonalToken(ctx context.Context, in *BindIdentityPersonalTokenRequest, opts ...grpc.CallOption) (*BindIdentityPersonalTokenResponse, error)
}
type accountServiceClient struct {
@@ -265,6 +272,16 @@ func (c *accountServiceClient) RevokePersonalToken(ctx context.Context, in *Revo
return out, nil
}
func (c *accountServiceClient) BindIdentityPersonalToken(ctx context.Context, in *BindIdentityPersonalTokenRequest, opts ...grpc.CallOption) (*BindIdentityPersonalTokenResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(BindIdentityPersonalTokenResponse)
err := c.cc.Invoke(ctx, AccountService_BindIdentityPersonalToken_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
// AccountServiceServer is the server API for AccountService service.
// All implementations must embed UnimplementedAccountServiceServer
// for forward compatibility.
@@ -290,6 +307,12 @@ type AccountServiceServer interface {
AddPersonalToken(context.Context, *AddPersonalTokenRequest) (*AddPersonalTokenResponse, error)
ListPersonalTokens(context.Context, *ListPersonalTokensRequest) (*ListPersonalTokensResponse, error)
RevokePersonalToken(context.Context, *RevokePersonalTokenRequest) (*RevokePersonalTokenResponse, error)
// BindIdentityPersonalToken binds a personal token to a NAMED identity
// instead of the caller's principal, which is how a service account gets
// its bearer credential. Only non-user targets are accepted: a person's
// token comes from AddPersonalToken. The audit actor is the caller's
// principal, never a request field.
BindIdentityPersonalToken(context.Context, *BindIdentityPersonalTokenRequest) (*BindIdentityPersonalTokenResponse, error)
mustEmbedUnimplementedAccountServiceServer()
}
@@ -357,6 +380,9 @@ func (UnimplementedAccountServiceServer) ListPersonalTokens(context.Context, *Li
func (UnimplementedAccountServiceServer) RevokePersonalToken(context.Context, *RevokePersonalTokenRequest) (*RevokePersonalTokenResponse, error) {
return nil, status.Error(codes.Unimplemented, "method RevokePersonalToken not implemented")
}
func (UnimplementedAccountServiceServer) BindIdentityPersonalToken(context.Context, *BindIdentityPersonalTokenRequest) (*BindIdentityPersonalTokenResponse, error) {
return nil, status.Error(codes.Unimplemented, "method BindIdentityPersonalToken not implemented")
}
func (UnimplementedAccountServiceServer) mustEmbedUnimplementedAccountServiceServer() {}
func (UnimplementedAccountServiceServer) testEmbeddedByValue() {}
@@ -720,6 +746,24 @@ func _AccountService_RevokePersonalToken_Handler(srv interface{}, ctx context.Co
return interceptor(ctx, in, info, handler)
}
func _AccountService_BindIdentityPersonalToken_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(BindIdentityPersonalTokenRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(AccountServiceServer).BindIdentityPersonalToken(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: AccountService_BindIdentityPersonalToken_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(AccountServiceServer).BindIdentityPersonalToken(ctx, req.(*BindIdentityPersonalTokenRequest))
}
return interceptor(ctx, in, info, handler)
}
// AccountService_ServiceDesc is the grpc.ServiceDesc for AccountService service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
@@ -803,6 +847,10 @@ var AccountService_ServiceDesc = grpc.ServiceDesc{
MethodName: "RevokePersonalToken",
Handler: _AccountService_RevokePersonalToken_Handler,
},
{
MethodName: "BindIdentityPersonalToken",
Handler: _AccountService_BindIdentityPersonalToken_Handler,
},
},
Streams: []grpc.StreamDesc{},
Metadata: "state/v1/account.proto",
+26 -4
View File
@@ -211,8 +211,12 @@ func (x *EventsSinceResponse) GetEvents() []*Event {
}
type TaskEventsRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
TaskId int64 `protobuf:"varint,1,opt,name=task_id,json=taskId,proto3" json:"task_id,omitempty"`
state protoimpl.MessageState `protogen:"open.v1"`
TaskId int64 `protobuf:"varint,1,opt,name=task_id,json=taskId,proto3" json:"task_id,omitempty"`
// transcript selects the agent_transcript entries of one attempt instead of
// the timeline, which leaves them out.
Transcript bool `protobuf:"varint,2,opt,name=transcript,proto3" json:"transcript,omitempty"`
Attempt int64 `protobuf:"varint,3,opt,name=attempt,proto3" json:"attempt,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -254,6 +258,20 @@ func (x *TaskEventsRequest) GetTaskId() int64 {
return 0
}
func (x *TaskEventsRequest) GetTranscript() bool {
if x != nil {
return x.Transcript
}
return false
}
func (x *TaskEventsRequest) GetAttempt() int64 {
if x != nil {
return x.Attempt
}
return 0
}
type TaskEventsResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
Events []*Event `protobuf:"bytes,1,rep,name=events,proto3" json:"events,omitempty"`
@@ -311,9 +329,13 @@ const file_state_v1_task_event_proto_rawDesc = "" +
"\bsince_id\x18\x01 \x01(\tR\asinceId\x12\x14\n" +
"\x05limit\x18\x02 \x01(\x03R\x05limit\">\n" +
"\x13EventsSinceResponse\x12'\n" +
"\x06events\x18\x01 \x03(\v2\x0f.state.v1.EventR\x06events\",\n" +
"\x06events\x18\x01 \x03(\v2\x0f.state.v1.EventR\x06events\"f\n" +
"\x11TaskEventsRequest\x12\x17\n" +
"\atask_id\x18\x01 \x01(\x03R\x06taskId\"=\n" +
"\atask_id\x18\x01 \x01(\x03R\x06taskId\x12\x1e\n" +
"\n" +
"transcript\x18\x02 \x01(\bR\n" +
"transcript\x12\x18\n" +
"\aattempt\x18\x03 \x01(\x03R\aattempt\"=\n" +
"\x12TaskEventsResponse\x12'\n" +
"\x06events\x18\x01 \x03(\v2\x0f.state.v1.EventR\x06events2\xf3\x01\n" +
"\x10TaskEventService\x12J\n" +
+16
View File
@@ -0,0 +1,16 @@
package daemon
import (
"context"
"testing"
)
// The systemd watchdog beats only while StalestLoop is fresh, so a cycle that
// doesn't stamp it gets archied killed shortly after every start.
func TestCycleMarksPass(t *testing.T) {
d := &Daemon{}
d.Cycle(context.Background())
if loop, at := d.StalestLoop(); loop != "cycle" || at.IsZero() {
t.Fatalf("StalestLoop() = %q, %v; want a cycle pass", loop, at)
}
}
+20 -17
View File
@@ -240,10 +240,10 @@ type Daemon struct {
// to use.
running runningTasks
// lastPollAt maps each poll loop (an identity ID, or "" for the root
// loop) to when its latest pass began, as Unix nanoseconds. LastPollAt
// reports the oldest, so one wedged loop is not hidden by a healthy one.
lastPollAt sync.Map
// loopPasses maps each run loop ("cycle", "drain") to when its latest
// pass began, as Unix nanoseconds. StalestLoop reports the oldest, so one
// wedged loop isn't hidden by a healthy one.
loopPasses sync.Map
}
// IssueSource is one forge connection's repositories as the identity
@@ -389,6 +389,7 @@ func (d *Daemon) drainLoop(ctx context.Context) {
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
d.markPass("drain")
d.drainNATS(ctx)
select {
case <-ctx.Done():
@@ -489,6 +490,7 @@ func (d *Daemon) RunOnce(ctx context.Context) {
// tasks; maintenance runs as built-in schedules. Draining queued tasks is the caller's job
// (drainLoop in Run, drainNATS in RunOnce) so there is one drainer.
func (d *Daemon) Cycle(ctx context.Context) {
d.markPass("cycle")
d.dispatchBindings(ctx)
d.drainReactions(ctx)
}
@@ -1205,7 +1207,6 @@ func (d *taskDispatcher) Wait() {
// afresh each cycle so a rotated credential or a new connection applies to
// the next poll.
func (d *Daemon) poll(ctx context.Context) error {
d.markPoll("")
// Webhook intake replaces polling; an empty pass is not a failure.
if d.Cfg.Get().Forge.Intake == config.ForgeIntakeWebhook {
return nil
@@ -2603,26 +2604,28 @@ func (d *Daemon) allowConcurrentForTask(task *workflow.Task) bool {
return ok && repo.AllowConcurrent
}
// LastPollAt reports when the least recently polled loop last began a pass,
// or the zero time when no pass has started yet.
func (d *Daemon) LastPollAt() time.Time {
// StalestLoop reports the run loop that least recently began a pass and when,
// or the zero time when no loop has started yet. The systemd watchdog beats
// only while this is fresh, so every loop Run starts must mark its passes.
func (d *Daemon) StalestLoop() (string, time.Time) {
var loop string
var oldest int64
d.lastPollAt.Range(func(_, v any) bool {
if ns, ok := v.(int64); ok && (oldest == 0 || ns < oldest) {
oldest = ns
d.loopPasses.Range(func(k, v any) bool {
name, okName := k.(string)
if ns, ok := v.(int64); ok && okName && (oldest == 0 || ns < oldest) {
loop, oldest = name, ns
}
return true
})
if oldest == 0 {
return time.Time{}
return "", time.Time{}
}
return time.Unix(0, oldest).UTC()
return loop, time.Unix(0, oldest).UTC()
}
// markPoll stamps the start of a poll pass for one loop, so a hung pass shows
// as stale.
func (d *Daemon) markPoll(loop string) {
d.lastPollAt.Store(loop, time.Now().UnixNano())
// markPass stamps the start of one loop's pass, so a hung pass shows as stale.
func (d *Daemon) markPass(loop string) {
d.loopPasses.Store(loop, time.Now().UnixNano())
}
// errContainerExited is the cancellation cause when a task's agent container
+59
View File
@@ -0,0 +1,59 @@
package access
import "github.com/samcharles93/archie-core/internal/domain/org"
// CanaryGrantID is the ID the canary's scoped grant is known by in the policy
// store and in denial records. A stored policy must not reuse it: the store's
// seeding and the reset command overwrite shipped IDs, and this one is
// installed by the operator from CanaryGrantPolicy's text.
const CanaryGrantID = "org-canary-grant"
// CanaryGrantForbidID is the matching forbid's ID: the grant only scopes the
// canary if everything else it could reach is forbidden, because the identity
// may also carry a role (API-created service accounts are assigned developer)
// whose shipped policy permits broadly within the org. A forbid wins within
// the level, so the pair scopes the identity regardless of its role.
const CanaryGrantForbidID = "org-canary-grant-forbid"
// canaryGrantActions is the exact action surface the canary needs, as one
// Cedar condition shared by the grant's permit and its forbid.
func canaryGrantActions() string {
return `(
(action == Archie::Action::"read" && ["dashboard", "task"].contains(resource.kind)) ||
(action == Archie::Action::"read" && resource == Archie::Object::"update-settings") ||
(action == Archie::Action::"update" && resource == Archie::Object::"update") ||
(action == Archie::Action::"manage_identities" && resource == Archie::Object::"update-settings")
)`
}
func canaryPrincipal(identityID string) string {
return `principal == Archie::Identity::"` + identityID + `"`
}
// CanaryGrantPolicy is the grant a CI canary carries instead of instance
// admin: exactly one named identity may read the dashboard's signal surface
// (chat update, alerts, logs, tasks), read and write the update-settings
// resource, and trigger an update install or deferral. Install it together
// with CanaryGrantForbidPolicy, which forbids the same identity from
// everything else. Every clause is guarded by the resource belonging to the
// principal's own org, which the engine enforces structurally anyway.
func CanaryGrantPolicy(orgID org.OrgID, identityID string) Policy {
text := `permit(principal, action, resource) when {
resource.org == principal.org &&
` + canaryPrincipal(identityID) + ` &&
` + canaryGrantActions() + `
};`
return Policy{ID: CanaryGrantID, Level: LevelOrg, OrgID: orgID, Text: text}
}
// CanaryGrantForbidPolicy forbids the canary identity from everything outside
// CanaryGrantPolicy's surface, so a role assigned to the identity (service
// accounts created through the API carry developer) cannot widen it.
func CanaryGrantForbidPolicy(orgID org.OrgID, identityID string) Policy {
text := `forbid(principal, action, resource) when {
resource.org == principal.org &&
` + canaryPrincipal(identityID) + ` &&
!` + canaryGrantActions() + `
};`
return Policy{ID: CanaryGrantForbidID, Level: LevelOrg, OrgID: orgID, Text: text}
}
+54
View File
@@ -0,0 +1,54 @@
package access
import (
"strings"
"testing"
"github.com/samcharles93/archie-core/internal/domain/org"
)
// TestCanaryGrantPolicy pins the shape of the canary grant: it names exactly
// the one identity it is built for, scoped to the org it serves, so a token
// carrying it cannot act as anyone else or reach another org.
func TestCanaryGrantPolicy(t *testing.T) {
p := CanaryGrantPolicy(org.DefaultOrgID, "canary-svc")
if err := p.Validate(); err != nil {
t.Fatalf("Validate: %v", err)
}
if p.ID != CanaryGrantID || p.Level != LevelOrg || p.OrgID != org.DefaultOrgID {
t.Fatalf("policy = %+v, want ID %q at org level in %q", p, CanaryGrantID, org.DefaultOrgID)
}
if !strings.Contains(p.Text, `principal == Archie::Identity::"canary-svc"`) {
t.Fatalf("text does not name the canary identity:\n%s", p.Text)
}
other := CanaryGrantPolicy(org.DefaultOrgID, "another")
if strings.Contains(other.Text, "canary-svc") || !strings.Contains(other.Text, `Archie::Identity::"another"`) {
t.Fatalf("grant for another identity must name that identity only:\n%s", other.Text)
}
}
// TestCanaryGrantForbidPolicy pins the forbid that keeps a role assigned to
// the canary identity from widening the grant: it names the same identity and
// negates exactly the permit's action surface.
func TestCanaryGrantForbidPolicy(t *testing.T) {
permit := CanaryGrantPolicy(org.DefaultOrgID, "canary-svc")
forbid := CanaryGrantForbidPolicy(org.DefaultOrgID, "canary-svc")
if err := forbid.Validate(); err != nil {
t.Fatalf("Validate: %v", err)
}
if forbid.ID != CanaryGrantForbidID || forbid.Level != LevelOrg || forbid.OrgID != org.DefaultOrgID {
t.Fatalf("forbid = %+v, want ID %q at org level in %q", forbid, CanaryGrantForbidID, org.DefaultOrgID)
}
if !strings.HasPrefix(forbid.Text, "forbid(") {
t.Fatalf("forbid text must be a forbid:\n%s", forbid.Text)
}
if !strings.Contains(forbid.Text, `principal == Archie::Identity::"canary-svc"`) {
t.Fatalf("forbid does not name the canary identity:\n%s", forbid.Text)
}
// The forbid must negate the permit's surface, not the permit itself.
permitBody := strings.TrimSuffix(strings.SplitN(permit.Text, "when {", 2)[1], "};")
forbidBody := strings.TrimSuffix(strings.SplitN(forbid.Text, "when {", 2)[1], "};")
if !strings.Contains(forbidBody, "!"+canaryGrantActions()) {
t.Fatalf("forbid does not negate the grant's action surface:\npermit:%s\nforbid:%s", permitBody, forbidBody)
}
}
+68 -6
View File
@@ -4,12 +4,13 @@ import "context"
// Runner executes one autonomous stage against an already prepared workspace.
type Runner interface {
// report, when non-nil, is notified once per completed tool call.
Run(ctx context.Context, workspace string, req Request, report ToolCallReporter) (Result, error)
// report receives what the run does as it happens; its zero value
// discards everything.
Run(ctx context.Context, workspace string, req Request, report Reporter) (Result, error)
}
// ToolCallReport is one completed tool invocation, reported through a
// ToolCallReporter.
// Reporter.
type ToolCallReport struct {
Tool string
// Detail is a clipped one-line summary: the tool's result, or
@@ -18,6 +19,67 @@ type ToolCallReport struct {
Failed bool
}
// ToolCallReporter receives one ToolCallReport per completed tool call. A nil
// reporter is valid and reports nothing.
type ToolCallReporter func(ToolCallReport)
// Reporter receives a run's activity. Either func may be nil.
type Reporter struct {
// ToolCall is notified once per completed tool call.
ToolCall func(ToolCallReport)
// Transcript is notified of each transcript entry, in order.
Transcript func(TranscriptEntry)
}
// ReportToolCall forwards report to r.ToolCall, if set.
func (r Reporter) ReportToolCall(report ToolCallReport) {
if r.ToolCall != nil {
r.ToolCall(report)
}
}
// ReportTranscript forwards entry to r.Transcript, if set.
func (r Reporter) ReportTranscript(entry TranscriptEntry) {
if r.Transcript != nil {
r.Transcript(entry)
}
}
// Transcript entry kinds.
const (
TranscriptSystem = "system" // Text: the system prompt
TranscriptUser = "user" // Text: the mission, or input the loop added later
TranscriptModel = "model" // Reasoning, Text and ToolCalls: one model response
TranscriptTool = "tool" // one tool call's arguments and result
TranscriptCompaction = "compaction" // Text: the summary that replaced the history
)
// TranscriptEntry is one ordered item of an agent run's conversation. Text
// fields are redacted and clipped; a *Bytes field holds the full size when its
// field was clipped.
type TranscriptEntry struct {
Seq int `json:"seq"`
Kind string `json:"kind"`
Text string `json:"text,omitempty"`
TextBytes int `json:"text_bytes,omitempty"`
ToolCalls []TranscriptToolCall `json:"tool_calls,omitempty"`
Reasoning string `json:"reasoning,omitempty"`
ReasoningBytes int `json:"reasoning_bytes,omitempty"`
CallID string `json:"call_id,omitempty"`
Tool string `json:"tool,omitempty"`
Args string `json:"args,omitempty"`
ArgsBytes int `json:"args_bytes,omitempty"`
Output string `json:"output,omitempty"`
OutputBytes int `json:"output_bytes,omitempty"`
Error string `json:"error,omitempty"`
FinishReason string `json:"finish_reason,omitempty"`
Usage *Usage `json:"usage,omitempty"`
LatencyMS int64 `json:"latency_ms,omitempty"`
}
// TranscriptToolCall is one tool call a model response requested.
type TranscriptToolCall struct {
ID string `json:"id,omitempty"`
Name string `json:"name"`
Args string `json:"args,omitempty"`
ArgsBytes int `json:"args_bytes,omitempty"`
}
+13 -1
View File
@@ -13,6 +13,10 @@ import (
// registry's activity surface and events.
const ActionToolCalled = "tool.called"
// ActionSummary is the Action.Type recorded for the pass's final text: the
// report a definition pass produces, kept inspectable in the activity trail.
const ActionSummary = "pass.summary"
// defaultDefinitionMaxSteps bounds the model/tool loop of one definition pass.
const defaultDefinitionMaxSteps = 10
@@ -84,7 +88,7 @@ func (e *DefinitionEngine) Pass(ctx context.Context, in PassInput) (PassResult,
return PassResult{}, err
}
actions := make([]Action, 0, len(res.ToolCalls))
actions := make([]Action, 0, len(res.ToolCalls)+1)
for _, call := range res.ToolCalls {
actions = append(actions, Action{
At: e.host.Clock.Now(),
@@ -93,6 +97,14 @@ func (e *DefinitionEngine) Pass(ctx context.Context, in PassInput) (PassResult,
Reason: in.Reason,
})
}
if summary := strings.TrimSpace(res.Text); summary != "" {
actions = append(actions, Action{
At: e.host.Clock.Now(),
Type: ActionSummary,
Detail: summary,
Reason: in.Reason,
})
}
return PassResult{Actions: actions}, nil
}
@@ -139,3 +139,31 @@ func TestInstructionOnlyRegistrarViewFiltered(t *testing.T) {
t.Error("registrar view has no clock; time is always bound")
}
}
// TestDefinitionPassRecordsSummary pins that a pass's final text is recorded
// as a pass.summary action, so a definition's report stays inspectable in the
// activity trail instead of being dropped when the model returns.
func TestDefinitionPassRecordsSummary(t *testing.T) {
llm := &fakeLLM{}
registry := NewRegistry(fullHost(llm))
engine := NewDefinitionEngine(instructionOnlyDefinition())
if err := registry.Register(engine); err != nil {
t.Fatal(err)
}
res, err := engine.Pass(t.Context(), PassInput{Reason: "test"})
if err != nil {
t.Fatalf("pass: %v", err)
}
var summary *Action
for i, a := range res.Actions {
if a.Type == ActionSummary {
summary = &res.Actions[i]
}
}
if summary == nil {
t.Fatalf("no %s action recorded; actions: %+v", ActionSummary, res.Actions)
}
if summary.Detail != "done" {
t.Errorf("summary detail = %q, want the model's final text %q", summary.Detail, "done")
}
}
+87
View File
@@ -0,0 +1,87 @@
package storecontract
import (
"context"
"time"
"github.com/samcharles93/archie-core/internal/domain/binding"
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
)
// BindingStore persists bindings and their draft -> pending_approval ->
// armed states.
type BindingStore interface {
InsertBinding(ctx context.Context, b binding.Binding) (string, error)
GetBinding(ctx context.Context, id string) (*binding.Binding, error)
ListBindings(ctx context.Context) ([]binding.Binding, error)
UpdateBinding(ctx context.Context, b binding.Binding) error
DeleteBinding(ctx context.Context, id string) error
ApproveBinding(ctx context.Context, id string) error
PauseBinding(ctx context.Context, id string) error
ResumeBinding(ctx context.Context, id string) error
}
// DispatchLedger reads the binding dispatch ledger.
type DispatchLedger interface {
// ListDispatches returns the newest ledger rows matching filter.
ListDispatches(ctx context.Context, filter DispatchFilter) ([]Dispatch, error)
}
// Dispatch is one binding/capture ledger row: the task it started, or the
// reason it started none.
type Dispatch struct {
BindingID string `json:"binding_id"`
BindingVersion int64 `json:"binding_version"`
CaptureID string `json:"capture_id"`
TaskID int64 `json:"task_id,omitempty"`
Reason string `json:"reason,omitempty"`
DispatchedAt time.Time `json:"dispatched_at"`
}
// DispatchFilter narrows ListDispatches; empty fields match any row.
type DispatchFilter struct {
BindingID string
CaptureID string
TaskID int64
Limit int
}
// BindingDispatcher looks up armed bindings, lists undispatched captures and
// records dispatches.
type BindingDispatcher interface {
ArmedBindingsForSource(ctx context.Context, source string) ([]binding.Binding, error)
RecordDispatch(
ctx context.Context,
bindingID string,
bindingVersion int64,
captureID string,
// taskID addresses the SQLite-owned task tables, which are not
// migrating: it stays an integer on purpose. A dispatch that starts
// a task is recorded by EnqueueBindingTask instead.
taskID int64,
// reason is empty for a dispatch and names the terminal outcome of
// an evaluation that started no task.
reason string,
) error
ListUndispatchedCaptures(ctx context.Context, bindings []string, limit int) ([]CapturedEvent, error)
}
// BindingTaskCreator is the single-method consumer-facing surface for
// enqueueing a task triggered by a binding. Splitting it off TaskLifecycle
// keeps the lifecycle surface narrow (8 methods, the interfacebloat limit)
// and keeps the binding-specific shape on the binding interfaces.
type BindingTaskCreator interface {
// EnqueueBindingTask claims the binding's dispatch of captureID in the
// ledger, enqueues the task and records its provenance as one unit: an
// error leaves none of them, and a delivery already dispatched for the
// binding returns ErrAlreadyDispatched.
EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity, bindingID string, bindingVersion int, captureID string, inputs map[string]any) (*task.Task, error)
}
// PlaybookDispatcher records a playbook action dispatch before its side
// effect runs, returning ErrAlreadyDispatched for a duplicate.
// DeletePlaybookDispatches deletes one playbook's records.
type PlaybookDispatcher interface {
RecordPlaybookDispatch(ctx context.Context, playbookID, playbookVersion, eventID, actionID string) error
DeletePlaybookDispatches(ctx context.Context, playbookID string) error
}
+73
View File
@@ -0,0 +1,73 @@
package storecontract
import (
"context"
"time"
"github.com/samcharles93/archie-core/internal/domain/eventtype"
)
// CaptureStore persists inbound webhook captures.
type CaptureStore interface {
InsertCapture(ctx context.Context, c CapturedEvent, retention time.Duration, maxEvents int) (string, error)
ListCaptures(ctx context.Context, limit int) ([]CapturedEvent, error)
}
// CaptureRefusalStore counts the events network rules refused, per source.
type CaptureRefusalStore interface {
RecordCaptureRefusal(ctx context.Context, source, addr string, at time.Time) error
CaptureRefusals(ctx context.Context, since time.Time) ([]CaptureRefusals, error)
}
// CaptureRefusals is one source's refusals and accepted captures since a
// time, and the addresses it refused.
type CaptureRefusals struct {
Source string `json:"source"`
Refused int64 `json:"refused"`
Accepted int64 `json:"accepted"`
Addrs []string `json:"addrs"`
}
// EventTypeStore persists event types. Insert and Update refuse a type whose
// rule overlaps another on the same source with eventtype.ErrOverlap, and a
// malformed one with eventtype.ErrInvalid. Update rewrites the name and rule;
// the schema is the one the type was created with.
type EventTypeStore interface {
InsertEventType(ctx context.Context, t eventtype.EventType) (string, error)
UpdateEventType(ctx context.Context, t eventtype.EventType) error
DeleteEventType(ctx context.Context, id string) error
ListEventTypes(ctx context.Context) ([]eventtype.EventType, error)
}
// CapturedEvent is one unbound inbound webhook capture: no workflow binding,
// no forge/task association -- just what arrived, from where, and when.
type CapturedEvent struct {
ID string `json:"id"`
ReceivedAt time.Time `json:"received_at"`
Source string `json:"source"`
RemoteAddr string `json:"remote_addr"`
ContentType string `json:"content_type"`
// Headers and Body are redacted (webhookguard.RedactPayload) before
// they ever reach InsertCapture; the store persists what it is given.
Headers string `json:"headers"`
Body string `json:"body"`
Authenticated bool `json:"authenticated"`
// EventType is the ID of the event type the capture was identified as
// when it arrived, or empty when it matched none. An unidentified capture
// is never dispatched.
EventType string `json:"event_type"`
// Unsigned marks an event that arrived on an approved unsigned source.
// It dispatches without Authenticated and is flagged wherever it shows.
Unsigned bool `json:"unsigned"`
// Delivery names the sender's delivery; a retry of one delivery shares
// it, so a binding dispatches each delivery once. Intake sets it from the
// source's delivery header; the store fills the capture ID when empty.
Delivery string `json:"delivery"`
// OrgID is honoured only for a schedule-fired capture, whose source is
// reserved and so names no org; every other capture takes its source's.
OrgID string `json:"org_id,omitempty"`
}
// Dispatchable reports whether a binding may start a task from this event:
// a valid signature, or a source approved to take unsigned events.
func (c CapturedEvent) Dispatchable() bool { return c.Authenticated || c.Unsigned }
@@ -0,0 +1,25 @@
package storecontract
import (
"context"
"time"
"github.com/samcharles93/archie-core/internal/domain/harnesssecret"
)
// HarnessSecretStore stores one org's OAuth token set per credential binding
// service, encrypted at rest.
type HarnessSecretStore interface {
GetHarnessSecret(ctx context.Context, org, service string) (harnesssecret.Secret, error)
PutHarnessSecret(ctx context.Context, s harnesssecret.Secret) error
}
// RunCredentialStore keeps run credentials by digest: the SHA-256 of the
// token a task's container presents. A lookup of an unknown or expired
// digest reports ErrRunCredentialUnknown.
type RunCredentialStore interface {
PutRunCredential(ctx context.Context, digest [32]byte, taskID int64, expires time.Time) error
DeleteRunCredential(ctx context.Context, digest [32]byte) error
RunCredentialTask(ctx context.Context, digest [32]byte, now time.Time) (int64, error)
DeleteExpiredRunCredentials(ctx context.Context, now time.Time) error
}
+90
View File
@@ -0,0 +1,90 @@
package storecontract
import (
"errors"
)
// Sentinel errors, part of the wire contract: staterpc's
// mapError/unmapError match these by (code, canonical message) across the
// gRPC boundary, so their message strings are load-bearing and must never
// change.
var (
// ErrStaleTransition is returned when a task transition's expected
// from status does not match the task's current status.
ErrStaleTransition = errors.New("store: stale transition: task status does not match expected from status")
// ErrIllegalTransition is returned when the transition table does not allow
// the status pair.
ErrIllegalTransition = errors.New("store: illegal transition: status pair is not in the transition table")
// ErrInvalidStep is returned when a step-execution request names a kind the
// store does not record or a step whose identifying fields do not resolve.
// Distinct from the transition sentinels, which cover a row's state.
ErrInvalidStep = errors.New("store: invalid step execution request")
// ErrResumeIncomplete is returned when a retry resumes after a stage whose
// last run did not complete, so there are no results to start from.
ErrResumeIncomplete = errors.New("store: the step before the resume point did not complete in its last run")
// ErrBindingNotFound is returned when a binding ID does not exist.
ErrBindingNotFound = errors.New("store: binding not found")
// ErrBindingOverlap is returned when a binding's matcher overlaps an
// existing binding for the same source.
ErrBindingOverlap = errors.New("store: binding overlaps existing binding for source")
// ErrBindingTransition is returned when a binding state transition is
// rejected by the draft -> pending_approval -> armed machine.
ErrBindingTransition = errors.New("store: binding state transition rejected")
// ErrAlreadyDispatched is returned when a dispatch ledger already holds the
// key.
ErrAlreadyDispatched = errors.New("store: binding already dispatched for capture")
// ErrCallNotYours is returned when a workflow.call status read names a
// task that is not a callee of the calling task, or the caller itself
// does not exist: a caller reads only the tasks it started
ErrCallNotYours = errors.New("store: task is not a callee of the calling task")
// ErrCallCallerNotRunning is returned when a workflow.call enqueue names
// a caller that is not running: a stale retry, or a bug in the caller.
ErrCallCallerNotRunning = errors.New("store: workflow.call caller is not running")
// ErrCallDepthExceeded is returned when a workflow.call enqueue would
// pass workflow.MaxCallDepth. The engine refuses the same call first;
// the store re-checks because it owns the table.
ErrCallDepthExceeded = errors.New("store: workflow.call would pass the depth limit")
// ErrMappingNotFound is returned when a mapping ID does not exist.
ErrMappingNotFound = errors.New("store: mapping not found")
// ErrEventTypeNotFound is returned when an event type ID does not exist.
ErrEventTypeNotFound = errors.New("store: event type not found")
// ErrSourceNotFound is returned when a source path does not exist.
ErrSourceNotFound = errors.New("store: source not found")
// ErrSourceInUse is returned when deleting a source an armed binding
// fires on.
ErrSourceInUse = errors.New("store: source has an armed binding")
// ErrSourcePathTaken is returned when a new source's path is in use.
ErrSourcePathTaken = errors.New("store: source path already taken")
// ErrSourceSigningStale is returned when a signing write's expected
// from state does not match the stored one.
ErrSourceSigningStale = errors.New("store: source signing does not match expected state")
// ErrHarnessSecretNotFound is returned when no OAuth token set is
// stored for an org/service pair -- the setup terminal has not
// captured one yet.
ErrHarnessSecretNotFound = errors.New("store: harness secret not found")
// ErrRereviewCapReached is returned when a review gate re-review would
// round past prreview.MaxRereviewRounds. The task stays waiting and its
// gate row is unchanged. The dashboard answers it 409, like every other
// conflict.
ErrRereviewCapReached = errors.New("store: re-review cap reached")
// ErrResourceNotFound is returned when a control-plane resource kind has
// no stored document. It is in-process only (not on the gRPC wire): the
// control plane runs in the daemon against the local store.
ErrResourceNotFound = errors.New("resource not found")
// ErrResourceVersionConflict is returned when a resource write carries an
// expected version that does not match the stored revision.
ErrResourceVersionConflict = errors.New("resource version conflict")
)
// ErrUnavailable means the store could not serve a request now: it is
// restarting or unreachable. It is not paired with a canonical message the way
// the sentinels above are -- staterpc derives it from the Unavailable code
// alone, whose text varies by transport -- and it is the one store error a
// caller with budget may answer by retrying rather than by failing its own
// work. The wrapping keeps the gRPC status code readable through the chain.
var ErrUnavailable = errors.New("store: state store unavailable")
// ErrRunCredentialUnknown is a run credential that was never issued, was
// revoked, or has expired.
var ErrRunCredentialUnknown = errors.New("run credential unknown or expired")
+26
View File
@@ -0,0 +1,26 @@
package storecontract
import (
"context"
"github.com/samcharles93/archie-core/internal/domain/mapping"
)
// MappingStore persists payload field mappings. Deliberately separate from
// TaskStore and CaptureStore for the same reason those are split: the
// dashboard's mapping editor should only acquire the mapping surface, not the
// full task or capture APIs. See
type MappingStore interface {
InsertMapping(ctx context.Context, m mapping.Mapping) (string, error)
GetMapping(ctx context.Context, id string) (*mapping.Mapping, error)
ListMappings(ctx context.Context) ([]mapping.Mapping, error)
UpdateMapping(ctx context.Context, m mapping.Mapping) error
DeleteMapping(ctx context.Context, id string) error
}
// MappingMatchRecorder counts the events each mapping resolved, which the
// store reports as Mapping.MatchCount and LastMatchedAt. Recording the same
// (mapping, capture) twice counts once.
type MappingMatchRecorder interface {
RecordMappingMatch(ctx context.Context, mappingID, captureID string) error
}
+50
View File
@@ -0,0 +1,50 @@
package storecontract
import (
"time"
)
// Resource is one control-plane resource document: the stored value for an
// org's kind, its revision, and the audit record written with the last write.
// Resource and ResourceWrite live here so the UI-side consumers can reference
// them without linking the PostgreSQL implementation.
type Resource struct {
OrgID string
Kind string
Value []byte
Version int64
Actor string
Source string
RequestID string
ExpectedVersion int64
CurrentVersion int64
At time.Time
}
// DefaultOrgID is the org every resource written before orgs existed belongs
// to, and the org a request that names none acts in.
const DefaultOrgID = "org-sys"
// ResourceAuditKey is the sys_audit record key of one org's resource kind. The
// default org keeps the bare kind, so audit rows written before resources were
// per-org still resolve.
func ResourceAuditKey(orgID, kind string) string {
if orgID == DefaultOrgID {
return kind
}
return orgID + "/" + kind
}
// ResourceWrite is a control-plane resource write request. ExpectedVersion is
// the optimistic-concurrency guard: a write whose expectation does not match
// the stored revision is refused with ErrResourceVersionConflict.
type ResourceWrite struct {
OrgID string
Kind string
Value []byte
Actor string
Source string
RequestID string
ExpectedVersion int64
At time.Time
}
+23
View File
@@ -0,0 +1,23 @@
package storecontract
import (
"context"
"github.com/samcharles93/archie-core/internal/domain/source"
)
// SourceStore persists capture sources keyed by path. Secrets are encrypted
// at rest; signing transitions use an expected-from guard.
type SourceStore interface {
InsertSource(ctx context.Context, s source.Source) error
// GetSource returns (nil, nil) for an unknown path.
GetSource(ctx context.Context, path string) (*source.Source, error)
ListSources(ctx context.Context) ([]source.Source, error)
SetSourceSigning(ctx context.Context, path string, from, to source.Signing) error
SetSourceSecret(ctx context.Context, path, secret string) error
SetSourceName(ctx context.Context, path, name string) error
SetSourceDeliveryHeader(ctx context.Context, path, header string) error
// DeleteSource refuses with ErrSourceInUse while an armed binding fires
// on the source.
DeleteSource(ctx context.Context, path string) error
}
+87
View File
@@ -0,0 +1,87 @@
package storecontract
import (
"context"
"time"
"github.com/samcharles93/archie-core/internal/domain/health"
)
// ConfigSnapshotStore holds the published configuration view.
type ConfigSnapshotStore interface {
PutConfigSnapshot(ctx context.Context, snapshot ConfigSnapshot) error
ConfigSnapshot(ctx context.Context) (ConfigSnapshot, bool, error)
}
// ChannelStatusStore holds channel runtime state published by the process
// hosting the channels.
type ChannelStatusStore interface {
// PutChannelStatus records the reporting process's whole set: a channel it no
// longer reports is removed rather than left behind, because a stale "running"
// row is a lie the dashboard would show indefinitely.
PutChannelStatus(ctx context.Context, channels []ChannelStatus) error
ChannelStatus(ctx context.Context) ([]ChannelStatus, error)
}
// ApplyStatusStore holds which version of each control-plane resource each
// process is running. Same split as ConfigSnapshotStore and for the same
// reason: the writers are the processes that apply a resource, the reader is
// the UI, and a task-scoped credential must reach neither.
type ApplyStatusStore interface {
PutApplyStatus(ctx context.Context, status ApplyStatus) error
ListApplyStatus(ctx context.Context) ([]ApplyStatus, error)
}
// PresenceStore holds one record per running service instance. Writers are
// the services themselves; readers are the UI and archied status.
type PresenceStore interface {
PutPresence(ctx context.Context, presence Presence) error
ListPresence(ctx context.Context) ([]Presence, error)
}
// ConfigSnapshot is the published, secret-free configuration view. Schema
// names Document's shape.
type ConfigSnapshot struct {
Schema string
Document []byte
PublishedAt time.Time
}
// ChannelStatus is one channel's runtime state as the process hosting it
// reports it. ReloadSupported is a capability declaration rather than a request:
// asking for a reload travels on its own surface, because a transient request and
// a durable state do not share a lifetime.
type ChannelStatus struct {
ID string
Name string
State string
Detail string
Configured bool
ReloadSupported bool
ObservedAt time.Time
}
// ApplyStatus is one process's report on one resource kind. On error,
// AppliedVersion stays at the last applied version. ReportedAt is
// re-stamped periodically.
type ApplyStatus struct {
Process string `json:"process"`
Kind string `json:"kind"`
AppliedVersion int64 `json:"applied_version"`
Error string `json:"error,omitempty"`
ReportedAt time.Time `json:"reported_at"`
}
// Presence is one running service instance. Ready and Detail summarise its
// readiness probes; Detail names the degraded ones.
type Presence struct {
Checks []health.Component `json:"checks"`
Service string `json:"service"`
InstanceID string `json:"instance_id"`
Version string `json:"version"`
InstallType string `json:"install_type"`
StartedAt time.Time `json:"started_at"`
ReportedAt time.Time `json:"reported_at"`
Ready bool `json:"ready"`
Detail string `json:"detail,omitempty"`
}
@@ -2,23 +2,6 @@
// without an implementation.
package storecontract
import (
"context"
"errors"
"io"
"time"
"github.com/samcharles93/archie-core/internal/domain/binding"
"github.com/samcharles93/archie-core/internal/domain/eventtype"
"github.com/samcharles93/archie-core/internal/domain/harnesssecret"
"github.com/samcharles93/archie-core/internal/domain/health"
"github.com/samcharles93/archie-core/internal/domain/mapping"
"github.com/samcharles93/archie-core/internal/domain/source"
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/logging"
)
// TaskStore is the full store surface the daemon needs.
// Consumers depend on TaskStore, never on a concrete type.
type TaskStore interface {
@@ -31,591 +14,3 @@ type TaskStore interface {
ReviewGateResponder
RemediationStarter
}
// TaskLifecycle manages the core task state machine and enqueuing.
// Binding-triggered task creation lives on BindingTaskCreator, not here,
// so the lifecycle surface stays narrow and non-binding consumers (forge
// poll, chat spawn, drain loop) do not acquire a binding-specific shape.
type TaskLifecycle interface {
EnqueueIssue(ctx context.Context, owner, repo string, number int, title, body, labels, identity string) (bool, error)
EnqueueChatTask(ctx context.Context, owner, repo, title, body, wf, identity, origin, draft string, inputs map[string]any) (*task.Task, error)
ClaimNext(ctx context.Context) (*task.Task, error)
ClaimByIssue(ctx context.Context, owner, repo string, number int) (*task.Task, error)
Transition(ctx context.Context, taskID int64, from, to, detail string) error
Update(ctx context.Context, t *task.Task) error
Requeue(ctx context.Context, taskID int64, fromStatus, workflow string) error
RecoverStale(ctx context.Context) (int64, error)
}
// TaskParker parks a task with a park class and the blocker the park site
// knows, if any. Unknown classes become needs_decision.
type TaskParker interface {
ParkTask(ctx context.Context, taskID int64, from, detail, class, blocker string) error
}
// StepRecorder records a running execution's steps.
type StepRecorder interface {
StartStep(ctx context.Context, s task.StepStart) (int64, events.Event, error)
FinishStep(ctx context.Context, s task.StepFinish) (events.Event, error)
}
// ExecutionCanceller cancels an execution and its current attempt's open
// steps in one transaction. It also cancels the callees the execution was
// waiting on, and returns their ids (nested callees included) so the caller
// can stop their runs too.
type ExecutionCanceller interface {
CancelExecution(ctx context.Context, taskID int64, reason, to string) ([]int64, error)
}
// StepReader lists one execution's recorded steps.
type StepReader interface {
ListSteps(ctx context.Context, executionID int64, attempt int) ([]task.StepExecution, error)
}
// HarnessSecretStore stores one org's OAuth token set per credential binding
// service, encrypted at rest.
type HarnessSecretStore interface {
GetHarnessSecret(ctx context.Context, org, service string) (harnesssecret.Secret, error)
PutHarnessSecret(ctx context.Context, s harnesssecret.Secret) error
}
// TaskArchiver removes one terminal task's local record with an optimistic
// status guard. It is separate from the already broad lifecycle contract so
// consumers that only run tasks do not acquire an operator-only capability.
type TaskArchiver interface {
OutcomePoster
ArchiveTask(ctx context.Context, taskID int64, fromStatus string, audit events.Event) (eventID int64, err error)
ClearTerminalTasks(ctx context.Context) (int64, error)
}
// TaskRetryer atomically requeues recoverable work and accounts for the new
// attempt so a partial write cannot evade the retry cap. retryMode is the
// operator's worktree choice for the next dispatch (taskstate.RetryMode),
// persisted on the row so the daemon reads it instead of inferring it. resume
// is where the attempt starts; a resume whose After stage did not complete in
// its last run is refused with ErrResumeIncomplete.
type TaskRetryer interface {
RetryTask(ctx context.Context, taskID int64, fromStatus, workflow, retryMode string, resume task.Resume, currentDefinition bool) error
}
// ReviewGateResponder records the operator's review-gate answer. A re-review
// increments rereview_rounds and requeues; one past the cap returns
// ErrRereviewCapReached and writes nothing.
type ReviewGateResponder interface {
RespondReviewGate(ctx context.Context, taskID int64, fromStatus, gate string, rereview bool, maxRounds int) error
}
// RemediationStarter queues a review-triggered remediation for an Archie-owned
// open-PR task: one guarded transition that carries the JSON-encoded review
// unit in the same write, so a claimed remediation always has its input. Runs
// for one task are serialised, so a distinct review that arrives while a
// remediation owns the task is queued behind it and remediated once that run
// finishes; a re-delivered unit is absorbed, not queued twice.
// UpdateReviewPayload merges late-arriving comments into the unclaimed unit
// for the same review, active or queued behind a running remediation.
type RemediationStarter interface {
BeginRemediation(ctx context.Context, taskID int64, payload string) error
UpdateReviewPayload(ctx context.Context, taskID int64, payload string) error
// SetReviewCursors persists the poll backstop's per-task review and
// comment high-water marks (pr-review-remediation.md decision 2). The
// pr_open guard is part of the contract: cursors only move while no
// remediation run owns the task.
SetReviewCursors(ctx context.Context, taskID, reviewCursor, commentCursor int64) error
}
// OutcomePoster tracks which finished chat tasks have reported back to their
// conversation.
type OutcomePoster interface {
// UnpostedOutcomes returns finished chat tasks whose outcome has not been
// posted to the conversation that created them.
UnpostedOutcomes(ctx context.Context) ([]*task.Task, error)
// MarkOutcomePosted records that the task's state at status was posted.
MarkOutcomePosted(ctx context.Context, taskID int64, status string) error
}
// TaskQueries groups read-only task accessors.
type TaskQueries interface {
TaskByIssue(ctx context.Context, owner, repo string, number int) (*task.Task, error)
OpenTaskByPR(ctx context.Context, owner, repo string, number int) (*task.Task, error)
TaskByID(ctx context.Context, taskID int64) (*task.Task, error)
OpenPRs(ctx context.Context) ([]task.Task, error)
// ActiveTasksByOrigin returns the queued and running tasks the
// conversation origin created, for its /stop.
ActiveTasksByOrigin(ctx context.Context, origin string) ([]*task.Task, error)
// Tasks returns the most recently updated tasks.
Tasks(ctx context.Context, limit int) ([]task.Task, error)
// TasksPage returns one page of tasks, most recently updated first.
TasksPage(ctx context.Context, page TaskPage) ([]task.Task, error)
StatusCounts(ctx context.Context) (map[string]int, error)
}
// TaskEvents groups observability and lifecycle methods.
type TaskEvents interface {
InsertEvent(ctx context.Context, e events.Event) (int64, error)
EventsSince(ctx context.Context, cursor string, limit int) ([]events.Event, error)
TaskEvents(ctx context.Context, taskID int64) ([]events.Event, error)
WorkflowStats(ctx context.Context) ([]WorkflowStat, error)
StageStats(ctx context.Context) ([]StageStat, error)
TokensByDay(ctx context.Context, days int) ([]DayTokens, error)
Close() error
}
// CaptureStore persists inbound webhook captures.
type CaptureStore interface {
InsertCapture(ctx context.Context, c CapturedEvent, retention time.Duration, maxEvents int) (string, error)
ListCaptures(ctx context.Context, limit int) ([]CapturedEvent, error)
}
// CaptureRefusalStore counts the events network rules refused, per source.
type CaptureRefusalStore interface {
RecordCaptureRefusal(ctx context.Context, source, addr string, at time.Time) error
CaptureRefusals(ctx context.Context, since time.Time) ([]CaptureRefusals, error)
}
// CaptureRefusals is one source's refusals and accepted captures since a
// time, and the addresses it refused.
type CaptureRefusals struct {
Source string `json:"source"`
Refused int64 `json:"refused"`
Accepted int64 `json:"accepted"`
Addrs []string `json:"addrs"`
}
// ConfigSnapshotStore holds the published configuration view.
type ConfigSnapshotStore interface {
PutConfigSnapshot(ctx context.Context, snapshot ConfigSnapshot) error
ConfigSnapshot(ctx context.Context) (ConfigSnapshot, bool, error)
}
// ChannelStatusStore holds channel runtime state published by the process
// hosting the channels.
type ChannelStatusStore interface {
// PutChannelStatus records the reporting process's whole set: a channel it no
// longer reports is removed rather than left behind, because a stale "running"
// row is a lie the dashboard would show indefinitely.
PutChannelStatus(ctx context.Context, channels []ChannelStatus) error
ChannelStatus(ctx context.Context) ([]ChannelStatus, error)
}
// ApplyStatusStore holds which version of each control-plane resource each
// process is running. Same split as ConfigSnapshotStore and for the same
// reason: the writers are the processes that apply a resource, the reader is
// the UI, and a task-scoped credential must reach neither.
type ApplyStatusStore interface {
PutApplyStatus(ctx context.Context, status ApplyStatus) error
ListApplyStatus(ctx context.Context) ([]ApplyStatus, error)
}
// PresenceStore holds one record per running service instance. Writers are
// the services themselves; readers are the UI and archied status.
type PresenceStore interface {
PutPresence(ctx context.Context, presence Presence) error
ListPresence(ctx context.Context) ([]Presence, error)
}
// MappingStore persists payload field mappings. Deliberately separate from
// TaskStore and CaptureStore for the same reason those are split: the
// dashboard's mapping editor should only acquire the mapping surface, not the
// full task or capture APIs. See
type MappingStore interface {
InsertMapping(ctx context.Context, m mapping.Mapping) (string, error)
GetMapping(ctx context.Context, id string) (*mapping.Mapping, error)
ListMappings(ctx context.Context) ([]mapping.Mapping, error)
UpdateMapping(ctx context.Context, m mapping.Mapping) error
DeleteMapping(ctx context.Context, id string) error
}
// MappingMatchRecorder counts the events each mapping resolved, which the
// store reports as Mapping.MatchCount and LastMatchedAt. Recording the same
// (mapping, capture) twice counts once.
type MappingMatchRecorder interface {
RecordMappingMatch(ctx context.Context, mappingID, captureID string) error
}
// EventTypeStore persists event types. Insert and Update refuse a type whose
// rule overlaps another on the same source with eventtype.ErrOverlap, and a
// malformed one with eventtype.ErrInvalid. Update rewrites the name and rule;
// the schema is the one the type was created with.
type EventTypeStore interface {
InsertEventType(ctx context.Context, t eventtype.EventType) (string, error)
UpdateEventType(ctx context.Context, t eventtype.EventType) error
DeleteEventType(ctx context.Context, id string) error
ListEventTypes(ctx context.Context) ([]eventtype.EventType, error)
}
// BindingStore persists bindings and their draft -> pending_approval ->
// armed states.
type BindingStore interface {
InsertBinding(ctx context.Context, b binding.Binding) (string, error)
GetBinding(ctx context.Context, id string) (*binding.Binding, error)
ListBindings(ctx context.Context) ([]binding.Binding, error)
UpdateBinding(ctx context.Context, b binding.Binding) error
DeleteBinding(ctx context.Context, id string) error
ApproveBinding(ctx context.Context, id string) error
PauseBinding(ctx context.Context, id string) error
ResumeBinding(ctx context.Context, id string) error
}
// SourceStore persists capture sources keyed by path. Secrets are encrypted
// at rest; signing transitions use an expected-from guard.
type SourceStore interface {
InsertSource(ctx context.Context, s source.Source) error
// GetSource returns (nil, nil) for an unknown path.
GetSource(ctx context.Context, path string) (*source.Source, error)
ListSources(ctx context.Context) ([]source.Source, error)
SetSourceSigning(ctx context.Context, path string, from, to source.Signing) error
SetSourceSecret(ctx context.Context, path, secret string) error
SetSourceName(ctx context.Context, path, name string) error
SetSourceDeliveryHeader(ctx context.Context, path, header string) error
// DeleteSource refuses with ErrSourceInUse while an armed binding fires
// on the source.
DeleteSource(ctx context.Context, path string) error
}
// DispatchLedger reads the binding dispatch ledger.
type DispatchLedger interface {
// ListDispatches returns the newest ledger rows matching filter.
ListDispatches(ctx context.Context, filter DispatchFilter) ([]Dispatch, error)
}
// TaskPage selects one page of the task list. Empty Statuses matches every
// status. A zero After starts at the newest task; otherwise the page begins
// after the task it names, in (updated_at, id) descending order.
type TaskPage struct {
Statuses []string
// Workflow, when set, keeps only that workflow's tasks.
Workflow string
After TaskCursor
Limit int
}
// TaskCursor is the position of the last task a page returned.
type TaskCursor struct {
UpdatedAt time.Time
ID int64
}
// Dispatch is one binding/capture ledger row: the task it started, or the
// reason it started none.
type Dispatch struct {
BindingID string `json:"binding_id"`
BindingVersion int64 `json:"binding_version"`
CaptureID string `json:"capture_id"`
TaskID int64 `json:"task_id,omitempty"`
Reason string `json:"reason,omitempty"`
DispatchedAt time.Time `json:"dispatched_at"`
}
// DispatchFilter narrows ListDispatches; empty fields match any row.
type DispatchFilter struct {
BindingID string
CaptureID string
TaskID int64
Limit int
}
// BindingDispatcher looks up armed bindings, lists undispatched captures and
// records dispatches.
type BindingDispatcher interface {
ArmedBindingsForSource(ctx context.Context, source string) ([]binding.Binding, error)
RecordDispatch(
ctx context.Context,
bindingID string,
bindingVersion int64,
captureID string,
// taskID addresses the SQLite-owned task tables, which are not
// migrating: it stays an integer on purpose. A dispatch that starts
// a task is recorded by EnqueueBindingTask instead.
taskID int64,
// reason is empty for a dispatch and names the terminal outcome of
// an evaluation that started no task.
reason string,
) error
ListUndispatchedCaptures(ctx context.Context, bindings []string, limit int) ([]CapturedEvent, error)
}
// BindingTaskCreator is the single-method consumer-facing surface for
// enqueueing a task triggered by a binding. Splitting it off TaskLifecycle
// keeps the lifecycle surface narrow (8 methods, the interfacebloat limit)
// and keeps the binding-specific shape on the binding interfaces.
type BindingTaskCreator interface {
// EnqueueBindingTask claims the binding's dispatch of captureID in the
// ledger, enqueues the task and records its provenance as one unit: an
// error leaves none of them, and a delivery already dispatched for the
// binding returns ErrAlreadyDispatched.
EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity, bindingID string, bindingVersion int, captureID string, inputs map[string]any) (*task.Task, error)
}
// WorkflowCaller starts a workflow.call callee and reads it back.
type WorkflowCaller = task.Caller
// PlaybookDispatcher records a playbook action dispatch before its side
// effect runs, returning ErrAlreadyDispatched for a duplicate.
// DeletePlaybookDispatches deletes one playbook's records.
type PlaybookDispatcher interface {
RecordPlaybookDispatch(ctx context.Context, playbookID, playbookVersion, eventID, actionID string) error
DeletePlaybookDispatches(ctx context.Context, playbookID string) error
}
// TaskLogStore reads one task attempt's log. Found=false means no log;
// ErrTaskLogsUnavailable means this process cannot read logs.
type TaskLogStore interface {
// TaskLog returns a page of one attempt's decoded log. attempt 0 selects
// the task's current attempt.
TaskLog(ctx context.Context, taskID int64, attempt int, q logging.Query) (logging.TaskLogPage, error)
// TaskLogContent writes one attempt's log verbatim to w, for a download.
// found is false, with a nil error, when the attempt has no log file.
TaskLogContent(ctx context.Context, taskID int64, attempt int, w io.Writer) (found bool, err error)
}
// CapturedEvent is one unbound inbound webhook capture: no workflow binding,
// no forge/task association -- just what arrived, from where, and when.
type CapturedEvent struct {
ID string `json:"id"`
ReceivedAt time.Time `json:"received_at"`
Source string `json:"source"`
RemoteAddr string `json:"remote_addr"`
ContentType string `json:"content_type"`
// Headers and Body are redacted (webhookguard.RedactPayload) before
// they ever reach InsertCapture; the store persists what it is given.
Headers string `json:"headers"`
Body string `json:"body"`
Authenticated bool `json:"authenticated"`
// EventType is the ID of the event type the capture was identified as
// when it arrived, or empty when it matched none. An unidentified capture
// is never dispatched.
EventType string `json:"event_type"`
// Unsigned marks an event that arrived on an approved unsigned source.
// It dispatches without Authenticated and is flagged wherever it shows.
Unsigned bool `json:"unsigned"`
// Delivery names the sender's delivery; a retry of one delivery shares
// it, so a binding dispatches each delivery once. Intake sets it from the
// source's delivery header; the store fills the capture ID when empty.
Delivery string `json:"delivery"`
// OrgID is honoured only for a schedule-fired capture, whose source is
// reserved and so names no org; every other capture takes its source's.
OrgID string `json:"org_id,omitempty"`
}
// Dispatchable reports whether a binding may start a task from this event:
// a valid signature, or a source approved to take unsigned events.
func (c CapturedEvent) Dispatchable() bool { return c.Authenticated || c.Unsigned }
// ConfigSnapshot is the published, secret-free configuration view. Schema
// names Document's shape.
type ConfigSnapshot struct {
Schema string
Document []byte
PublishedAt time.Time
}
// ChannelStatus is one channel's runtime state as the process hosting it
// reports it. ReloadSupported is a capability declaration rather than a request:
// asking for a reload travels on its own surface, because a transient request and
// a durable state do not share a lifetime.
type ChannelStatus struct {
ID string
Name string
State string
Detail string
Configured bool
ReloadSupported bool
ObservedAt time.Time
}
// ApplyStatus is one process's report on one resource kind. On error,
// AppliedVersion stays at the last applied version. ReportedAt is
// re-stamped periodically.
type ApplyStatus struct {
Process string `json:"process"`
Kind string `json:"kind"`
AppliedVersion int64 `json:"applied_version"`
Error string `json:"error,omitempty"`
ReportedAt time.Time `json:"reported_at"`
}
// Presence is one running service instance. Ready and Detail summarise its
// readiness probes; Detail names the degraded ones.
type Presence struct {
Checks []health.Component `json:"checks"`
Service string `json:"service"`
InstanceID string `json:"instance_id"`
Version string `json:"version"`
InstallType string `json:"install_type"`
StartedAt time.Time `json:"started_at"`
ReportedAt time.Time `json:"reported_at"`
Ready bool `json:"ready"`
Detail string `json:"detail,omitempty"`
}
// WorkflowStat is one row of the per-workflow metrics table.
type WorkflowStat struct {
Workflow string `json:"workflow"`
Runs int `json:"runs"`
Merged int `json:"merged"`
Completed int `json:"completed"`
PROpen int `json:"pr_open"`
Parked int `json:"parked"`
AvgTokens int `json:"avg_tokens"`
AvgSteps float64 `json:"avg_steps"`
TotalToken int `json:"total_tokens"`
// Running and LastStatus describe the workflow now: how many of its runs
// are working, and how its newest one ended.
Running int `json:"running"`
LastStatus string `json:"last_status"`
}
// StageStat is average stage duration and failure counts per stage.
type StageStat struct {
Workflow string `json:"workflow"`
Stage string `json:"stage"`
Runs int `json:"runs"`
AvgMs int `json:"avg_ms"`
Errors int `json:"errors"`
}
// DayTokens is token spend per UTC day.
type DayTokens struct {
Day string `json:"day"`
Tokens int `json:"tokens"`
}
// Sentinel errors, part of the wire contract: staterpc's
// mapError/unmapError match these by (code, canonical message) across the
// gRPC boundary, so their message strings are load-bearing and must never
// change.
var (
// ErrStaleTransition is returned when a task transition's expected
// from status does not match the task's current status.
ErrStaleTransition = errors.New("store: stale transition: task status does not match expected from status")
// ErrIllegalTransition is returned when the transition table does not allow
// the status pair.
ErrIllegalTransition = errors.New("store: illegal transition: status pair is not in the transition table")
// ErrInvalidStep is returned when a step-execution request names a kind the
// store does not record or a step whose identifying fields do not resolve.
// Distinct from the transition sentinels, which cover a row's state.
ErrInvalidStep = errors.New("store: invalid step execution request")
// ErrResumeIncomplete is returned when a retry resumes after a stage whose
// last run did not complete, so there are no results to start from.
ErrResumeIncomplete = errors.New("store: the step before the resume point did not complete in its last run")
// ErrBindingNotFound is returned when a binding ID does not exist.
ErrBindingNotFound = errors.New("store: binding not found")
// ErrBindingOverlap is returned when a binding's matcher overlaps an
// existing binding for the same source.
ErrBindingOverlap = errors.New("store: binding overlaps existing binding for source")
// ErrBindingTransition is returned when a binding state transition is
// rejected by the draft -> pending_approval -> armed machine.
ErrBindingTransition = errors.New("store: binding state transition rejected")
// ErrAlreadyDispatched is returned when a dispatch ledger already holds the
// key.
ErrAlreadyDispatched = errors.New("store: binding already dispatched for capture")
// ErrCallNotYours is returned when a workflow.call status read names a
// task that is not a callee of the calling task, or the caller itself
// does not exist: a caller reads only the tasks it started
ErrCallNotYours = errors.New("store: task is not a callee of the calling task")
// ErrCallCallerNotRunning is returned when a workflow.call enqueue names
// a caller that is not running: a stale retry, or a bug in the caller.
ErrCallCallerNotRunning = errors.New("store: workflow.call caller is not running")
// ErrCallDepthExceeded is returned when a workflow.call enqueue would
// pass workflow.MaxCallDepth. The engine refuses the same call first;
// the store re-checks because it owns the table.
ErrCallDepthExceeded = errors.New("store: workflow.call would pass the depth limit")
// ErrMappingNotFound is returned when a mapping ID does not exist.
ErrMappingNotFound = errors.New("store: mapping not found")
// ErrEventTypeNotFound is returned when an event type ID does not exist.
ErrEventTypeNotFound = errors.New("store: event type not found")
// ErrSourceNotFound is returned when a source path does not exist.
ErrSourceNotFound = errors.New("store: source not found")
// ErrSourceInUse is returned when deleting a source an armed binding
// fires on.
ErrSourceInUse = errors.New("store: source has an armed binding")
// ErrSourcePathTaken is returned when a new source's path is in use.
ErrSourcePathTaken = errors.New("store: source path already taken")
// ErrSourceSigningStale is returned when a signing write's expected
// from state does not match the stored one.
ErrSourceSigningStale = errors.New("store: source signing does not match expected state")
// ErrHarnessSecretNotFound is returned when no OAuth token set is
// stored for an org/service pair -- the setup terminal has not
// captured one yet.
ErrHarnessSecretNotFound = errors.New("store: harness secret not found")
// ErrRereviewCapReached is returned when a review gate re-review would
// round past prreview.MaxRereviewRounds. The task stays waiting and its
// gate row is unchanged. The dashboard answers it 409, like every other
// conflict.
ErrRereviewCapReached = errors.New("store: re-review cap reached")
// ErrResourceNotFound is returned when a control-plane resource kind has
// no stored document. It is in-process only (not on the gRPC wire): the
// control plane runs in the daemon against the local store.
ErrResourceNotFound = errors.New("resource not found")
// ErrResourceVersionConflict is returned when a resource write carries an
// expected version that does not match the stored revision.
ErrResourceVersionConflict = errors.New("resource version conflict")
)
// ErrUnavailable means the store could not serve a request now: it is
// restarting or unreachable. It is not paired with a canonical message the way
// the sentinels above are -- staterpc derives it from the Unavailable code
// alone, whose text varies by transport -- and it is the one store error a
// caller with budget may answer by retrying rather than by failing its own
// work. The wrapping keeps the gRPC status code readable through the chain.
var ErrUnavailable = errors.New("store: state store unavailable")
// Resource is one control-plane resource document: the stored value for an
// org's kind, its revision, and the audit record written with the last write.
// Resource and ResourceWrite live here so the UI-side consumers can reference
// them without linking the PostgreSQL implementation.
type Resource struct {
OrgID string
Kind string
Value []byte
Version int64
Actor string
Source string
RequestID string
ExpectedVersion int64
CurrentVersion int64
At time.Time
}
// DefaultOrgID is the org every resource written before orgs existed belongs
// to, and the org a request that names none acts in.
const DefaultOrgID = "org-sys"
// ResourceAuditKey is the sys_audit record key of one org's resource kind. The
// default org keeps the bare kind, so audit rows written before resources were
// per-org still resolve.
func ResourceAuditKey(orgID, kind string) string {
if orgID == DefaultOrgID {
return kind
}
return orgID + "/" + kind
}
// ResourceWrite is a control-plane resource write request. ExpectedVersion is
// the optimistic-concurrency guard: a write whose expectation does not match
// the stored revision is refused with ErrResourceVersionConflict.
type ResourceWrite struct {
OrgID string
Kind string
Value []byte
Actor string
Source string
RequestID string
ExpectedVersion int64
At time.Time
}
// ErrRunCredentialUnknown is a run credential that was never issued, was
// revoked, or has expired.
var ErrRunCredentialUnknown = errors.New("run credential unknown or expired")
// RunCredentialStore keeps run credentials by digest: the SHA-256 of the
// token a task's container presents. A lookup of an unknown or expired
// digest reports ErrRunCredentialUnknown.
type RunCredentialStore interface {
PutRunCredential(ctx context.Context, digest [32]byte, taskID int64, expires time.Time) error
DeleteRunCredential(ctx context.Context, digest [32]byte) error
RunCredentialTask(ctx context.Context, digest [32]byte, now time.Time) (int64, error)
DeleteExpiredRunCredentials(ctx context.Context, now time.Time) error
}
+198
View File
@@ -0,0 +1,198 @@
package storecontract
import (
"context"
"io"
"time"
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/logging"
)
// TaskLifecycle manages the core task state machine and enqueuing.
// Binding-triggered task creation lives on BindingTaskCreator, not here,
// so the lifecycle surface stays narrow and non-binding consumers (forge
// poll, chat spawn, drain loop) do not acquire a binding-specific shape.
type TaskLifecycle interface {
EnqueueIssue(ctx context.Context, owner, repo string, number int, title, body, labels, identity string) (bool, error)
EnqueueChatTask(ctx context.Context, owner, repo, title, body, wf, identity, origin, draft string, inputs map[string]any) (*task.Task, error)
ClaimNext(ctx context.Context) (*task.Task, error)
ClaimByIssue(ctx context.Context, owner, repo string, number int) (*task.Task, error)
Transition(ctx context.Context, taskID int64, from, to, detail string) error
Update(ctx context.Context, t *task.Task) error
Requeue(ctx context.Context, taskID int64, fromStatus, workflow string) error
RecoverStale(ctx context.Context) (int64, error)
}
// TaskParker parks a task with a park class and the blocker the park site
// knows, if any. Unknown classes become needs_decision.
type TaskParker interface {
ParkTask(ctx context.Context, taskID int64, from, detail, class, blocker string) error
}
// StepRecorder records a running execution's steps.
type StepRecorder interface {
StartStep(ctx context.Context, s task.StepStart) (int64, events.Event, error)
FinishStep(ctx context.Context, s task.StepFinish) (events.Event, error)
}
// ExecutionCanceller cancels an execution and its current attempt's open
// steps in one transaction. It also cancels the callees the execution was
// waiting on, and returns their ids (nested callees included) so the caller
// can stop their runs too.
type ExecutionCanceller interface {
CancelExecution(ctx context.Context, taskID int64, reason, to string) ([]int64, error)
}
// StepReader lists one execution's recorded steps.
type StepReader interface {
ListSteps(ctx context.Context, executionID int64, attempt int) ([]task.StepExecution, error)
}
// TaskArchiver removes one terminal task's local record with an optimistic
// status guard. It is separate from the already broad lifecycle contract so
// consumers that only run tasks do not acquire an operator-only capability.
type TaskArchiver interface {
OutcomePoster
ArchiveTask(ctx context.Context, taskID int64, fromStatus string, audit events.Event) (eventID int64, err error)
ClearTerminalTasks(ctx context.Context) (int64, error)
}
// TaskRetryer atomically requeues recoverable work and accounts for the new
// attempt so a partial write cannot evade the retry cap. retryMode is the
// operator's worktree choice for the next dispatch (taskstate.RetryMode),
// persisted on the row so the daemon reads it instead of inferring it. resume
// is where the attempt starts; a resume whose After stage did not complete in
// its last run is refused with ErrResumeIncomplete.
type TaskRetryer interface {
RetryTask(ctx context.Context, taskID int64, fromStatus, workflow, retryMode string, resume task.Resume, currentDefinition bool) error
}
// ReviewGateResponder records the operator's review-gate answer. A re-review
// increments rereview_rounds and requeues; one past the cap returns
// ErrRereviewCapReached and writes nothing.
type ReviewGateResponder interface {
RespondReviewGate(ctx context.Context, taskID int64, fromStatus, gate string, rereview bool, maxRounds int) error
}
// RemediationStarter queues a review-triggered remediation for an Archie-owned
// open-PR task: one guarded transition that carries the JSON-encoded review
// unit in the same write, so a claimed remediation always has its input. Runs
// for one task are serialised, so a distinct review that arrives while a
// remediation owns the task is queued behind it and remediated once that run
// finishes; a re-delivered unit is absorbed, not queued twice.
// UpdateReviewPayload merges late-arriving comments into the unclaimed unit
// for the same review, active or queued behind a running remediation.
type RemediationStarter interface {
BeginRemediation(ctx context.Context, taskID int64, payload string) error
UpdateReviewPayload(ctx context.Context, taskID int64, payload string) error
// SetReviewCursors persists the poll backstop's per-task review and
// comment high-water marks (pr-review-remediation.md decision 2). The
// pr_open guard is part of the contract: cursors only move while no
// remediation run owns the task.
SetReviewCursors(ctx context.Context, taskID, reviewCursor, commentCursor int64) error
}
// OutcomePoster tracks which finished chat tasks have reported back to their
// conversation.
type OutcomePoster interface {
// UnpostedOutcomes returns finished chat tasks whose outcome has not been
// posted to the conversation that created them.
UnpostedOutcomes(ctx context.Context) ([]*task.Task, error)
// MarkOutcomePosted records that the task's state at status was posted.
MarkOutcomePosted(ctx context.Context, taskID int64, status string) error
}
// TaskQueries groups read-only task accessors.
type TaskQueries interface {
TaskByIssue(ctx context.Context, owner, repo string, number int) (*task.Task, error)
OpenTaskByPR(ctx context.Context, owner, repo string, number int) (*task.Task, error)
TaskByID(ctx context.Context, taskID int64) (*task.Task, error)
OpenPRs(ctx context.Context) ([]task.Task, error)
// ActiveTasksByOrigin returns the queued and running tasks the
// conversation origin created, for its /stop.
ActiveTasksByOrigin(ctx context.Context, origin string) ([]*task.Task, error)
// Tasks returns the most recently updated tasks.
Tasks(ctx context.Context, limit int) ([]task.Task, error)
// TasksPage returns one page of tasks, most recently updated first.
TasksPage(ctx context.Context, page TaskPage) ([]task.Task, error)
StatusCounts(ctx context.Context) (map[string]int, error)
}
// TaskEvents groups observability and lifecycle methods.
type TaskEvents interface {
InsertEvent(ctx context.Context, e events.Event) (int64, error)
EventsSince(ctx context.Context, cursor string, limit int) ([]events.Event, error)
// TaskEvents returns a task's timeline, without agent transcripts.
TaskEvents(ctx context.Context, taskID int64) ([]events.Event, error)
// TaskTranscript returns one attempt's agent transcript entries in order.
TaskTranscript(ctx context.Context, taskID int64, attempt int) ([]events.Event, error)
WorkflowStats(ctx context.Context) ([]WorkflowStat, error)
StageStats(ctx context.Context) ([]StageStat, error)
TokensByDay(ctx context.Context, days int) ([]DayTokens, error)
Close() error
}
// TaskPage selects one page of the task list. Empty Statuses matches every
// status. A zero After starts at the newest task; otherwise the page begins
// after the task it names, in (updated_at, id) descending order.
type TaskPage struct {
Statuses []string
// Workflow, when set, keeps only that workflow's tasks.
Workflow string
After TaskCursor
Limit int
}
// TaskCursor is the position of the last task a page returned.
type TaskCursor struct {
UpdatedAt time.Time
ID int64
}
// WorkflowCaller starts a workflow.call callee and reads it back.
type WorkflowCaller = task.Caller
// TaskLogStore reads one task attempt's log. Found=false means no log;
// ErrTaskLogsUnavailable means this process cannot read logs.
type TaskLogStore interface {
// TaskLog returns a page of one attempt's decoded log. attempt 0 selects
// the task's current attempt.
TaskLog(ctx context.Context, taskID int64, attempt int, q logging.Query) (logging.TaskLogPage, error)
// TaskLogContent writes one attempt's log verbatim to w, for a download.
// found is false, with a nil error, when the attempt has no log file.
TaskLogContent(ctx context.Context, taskID int64, attempt int, w io.Writer) (found bool, err error)
}
// WorkflowStat is one row of the per-workflow metrics table.
type WorkflowStat struct {
Workflow string `json:"workflow"`
Runs int `json:"runs"`
Merged int `json:"merged"`
Completed int `json:"completed"`
PROpen int `json:"pr_open"`
Parked int `json:"parked"`
AvgTokens int `json:"avg_tokens"`
AvgSteps float64 `json:"avg_steps"`
TotalToken int `json:"total_tokens"`
// Running and LastStatus describe the workflow now: how many of its runs
// are working, and how its newest one ended.
Running int `json:"running"`
LastStatus string `json:"last_status"`
}
// StageStat is average stage duration and failure counts per stage.
type StageStat struct {
Workflow string `json:"workflow"`
Stage string `json:"stage"`
Runs int `json:"runs"`
AvgMs int `json:"avg_ms"`
Errors int `json:"errors"`
}
// DayTokens is token spend per UTC day.
type DayTokens struct {
Day string `json:"day"`
Tokens int `json:"tokens"`
}
+1 -1
View File
@@ -71,7 +71,7 @@ func (a AgentStage) Stage() Stage {
// The agent call is a child of the stage's recorded step: the store
// says what the stage's runtime did, not just what the stage did.
res, runErr := tc.RunAgentChild(ctx, a.Name, a.Model, modelRef, func() (agentrun.Result, error) {
return tc.Agent.Run(ctx, tc.Dir, req, tc.toolCallReporter(ctx, a.Name))
return tc.Agent.Run(ctx, tc.Dir, req, tc.reporter(ctx, a.Name))
})
return a.handleResult(ctx, tc, req, res, runErr, modelRef)
}}
@@ -15,7 +15,7 @@ import (
// fakeBudgetRunner is an agent runner whose stage exhausted its budget.
type fakeBudgetRunner struct{ res agentrun.Result }
func (f fakeBudgetRunner) Run(context.Context, string, agentrun.Request, agentrun.ToolCallReporter) (agentrun.Result, error) {
func (f fakeBudgetRunner) Run(context.Context, string, agentrun.Request, agentrun.Reporter) (agentrun.Result, error) {
return f.res, nil
}
+1 -1
View File
@@ -97,7 +97,7 @@ func runBaselineFix(ctx context.Context, tc *TaskContext, argv []string, out []b
}
res, agentErr := tc.RunAgentChild(ctx, "baseline-fix", "", modelRef, func() (agentrun.Result, error) {
return tc.Agent.Run(ctx, tc.Dir, req, tc.toolCallReporter(ctx, "baseline-fix"))
return tc.Agent.Run(ctx, tc.Dir, req, tc.reporter(ctx, "baseline-fix"))
})
if agentErr != nil && res.Version == 0 {
return baselineFixFailedError(argv, string(out), fmt.Sprintf("the builder could not run: %v", agentErr))
+2 -2
View File
@@ -579,7 +579,7 @@ func runReviewerAgent(
if err != nil {
return agentrun.Result{}, err
}
res, runErr := tc.Agent.Run(ctx, tc.prReview.snapshotDir, req, tc.toolCallReporter(ctx, name))
res, runErr := tc.Agent.Run(ctx, tc.prReview.snapshotDir, req, tc.reporter(ctx, name))
tc.prReview.tokensSpent.Add(int64(res.TokensUsed))
to, detail := taskstate.StepSucceeded, res.Summary
switch {
@@ -805,7 +805,7 @@ func runPRReviewAgentRecorded(
CaptureTools: captureTools,
}
res, err := tc.RunAgentChild(ctx, name, "", modelRef, func() (agentrun.Result, error) {
return tc.Agent.Run(ctx, workspace, req, tc.toolCallReporter(ctx, name))
return tc.Agent.Run(ctx, workspace, req, tc.reporter(ctx, name))
})
tc.prReview.tokensSpent.Add(int64(res.TokensUsed))
return res, err
+5
View File
@@ -45,6 +45,11 @@ type recordedForge struct {
deliveries Deliveries
}
func (f recordedForge) CreateIssue(ctx context.Context, owner, repo, title, _ string, _ []string) (int, error) {
f.deliveries.Record(ctx, fmt.Sprintf("open issue %q in %s/%s", title, owner, repo))
return 0, nil
}
func (f recordedForge) CloseIssue(ctx context.Context, owner, repo string, number int, _ string) error {
f.deliveries.Record(ctx, fmt.Sprintf("close issue %s/%s#%d", owner, repo, number))
return nil
@@ -64,7 +64,7 @@ type fakePromptRunner struct {
mission string
}
func (f *fakePromptRunner) Run(_ context.Context, workspace string, req agentrun.Request, _ agentrun.ToolCallReporter) (agentrun.Result, error) {
func (f *fakePromptRunner) Run(_ context.Context, workspace string, req agentrun.Request, _ agentrun.Reporter) (agentrun.Result, error) {
f.workspace = workspace
f.mission = req.Mission
return f.res, nil
+33 -10
View File
@@ -206,7 +206,7 @@ func (tc *TaskContext) Emit(ctx context.Context, kind, stage, detail string, dat
// Activity belongs to the recorded stage, even when an agent uses its own name.
func (tc *TaskContext) activityData(kind string, data map[string]any) map[string]any {
if tc.StepID == 0 || (kind != events.KindToolCall && kind != events.KindAgentFinish) {
if tc.StepID == 0 || (kind != events.KindToolCall && kind != events.KindAgentFinish && kind != events.KindAgentTranscript) {
return data
}
out := make(map[string]any, len(data)+2)
@@ -239,18 +239,41 @@ func (tc *TaskContext) EmitDurable(ctx context.Context, kind, stage, detail stri
return nil
}
// toolCallReporter builds the agentrun.ToolCallReporter an agent stage passes
// to tc.Agent.Run, so every completed tool call during that run surfaces as a
// tool_call event on the task timeline. Safe to call on a nil Bus: Emit no-ops.
func (tc *TaskContext) toolCallReporter(ctx context.Context, stage string) agentrun.ToolCallReporter {
return func(report agentrun.ToolCallReport) {
tc.Emit(ctx, events.KindToolCall, stage, report.Detail, map[string]any{
"tool": report.Tool,
"failed": report.Failed,
})
// reporter builds the agentrun.Reporter an agent stage passes to
// tc.Agent.Run: every completed tool call surfaces as a tool_call event on the
// task timeline, and every transcript entry is persisted as an
// agent_transcript event so it outlives the container. A transcript entry
// that cannot be persisted is logged and dropped; it never fails the run.
func (tc *TaskContext) reporter(ctx context.Context, stage string) agentrun.Reporter {
return agentrun.Reporter{
ToolCall: func(report agentrun.ToolCallReport) {
tc.Emit(ctx, events.KindToolCall, stage, report.Detail, map[string]any{
"tool": report.Tool,
"failed": report.Failed,
})
},
Transcript: func(entry agentrun.TranscriptEntry) {
data, err := transcriptData(entry)
if err == nil {
err = tc.EmitDurable(ctx, events.KindAgentTranscript, stage, entry.Kind, data)
}
if err != nil && tc.Log != nil {
tc.Log.WarnContext(ctx, "agent transcript entry not recorded",
"task", tc.Task.ID, "stage", stage, "seq", entry.Seq, "err", err)
}
},
}
}
func transcriptData(entry agentrun.TranscriptEntry) (map[string]any, error) {
encoded, err := json.Marshal(entry)
if err != nil {
return nil, err
}
var data map[string]any
return data, json.Unmarshal(encoded, &data)
}
// Outcome is a stage's terminal decision for the whole workflow. Stages
// that don't end the workflow leave it zero.
type Outcome struct {
+5 -1
View File
@@ -22,8 +22,12 @@ const (
KindAgentFinish = "agent_finish" // data: status, stop_reason, tokens, iterations, model, prompt_tokens, completion_tokens, cached_tokens, cache_creation_tokens
// KindToolCall marks one completed tool invocation during an agent stage
// run. Only emitted for runners that execute in the same process as their
// tools; see agentexec.ToolCallReporter.
// tools; see agentrun.Reporter.
KindToolCall = "tool_call" // data: tool, failed
// KindAgentTranscript is one entry of an agent run's conversation: a
// prompt, a model response, a tool call or a compaction. detail: the
// entry kind; data: agentrun.TranscriptEntry's fields.
KindAgentTranscript = "agent_transcript"
// KindDeliveryRecorded records one outward delivery a test run did not
// perform: a forge write, a push, an HTTP request, a central tool call.
// detail: what it would have done, phrased "would ...".
+4
View File
@@ -38,6 +38,10 @@ type IssueForge interface {
// Comment posts an issue (or PR) comment and returns its id.
Comment(ctx context.Context, owner, repo string, number int, body string) (int64, error)
// CreateIssue opens an issue with title, body and optional labels,
// returning its number.
CreateIssue(ctx context.Context, owner, repo, title, body string, labels []string) (int, error)
// CloseIssue closes an issue with an optional final comment.
CloseIssue(ctx context.Context, owner, repo string, number int, comment string) error
+4
View File
@@ -30,6 +30,10 @@ func (n *NoopForge) Comment(ctx context.Context, owner, repo string, number int,
return 0, nil
}
func (n *NoopForge) CreateIssue(ctx context.Context, owner, repo, title, body string, labels []string) (int, error) {
return 0, nil
}
func (n *NoopForge) CloseIssue(ctx context.Context, owner, repo string, number int, comment string) error {
return nil
}
+43 -2
View File
@@ -23,6 +23,7 @@ import (
const (
SubjectComment = "archie.forge.comment"
SubjectCreateIssue = "archie.forge.create_issue"
SubjectCloseIssue = "archie.forge.close_issue"
SubjectCreatePR = "archie.forge.create_pr"
SubjectLinkBranch = "archie.forge.link_branch"
@@ -35,8 +36,9 @@ const (
// set a task of that identity may publish to.
func Subjects(identity string) []string {
bases := []string{
SubjectComment, SubjectCloseIssue, SubjectCreatePR, SubjectLinkBranch,
SubjectSetStateLabel, SubjectCreateReviewComment, SubjectReplyToReview,
SubjectComment, SubjectCreateIssue, SubjectCloseIssue, SubjectCreatePR,
SubjectLinkBranch, SubjectSetStateLabel, SubjectCreateReviewComment,
SubjectReplyToReview,
}
out := make([]string, len(bases))
for i, base := range bases {
@@ -80,6 +82,17 @@ type CommentResponse struct {
natsrpc.Envelope
}
type CreateIssueRequest struct {
Target
Title, Body string
Labels []string
}
type CreateIssueResponse struct {
Number int `json:"number"`
natsrpc.Envelope
}
type CloseIssueRequest struct {
Target
Number int
@@ -168,6 +181,7 @@ func (s *Server) RegisterFor(nc *nats.Conn, identity string) (unsubscribe func()
}
return natsrpc.RegisterAll(nc, []natsrpc.Registration{
{Subject: serverSubject(identity, SubjectComment), Handler: as(s.handleComment)},
{Subject: serverSubject(identity, SubjectCreateIssue), Handler: as(s.handleCreateIssue)},
{Subject: serverSubject(identity, SubjectCloseIssue), Handler: as(s.handleCloseIssue)},
{Subject: serverSubject(identity, SubjectCreatePR), Handler: as(s.handleCreatePR)},
{Subject: serverSubject(identity, SubjectLinkBranch), Handler: as(s.handleLinkBranch)},
@@ -249,6 +263,21 @@ func (s *Server) handleComment(ctx context.Context, identity string, msg *nats.M
s.respond(msg, CommentResponse{ID: id, Envelope: natsrpc.NewEnvelope(err)})
}
func (s *Server) handleCreateIssue(ctx context.Context, identity string, msg *nats.Msg) {
var req CreateIssueRequest
if err := json.Unmarshal(msg.Data, &req); err != nil {
s.respond(msg, CreateIssueResponse{Envelope: natsrpc.NewEnvelope(fmt.Errorf("decode create_issue request: %w", err))})
return
}
client, err := s.authorizedForge(ctx, identity, req.Target, "comment")
if err != nil {
s.respond(msg, CreateIssueResponse{Envelope: natsrpc.NewEnvelope(err)})
return
}
number, err := client.CreateIssue(ctx, req.Owner, req.Repo, req.Title, req.Body, req.Labels)
s.respond(msg, CreateIssueResponse{Number: number, Envelope: natsrpc.NewEnvelope(err)})
}
func (s *Server) handleCloseIssue(ctx context.Context, identity string, msg *nats.Msg) {
var req CloseIssueRequest
if err := json.Unmarshal(msg.Data, &req); err != nil {
@@ -392,6 +421,18 @@ func (c *Client) Comment(ctx context.Context, owner, repo string, number int, bo
return resp.ID, nil
}
func (c *Client) CreateIssue(ctx context.Context, owner, repo, title, body string, labels []string) (int, error) {
req := CreateIssueRequest{Target: c.target(owner, repo), Title: title, Body: body, Labels: labels}
resp, err := natsrpc.Call[CreateIssueResponse](ctx, c.rpc(), c.subject(SubjectCreateIssue), req)
if err != nil {
return 0, err
}
if err := resp.Err(); err != nil {
return 0, err
}
return resp.Number, nil
}
func (c *Client) CloseIssue(ctx context.Context, owner, repo string, number int, comment string) error {
req := CloseIssueRequest{Target: c.target(owner, repo), Number: number, Comment: comment}
resp, err := natsrpc.Call[Response](ctx, c.rpc(), c.subject(SubjectCloseIssue), req)
+137
View File
@@ -0,0 +1,137 @@
package gateway
import (
"context"
"fmt"
"time"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/tools"
)
// RunSource is the state-store surface the run-inspection tools use. The
// State Store client satisfies it directly.
type RunSource interface {
// Tasks returns the most recently updated tasks.
Tasks(ctx context.Context, limit int) ([]task.Task, error)
// TaskEvents returns one task's events, oldest first.
TaskEvents(ctx context.Context, taskID int64) ([]events.Event, error)
}
// RunSummary is one task as run_list shows it.
type RunSummary struct {
ID int64 `json:"id"`
Repo string `json:"repo"`
Title string `json:"title"`
Status string `json:"status"`
Workflow string `json:"workflow,omitempty"`
Issue int `json:"issue,omitempty"`
PRNumber int `json:"pr_number,omitempty"`
ParkReason string `json:"park_reason,omitempty"`
UpdatedAt time.Time `json:"updated_at"`
}
// RunEvent is one task event as run_events shows it.
type RunEvent struct {
At time.Time `json:"at"`
Kind string `json:"kind"`
Stage string `json:"stage,omitempty"`
Detail string `json:"detail,omitempty"`
}
// RunTools returns the read-only run-inspection tools for long-running
// curators: what ran recently, and each run's event trail. Reads are checked
// as reading tasks, the same check the dashboard's task list gets. Nil
// source or authorizer registers none.
func RunTools(source RunSource, authz access.Authorizer) []tools.ToolEntry {
if source == nil || authz == nil {
return nil
}
read := func(entry tools.ToolEntry) tools.ToolEntry {
return guarded(authz, access.ActionRead, access.KindTask, nil, entry)
}
return []tools.ToolEntry{
read(runListTool(source)),
read(runEventsTool(source)),
}
}
func runListTool(source RunSource) tools.ToolEntry {
return tools.ToolEntry{
Name: "run_list",
Toolset: "runs",
Description: "List the most recently updated task runs across the instance, most recent first. " +
"Use it to survey what ran, what parked and why.",
Classification: tools.ClassIdempotent,
Schema: tools.JSONSchema{
"type": "object",
"properties": map[string]any{
"limit": map[string]any{
"type": "integer",
"description": fmt.Sprintf(
"Maximum runs to return. Defaults to %d, capped at %d.",
defaultTaskListLimit, maxTaskListLimit,
),
},
},
},
Handler: func(ctx context.Context, input map[string]any) (any, error) {
rows, err := source.Tasks(ctx, taskListLimit(input))
if err != nil {
return nil, fmt.Errorf("run_list: %w", err)
}
out := make([]RunSummary, 0, len(rows))
for _, t := range rows {
out = append(out, RunSummary{
ID: t.ID,
Repo: t.Owner + "/" + t.Repo,
Title: t.Title,
Status: t.Status,
Workflow: t.Workflow,
Issue: t.IssueNumber,
PRNumber: t.PRNumber,
ParkReason: t.ParkReason,
UpdatedAt: t.UpdatedAt,
})
}
return out, nil
},
}
}
func runEventsTool(source RunSource) tools.ToolEntry {
return tools.ToolEntry{
Name: "run_events",
Toolset: "runs",
Description: "List one task run's event trail, oldest first: stage transitions, deliveries, " +
"failures and park reasons. Use it to see why a run ended the way it did.",
Classification: tools.ClassIdempotent,
Schema: tools.JSONSchema{
"type": "object",
"properties": map[string]any{
"task_id": map[string]any{
"type": "integer",
"description": "The task run's ID, as run_list reports it.",
},
},
"required": []string{"task_id"},
},
Handler: func(ctx context.Context, input map[string]any) (any, error) {
id, ok := asInt64(input["task_id"])
if !ok || id <= 0 {
return nil, fmt.Errorf("run_events: task_id must be a positive integer")
}
evs, err := source.TaskEvents(ctx, id)
if err != nil {
return nil, fmt.Errorf("run_events: %w", err)
}
out := make([]RunEvent, 0, len(evs))
for _, e := range evs {
out = append(out, RunEvent{At: e.At, Kind: e.Kind, Stage: e.Stage, Detail: e.Detail})
}
return out, nil
},
}
}
@@ -112,3 +112,84 @@ func TestWorkspaceEnvironment(t *testing.T) {
})
}
}
// TestCanaryGrant pins the scoped canary grant: the one identity it names may
// read the signal surface, read and write update-settings, and trigger an
// install -- and nothing else, neither for it nor for anyone else. The canary
// principal carries the developer role (API-created service accounts are
// assigned it); the grant's forbid must keep the role from widening access.
func TestCanaryGrant(t *testing.T) {
const canaryID = identity.IdentityID("canary-svc")
grant := access.CanaryGrantPolicy(org.DefaultOrgID, string(canaryID))
forbid := access.CanaryGrantForbidPolicy(org.DefaultOrgID, string(canaryID))
engine := New(append(access.ShippedOrgPolicies(org.DefaultOrgID), grant, forbid))
if len(engine.Problems()) != 0 {
t.Fatalf("problems = %v, want the canary grant to compile", engine.Problems())
}
canary := access.Principal{
IdentityID: canaryID, Org: org.DefaultOrgID,
Memberships: []org.Membership{{IdentityID: canaryID, OrgID: org.DefaultOrgID, Role: org.RoleDeveloper}},
}
other := access.Principal{IdentityID: "someone-else", Org: org.DefaultOrgID}
at := func(kind access.ResourceKind, id string) access.Resource {
return access.Resource{Kind: kind, ID: id, Org: org.DefaultOrgID}
}
tests := []struct {
name string
principal access.Principal
action access.Action
resource access.Resource
allowed bool
}{
{"read the dashboard", canary, access.ActionRead, at(access.KindDashboard, ""), true},
{"read tasks", canary, access.ActionRead, at(access.KindTask, ""), true},
{"read one task", canary, access.ActionRead, at(access.KindTask, "7"), true},
{"read update-settings", canary, access.ActionRead, at(access.KindPolicy, "update-settings"), true},
{"install an update", canary, access.ActionUpdate, at(access.KindDashboard, "update"), true},
{"replace update-settings", canary, access.ActionManageIdentities, at(access.KindPolicy, "update-settings"), true},
{"read a secret", canary, access.ActionReadSecret, at(access.KindSecret, "github-token"), false},
{"delete a task", canary, access.ActionDelete, at(access.KindTask, "7"), false},
{"manage policies", canary, access.ActionManagePolicies, at(access.KindPolicy, "org-role-read"), false},
{"run a workflow", canary, access.ActionRun, at(access.KindWorkflow, "triage"), false},
{"create anything", canary, access.ActionCreate, at(access.KindTask, ""), false},
{"update another object", canary, access.ActionUpdate, at(access.KindDashboard, "other"), false},
{"read another object", canary, access.ActionRead, at(access.KindPolicy, "other-policy"), false},
{"manage identities on another object", canary, access.ActionManageIdentities, at(access.KindIdentity, "someone-else"), false},
{"read logs", canary, access.ActionReadLogs, at(access.KindTask, "7"), false},
{"read the audit feed", canary, access.ActionReadAudit, at(access.KindAudit, ""), false},
{"a resource of another org", canary, access.ActionRead, access.Resource{Kind: access.KindDashboard, Org: "acme"}, false},
{"another identity reads the dashboard", other, access.ActionRead, at(access.KindDashboard, ""), false},
{"another identity installs an update", other, access.ActionUpdate, at(access.KindDashboard, "update"), false},
{"another identity replaces update-settings", other, access.ActionManageIdentities, at(access.KindPolicy, "update-settings"), false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := engine.Authorize(tt.principal, tt.action, tt.resource, access.Context{})
if got.Allowed != tt.allowed {
t.Fatalf("allowed = %v, want %v (%+v)", got.Allowed, tt.allowed, got)
}
})
}
}
// TestCanaryGrantValidates pins that the canary grant passes the domain's
// structural check and the engine's schema validation, so it can be installed
// through the policy store.
func TestCanaryGrantValidates(t *testing.T) {
grant := access.CanaryGrantPolicy(org.DefaultOrgID, "canary-svc")
forbid := access.CanaryGrantForbidPolicy(org.DefaultOrgID, "canary-svc")
for _, p := range []access.Policy{grant, forbid} {
if err := p.Validate(); err != nil {
t.Fatalf("Validate %s: %v", p.ID, err)
}
}
engine := New([]access.Policy{grant, forbid})
for _, p := range []access.Policy{grant, forbid} {
if err := engine.Validate(p); err != nil {
t.Fatalf("engine Validate %s: %v", p.ID, err)
}
}
if len(engine.Problems()) != 0 {
t.Fatalf("problems = %v, want none", engine.Problems())
}
}
@@ -52,3 +52,13 @@ func DefaultUpdateStateDir() string {
}
return filepath.Join(dir, InstanceName(), "update")
}
// UpdateStateDir is the one resolution of the host's update state directory:
// ARCHIE_UPDATE_STATE_DIR when set, else the default. The daemon's
// transactions and the dashboard's update snapshot must read the same one.
func UpdateStateDir() string {
if dir := os.Getenv("ARCHIE_UPDATE_STATE_DIR"); dir != "" {
return dir
}
return DefaultUpdateStateDir()
}
@@ -136,24 +136,56 @@ func RunParams(ctx context.Context, p Prompter, discovery ModelDiscovery, secret
}
// stepDatabase returns the PostgreSQL URL. A blank answer on a fresh install
// means the bundled database: no network, reached through its Unix socket, with
// a generated password that goes only to the env file as PGPASSWORD, which pgx
// and Compose both read, so config.toml holds no secret.
// means the bundled database. On a re-run with an existing URL the bundled
// database is an explicit choice rather than something reachable only by
// typing its socket URL; keeping the existing URL is the first option, so
// the default answer is unchanged.
func stepDatabase(ctx context.Context, p Prompter, secrets SecretSink, existing, param string) (string, error) {
if strings.TrimSpace(param) != "" {
return param, nil
}
prompt := "PostgreSQL 18 URL (blank for the bundled database with a generated password): "
if existing != "" {
prompt = "PostgreSQL 18 URL: "
if strings.TrimSpace(existing) != "" {
choice, err := p.Select(ctx, "Database:", []string{
"Keep the existing database URL",
"Bundled database (Unix socket, generated password in the env file)",
"Enter a different PostgreSQL 18 URL",
})
if err != nil {
return "", fmt.Errorf("setup: database choice: %w", err)
}
switch choice {
case 0:
return existing, nil
case 1:
return bundledDatabase(secrets)
default:
url, err := p.ReadLine(ctx, "PostgreSQL 18 URL: ", "")
if err != nil {
return "", fmt.Errorf("setup: database url: %w", err)
}
if strings.TrimSpace(url) == "" {
// A blank answer keeps the existing URL, as it always has.
return existing, nil
}
return url, nil
}
}
url, err := p.ReadLine(ctx, prompt, existing)
prompt := "PostgreSQL 18 URL (blank for the bundled database with a generated password): "
url, err := p.ReadLine(ctx, prompt, "")
if err != nil {
return "", fmt.Errorf("setup: database url: %w", err)
}
if strings.TrimSpace(url) != "" {
return url, nil
}
return bundledDatabase(secrets)
}
// bundledDatabase switches to the bundled database: no network, reached
// through its Unix socket, with a generated password that goes only to the
// env file as PGPASSWORD, which pgx and Compose both read, so config.toml
// holds no secret.
func bundledDatabase(secrets SecretSink) (string, error) {
pw := make([]byte, 24)
if _, err := rand.Read(pw); err != nil {
return "", fmt.Errorf("setup: generate database password: %w", err)
@@ -0,0 +1,212 @@
package setup
import (
"context"
"strings"
"testing"
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
"github.com/samcharles93/archie-core/internal/infrastructure/configuration/tomlwrite"
)
// fakePrompter answers prompts from maps keyed by the exact prompt text and
// records what it was asked, so a test can pin both the answers and that the
// bundled database was actually offered.
type fakePrompter struct {
selectAnswers map[string]int
lineAnswers map[string]string
confirmAnswer bool
selects []recordedSelect
lines []recordedLine
}
type recordedSelect struct {
prompt string
options []string
}
type recordedLine struct {
prompt string
defaultValue string
}
func (f *fakePrompter) Select(_ context.Context, prompt string, options []string) (int, error) {
f.selects = append(f.selects, recordedSelect{prompt: prompt, options: options})
if ans, ok := f.selectAnswers[prompt]; ok {
return ans, nil
}
return 0, nil
}
func (f *fakePrompter) ReadLine(_ context.Context, prompt, defaultValue string) (string, error) {
f.lines = append(f.lines, recordedLine{prompt: prompt, defaultValue: defaultValue})
if ans, ok := f.lineAnswers[prompt]; ok {
return ans, nil
}
return defaultValue, nil
}
func (f *fakePrompter) ReadSecret(context.Context, string) (string, error) { return "", nil }
func (f *fakePrompter) Confirm(context.Context, string, bool) (bool, error) {
return f.confirmAnswer, nil
}
// fakeSink records Put calls by engine/key.
type fakeSink struct {
puts map[string]string
}
func (s *fakeSink) Put(engine, key, value string) error {
if s.puts == nil {
s.puts = map[string]string{}
}
s.puts[engine+"/"+key] = value
return nil
}
func (s *fakeSink) Commit() error { return nil }
// databasePrompt is the re-run database choice; bundledChoice is the index of
// the bundled database option in that menu.
const (
databasePrompt = "Database:"
bundledChoice = 1
differentURL = 2
)
func bundledURL() string {
return "postgres://archie@/archie?host=" + configuration.DefaultPostgresSocketDir()
}
// TestRunParamsDatabaseChoice pins the re-run database step at the setup API
// level: with an existing database_url the flow must offer the bundled
// database as an explicit choice, switching to it must generate PGPASSWORD
// into the sink, and keeping must leave the existing URL untouched.
func TestRunParamsDatabaseChoice(t *testing.T) {
const existing = "postgres://operator:secret@db.internal:5432/archie"
tests := []struct {
name string
// existing is the prefilled database_url; empty means fresh install.
existing string
// param is Params.DatabaseURL (the -database-url flag).
param string
selectAnswers map[string]int
lineAnswers map[string]string
wantURL string
wantPassword bool
wantChoice bool // a database Select menu must be presented
}{
{
name: "re-run keep existing is the default choice",
existing: existing,
selectAnswers: map[string]int{databasePrompt: 0},
wantURL: existing,
wantChoice: true,
},
{
name: "re-run switch to the bundled database",
existing: existing,
selectAnswers: map[string]int{databasePrompt: bundledChoice},
wantURL: bundledURL(),
wantPassword: true,
wantChoice: true,
},
{
name: "re-run enter a different URL",
existing: existing,
selectAnswers: map[string]int{databasePrompt: differentURL},
lineAnswers: map[string]string{"PostgreSQL 18 URL: ": "postgres://other.internal/archie"},
wantURL: "postgres://other.internal/archie",
wantChoice: true,
},
{
name: "re-run blank answer on the different-URL prompt keeps existing",
existing: existing,
selectAnswers: map[string]int{databasePrompt: differentURL},
lineAnswers: map[string]string{"PostgreSQL 18 URL: ": ""},
wantURL: existing,
wantChoice: true,
},
{
name: "fresh install blank answer means the bundled database",
existing: "",
lineAnswers: map[string]string{"PostgreSQL 18 URL (blank for the bundled database with a generated password): ": ""},
wantURL: bundledURL(),
wantPassword: true,
wantChoice: false,
},
{
name: "database-url flag wins without prompting",
existing: existing,
param: "postgres://flag.internal/archie",
wantURL: "postgres://flag.internal/archie",
wantChoice: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
p := &fakePrompter{
selectAnswers: tt.selectAnswers,
lineAnswers: tt.lineAnswers,
confirmAnswer: false,
}
sink := &fakeSink{}
params := Params{
DatabaseURL: tt.param,
BotUser: "archie-bot",
ForgeType: "none",
Provider: "ollama",
Model: "llama3",
}
edits, err := RunParams(context.Background(), p, nil, sink, ExistingValues{DatabaseURL: tt.existing}, params)
if err != nil {
t.Fatalf("RunParams: %v", err)
}
var gotURL string
for _, e := range edits {
if e.Table == "" && e.Key == "database_url" {
gotURL = e.Value
}
}
if gotURL != tomlwrite.String(tt.wantURL) {
t.Errorf("database_url = %s, want %s", gotURL, tomlwrite.String(tt.wantURL))
}
password, hasPassword := sink.puts["env/PGPASSWORD"]
if hasPassword != tt.wantPassword {
t.Fatalf("PGPASSWORD stored = %v, want %v", hasPassword, tt.wantPassword)
}
if tt.wantPassword && len(password) != 48 {
t.Errorf("PGPASSWORD = %q, want a 48-character hex password", password)
}
var offered []string
for _, sel := range p.selects {
if sel.prompt == databasePrompt {
offered = sel.options
}
}
if tt.wantChoice {
if offered == nil {
t.Fatalf("no database choice was presented on a re-run; prompts seen: %+v", p.selects)
}
bundled := false
for _, opt := range offered {
if strings.Contains(strings.ToLower(opt), "bundled") {
bundled = true
}
}
if !bundled {
t.Errorf("database choice options %v do not offer the bundled database", offered)
}
} else if offered != nil {
t.Errorf("database choice was presented (%v) where the flow should not ask", offered)
}
})
}
}
@@ -195,6 +195,16 @@ func (f *Forge) Comment(ctx context.Context, owner, repo string, number int, bod
return resp.GetId(), err
}
func (f *Forge) CreateIssue(ctx context.Context, owner, repo, title, body string, labels []string) (int, error) {
resp, err := call(ctx, f, "create issue", func(c forgev1.ForgeServiceClient) (*forgev1.CreateIssueResponse, error) {
return c.CreateIssue(ctx, &forgev1.CreateIssueRequest{Repo: ref(owner, repo), Title: title, Body: body, Labels: labels})
})
if err != nil {
return 0, err
}
return int(resp.GetNumber()), nil
}
func (f *Forge) CloseIssue(ctx context.Context, owner, repo string, number int, comment string) error {
_, err := call(ctx, f, "close issue", func(c forgev1.ForgeServiceClient) (*forgev1.CloseIssueResponse, error) {
return c.CloseIssue(ctx, &forgev1.CloseIssueRequest{Repo: ref(owner, repo), Number: int32(number), Comment: comment}) // issue numbers fit int32
+11 -1
View File
@@ -90,7 +90,17 @@ func eventsFromRows(rows []postgresdb.Event) []events.Event {
return out
}
// TaskEvents returns a task's full timeline, oldest first.
// TaskTranscript returns one attempt's agent transcript entries, oldest first.
func (s *Store) TaskTranscript(ctx context.Context, taskID int64, attempt int) ([]events.Event, error) {
rows, err := s.queries().TaskTranscript(ctx, postgresdb.TaskTranscriptParams{TaskID: taskID, Attempt: int64(attempt), ScopeOrg: scopeOrg(ctx)})
if err != nil {
return nil, err
}
return eventsFromRows(rows), nil
}
// TaskEvents returns a task's timeline without agent transcripts, oldest
// first.
func (s *Store) TaskEvents(ctx context.Context, taskID int64) ([]events.Event, error) {
rows, err := s.queries().TaskEventsByID(ctx, postgresdb.TaskEventsByIDParams{TaskID: taskID, ScopeOrg: scopeOrg(ctx)})
if err != nil {
@@ -228,7 +228,7 @@ func (q *Queries) StageStats(ctx context.Context, scopeOrg pgtype.Text) ([]Stage
const taskEventsByID = `-- name: TaskEventsByID :many
SELECT id, at, kind, task_id, repo, issue, workflow, stage, attempt, actor_id, actor_kind, principal_id, detail, data, org_id, workspace_id
FROM events
WHERE task_id = $1 AND ($2::text IS NULL OR org_id = $2)
WHERE task_id = $1 AND kind <> 'agent_transcript' AND ($2::text IS NULL OR org_id = $2)
ORDER BY id
`
@@ -274,6 +274,56 @@ func (q *Queries) TaskEventsByID(ctx context.Context, arg TaskEventsByIDParams)
return items, nil
}
const taskTranscript = `-- name: TaskTranscript :many
SELECT id, at, kind, task_id, repo, issue, workflow, stage, attempt, actor_id, actor_kind, principal_id, detail, data, org_id, workspace_id
FROM events
WHERE task_id = $1 AND attempt = $2 AND kind = 'agent_transcript' AND ($3::text IS NULL OR org_id = $3)
ORDER BY id
`
type TaskTranscriptParams struct {
TaskID int64
Attempt int64
ScopeOrg pgtype.Text
}
func (q *Queries) TaskTranscript(ctx context.Context, arg TaskTranscriptParams) ([]Event, error) {
rows, err := q.db.Query(ctx, taskTranscript, arg.TaskID, arg.Attempt, arg.ScopeOrg)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Event
for rows.Next() {
var i Event
if err := rows.Scan(
&i.ID,
&i.At,
&i.Kind,
&i.TaskID,
&i.Repo,
&i.Issue,
&i.Workflow,
&i.Stage,
&i.Attempt,
&i.ActorID,
&i.ActorKind,
&i.PrincipalID,
&i.Detail,
&i.Data,
&i.OrgID,
&i.WorkspaceID,
); err != nil {
return nil, err
}
items = append(items, i)
}
if err := rows.Err(); err != nil {
return nil, err
}
return items, nil
}
const tokensByDay = `-- name: TokensByDay :many
WITH event_totals AS (
SELECT task_id, to_char(at AT TIME ZONE 'UTC', 'YYYY-MM-DD') AS day,
@@ -35,7 +35,13 @@ LIMIT sqlc.arg(page_limit);
-- name: TaskEventsByID :many
SELECT id, at, kind, task_id, repo, issue, workflow, stage, attempt, actor_id, actor_kind, principal_id, detail, data, org_id, workspace_id
FROM events
WHERE task_id = sqlc.arg(task_id) AND (sqlc.narg(scope_org)::text IS NULL OR org_id = sqlc.narg(scope_org))
WHERE task_id = sqlc.arg(task_id) AND kind <> 'agent_transcript' AND (sqlc.narg(scope_org)::text IS NULL OR org_id = sqlc.narg(scope_org))
ORDER BY id;
-- name: TaskTranscript :many
SELECT id, at, kind, task_id, repo, issue, workflow, stage, attempt, actor_id, actor_kind, principal_id, detail, data, org_id, workspace_id
FROM events
WHERE task_id = sqlc.arg(task_id) AND attempt = sqlc.arg(attempt) AND kind = 'agent_transcript' AND (sqlc.narg(scope_org)::text IS NULL OR org_id = sqlc.narg(scope_org))
ORDER BY id;
-- name: WorkflowStats :many
@@ -90,6 +90,40 @@ func (s *accountServer) RevokePersonalToken(ctx context.Context, r *pb.RevokePer
return &pb.RevokePersonalTokenResponse{}, nil
}
// BindIdentityPersonalToken binds a token to a named non-user identity, the
// credential a service account acts with. The audit actor is the caller's
// principal, so the request never names who did it.
func (s *accountServer) BindIdentityPersonalToken(ctx context.Context, r *pb.BindIdentityPersonalTokenRequest) (*pb.BindIdentityPersonalTokenResponse, error) {
if s.deps.PersonalTokens == nil {
return nil, status.Error(codes.Unavailable, "personal tokens unavailable")
}
if s.deps.Identities == nil {
return nil, status.Error(codes.Unavailable, "identity store unavailable")
}
caller, ok := access.PrincipalFromContext(ctx)
if !ok || caller.IdentityID == "" {
return nil, status.Error(codes.Unauthenticated, "binding an identity token requires a principal")
}
target, err := s.deps.Identities.Get(ctx, identity.IdentityID(r.GetIdentityId()))
if err != nil {
return nil, s.identityStatus(ctx, err)
}
if target.Kind == identity.KindUser {
return nil, s.identityStatus(ctx, fmt.Errorf("%w: personal tokens belong to a signed-in person", identity.ErrInvalid))
}
if !target.CanAct() {
return nil, s.identityStatus(ctx, fmt.Errorf("%w: identity is not active", identity.ErrIllegalTransition))
}
subject, err := tokenSubject(r.GetId())
if err != nil {
return nil, err
}
if err := s.deps.PersonalTokens.BindPersonalToken(ctx, target.ID, subject, r.GetName(), optionalTime(r.GetExpiresAt()), tokenAudit(ctx, caller.IdentityID, "bind", r.GetId())); err != nil {
return nil, s.identityStatus(ctx, err)
}
return &pb.BindIdentityPersonalTokenResponse{}, nil
}
func (c *Client) AddPersonalToken(ctx context.Context, subject identity.Subject, name string, expiresAt *time.Time) error {
_, err := c.account.AddPersonalToken(ctx, &pb.AddPersonalTokenRequest{Id: subject.Subject, Name: name, ExpiresAt: optionalTimestamp(expiresAt)})
return identityClientError(err)
@@ -115,6 +149,15 @@ func (c *Client) RevokePersonalToken(ctx context.Context, id string) error {
return identityClientError(err)
}
// BindPersonalToken binds a token to a named identity rather than the caller's
// principal. The audit argument is accepted for the caller-side contract but
// never sent: the store attributes the bind to the caller's own principal, so
// a request cannot claim an actor.
func (c *Client) BindPersonalToken(ctx context.Context, id identity.IdentityID, subject identity.Subject, name string, expiresAt *time.Time, _ identity.Audit) error {
_, err := c.account.BindIdentityPersonalToken(ctx, &pb.BindIdentityPersonalTokenRequest{IdentityId: string(id), Id: subject.Subject, Name: name, ExpiresAt: optionalTimestamp(expiresAt)})
return identityClientError(err)
}
func optionalTimestamp(t *time.Time) *timestamppb.Timestamp {
if t == nil {
return nil
@@ -6,6 +6,10 @@ import (
"testing"
"time"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/timestamppb"
pb "github.com/samcharles93/archie-core/internal/contracts/state/v1"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/identity"
@@ -49,3 +53,88 @@ func TestPersonalTokenOwner(t *testing.T) {
})
}
}
type bindCalls struct {
identity.PersonalTokenStore
bound bool
id identity.IdentityID
subject identity.Subject
name string
expires *time.Time
audit identity.Audit
}
func (b *bindCalls) BindPersonalToken(_ context.Context, id identity.IdentityID, subject identity.Subject, name string, expiresAt *time.Time, audit identity.Audit) error {
b.bound, b.id, b.subject, b.name, b.expires, b.audit = true, id, subject, name, expiresAt, audit
return nil
}
type bindIdentityRepo struct {
identity.Repository
values map[identity.IdentityID]identity.Identity
}
func (r bindIdentityRepo) Get(_ context.Context, id identity.IdentityID) (identity.Identity, error) {
if value, ok := r.values[id]; ok {
return value, nil
}
return identity.Identity{}, identity.ErrNotFound
}
// TestBindIdentityPersonalToken pins the named-identity bind: a service
// account gets the token with the caller as audit actor, while a user target,
// an unknown or inactive identity, a malformed token id and a caller with no
// principal are all refused without binding.
func TestBindIdentityPersonalToken(t *testing.T) {
id := strings.Repeat("ab", 32)
sa := identity.Identity{ID: "sa1", Kind: identity.KindServiceAccount, DisplayName: "Canary", Lifecycle: identity.LifecycleActive}
user := identity.Identity{ID: "u1", Kind: identity.KindUser, DisplayName: "Alice", Lifecycle: identity.LifecycleActive}
suspended := identity.Identity{ID: "sa2", Kind: identity.KindBot, DisplayName: "Old", Lifecycle: identity.LifecycleSuspended}
repo := bindIdentityRepo{values: map[identity.IdentityID]identity.Identity{sa.ID: sa, user.ID: user, suspended.ID: suspended}}
tests := []struct {
name string
target identity.IdentityID
tokenID string
noPrincipal bool
wantCode codes.Code
wantBound bool
}{
{"service account binds with the caller as actor", sa.ID, id, false, codes.OK, true},
{"user target is refused", user.ID, id, false, codes.FailedPrecondition, false},
{"unknown identity is refused", "nope", id, false, codes.NotFound, false},
{"inactive identity is refused", suspended.ID, id, false, codes.FailedPrecondition, false},
{"malformed token id is refused", sa.ID, "not-hex", false, codes.FailedPrecondition, false},
{"no principal is refused", sa.ID, id, true, codes.Unauthenticated, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
store := &bindCalls{}
s := &accountServer{server: &server{deps: Deps{PersonalTokens: store, Identities: repo}}}
ctx := context.Background()
if !tt.noPrincipal {
ctx = access.WithPrincipal(ctx, access.Principal{IdentityID: "alice"})
}
expires := time.Now().UTC().Add(30 * 24 * time.Hour).Truncate(time.Second)
_, err := s.BindIdentityPersonalToken(ctx, &pb.BindIdentityPersonalTokenRequest{IdentityId: string(tt.target), Id: tt.tokenID, Name: "ci", ExpiresAt: timestamppb.New(expires)})
if got := status.Code(err); got != tt.wantCode {
t.Fatalf("code=%v want=%v err=%v", got, tt.wantCode, err)
}
if store.bound != tt.wantBound {
t.Fatalf("bound=%v want=%v", store.bound, tt.wantBound)
}
if !tt.wantBound {
return
}
wantSubject := identity.Subject{Issuer: identity.PersonalTokenIssuer, Subject: id}
if store.id != sa.ID || store.name != "ci" || store.subject != wantSubject {
t.Fatalf("bind id=%q name=%q subject=%+v, want %q %q %+v", store.id, store.name, store.subject, sa.ID, "ci", wantSubject)
}
if store.expires == nil || !store.expires.Equal(expires) {
t.Fatalf("expires=%v want %v", store.expires, expires)
}
if store.audit.ActorID != "alice" || store.audit.RequestID != "personal-token:bind:"+id {
t.Fatalf("audit=%+v, want actor alice request personal-token:bind:<id>", store.audit)
}
})
}
}
@@ -29,7 +29,13 @@ func (s *taskEventServer) EventsSince(ctx context.Context, r *pb.EventsSinceRequ
}
func (s *taskEventServer) TaskEvents(ctx context.Context, r *pb.TaskEventsRequest) (*pb.TaskEventsResponse, error) {
evs, err := s.deps.Tasks.TaskEvents(ctx, r.TaskId)
var evs []events.Event
var err error
if r.Transcript {
evs, err = s.deps.Tasks.TaskTranscript(ctx, r.TaskId, int(r.Attempt))
} else {
evs, err = s.deps.Tasks.TaskEvents(ctx, r.TaskId)
}
if err != nil {
return nil, s.logErr("TaskEvents", err)
}
@@ -59,3 +65,11 @@ func (c *Client) TaskEvents(ctx context.Context, taskID int64) ([]events.Event,
}
return mapValues(r.Events, eventValue), nil
}
func (c *Client) TaskTranscript(ctx context.Context, taskID int64, attempt int) ([]events.Event, error) {
r, err := c.taskEvent.TaskEvents(ctx, &pb.TaskEventsRequest{TaskId: taskID, Transcript: true, Attempt: int64(attempt)})
if err != nil {
return nil, unmapError(err)
}
return mapValues(r.Events, eventValue), nil
}
+152
View File
@@ -0,0 +1,152 @@
package releaseupdate
import (
"fmt"
"time"
)
// The canary host installs each release from the next channel. Its behaviour
// in the window after a release was installed is compared with its own
// behaviour in the window before, because there's only one production host to
// compare with.
const (
// CanaryWindow is the default window a release runs on the canary before
// it can be promoted. A day covers one full cycle of schedules and
// traffic.
CanaryWindow = 24 * time.Hour
// canarySettle is how long the install and restart get, counted from when
// the release was installed on the canary, before any check can fail the
// release.
canarySettle = 30 * time.Minute
// minFinished is the fewest finished tasks needed to judge the park rate.
minFinished = 5
// minLogCoverage is how far back into the baseline window the error log
// must reach before error counts are compared.
minLogCoverage = time.Hour
)
// CanaryDecision is what the evaluation says to do with a release.
type CanaryDecision string
const (
CanaryWait CanaryDecision = "wait"
CanaryPromote CanaryDecision = "promote"
CanaryRollBack CanaryDecision = "roll back"
)
// CanaryWindowSignals is one window's task and error-log counts.
type CanaryWindowSignals struct {
// Finished counts tasks that reached a terminal or review status; Bad
// is those that parked or died.
Finished, Bad int
// Errors counts error-level log lines within Covered, the part of the
// window that the log still reaches.
Errors int
Covered time.Duration
}
// CanarySignals is what the canary reported at one check.
type CanarySignals struct {
Installed string
Ready bool
// NewActionAlerts are open action-severity alerts first raised since
// the release was installed.
NewActionAlerts []string
Baseline CanaryWindowSignals
Canary CanaryWindowSignals
}
// CanaryVerdict is a decision and the signals that decided it.
type CanaryVerdict struct {
Decision CanaryDecision
Reasons []string
}
// JudgeCanaryNow decides from the current signals alone, for an operator
// promoting without waiting out the window: the settle and window waits are
// skipped by construction, every hard and relative check still applies.
func JudgeCanaryNow(version string, now time.Time, window time.Duration, s CanarySignals) CanaryVerdict {
return JudgeCanary(version, now.Add(-(window + canarySettle)), now, window, s)
}
// JudgeCanary decides what to do with a release, given when it was installed
// on the canary host. Once the release has settled, a failed hard check rolls
// it back at any time. Error and park rates are judged against the baseline
// whenever both windows carry enough data, so a breach rolls the release back
// even mid-window; a full window with no failure promotes it.
func JudgeCanary(version string, installedAt, now time.Time, window time.Duration, s CanarySignals) CanaryVerdict {
elapsed := now.Sub(installedAt)
if elapsed < canarySettle {
return CanaryVerdict{Decision: CanaryWait, Reasons: []string{"settling after install"}}
}
var failed []string
if s.Installed != version {
failed = append(failed, fmt.Sprintf("canary runs %q, not %s: the install failed or was rolled back", s.Installed, version))
}
if !s.Ready {
failed = append(failed, "readiness is degraded")
}
for _, alert := range s.NewActionAlerts {
failed = append(failed, "new action alert: "+alert)
}
if len(failed) > 0 {
return CanaryVerdict{Decision: CanaryRollBack, Reasons: failed}
}
var notes []string
if reason, ok := judgeErrors(s.Baseline, s.Canary, window); !ok {
failed = append(failed, reason)
} else {
notes = append(notes, reason)
}
if reason, ok := judgeParks(s.Baseline, s.Canary); !ok {
failed = append(failed, reason)
} else {
notes = append(notes, reason)
}
if len(failed) > 0 {
return CanaryVerdict{Decision: CanaryRollBack, Reasons: failed}
}
if elapsed < window {
reasons := append([]string{fmt.Sprintf("%s of %s elapsed since the install", elapsed.Round(time.Minute), window)}, notes...)
return CanaryVerdict{Decision: CanaryWait, Reasons: reasons}
}
return CanaryVerdict{Decision: CanaryPromote, Reasons: notes}
}
// judgeErrors fails when the canary logged more than twice the baseline's
// error lines plus ten. Each count is scaled up to a full window when the log
// covers only part of its window.
func judgeErrors(baseline, canary CanaryWindowSignals, window time.Duration) (string, bool) {
if baseline.Covered < minLogCoverage || canary.Covered <= 0 {
return "error log doesn't reach back over the baseline; error signal skipped", true
}
scale := func(w CanaryWindowSignals) float64 {
return float64(w.Errors) * float64(window) / float64(w.Covered)
}
before, after := scale(baseline), scale(canary)
limit := 2*before + 10
if after > limit {
return fmt.Sprintf("%.0f error log lines per window against a limit of %.0f (baseline %.0f)", after, limit, before), false
}
return fmt.Sprintf("%.0f error log lines per window, limit %.0f", after, limit), true
}
// judgeParks fails when the share of finished tasks that parked or died rose
// more than ten points over the baseline. Both windows need the minimum
// finished tasks before the rates are compared.
func judgeParks(baseline, canary CanaryWindowSignals) (string, bool) {
if canary.Finished < minFinished || baseline.Finished < minFinished {
return fmt.Sprintf("%d finished tasks on the canary and %d in the baseline, fewer than %d each; park signal skipped", canary.Finished, baseline.Finished, minFinished), true
}
rate := func(w CanaryWindowSignals) float64 {
if w.Finished == 0 {
return 0
}
return float64(w.Bad) / float64(w.Finished)
}
before, after := rate(baseline), rate(canary)
if after > before+0.10 {
return fmt.Sprintf("%.0f%% of tasks parked or died against %.0f%% before", 100*after, 100*before), false
}
return fmt.Sprintf("%.0f%% of tasks parked or died, %.0f%% before", 100*after, 100*before), true
}
+333
View File
@@ -0,0 +1,333 @@
package releaseupdate
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
)
// CanaryDashboard reaches the canary through its dashboard API, the same
// contract an operator uses: it reads the signals, edits the update settings
// through the control plane and installs through the dashboard updater.
type CanaryDashboard struct {
URL string
Token string
HTTP *http.Client
}
// Statuses a finished task can be in, and the subset that counts against a
// release.
var (
finishedStatuses = map[string]bool{"completed": true, "merged": true, "pr_open": true, "rejected": true, "closed_wont_do": true, "parked": true, "dead": true}
badStatuses = map[string]bool{"parked": true, "dead": true}
)
const updateSettingsPath = "/api/control-plane/resources/update-settings"
// Signals reads the canary's current state. Tasks and error lines are split
// into the window before the release was installed on the canary and the time
// since.
func (d CanaryDashboard) Signals(ctx context.Context, installedAt, now time.Time, window time.Duration) (CanarySignals, error) {
var s CanarySignals
var err error
if s.Installed, _, _, err = d.Installed(ctx); err != nil {
return s, err
}
if s.Ready, err = d.ready(ctx); err != nil {
return s, err
}
if s.NewActionAlerts, err = d.actionAlerts(ctx, installedAt); err != nil {
return s, err
}
if err := d.tasks(ctx, installedAt, window, &s); err != nil {
return s, err
}
return s, d.errors(ctx, installedAt, now, window, &s)
}
// Installed returns the daemon version the canary runs, when that release was
// installed on the canary host (zero when the host doesn't report it), and
// the raw update snapshot it offers, which an install request must send back
// unchanged.
func (d CanaryDashboard) Installed(ctx context.Context) (version string, installedAt time.Time, snapshot json.RawMessage, err error) {
var body struct {
Snapshot json.RawMessage `json:"snapshot"`
}
if _, err := d.call(ctx, http.MethodGet, "/api/chat/update", nil, &body); err != nil {
return "", time.Time{}, nil, err
}
var snap Snapshot
if err := json.Unmarshal(body.Snapshot, &snap); err != nil {
return "", time.Time{}, nil, fmt.Errorf("decode update snapshot: %w", err)
}
for _, c := range snap.Components {
if c.ID == "daemon" {
return c.Installed, snap.InstalledAt, body.Snapshot, nil
}
}
return "", time.Time{}, body.Snapshot, errors.New("canary reports no daemon version")
}
func (d CanaryDashboard) ready(ctx context.Context) (bool, error) {
var report struct {
Status string `json:"status"`
}
if _, err := d.call(ctx, http.MethodGet, "/health/detailed", nil, &report); err != nil {
return false, err
}
return report.Status == "ok", nil
}
func (d CanaryDashboard) actionAlerts(ctx context.Context, since time.Time) ([]string, error) {
var body struct {
Alerts []struct {
Severity string `json:"severity"`
Title string `json:"title"`
FirstRaised time.Time `json:"first_raised"`
} `json:"alerts"`
}
if _, err := d.call(ctx, http.MethodGet, "/api/alerts", nil, &body); err != nil {
return nil, err
}
var titles []string
for _, a := range body.Alerts {
if a.Severity == "action" && !a.FirstRaised.Before(since) {
titles = append(titles, a.Title)
}
}
return titles, nil
}
// tasks reads tasks, newest update first, until it reaches tasks older than
// the baseline window.
func (d CanaryDashboard) tasks(ctx context.Context, at time.Time, window time.Duration, s *CanarySignals) error {
from := at.Add(-window)
cursor := ""
for range 50 {
var page []struct {
Status string `json:"status"`
UpdatedAt time.Time `json:"updated_at"`
}
path := "/api/tasks?limit=500"
if cursor != "" {
path += "&cursor=" + url.QueryEscape(cursor)
}
header, err := d.call(ctx, http.MethodGet, path, nil, &page)
if err != nil {
return err
}
for _, t := range page {
if t.UpdatedAt.Before(from) || !finishedStatuses[t.Status] {
continue
}
w := &s.Canary
if t.UpdatedAt.Before(at) {
w = &s.Baseline
}
w.Finished++
if badStatuses[t.Status] {
w.Bad++
}
}
cursor = header.Get("X-Next-Cursor")
if cursor == "" || len(page) == 0 || page[len(page)-1].UpdatedAt.Before(from) {
return nil
}
}
return nil
}
// errors counts error-level lines in each window. A log read returns a
// limited number of lines, so a truncated read only reaches back as far as
// its oldest line.
func (d CanaryDashboard) errors(ctx context.Context, at, now time.Time, window time.Duration, s *CanarySignals) error {
var body struct {
Entries []struct {
Time time.Time `json:"time"`
} `json:"entries"`
Truncated bool `json:"truncated"`
}
if _, err := d.call(ctx, http.MethodGet, "/api/logs?level=error&limit=2000", nil, &body); err != nil {
return err
}
from := at.Add(-window)
if body.Truncated {
from = now
if len(body.Entries) > 0 {
from = later(body.Entries[0].Time, at.Add(-window))
}
}
for _, e := range body.Entries {
switch {
case e.Time.Before(from):
case e.Time.Before(at):
s.Baseline.Errors++
default:
s.Canary.Errors++
}
}
s.Baseline.Covered = max(at.Sub(from), 0)
s.Canary.Covered = now.Sub(later(from, at))
return nil
}
func later(a, b time.Time) time.Time {
if a.After(b) {
return a
}
return b
}
// Follow sets the canary's update channel, keeping its release source and
// key. A pin is used only by exact-pin.
func (d CanaryDashboard) Follow(ctx context.Context, channel, pin string) error {
var current struct {
Resource struct {
Version int64 `json:"version"`
Value json.RawMessage `json:"value"`
} `json:"resource"`
}
if _, err := d.call(ctx, http.MethodGet, updateSettingsPath, nil, &current); err != nil {
return err
}
value := map[string]any{}
if len(current.Resource.Value) > 0 && string(current.Resource.Value) != "null" {
if err := json.Unmarshal(current.Resource.Value, &value); err != nil {
return fmt.Errorf("decode update settings: %w", err)
}
}
value["channel"], value["pin"] = channel, pin
_, err := d.call(ctx, http.MethodPost, updateSettingsPath+"/commands/replace",
map[string]any{"value": value, "expected_version": current.Resource.Version}, nil)
return err
}
// Channel is the canary's update channel and pin.
func (d CanaryDashboard) Channel(ctx context.Context) (channel, pin string, err error) {
var current struct {
Resource struct {
Value struct {
Channel string `json:"channel"`
Pin string `json:"pin"`
} `json:"value"`
} `json:"resource"`
}
if _, err := d.call(ctx, http.MethodGet, updateSettingsPath, nil, &current); err != nil {
return "", "", err
}
if current.Resource.Value.Channel == "" {
return "stable", "", nil
}
return current.Resource.Value.Channel, current.Resource.Value.Pin, nil
}
// Install has the canary install version through its updater and waits until
// it reports running it. The install restarts the dashboard, so a dropped
// connection after the request is sent is expected.
func (d CanaryDashboard) Install(ctx context.Context, version string, wait time.Duration) error {
deadline := time.Now().Add(wait)
requested := false
for {
installed, _, snapshot, err := d.Installed(ctx)
switch {
case err != nil:
// The dashboard drops off while the install restarts it.
if !requested {
return err
}
case installed == version:
return nil
case !requested:
// The canary caches its registry lookup, so a release just
// published can take a few minutes to be offered.
offered, err := offers(snapshot, version)
if err != nil {
return err
}
if offered {
_, err := d.call(ctx, http.MethodPost, "/api/chat/update/install", map[string]json.RawMessage{"snapshot": snapshot}, nil)
if _, ok := errors.AsType[statusError](err); ok {
return err
}
requested = true
}
}
if !time.Now().Before(deadline) {
break
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(15 * time.Second):
}
}
if !requested {
return fmt.Errorf("canary didn't offer %s within %s", version, wait)
}
return fmt.Errorf("canary didn't report %s within %s", version, wait)
}
// offers reports whether an update snapshot offers version.
func offers(snapshot json.RawMessage, version string) (bool, error) {
var offered Snapshot
if err := json.Unmarshal(snapshot, &offered); err != nil {
return false, fmt.Errorf("decode update snapshot: %w", err)
}
available := offered.Available()
return len(available) > 0 && available[0].Available == version, nil
}
type statusError struct {
code int
body string
}
func (e statusError) Error() string { return fmt.Sprintf("dashboard answered %d: %s", e.code, e.body) }
func (d CanaryDashboard) call(ctx context.Context, method, path string, in, out any) (http.Header, error) {
var body io.Reader
if in != nil {
data, err := json.Marshal(in)
if err != nil {
return nil, err
}
body = bytes.NewReader(data)
}
ctx, cancel := context.WithTimeout(ctx, 2*time.Minute)
defer cancel()
req, err := http.NewRequestWithContext(ctx, method, strings.TrimRight(d.URL, "/")+path, body)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+d.Token)
if in != nil {
req.Header.Set("Content-Type", "application/json")
}
client := d.HTTP
if client == nil {
client = http.DefaultClient
}
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("%s %s: %w", method, path, err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode/100 != 2 {
text, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
return nil, fmt.Errorf("%s %s: %w", method, path, statusError{resp.StatusCode, strings.TrimSpace(string(text))})
}
if out == nil {
return resp.Header, nil
}
if err := json.NewDecoder(resp.Body).Decode(out); err != nil {
return nil, fmt.Errorf("%s %s: decode: %w", method, path, err)
}
return resp.Header, nil
}
+89
View File
@@ -0,0 +1,89 @@
package releaseupdate
import (
"testing"
"time"
)
// A release promotes only after a full window counted from its install with
// no hard failure and no relative signal past its threshold; any failure after
// settling rolls it back at once, even mid-window.
func TestJudgeCanary(t *testing.T) {
installed := time.Date(2026, 1, 2, 12, 0, 0, 0, time.UTC)
healthy := func() CanarySignals {
return CanarySignals{
Installed: "1.1.0", Ready: true,
Baseline: CanaryWindowSignals{Finished: 20, Bad: 2, Errors: 10, Covered: CanaryWindow},
Canary: CanaryWindowSignals{Finished: 20, Bad: 2, Errors: 12, Covered: CanaryWindow},
}
}
with := func(change func(*CanarySignals)) CanarySignals {
s := healthy()
change(&s)
return s
}
cases := []struct {
name string
elapsed time.Duration // now minus the install time
window time.Duration
signals CanarySignals
want CanaryDecision
}{
{"settling counts from the install", 10 * time.Minute, CanaryWindow, with(func(s *CanarySignals) { s.Ready = false }), CanaryWait},
{"healthy mid-window waits", 2 * time.Hour, CanaryWindow, healthy(), CanaryWait},
{"healthy at the window promotes", CanaryWindow, CanaryWindow, healthy(), CanaryPromote},
{"healthy at a custom window promotes", 12 * time.Hour, 12 * time.Hour, healthy(), CanaryPromote},
{"the watchdog rolled the install back", 2 * time.Hour, CanaryWindow, with(func(s *CanarySignals) { s.Installed = "1.0.0" }), CanaryRollBack},
{"readiness degraded", 2 * time.Hour, CanaryWindow, with(func(s *CanarySignals) { s.Ready = false }), CanaryRollBack},
{"a new action alert", 2 * time.Hour, CanaryWindow, with(func(s *CanarySignals) { s.NewActionAlerts = []string{"queue stalled"} }), CanaryRollBack},
{"errors past twice the baseline plus ten", CanaryWindow, CanaryWindow, with(func(s *CanarySignals) { s.Canary.Errors = 31 }), CanaryRollBack},
{"errors at the limit", CanaryWindow, CanaryWindow, with(func(s *CanarySignals) { s.Canary.Errors = 30 }), CanaryPromote},
{"errors scaled from a partly covered log", CanaryWindow, CanaryWindow, with(func(s *CanarySignals) {
s.Canary.Errors, s.Canary.Covered = 8, 6*time.Hour
}), CanaryRollBack},
{"a baseline the log doesn't reach skips errors", CanaryWindow, CanaryWindow, with(func(s *CanarySignals) {
s.Baseline.Covered, s.Canary.Errors = 30*time.Minute, 500
}), CanaryPromote},
{"a mid-window error breach rolls back early", 2 * time.Hour, CanaryWindow, with(func(s *CanarySignals) { s.Canary.Errors = 31 }), CanaryRollBack},
{"park rate up more than ten points", CanaryWindow, CanaryWindow, with(func(s *CanarySignals) { s.Canary.Bad = 5 }), CanaryRollBack},
{"park rate up ten points", CanaryWindow, CanaryWindow, with(func(s *CanarySignals) { s.Canary.Bad = 4 }), CanaryPromote},
{"too few canary tasks skips the park rate", CanaryWindow, CanaryWindow, with(func(s *CanarySignals) { s.Canary = CanaryWindowSignals{Finished: 4, Bad: 4, Covered: CanaryWindow} }), CanaryPromote},
{"a thin baseline skips the park rate", CanaryWindow, CanaryWindow, with(func(s *CanarySignals) {
s.Baseline.Finished, s.Baseline.Bad = 2, 0
s.Canary.Bad = 5
}), CanaryPromote},
{"a mid-window park breach rolls back early", 2 * time.Hour, CanaryWindow, with(func(s *CanarySignals) { s.Canary.Bad = 5 }), CanaryRollBack},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := JudgeCanary("1.1.0", installed, installed.Add(tc.elapsed), tc.window, tc.signals)
if got.Decision != tc.want {
t.Fatalf("decision = %s, want %s (%v)", got.Decision, tc.want, got.Reasons)
}
})
}
}
// Judging "now" skips the settle and window waits but still applies every
// hard and relative check.
func TestJudgeCanaryNow(t *testing.T) {
healthy := CanarySignals{
Installed: "1.1.0", Ready: true,
Baseline: CanaryWindowSignals{Finished: 20, Bad: 2, Errors: 10, Covered: CanaryWindow},
Canary: CanaryWindowSignals{Finished: 20, Bad: 2, Errors: 12, Covered: CanaryWindow},
}
now := time.Date(2026, 1, 2, 12, 10, 0, 0, time.UTC)
if got := JudgeCanaryNow("1.1.0", now, CanaryWindow, healthy); got.Decision != CanaryPromote {
t.Fatalf("healthy now = %s, want promote (%v)", got.Decision, got.Reasons)
}
degraded := healthy
degraded.Ready = false
if got := JudgeCanaryNow("1.1.0", now, CanaryWindow, degraded); got.Decision != CanaryRollBack {
t.Fatalf("degraded now = %s, want roll back (%v)", got.Decision, got.Reasons)
}
breached := healthy
breached.Canary.Bad = 10
if got := JudgeCanaryNow("1.1.0", now, CanaryWindow, breached); got.Decision != CanaryRollBack {
t.Fatalf("park breach now = %s, want roll back (%v)", got.Decision, got.Reasons)
}
}
+47
View File
@@ -0,0 +1,47 @@
package releaseupdate
import (
"context"
"fmt"
digest "github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"oras.land/oras-go/v2/registry"
"github.com/samcharles93/archie-core/internal/domain/distribution"
)
// Installable reports whether a release can still be installed from source:
// the zip blob its statement names must be present in the release repository
// and the agent image it pins must still resolve. A canary must never be
// pinned to a release whose artifacts are gone, or it would be stranded on a
// version it can never install again.
func Installable(ctx context.Context, source distribution.Source, release Release) error {
zipDigest, err := digest.Parse(release.Statement.Zip.Digest)
if err != nil {
return fmt.Errorf("zip digest %q: %w", release.Statement.Zip.Digest, err)
}
repo, err := pushRepository(source.Reference)
if err != nil {
return err
}
exists, err := repo.Exists(ctx, ocispec.Descriptor{Digest: zipDigest})
if err != nil {
return fmt.Errorf("resolve zip blob %s in %s: %w", release.Statement.Zip.Digest, source.Reference, err)
}
if !exists {
return fmt.Errorf("zip blob %s (%s) is absent from %s", release.Statement.Zip.Digest, release.Statement.Zip.Name, source.Reference)
}
image, err := registry.ParseReference(release.Statement.AgentImage)
if err != nil {
return fmt.Errorf("agent image %q: %w", release.Statement.AgentImage, err)
}
imageRepo, err := pushRepository(image.Registry + "/" + image.Repository)
if err != nil {
return err
}
if _, err := imageRepo.Resolve(ctx, image.Reference); err != nil {
return fmt.Errorf("agent image %s: %w", release.Statement.AgentImage, err)
}
return nil
}
+115
View File
@@ -0,0 +1,115 @@
package releaseupdate
import (
"context"
"crypto/ed25519"
"encoding/json"
"errors"
"fmt"
"io"
"strings"
digest "github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"oras.land/oras-go/v2/errdef"
"oras.land/oras-go/v2/registry/remote"
"github.com/samcharles93/archie-core/internal/domain/distribution"
"github.com/samcharles93/archie-core/internal/infrastructure/signedoci"
)
// Promote points the stable tag at a published release. The signature covers
// the release statement, not the tag, so the release doesn't need signing
// again. It refuses a prerelease, and a release issued before the one stable
// already points at, which hosts would reject as a replay.
//
// Each image repository in latest gets its v<version> tag copied to latest
// before stable moves, so if that fails stable is unchanged and running
// Promote again finishes the job.
func Promote(ctx context.Context, repository string, key ed25519.PublicKey, version string, latest []string) (Statement, error) {
source := distribution.Source{Reference: repository, Key: key}
release, err := Fetch(ctx, source, Tag("exact-pin", version))
if err != nil {
return Statement{}, err
}
if release.Statement.Prerelease {
return Statement{}, fmt.Errorf("%s is a prerelease and never reaches stable", version)
}
current, err := Fetch(ctx, source, "stable")
switch {
case errors.Is(err, errdef.ErrNotFound):
case err != nil:
return Statement{}, err
case release.Statement.IssuedAt.Before(current.Statement.IssuedAt):
return Statement{}, fmt.Errorf("%s was issued before %s, which stable names", version, current.Statement.Version)
}
repo, err := pushRepository(repository)
if err != nil {
return Statement{}, err
}
desc, err := verifiedManifest(ctx, repo, version, release.Artifact.Statement, key)
if err != nil {
return Statement{}, err
}
for _, image := range latest {
if err := promoteImage(ctx, image, version, release.Statement.AgentImage); err != nil {
return Statement{}, err
}
}
if err := repo.Tag(ctx, desc, "stable"); err != nil {
return Statement{}, fmt.Errorf("tag stable: %w", err)
}
return release.Statement, nil
}
// promoteImage tags image:v<version> as latest. The agent image must be the
// one the statement pins, or latest would name an image the release never
// signed for.
func promoteImage(ctx context.Context, image, version, agentImage string) error {
repo, err := pushRepository(image)
if err != nil {
return err
}
desc, err := repo.Resolve(ctx, Tag("exact-pin", version))
if err != nil {
return fmt.Errorf("resolve %s:v%s: %w", image, version, err)
}
if pinned, digest, ok := strings.Cut(agentImage, "@"); ok && pinned == image && desc.Digest.String() != digest {
return fmt.Errorf("%s:v%s is %s, not the %s the release pins", image, version, desc.Digest, digest)
}
if err := repo.Tag(ctx, desc, "latest"); err != nil {
return fmt.Errorf("tag %s:latest: %w", image, err)
}
return nil
}
// verifiedManifest returns the manifest the version tag points at now, after
// checking it carries the statement that was already verified. This stops a
// tag that moved in the meantime from putting an unverified manifest on
// stable.
func verifiedManifest(ctx context.Context, repo *remote.Repository, version string, statement []byte, key ed25519.PublicKey) (ocispec.Descriptor, error) {
desc, err := repo.Resolve(ctx, Tag("exact-pin", version))
if err != nil {
return ocispec.Descriptor{}, fmt.Errorf("resolve %s: %w", version, err)
}
stream, err := repo.Fetch(ctx, desc)
if err != nil {
return ocispec.Descriptor{}, fmt.Errorf("fetch %s manifest: %w", version, err)
}
raw, err := io.ReadAll(io.LimitReader(stream, desc.Size))
_ = stream.Close()
if err != nil {
return ocispec.Descriptor{}, fmt.Errorf("read %s manifest: %w", version, err)
}
if digest.FromBytes(raw) != desc.Digest {
return ocispec.Descriptor{}, fmt.Errorf("%s manifest doesn't match its digest", version)
}
var manifest ocispec.Manifest
if err := json.Unmarshal(raw, &manifest); err != nil {
return ocispec.Descriptor{}, fmt.Errorf("%s manifest: %w", version, err)
}
if err := signedoci.Verify(manifest, statement, key, Annotation); err != nil {
return ocispec.Descriptor{}, fmt.Errorf("%s changed while promoting: %w", version, err)
}
return desc, nil
}
+167
View File
@@ -0,0 +1,167 @@
package releaseupdate
import (
"crypto/ed25519"
"encoding/base64"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"sync"
"testing"
"time"
digest "github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/samcharles93/archie-core/internal/domain/distribution"
"github.com/samcharles93/archie-core/internal/infrastructure/signedoci"
)
// stable moves only to a signed, non-prerelease release issued no earlier
// than the one it names, and it moves to the very manifest that was verified.
func TestPromote(t *testing.T) {
trusted, signer, _ := ed25519.GenerateKey(nil)
_, stranger, _ := ed25519.GenerateKey(nil)
jan1, jan2 := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC), time.Date(2026, 1, 2, 0, 0, 0, 0, time.UTC)
type published struct {
version string
issued time.Time
prerelease bool
signer ed25519.PrivateKey
tags []string
}
cases := []struct {
name string
release []published
promote string
want string // error substring, or "" when stable must name promote
}{
{name: "a newer release", release: []published{{"1.0.0", jan1, false, signer, []string{"stable"}}, {"1.1.0", jan2, false, signer, []string{"next"}}}, promote: "1.1.0"},
{name: "the first release", release: []published{{"1.0.0", jan1, false, signer, []string{"next"}}}, promote: "1.0.0"},
{name: "a release issued before stable", release: []published{{"1.1.0", jan2, false, signer, []string{"stable"}}, {"1.0.0", jan1, false, signer, nil}}, promote: "1.0.0", want: "issued before"},
{name: "a prerelease", release: []published{{"1.1.0-rc.1", jan2, true, signer, []string{"next"}}}, promote: "1.1.0-rc.1", want: "prerelease"},
{name: "a release another key signed", release: []published{{"1.1.0", jan2, false, stranger, []string{"next"}}}, promote: "1.1.0", want: "signature"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Setenv("XDG_STATE_HOME", t.TempDir())
t.Setenv("DOCKER_CONFIG", t.TempDir())
registry := newMemoryRegistry()
server := httptest.NewServer(registry)
defer server.Close()
for _, r := range tc.release {
registry.publish(t, r.version, r.issued, r.prerelease, r.signer, r.tags...)
}
repository := strings.TrimPrefix(server.URL, "http://") + "/archie"
_, err := Promote(t.Context(), repository, trusted, tc.promote, nil)
switch {
case tc.want == "" && err != nil:
t.Fatalf("Promote: %v", err)
case tc.want != "" && (err == nil || !strings.Contains(err.Error(), tc.want)):
t.Fatalf("Promote error = %v, want %q", err, tc.want)
}
stable, ok := registry.statement("stable")
promoted := ok && stable.Version == tc.promote
if promoted != (tc.want == "") {
t.Fatalf("stable names %q (found %v) after promoting %s", stable.Version, ok, tc.promote)
}
if tc.want == "" {
if _, err := Fetch(t.Context(), distribution.Source{Reference: repository, Key: trusted}, "stable"); err != nil {
t.Fatalf("hosts refuse the promoted stable: %v", err)
}
}
})
}
}
// memoryRegistry serves manifests by tag or digest and accepts manifest
// pushes, which is all fetching and retagging a release needs.
type memoryRegistry struct {
mu sync.Mutex
manifests map[string][]byte
blobs map[string][]byte
}
func newMemoryRegistry() *memoryRegistry {
return &memoryRegistry{manifests: map[string][]byte{}, blobs: map[string][]byte{}}
}
func (m *memoryRegistry) publish(t *testing.T, version string, issued time.Time, prerelease bool, signer ed25519.PrivateKey, tags ...string) {
t.Helper()
archive := []byte("zip " + version)
statement, err := json.Marshal(Statement{
Version: version, Prerelease: prerelease, IssuedAt: issued, Platform: Platform, AgentImage: newImage,
Zip: Blob{Name: ZipName(version), Digest: signedoci.Digest(archive), Size: int64(len(archive))},
})
if err != nil {
t.Fatal(err)
}
manifest, err := json.Marshal(ocispec.Manifest{
SchemaVersion: 2, MediaType: ocispec.MediaTypeImageManifest, ArtifactType: ArtifactType, Config: ocispec.DescriptorEmptyJSON,
Layers: []ocispec.Descriptor{
{MediaType: StatementMediaType, Digest: digest.FromBytes(statement), Size: int64(len(statement))},
{MediaType: "application/zip", Digest: digest.FromBytes(archive), Size: int64(len(archive))},
},
Annotations: map[string]string{Annotation: base64.StdEncoding.EncodeToString(ed25519.Sign(signer, statement))},
})
if err != nil {
t.Fatal(err)
}
m.mu.Lock()
defer m.mu.Unlock()
m.blobs[signedoci.Digest(statement)], m.blobs[signedoci.Digest(archive)] = statement, archive
m.manifests[signedoci.Digest(manifest)] = manifest
for _, tag := range append(tags, "v"+version) {
m.manifests[tag] = manifest
}
}
func (m *memoryRegistry) statement(tag string) (Statement, bool) {
m.mu.Lock()
defer m.mu.Unlock()
var manifest ocispec.Manifest
if json.Unmarshal(m.manifests[tag], &manifest) != nil || len(manifest.Layers) == 0 {
return Statement{}, false
}
var s Statement
return s, json.Unmarshal(m.blobs[manifest.Layers[0].Digest.String()], &s) == nil
}
func (m *memoryRegistry) ServeHTTP(w http.ResponseWriter, r *http.Request) {
m.mu.Lock()
defer m.mu.Unlock()
if ref, ok := strings.CutPrefix(r.URL.Path, "/v2/archie/blobs/"); ok {
blob, found := m.blobs[ref]
if !found {
http.NotFound(w, r)
return
}
_, _ = w.Write(blob)
return
}
ref, ok := strings.CutPrefix(r.URL.Path, "/v2/archie/manifests/")
if !ok {
http.NotFound(w, r)
return
}
if r.Method == http.MethodPut {
body, _ := io.ReadAll(r.Body)
m.manifests[ref], m.manifests[signedoci.Digest(body)] = body, body
w.WriteHeader(http.StatusCreated)
return
}
manifest, found := m.manifests[ref]
if !found {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", ocispec.MediaTypeImageManifest)
w.Header().Set("Docker-Content-Digest", signedoci.Digest(manifest))
w.Header().Set("Content-Length", strconv.Itoa(len(manifest)))
if r.Method != http.MethodHead {
_, _ = w.Write(manifest)
}
}
+3
View File
@@ -44,6 +44,9 @@ type Component struct {
type Snapshot struct {
Components []Component
Deferred bool
// InstalledAt is when the installed release was committed on this host,
// from its installed.json. Zero when the host doesn't record one.
InstalledAt time.Time `json:"installed_at"`
}
func (s Snapshot) Available() []Component {
+54
View File
@@ -40,6 +40,7 @@ const (
journalDir = "transaction"
journalFile = "journal.json"
snapshotFile = "db.pre-update.dump"
installedFile = "installed.json"
lockWait = 60 * time.Second
)
@@ -126,6 +127,39 @@ type journal struct {
// JournalPath is where an unfinished transaction lives.
func JournalPath(stateDir string) string { return filepath.Join(stateDir, journalDir) }
// installedRecord is the host's durable note of which release it last
// committed (or rolled back to) and when, so a canary can count its window
// from the actual install rather than the release statement.
type installedRecord struct {
Version string `json:"version"`
CommittedAt time.Time `json:"committed_at"`
}
// recordInstall writes installed.json atomically: the file is the source of
// truth for when the running release was installed.
func recordInstall(stateDir, version string, at time.Time) error {
data, err := json.Marshal(installedRecord{Version: version, CommittedAt: at.UTC()})
if err != nil {
return err
}
return fsutil.WriteFileAtomic(filepath.Join(stateDir, installedFile), data, 0o600)
}
// LastInstall reads the host's installed record. ok is false when the file is
// missing or malformed, as on a host that updated before install times were
// recorded.
func LastInstall(stateDir string) (version string, at time.Time, ok bool) {
data, err := os.ReadFile(filepath.Join(stateDir, installedFile))
if err != nil {
return "", time.Time{}, false
}
var record installedRecord
if err := json.Unmarshal(data, &record); err != nil || record.Version == "" || record.CommittedAt.IsZero() {
return "", time.Time{}, false
}
return record.Version, record.CommittedAt, true
}
// Run drives the transaction in the journal to an outcome. recovering is set
// on boot: it never waits for the lock, and a transaction that was prepared
// but never started is abandoned rather than started unattended.
@@ -158,6 +192,17 @@ func (t *Transaction) Run(ctx context.Context, recovering bool) error {
if err := t.startUnits(ctx, j, ""); err != nil {
return err
}
// A crash between the outcome phase and recordInstall must not lose
// the install record, so the recovery path writes it too.
version := j.Version
if j.Phase == phaseRolledBack {
version = j.Previous[ComponentDaemon]
}
if version != "" {
if err := recordInstall(t.StateDir, version, time.Now().UTC()); err != nil {
return err
}
}
return t.remove()
default:
return t.rollback(ctx, j)
@@ -182,6 +227,9 @@ func (t *Transaction) promote(ctx context.Context, j journal) error {
if err := t.setPhase(&j, phaseCommitted); err != nil {
return err
}
if err := recordInstall(t.StateDir, j.Version, time.Now().UTC()); err != nil {
return fmt.Errorf("record the install: %w", err)
}
return t.remove()
}
@@ -252,6 +300,12 @@ func (t *Transaction) rollback(ctx context.Context, j journal) error {
if err := t.setPhase(&j, phaseRolledBack); err != nil {
return t.incomplete(err)
}
// The host is back on the previous release; installed.json follows it.
if version := j.Previous[ComponentDaemon]; version != "" {
if err := recordInstall(t.StateDir, version, time.Now().UTC()); err != nil {
return t.incomplete(err)
}
}
if err := t.startUnits(ctx, j, started); err != nil {
return t.incomplete(err)
}
@@ -21,6 +21,7 @@ import (
digest "github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/samcharles93/archie-core/internal/buildinfo"
"github.com/samcharles93/archie-core/internal/domain/distribution"
"github.com/samcharles93/archie-core/internal/infrastructure/signedoci"
)
@@ -146,6 +147,11 @@ func TestTransaction(t *testing.T) {
// the env file beside its config; pin both to this test's own files so
// the result never depends on the machine's real archie config.
t.Setenv(DatabasePasswordEnv, "test-password")
// Stamp this binary as a release, as a host installed from one is, so the
// journal's previous daemon version is a real version.
stamped := buildinfo.Version
buildinfo.Version = "1.0.0"
defer func() { buildinfo.Version = stamped }()
configPath := filepath.Join(t.TempDir(), "config.toml")
if err := os.WriteFile(configPath, []byte("database_url = \"postgres://archie@/archie?host=/tmp/archie-test\"\n"), 0o600); err != nil {
t.Fatal(err)
@@ -257,10 +263,51 @@ func TestTransaction(t *testing.T) {
found && (got.HealthCheck != tt.want.report || got.RolledBack != (tt.want.report == "failed") || got.RestoredDatabase != tt.want.restoredDatabase) {
t.Errorf("report = %+v (found %v), want %+v", got, found, tt.want)
}
// A transaction that ran to an outcome records which release the
// host now runs and when it was installed.
if tt.prepare == "" && tt.crashAt == "" {
if version, at, ok := LastInstall(tx.StateDir); !ok || version != tt.want.version || at.IsZero() {
t.Errorf("installed.json after the transaction = %q, %v, %v; want %s", version, at, ok, tt.want.version)
}
}
})
}
}
// installed.json is the host's durable record of which release it last
// committed or rolled back to, and when.
func TestLastInstall(t *testing.T) {
dir := t.TempDir()
if _, _, ok := LastInstall(dir); ok {
t.Fatal("a missing installed.json reports ok")
}
at := time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC)
if err := recordInstall(dir, "1.2.3", at); err != nil {
t.Fatal(err)
}
var raw map[string]any
data, err := os.ReadFile(filepath.Join(dir, installedFile))
if err != nil {
t.Fatal(err)
}
if err := json.Unmarshal(data, &raw); err != nil {
t.Fatalf("installed.json is not JSON: %v", err)
}
if raw["version"] != "1.2.3" || raw["committed_at"] != "2026-01-02T03:04:05Z" {
t.Fatalf("installed.json = %v, want version 1.2.3 and committed_at 2026-01-02T03:04:05Z", raw)
}
version, got, ok := LastInstall(dir)
if !ok || version != "1.2.3" || !got.Equal(at) {
t.Fatalf("LastInstall = %q, %v, %v; want 1.2.3, %v, true", version, got, ok, at)
}
if err := os.WriteFile(filepath.Join(dir, installedFile), []byte("{not json"), 0o600); err != nil {
t.Fatal(err)
}
if _, _, ok := LastInstall(dir); ok {
t.Fatal("a malformed installed.json reports ok")
}
}
// releaseHandler serves one release's zip and SHA256SUMS the way GitHub
// release downloads are laid out.
func releaseHandler(t *testing.T, candidate string, corrupt bool) (http.Handler, ed25519.PublicKey) {
+5 -5
View File
@@ -114,9 +114,9 @@ type Watchdog struct {
Interval time.Duration
// Cadence reports how long the loop may go between passes.
Cadence func() time.Duration
// Progress reports the start of the loop's most recent pass, or the zero
// time when no pass has begun yet.
Progress func() time.Time
// Progress reports the stalest loop and the start of its most recent
// pass, or the zero time when no pass has begun yet.
Progress func() (string, time.Time)
// Beat emits one WATCHDOG=1.
Beat func()
Log *slog.Logger
@@ -138,7 +138,7 @@ func (w Watchdog) Run(ctx context.Context) error {
case <-ticker.C:
}
last := w.Progress()
loop, last := w.Progress()
if last.IsZero() {
// No pass has begun at all. Measure staleness from arming, so a
// loop that never starts polling is caught like one that stopped.
@@ -149,7 +149,7 @@ func (w Watchdog) Run(ctx context.Context) error {
case age > cadence:
if !stalled {
w.Log.WarnContext(ctx, "systemd watchdog: run loop has not begun a poll pass within its interval; withholding the heartbeat until it does",
"stale_for", age.String(), "loop_cadence", cadence.String())
"loop", loop, "stale_for", age.String(), "loop_cadence", cadence.String())
stalled = true
}
default:
+14
View File
@@ -52,6 +52,20 @@ func RedactString(value string) string {
return value
}
// pemPrivateKeyRe matches a PEM private-key block, header to footer.
var pemPrivateKeyRe = regexp.MustCompile(`-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----`)
// RedactText replaces each secret-shaped span inside free text, keeping the
// rest, for long text such as tool output where blanking the whole value would
// hide everything around the secret.
func RedactText(value string) string {
if len(value) < 20 {
return value
}
value = jwtTokenRe.ReplaceAllString(value, redactedValue)
return pemPrivateKeyRe.ReplaceAllString(value, redactedValue)
}
func redactValue(value any) any {
switch typed := value.(type) {
case map[string]any:
+5 -1
View File
@@ -58,8 +58,12 @@ func (s *Server) accountIdentity(r *http.Request) (identity.Identity, error) {
func (s *Server) requireAccount(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Any active identity may authenticate: a person, or a service
// account whose bearer token carries a scoped grant. What the
// identity may then do is the policy chain's call, not the
// credential kind's; an inactive identity is refused here.
value, err := s.accountIdentity(r)
if err != nil || value.Kind != identity.KindUser || !value.CanAct() {
if err != nil || !value.CanAct() {
if err == nil {
err = identity.ErrCredentialRejected
}
+144
View File
@@ -2,6 +2,7 @@ package webui
import (
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
@@ -131,6 +132,149 @@ func TestAccountSessionFailureIsClosed(t *testing.T) {
}
}
// TestRequireAccountAdmitsAnyActiveIdentity pins the bearer-token gate: any
// active identity -- a person or a service account -- is admitted and rides
// the context as the acting identity, while an inactive one is refused, so a
// scoped service-account token can use the API without being a user.
func TestRequireAccountAdmitsAnyActiveIdentity(t *testing.T) {
tests := []struct {
name string
value identity.Identity
want int
}{
{"active user", identity.Identity{ID: "u1", Kind: identity.KindUser, DisplayName: "User", Lifecycle: identity.LifecycleActive}, http.StatusOK},
{"active service account", identity.Identity{ID: "sa1", Kind: identity.KindServiceAccount, DisplayName: "Canary", Lifecycle: identity.LifecycleActive}, http.StatusOK},
{"active bot", identity.Identity{ID: "b1", Kind: identity.KindBot, DisplayName: "Bot", Lifecycle: identity.LifecycleActive}, http.StatusOK},
{"suspended user", identity.Identity{ID: "u2", Kind: identity.KindUser, DisplayName: "User", Lifecycle: identity.LifecycleSuspended}, http.StatusUnauthorized},
{"retired service account", identity.Identity{ID: "sa2", Kind: identity.KindServiceAccount, DisplayName: "Canary", Lifecycle: identity.LifecycleRetired}, http.StatusUnauthorized},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
srv := &Server{
Authenticate: func(context.Context, string) (identity.Identity, error) { return tt.value, nil },
}
var acting identity.IdentityID
handler := srv.requireAccount(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) {
if value, ok := ActingIdentity(r.Context()); ok {
acting = value.ID
}
}))
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "https://archie.example/api/summary", nil)
req.Header.Set("Authorization", "Bearer archie_pat_test")
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)
if rec.Code != tt.want {
t.Fatalf("status=%d want=%d body=%s", rec.Code, tt.want, rec.Body.String())
}
if tt.want == http.StatusOK && acting != tt.value.ID {
t.Fatalf("acting identity = %q, want %q", acting, tt.value.ID)
}
})
}
}
// identityRepo answers Get for the identities a test names.
type identityRepo struct {
identity.Repository
values map[identity.IdentityID]identity.Identity
}
func (s identityRepo) Get(_ context.Context, id identity.IdentityID) (identity.Identity, error) {
value, ok := s.values[id]
if !ok {
return identity.Identity{}, identity.ErrNotFound
}
return value, nil
}
// identityTokenStore carries both token contracts: the remote caller's view
// (embedded) and the store side, whose BindPersonalToken names an identity.
type identityTokenStore struct {
identity.PersonalTokens
bound bool
id identity.IdentityID
subject identity.Subject
name string
}
func (s *identityTokenStore) BindPersonalToken(_ context.Context, id identity.IdentityID, subject identity.Subject, name string, _ *time.Time, _ identity.Audit) error {
s.bound, s.id, s.subject, s.name = true, id, subject, name
return nil
}
func (s *identityTokenStore) SubjectsOf(context.Context, identity.IdentityID, string) ([]identity.PersonalToken, error) {
return nil, nil
}
func (s *identityTokenStore) UnbindSubject(context.Context, identity.IdentityID, identity.Subject, identity.Audit) error {
return nil
}
func (s *identityTokenStore) PersonalTokenOf(context.Context, identity.Subject) (identity.PersonalToken, error) {
return identity.PersonalToken{}, identity.ErrNotFound
}
func (s *identityTokenStore) TouchPersonalToken(context.Context, identity.Subject, time.Time) error {
return nil
}
// TestIdentityTokenCreate pins the per-identity token route: an active
// service account gets its raw token exactly once and the store binds only
// its hash, while an unknown or inactive identity is refused.
func TestIdentityTokenCreate(t *testing.T) {
active := identity.Identity{ID: "sa1", Kind: identity.KindServiceAccount, DisplayName: "Canary", Lifecycle: identity.LifecycleActive}
suspended := identity.Identity{ID: "sa2", Kind: identity.KindServiceAccount, DisplayName: "Old", Lifecycle: identity.LifecycleSuspended}
tests := []struct {
name, id, body string
want int
}{
{"active service account", "sa1", `{"name":"ci","days":30}`, http.StatusCreated},
{"unknown identity", "nope", `{"name":"ci","days":30}`, http.StatusBadRequest},
{"inactive identity", "sa2", `{"name":"ci","days":30}`, http.StatusConflict},
{"bad name", "sa1", `{"name":" ","days":30}`, http.StatusBadRequest},
{"bad lifetime", "sa1", `{"name":"ci","days":10}`, http.StatusBadRequest},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
store := &identityTokenStore{}
srv := &Server{
Identities: identityRepo{values: map[identity.IdentityID]identity.Identity{active.ID: active, suspended.ID: suspended}},
PersonalTokens: store,
}
req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "https://archie.example/api/identities/"+tt.id+"/tokens", strings.NewReader(tt.body))
req.Header.Set("Content-Type", "application/json")
req.SetPathValue("id", tt.id)
rec := httptest.NewRecorder()
srv.handleIdentityTokenCreate(rec, req)
if rec.Code != tt.want {
t.Fatalf("status=%d want=%d body=%s", rec.Code, tt.want, rec.Body.String())
}
if tt.want != http.StatusCreated {
if store.bound {
t.Fatal("a refused request still bound a token")
}
return
}
var body struct {
ID string `json:"id"`
Token string `json:"token"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatalf("response: %v (%s)", err, rec.Body.String())
}
if !strings.HasPrefix(body.Token, "archie_pat_") || body.ID == "" {
t.Fatalf("response must show the raw token once: %+v", body)
}
if !store.bound || store.id != active.ID || store.name != "ci" {
t.Fatalf("bound = %+v, want id %q name %q", store, active.ID, "ci")
}
if store.subject.Issuer != identity.PersonalTokenIssuer || store.subject.Subject != body.ID {
t.Fatalf("store kept subject %+v, want the token's hash %q under %q", store.subject, body.ID, identity.PersonalTokenIssuer)
}
})
}
}
type unavailableAccounts struct{ identity.Accounts }
func (unavailableAccounts) AccountSetupRequired(context.Context) (bool, error) {
+13
View File
@@ -15,6 +15,7 @@ import (
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
"github.com/samcharles93/archie-core/internal/releaseupdate"
)
@@ -513,6 +514,11 @@ func (s *Server) handleChatUpdate(w http.ResponseWriter, r *http.Request) {
writeUpdateCheckError(w, err)
return
}
// The snapshot carries when the installed release was committed on this
// host, so a canary can count its window from the actual install.
if _, at, ok := releaseupdate.LastInstall(updateStateDir()); ok {
snapshot.InstalledAt = at
}
writeJSON(w, map[string]any{
"snapshot": snapshot,
"available": snapshot.Available(),
@@ -520,6 +526,13 @@ func (s *Server) handleChatUpdate(w http.ResponseWriter, r *http.Request) {
})
}
// updateStateDir resolves the host's update state directory the same way the
// daemon does, so the dashboard reads the installed.json its transactions
// write.
func updateStateDir() string {
return configuration.UpdateStateDir()
}
func (s *Server) handleChatUpdateDefer(w http.ResponseWriter, r *http.Request) {
chat, ok := s.chatReady(w)
if !ok {
+66
View File
@@ -7,12 +7,25 @@ import (
"errors"
"net/http"
"strings"
"time"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/identity"
"github.com/samcharles93/archie-core/internal/domain/org"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
)
// identityTokenBinder is the one store-side operation the per-identity token
// route needs: binding a token to a NAMED identity. The remote client's
// caller-scoped PersonalTokens view cannot express that, so the store fronts
// it as its own RPC and the Client carries it under this name.
type identityTokenBinder interface {
BindPersonalToken(ctx context.Context, id identity.IdentityID, subject identity.Subject, name string, expiresAt *time.Time, audit identity.Audit) error
}
// The production store client satisfies the route's narrow contract.
var _ identityTokenBinder = (*staterpc.Client)(nil)
type identityRequest struct {
DisplayName string `json:"display_name"`
Kind identity.Kind `json:"kind"`
@@ -81,6 +94,59 @@ func (s *Server) handleIdentityCommand(w http.ResponseWriter, r *http.Request) {
writeJSON(w, value)
}
// handleIdentityTokenCreate issues an API token for one named identity, which
// is how a non-user identity gets its bearer credential: a person's token
// comes from /api/tokens, a service account's from here. The raw token is
// returned exactly once; the store keeps only its hash.
func (s *Server) handleIdentityTokenCreate(w http.ResponseWriter, r *http.Request) {
id := identity.IdentityID(r.PathValue("id"))
if s.Identities == nil {
http.Error(w, "identity store unavailable", http.StatusServiceUnavailable)
return
}
value, err := s.Identities.Get(r.Context(), id)
if errors.Is(err, identity.ErrNotFound) {
http.Error(w, "No such identity.", http.StatusBadRequest)
return
}
if err != nil {
writeIdentityError(w, err)
return
}
if !value.CanAct() {
http.Error(w, "That identity may not act.", http.StatusConflict)
return
}
name, expiresAt, ok := decodeTokenRequest(w, r)
if !ok {
return
}
// The store side of the PersonalTokens contract binds a token to a named
// identity; the remote caller's view only ever names its own principal.
binder, ok := s.PersonalTokens.(identityTokenBinder)
if !ok {
http.Error(w, "identity token binding unavailable", http.StatusServiceUnavailable)
return
}
raw, subject, err := identity.NewPersonalToken()
if err != nil {
http.Error(w, "cannot create token", http.StatusInternalServerError)
return
}
audit, err := webAudit(r.Context())
if err != nil {
http.Error(w, "cannot create request ID", http.StatusInternalServerError)
return
}
if err := binder.BindPersonalToken(r.Context(), id, subject, name, expiresAt, audit); err != nil {
writeIdentityError(w, err)
return
}
w.Header().Set("Cache-Control", "no-store")
w.WriteHeader(http.StatusCreated)
writeJSON(w, map[string]string{"id": subject.Subject, "token": raw})
}
// webAudit attributes a dashboard write to the request's own principal: the
// person the dashboard acts as. Only a request with no principal at all falls
// back to the System identity, which is a real identity the repository
+31 -21
View File
@@ -41,27 +41,8 @@ func (s *Server) handlePersonalTokenCreate(w http.ResponseWriter, r *http.Reques
if !s.personalTokensReady(w, r) {
return
}
var body struct {
Name string `json:"name"`
Days int `json:"days"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4<<10)).Decode(&body); err != nil {
http.Error(w, "invalid request", http.StatusBadRequest)
return
}
name := strings.TrimSpace(body.Name)
if name == "" || utf8.RuneCountInString(name) > maxTokenNameLength {
http.Error(w, "Name a token in up to 40 characters.", http.StatusBadRequest)
return
}
var expiresAt *time.Time
switch body.Days {
case 0:
case 7, 30, 90:
t := time.Now().UTC().AddDate(0, 0, body.Days)
expiresAt = &t
default:
http.Error(w, "Lifetime is 7, 30 or 90 days, or none.", http.StatusBadRequest)
name, expiresAt, ok := decodeTokenRequest(w, r)
if !ok {
return
}
raw, subject, err := identity.NewPersonalToken()
@@ -78,6 +59,35 @@ func (s *Server) handlePersonalTokenCreate(w http.ResponseWriter, r *http.Reques
writeJSON(w, map[string]string{"id": subject.Subject, "token": raw})
}
// decodeTokenRequest reads a token request's name and lifetime, the shape the
// personal-token route and the per-identity token route share.
func decodeTokenRequest(w http.ResponseWriter, r *http.Request) (string, *time.Time, bool) {
var body struct {
Name string `json:"name"`
Days int `json:"days"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4<<10)).Decode(&body); err != nil {
http.Error(w, "invalid request", http.StatusBadRequest)
return "", nil, false
}
name := strings.TrimSpace(body.Name)
if name == "" || utf8.RuneCountInString(name) > maxTokenNameLength {
http.Error(w, "Name a token in up to 40 characters.", http.StatusBadRequest)
return "", nil, false
}
var expiresAt *time.Time
switch body.Days {
case 0:
case 7, 30, 90:
t := time.Now().UTC().AddDate(0, 0, body.Days)
expiresAt = &t
default:
http.Error(w, "Lifetime is 7, 30 or 90 days, or none.", http.StatusBadRequest)
return "", nil, false
}
return name, expiresAt, true
}
func (s *Server) handlePersonalTokenRevoke(w http.ResponseWriter, r *http.Request) {
if !s.personalTokensReady(w, r) {
return
+161 -4
View File
@@ -2,8 +2,15 @@ package webui
import (
"context"
"encoding/json"
"fmt"
"maps"
"net/http"
"slices"
"strings"
"github.com/samcharles93/archie-core/internal/config"
controlpb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/identity"
"github.com/samcharles93/archie-core/internal/domain/org"
)
@@ -23,6 +30,44 @@ type ModelReader interface {
CallerModels(ctx context.Context) (org string, aliases map[string]string, err error)
}
// The control-plane resources the effective-model read layers: the org's own
// providers and the credential bindings that carry their keys. Spelled here
// rather than imported so the dashboard package stays a leaf, as
// api_harness.go does for "credential-bindings".
const (
providerSettingsKind = "provider-settings"
credentialBindingsKind = "credential-bindings"
)
// providerDoc is the part of one provider-settings entry the read needs: where
// the provider's key comes from.
type providerDoc struct {
Class string `json:"class"`
APIKeyEnv string `json:"api_key_env,omitempty"`
APIKeyRef config.SecretRef `json:"api_key_ref"`
}
// credentialBindingDoc is the part of one credential-bindings entry the read
// needs: the org secret the service's key resolves through.
type credentialBindingDoc struct {
Service string `json:"service"`
Secret config.SecretRef `json:"secret"`
}
// effectiveModel is one model alias the caller's org may use, with the state
// of the provider credential it runs on. An alias whose provider's credential
// is missing or does not resolve cannot run a task, so the Model setup step
// stays open and names it.
type effectiveModel struct {
Alias string
Model string
Provider string
// Credential reports that the provider's credential is present and
// resolves; Detail names what is missing when it does not.
Credential bool
Detail string
}
// handleSetup reports which setup steps are still missing in the caller's
// organisation, from live state. Optional capabilities (a chat channel, a
// repository) are set up where they are used, not listed here.
@@ -43,15 +88,127 @@ func (s *Server) handleSetup(w http.ResponseWriter, r *http.Request) {
steps = append(steps, SetupStep{Title: "First task", Done: total > 0, Path: "/tasks"})
}
if s.Models != nil {
// Shown only while the org has no model to run, so a working setup
// carries no extra step.
if _, aliases, err := s.Models.CallerModels(r.Context()); err == nil && len(aliases) == 0 {
steps = append(steps, SetupStep{Title: "Model", Path: "/org/models"})
// Shown only while the org has no runnable model, so a working setup
// carries no extra step: an alias whose provider credential does not
// resolve is no more runnable than no alias at all.
if models, err := s.callerEffectiveModels(r.Context()); err == nil {
if step, open := modelSetupStep(models); open {
steps = append(steps, step)
}
}
}
writeJSON(w, map[string]any{"steps": steps})
}
// modelSetupStep is the Model checklist step for the caller's effective
// models. The step stays open until every alias's provider credential
// resolves; its detail names the credential that does not.
func modelSetupStep(models []effectiveModel) (SetupStep, bool) {
if len(models) == 0 {
return SetupStep{Title: "Model", Path: "/org/models", Detail: "no model alias configured"}, true
}
for _, model := range models {
if !model.Credential {
return SetupStep{Title: "Model", Path: "/org/models", Detail: model.Detail}, true
}
}
return SetupStep{}, false
}
// callerEffectiveModels reads the caller's effective models: the aliases the
// org may use, each with the validity of the provider credential it runs on.
// The credential surfaces are optional; without them the read reports the
// aliases alone with every credential taken as valid.
func (s *Server) callerEffectiveModels(ctx context.Context) ([]effectiveModel, error) {
_, aliases, err := s.Models.CallerModels(ctx)
if err != nil {
return nil, err
}
models := make([]effectiveModel, 0, len(aliases))
for _, alias := range slices.Sorted(maps.Keys(aliases)) {
provider, _, _ := strings.Cut(aliases[alias], "/")
models = append(models, effectiveModel{Alias: alias, Model: aliases[alias], Provider: provider, Credential: true})
}
if s.ControlPlane == nil || s.CheckSecret == nil || len(models) == 0 {
return models, nil
}
orgID := string(org.OrgFromContext(ctx))
var providers map[string]providerDoc
if err := s.queryResourceValue(ctx, providerSettingsKind, &providers); err != nil {
return nil, err
}
if providers == nil {
providers = map[string]providerDoc{}
}
var bindings []credentialBindingDoc
if err := s.queryResourceValue(ctx, credentialBindingsKind, &bindings); err != nil {
return nil, err
}
for i := range models {
models[i].Credential, models[i].Detail = s.providerCredential(ctx, orgID, models[i].Provider, providers, bindings)
}
return models, nil
}
// queryResourceValue decodes one control-plane resource of the caller's org
// into into; an absent resource leaves into untouched.
func (s *Server) queryResourceValue(ctx context.Context, kind string, into any) error {
response, err := s.ControlPlane.Query(ctx, &controlpb.QueryRequest{Kind: kind})
if err != nil {
return err
}
if response.Resource == nil || len(response.Resource.ValueJson) == 0 {
return nil
}
return json.Unmarshal(response.Resource.ValueJson, into)
}
// providerCredential reports whether provider's credential is present and
// resolves for orgID, and names what is missing when it does not. The key
// comes from the provider's own api_key_ref, or from the org's credential
// binding named after the provider, or from its api_key_env; a binding with no
// secret is OAuth, whose credential is the captured token set. A provider
// that is not the org's own and has no binding is an instance provider: its
// credential is the instance admin's, not this org's to configure.
func (s *Server) providerCredential(ctx context.Context, orgID, provider string, providers map[string]providerDoc, bindings []credentialBindingDoc) (bool, string) {
doc, own := providers[provider]
if own && doc.APIKeyRef != (config.SecretRef{}) {
return s.secretCredential(ctx, provider, doc.APIKeyRef)
}
for _, binding := range bindings {
if binding.Service != provider {
continue
}
if binding.Secret != (config.SecretRef{}) {
return s.secretCredential(ctx, provider, binding.Secret)
}
if s.HarnessSecrets != nil {
if token, err := s.HarnessSecrets.GetHarnessSecret(ctx, orgID, provider); err == nil && token.Validate() == nil {
return true, ""
}
return false, fmt.Sprintf("provider %q has no captured sign-in for credential binding %q", provider, provider)
}
return false, fmt.Sprintf("credential binding %q names no secret and no sign-in is captured", provider)
}
if own {
if doc.APIKeyEnv != "" {
return s.secretCredential(ctx, provider, config.SecretRef{Engine: "env", Key: doc.APIKeyEnv})
}
return false, fmt.Sprintf("provider %q declares no credential: set its api_key_ref or add a credential binding named %q", provider, provider)
}
return true, ""
}
// secretCredential reports whether ref resolves through the gateway's secret
// engines, naming the reference when it does not.
func (s *Server) secretCredential(ctx context.Context, provider string, ref config.SecretRef) (bool, string) {
resolved, err := s.CheckSecret(ctx, ref)
if err == nil && resolved {
return true, ""
}
return false, fmt.Sprintf("provider %q credential %s:%s does not resolve", provider, ref.Engine, ref.Key)
}
// hasActiveAgent reports whether the caller's org has an active bot or
// service account assigned; ok is false when that cannot be read.
func (s *Server) hasActiveAgent(r *http.Request) (done, ok bool) {
+230
View File
@@ -0,0 +1,230 @@
package webui
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"google.golang.org/grpc"
controlpb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/harnesssecret"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/secret"
)
type setupStore struct {
storecontract.TaskStore
}
func (setupStore) StatusCounts(context.Context) (map[string]int, error) {
return map[string]int{"done": 1}, nil
}
type setupModels struct {
aliases map[string]string
err error
}
func (m setupModels) CallerModels(context.Context) (string, map[string]string, error) {
return "org", m.aliases, m.err
}
// setupControl answers provider-settings and credential-bindings queries from
// fixed documents, the way the State Store would for one org.
type setupControl struct {
controlpb.ControlPlaneServiceClient
providers string
bindings string
}
func (c setupControl) Query(_ context.Context, r *controlpb.QueryRequest, _ ...grpc.CallOption) (*controlpb.QueryResponse, error) {
var value string
switch r.Kind {
case providerSettingsKind:
value = c.providers
case credentialBindingsKind:
value = c.bindings
default:
value = "null"
}
return &controlpb.QueryResponse{Resource: &controlpb.Resource{Kind: r.Kind, ValueJson: []byte(value)}}, nil
}
type setupHarnessSecrets struct {
storecontract.HarnessSecretStore
tokens map[string]harnesssecret.Secret
}
func (h setupHarnessSecrets) GetHarnessSecret(_ context.Context, _, service string) (harnesssecret.Secret, error) {
if token, ok := h.tokens[service]; ok {
return token, nil
}
return harnesssecret.Secret{}, storecontract.ErrHarnessSecretNotFound
}
func setupRequest(t *testing.T, srv *Server) []SetupStep {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/api/setup", nil)
response := httptest.NewRecorder()
srv.handleSetup(response, req)
if response.Code != http.StatusOK {
t.Fatalf("response = %d %s", response.Code, response.Body.String())
}
var body struct {
Steps []SetupStep `json:"steps"`
}
if err := json.Unmarshal(response.Body.Bytes(), &body); err != nil {
t.Fatalf("decode response %q: %v", response.Body.String(), err)
}
return body.Steps
}
func modelStep(steps []SetupStep) *SetupStep {
for i := range steps {
if steps[i].Title == "Model" {
return &steps[i]
}
}
return nil
}
func TestHandleSetupModelStep(t *testing.T) {
resolving := func(_ context.Context, ref secret.SecretRef) (bool, error) {
return ref == (secret.SecretRef{Engine: "env", Key: "GOOD_KEY"}), nil
}
for _, tt := range []struct {
name string
aliases map[string]string
providers string
bindings string
tokens map[string]harnesssecret.Secret
check func(context.Context, secret.SecretRef) (bool, error)
open bool
detail string
}{
{
name: "no aliases leaves the step open",
aliases: map[string]string{},
open: true,
detail: "no model alias configured",
},
{
name: "resolving api_key_ref completes the step",
aliases: map[string]string{"default": "openai/gpt-5"},
providers: `{"openai":{"class":"openai","api_key_ref":{"engine":"env","key":"GOOD_KEY"}}}`,
check: resolving,
open: false,
},
{
name: "unresolving api_key_ref keeps the step open and names it",
aliases: map[string]string{"default": "openai/gpt-5"},
providers: `{"openai":{"class":"openai","api_key_ref":{"engine":"env","key":"BAD_KEY"}}}`,
check: resolving,
open: true,
detail: `provider "openai" credential env:BAD_KEY does not resolve`,
},
{
name: "provider declaring no credential keeps the step open",
aliases: map[string]string{"default": "openai/gpt-5"},
providers: `{"openai":{"class":"openai"}}`,
check: resolving,
open: true,
detail: `provider "openai" declares no credential`,
},
{
name: "credential binding secret resolves for an org provider",
aliases: map[string]string{"default": "acme/gpt-5"},
providers: `{"acme":{"class":"openai"}}`,
bindings: `[{"service":"acme","secret":{"engine":"env","key":"GOOD_KEY"}}]`,
check: resolving,
open: false,
},
{
name: "credential binding secret that does not resolve keeps the step open",
aliases: map[string]string{"default": "acme/gpt-5"},
providers: `{"acme":{"class":"openai"}}`,
bindings: `[{"service":"acme","secret":{"engine":"env","key":"BAD_KEY"}}]`,
check: resolving,
open: true,
detail: `provider "acme" credential env:BAD_KEY does not resolve`,
},
{
name: "org provider without a binding keeps the step open",
aliases: map[string]string{"default": "acme/gpt-5"},
providers: `{"acme":{"class":"openai"}}`,
check: resolving,
open: true,
detail: `provider "acme" declares no credential`,
},
{
name: "instance provider is not the org's credential to configure",
aliases: map[string]string{"default": "shared/gpt-5"},
providers: `{}`,
check: resolving,
open: false,
},
{
name: "oauth binding with a captured token set completes the step",
aliases: map[string]string{"default": "acme/gpt-5"},
providers: `{"acme":{"class":"openai"}}`,
bindings: `[{"service":"acme"}]`,
tokens: map[string]harnesssecret.Secret{"acme": {Org: "org", Service: "acme", AccessToken: "token"}},
check: resolving,
open: false,
},
{
name: "oauth binding without a captured token set keeps the step open",
aliases: map[string]string{"default": "acme/gpt-5"},
providers: `{"acme":{"class":"openai"}}`,
bindings: `[{"service":"acme"}]`,
tokens: map[string]harnesssecret.Secret{},
check: resolving,
open: true,
detail: `provider "acme" has no captured sign-in for credential binding "acme"`,
},
{
name: "api_key_env resolves through the env engine",
aliases: map[string]string{"default": "openai/gpt-5"},
providers: `{"openai":{"class":"openai","api_key_env":"GOOD_KEY"}}`,
check: resolving,
open: false,
},
{
name: "without the control plane the aliases alone decide",
aliases: map[string]string{"default": "openai/gpt-5"},
check: resolving,
open: false,
},
} {
t.Run(tt.name, func(t *testing.T) {
srv := &Server{Store: setupStore{}, Models: setupModels{aliases: tt.aliases}}
if tt.providers != "" || tt.bindings != "" {
srv.ControlPlane = setupControl{providers: tt.providers, bindings: tt.bindings}
}
srv.CheckSecret = tt.check
if tt.tokens != nil {
srv.HarnessSecrets = setupHarnessSecrets{tokens: tt.tokens}
}
step := modelStep(setupRequest(t, srv))
if !tt.open {
if step != nil {
t.Fatalf("Model step = %+v, want it complete (absent)", *step)
}
return
}
if step == nil {
t.Fatalf("Model step absent, want it open with detail %q", tt.detail)
}
if step.Done {
t.Fatalf("Model step done, want it open")
}
if !strings.Contains(step.Detail, tt.detail) {
t.Fatalf("Model step detail = %q, want it to contain %q", step.Detail, tt.detail)
}
})
}
}
+31
View File
@@ -0,0 +1,31 @@
package webui
import (
"net/http"
"github.com/samcharles93/archie-core/internal/events"
)
// taskTranscriptView is GET /api/tasks/{id}/transcript: one attempt's agent
// transcript entries, oldest first.
type taskTranscriptView struct {
TaskID int64 `json:"task_id"`
Attempt int `json:"attempt"`
Entries []events.Event `json:"entries"`
}
func (s *Server) handleTaskTranscript(w http.ResponseWriter, r *http.Request) {
t, attempt, ok := s.taskAttemptTarget(w, r)
if !ok {
return
}
entries, err := s.Store.TaskTranscript(r.Context(), t.ID, attempt)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
if entries == nil {
entries = []events.Event{}
}
writeJSON(w, taskTranscriptView{TaskID: t.ID, Attempt: attempt, Entries: entries})
}
@@ -86,6 +86,7 @@ func TestHTTPPolicyCannotBeBypassed(t *testing.T) {
{"GET", "/api/tasks/7/logs"},
{"GET", "/api/tasks/7/logs/download"},
{"GET", "/api/tasks/7/sandbox"},
{"GET", "/api/tasks/7/transcript"},
{"GET", "/api/tokens"},
{"GET", "/api/version"},
{"GET", "/api/workflows"},
Loaded 100 of 238 files, more files were not shown because too many files have changed in this diff. Show more