Compare commits

..
70 Commits
Author SHA1 Message Date
sam e2e24f9d56 ci(deploy): pull the container image the uncached test suite needs
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-11 04:26:08 +11:00
sam 5aa5c2a6c8 chore(release): v1.53.0 2026-10-11 04:23:10 +11:00
sam 41b2daed0c Merge branch 'work/53jc' 2026-10-11 04:11:43 +11:00
sam e0a79b3990 Merge branch 'work/ry4k' 2026-10-11 04:11:43 +11:00
sam e2d6b75b10 fix(prreview): restore the benchmark runner on configured models and public PRs 2026-10-11 04:11:40 +11:00
sam b4be999777 fix(gateway): remove the web /restart command that no process could serve 2026-10-11 04:11:37 +11:00
sam 5c3bf0b642 feat(storepkg): auto-update policy, applied by a State Store ticker 2026-10-11 04:10:53 +11:00
sam be2b897149 refactor(servicediscovery): delete unreachable contract and NATS implementation 2026-10-11 04:09:56 +11:00
sam f5a39cfb9b fix(daemon): bound a package workflow's triggers by its accepted authority 2026-10-11 04:08:02 +11:00
sam f5a6fa5b32 fix(postgres): number the outcome-posted migration after package updates 2026-10-11 04:04:45 +11:00
sam 93ec6d916b Merge branch 'work/mt9xo' 2026-10-11 04:03:52 +11:00
sam c36fb29683 Merge branch 'work/zcg8' 2026-10-11 04:03:52 +11:00
sam cac0f43b58 Merge branch 'work/mqti13'
# Conflicts:
#	internal/contracts/state/v1/state.pb.go
2026-10-11 04:03:52 +11:00
sam 805364f201 feat(archied): post a chat task's outcome back to its chat 2026-10-11 04:03:30 +11:00
sam 3b2a3a7630 fix(webui): the setup checklist and skill catalogue are system-org only 2026-10-11 04:01:46 +11:00
sam bf29d3640e fix(daemon): reown worktrees a killed container left root-owned 2026-10-11 04:01:19 +11:00
sam de15bffc2f fix(telegram): publish /approve, which the router already serves 2026-10-11 03:57:10 +11:00
sam 1377e7ebf2 feat(webui): add a member by provider subject or email
POST /api/orgs/{id}/members binds a person by subject, or by email as a
placeholder. The first sign-in whose provider vouches for that email
claims it: the State Store moves the binding onto the provider's subject
in one statement, so a second account with the same address finds
nothing to claim and later sign-ins rest on the subject alone.
2026-10-11 03:54:49 +11:00
sam d53ebbda95 fix(daemon): last poll reports the least recently polled loop 2026-10-11 03:53:11 +11:00
sam b227df3c3f feat(storepkg): widening updates wait for approval, rollback restores the previous pin 2026-10-11 03:48:31 +11:00
sam a3e0e5c1bb fix(daemon): bound a package workflow's tools by its accepted authority 2026-10-11 03:48:31 +11:00
sam 116f8a10fb style(webui): rewrap the assets comment 2026-10-11 03:48:17 +11:00
sam 116b03534f fix(staterpc): a task's own credential cannot replace its workflow pin
The pinned definition carries the package a run's authority is read
from. A task-scoped Update now reaches the store without a definition,
and UpdateTask keeps the stored pin when none is sent; the workflow name
still moves for handoffs and the daemon re-pins it.
2026-10-11 03:47:17 +11:00
sam b697a12b89 fix(webui): an unserved /api path is a 404, not the dashboard page 2026-10-11 03:46:40 +11:00
sam 7b64699ff1 refactor(workflow): delete routing overrides nothing applied 2026-10-11 03:43:25 +11:00
sam 1598b3cc74 docs(agents): extend the finish-and-shrink rule to 2026-10-20 2026-10-11 03:39:31 +11:00
sam 212742e880 feat(forgerpc): bound a package workflow's forge calls by its accepted authority 2026-10-11 03:37:16 +11:00
sam 6693e60f6f fix(access): an unloaded chain is a refusing engine, not nil
Webhook intake answers 503 and counts no refusal while the chain loads.
The readiness probe reads the chain on every check. The UI never exits on
a first-load failure. An invalid instance policy closes the instance but
lets an instance admin repair policies. Outage pages are 503, and the
health detail names the unloaded chain.
2026-10-10 22:19:02 +11:00
sam c097e50405 fix(archieui): warn when the dashboard is reachable off-host over plain HTTP
The dashboard serves plain HTTP, so a non-loopback bind carries the
session cookie and provider token in clear unless TLS terminates in front.
Warn at startup unless the operator declares a proxy with
[web].trust_forwarded_headers, and record the choice in deployments/README
where the dashboard is deployed. Point the non-loopback token error at
that surviving doc, not the deleted UI-boundary PRD.
2026-10-10 22:09:05 +11:00
sam ccbc9c7844 fix(webui): the wildcard listen address is not loopback
IsLoopback(":8484") returned true: the empty host fell through the
switch as loopback, so a wildcard bind skipped the non-loopback token
requirement and served every interface unauthenticated. An empty host is
the wildcard, not loopback.
2026-10-10 22:09:05 +11:00
sam 948a8d34fd test(worktree): give each transport its own branch
The four subtests share one bare repo but pushed the same feat/1-test
branch. Their commits have the same tree, message, author and parent, so
two commits in the same second hash identically and the second push is a
no-op -- passing by luck -- while a second of clock drift makes it a
non-fast-forward failure. One issue per subtest gives each its own branch.
2026-10-10 22:05:22 +11:00
sam e078b5c4bc fix(webui): refuse API requests until the access chain loads
A State Store that was unreachable at startup left the access chain nil,
and authorize treated nil as no chain at all: every request passed the
credential check, no principal or org reached the context, the State Store
defaulted each call to org-sys, and the instance-only surfaces were served.
The dashboard stayed open until a restart.

Build a chain with no engine when the first load is Unavailable; it refuses
every request with 503 (or the notice page for a browser) until the first
load succeeds, and Run keeps retrying so the dashboard recovers without a
restart. A denial is not recorded, because nothing was evaluated.
2026-10-10 21:55:02 +11:00
sam 4a24c0b31f fix(memory): each org's agent memory is its own 2026-10-10 21:20:37 +11:00
sam e988d3cc40 fix: close cross-org writes and move existing rows into their owner's org 2026-10-10 20:31:54 +11:00
sam cd20aeadad fix(webui): instance health, logs, config and identities are system-org only 2026-10-10 20:20:07 +11:00
sam 1bb7ba1248 fix(postgres): tasks, events, steps and run stats stay in their org 2026-10-10 20:18:29 +11:00
sam fdafec3d9d fix(postgres): event sources, types, mappings and bindings stay in their org 2026-10-10 20:14:23 +11:00
sam 0fcf792db3 feat(gateway): chat sessions belong to one org
Sessions carry an org; listing, reading and writing a session, its
messages and its turns are limited to the caller's org. Another org's
session reads as not found and cannot be overwritten or appended to.
Sessions from before orgs belong to the system org.
2026-10-10 18:57:26 +11:00
sam 9cd52dd9aa feat(archiegateway): chat resolves models in the caller's org and bills it
The State Store answers the calling principal's org aliases
(CallerModels) and bills chat usage to that org; usage records no longer
carry a caller-named org. Chat lists and resolves aliases for the
caller's org and fails rather than substitute one it may not use.
2026-10-10 18:53:57 +11:00
sam 5290dab199 feat(controlplane): per-org model access, org aliases and org providers
The system org sets each org's allowed models and whether it may add its
own providers (org-model-policy); an org with no entry may use nothing.
An org's aliases and providers are checked against that policy on save
and filtered by it at resolution. Tasks get their org's providers, keyed
by the org's credential binding through the egress proxy. Org Settings
gains a Models tab; instance Models gains Org access.
2026-10-10 18:48:40 +11:00
sam e17c7895e4 feat(daemon): resolve a task's models and credentials in the task's own org
The State Store answers a task's effective model aliases (instance aliases
with the task org's own over them), deriving the org from the task. Model
egress and the run credential resolver use the task's stored org instead
of the identity mapping in config.
2026-10-10 18:40:53 +11:00
sam e4ea2eaabc feat(statestore): record a usage row for every model call
Task agent runs and MCP sampling record against their task (billed to the
task's org); gateway chat turns, titles, sampling and curators record
against the system org. Rows carry source, workflow, step, alias,
provider/model and tokens for later chargeback.
2026-10-10 18:32:43 +11:00
sam 12b9a57404 feat(config): model aliases replace model roles, resolved by one resolver
Agent steps, implement, pr-review, MCP sampling, chat, embedding and
transcription resolve a model alias (empty means the purpose's default)
and fail on an unknown alias instead of falling back. The step editor and
chat pick aliases; workflow definitions naming an unknown alias are
refused on save.
2026-10-10 17:29:52 +11:00
sam 514c6cf61c fix(daemon): keep model provider keys out of agent containers via the egress proxy 2026-10-10 17:12:09 +11:00
sam bef0ecea3a feat(ui): workflow list shows running count and latest run result 2026-10-10 11:50:27 +11:00
sam 5bb5d9b660 feat(ui): quieter workflows page: hover data edges, apply chip, restore behind confirm 2026-10-10 11:49:32 +11:00
sam f30405ea2b fix(ui): input group addon no longer steals clicks from its select 2026-10-10 11:26:42 +11:00
sam f9efae8a5e fix(archiegateway): keep the chat model list non-nil when empty 2026-10-10 11:17:37 +11:00
sam 819cfc655b refactor: slices.Sorted over map keys and one shared IsNil check 2026-10-10 11:14:36 +11:00
sam 5afe5bcf99 refactor: use slices.Index/Delete and slices.Clone for small copies 2026-10-10 11:14:36 +11:00
sam 0c25b2f4a6 refactor(workflow): decodeSettings returns a typed value 2026-10-10 11:14:36 +11:00
sam 252895e01c refactor(staterpc): use mapValues for proto list conversions 2026-10-10 11:05:59 +11:00
sam 3ea49c7e07 refactor(gateway): drop duplicate string arg helpers 2026-10-10 11:05:59 +11:00
sam 66a27bd7ef refactor(controlplane): make layerResource decode into a typed value 2026-10-10 11:05:59 +11:00
sam b070ce4715 refactor(eda): use slices.Sorted over maps.Keys for ordered keys 2026-10-10 11:05:58 +11:00
sam 84b581fde6 refactor(webui): decode JSON request bodies with generic decodeBody 2026-10-10 11:05:58 +11:00
sam af5e11a734 refactor(domain): share one generic lifecycle registry for curators and memory engines 2026-10-10 11:05:58 +11:00
sam 522ced45ff feat(ui): lanes for parallel and switch steps on the workflow canvas 2026-10-10 10:52:52 +11:00
sam 79d7f2fa68 feat(workflow): switch step runs the case matching a reference 2026-10-10 10:52:52 +11:00
sam eb3ee14a0e feat(ui): step roles, numbers and run trace on workflow canvas 2026-10-10 10:37:47 +11:00
sam 6ed6b28d02 build(ui): refresh dashboard bundle 2026-10-09 20:06:01 +11:00
sam 2f4e9bbe14 Merge branch 'main' into codex/control-plane-4q28 2026-10-09 20:03:52 +11:00
sam 3057f054b2 style: remove space between icons and labels 2026-10-09 19:56:27 +11:00
sam a4e7ea8cc3 fix(kit): migrate runtime to sandbox kit m8 2026-10-09 18:31:56 +11:00
sam c812192800 feat(ui): author parallel workflow branches on canvas 2026-10-09 18:18:45 +11:00
sam 81121d03b0 docs(agents): omit status the maintainer can already see 2026-10-09 18:16:45 +11:00
sam e5f03587c9 docs(agents): post externally only when told 2026-10-09 18:09:11 +11:00
sam c25e70a588 docs(agents): act on clear direction and keep replies short 2026-10-09 18:04:38 +11:00
sam b97c79aa0b feat(ui): edit workflow call interfaces on the canvas 2026-10-09 18:04:08 +11:00
sam 2d2c39f9e5 fix(daemon): reconcile call waits from durable steps 2026-10-09 17:56:15 +11:00
301 changed files with 12704 additions and 4864 deletions

No files matched your search

+6
View File
@@ -103,6 +103,12 @@ jobs:
- name: Allow unprivileged user namespaces
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
# The gate's uncached `go test ./...` runs the Docker-backed tests that
# -short skips, and those start a container from alpine:3. A fresh runner
# has no such image, so the suite failed on "No such image: alpine:3".
- name: Pull test container images
run: docker pull alpine:3
- name: Quality gate
run: |
# The formatter sequence has one owner: task fmt, which drives go fix
+25
View File
@@ -5,6 +5,31 @@ release's per-component sections are labelled beneath its heading.
## [Unreleased]
## [1.53.0] - 2026-10-11
### archied
- Every organisation's data is its own. Chats, tasks, events, transitions, steps, tool calls, run statistics, event sources, event types, mappings, bindings and agent memory are read and written in the org that owns them, and a request can no longer write into another org. Rows written before orgs existed move to their owner's org on upgrade.
- An instance admin decides what each org may reach: the models it can use, and whether it may register providers under its own credentials. Org Settings gains a Models tab and instance Models gains Org access. The system org is never restricted; another org uses nothing until it is granted.
- Models are addressed by alias. Aliases replace model roles, one resolver serves every purpose (agent steps, implement, PR review, MCP sampling, chat, embedding, transcription), and an unknown alias fails instead of falling back to a default. The step editor and the chat picker offer aliases, and a workflow naming one that does not exist is refused when saved.
- Chat sessions belong to one org; a chat resolves its models in the caller's org and is billed to it, and a task run from chat posts its outcome back to that chat.
- Add a member by provider subject, or by email as a placeholder that the first sign-in the provider vouches for claims.
- The workflow canvas authors parallel branches as lanes, adds a switch step that runs the case matching a reference, and edits a call's interface. A step shows its role, its number and its run trace.
- The workflows list shows each workflow's running count and latest run result, the canvas reveals its data-edge handles on hover, and restoring a version asks for confirmation.
- Store packages can auto-update: a package given the policy is moved to the catalogue's newest digest by a State Store ticker. An update asking for more authority than was accepted waits as a pending update with the running version untouched; approving it accepts the new authority, and a rollback restores the previous version with the authority accepted against it.
- Model provider keys never enter an agent container: the runtime reaches its providers through the egress proxy, which holds the keys. A task's own credential can no longer replace the workflow definition its run is pinned to, and a package workflow's triggers, tools and forge calls are bounded by the authority accepted for the package.
- The dashboard warns when it is reachable from off-host over plain HTTP, and the wildcard listen address counts as off-host rather than loopback. Instance health, logs, config and identities, the setup checklist and the skill catalogue are system-org only, and API requests are refused until the access chain has loaded.
- `/approve` reaches Telegram; the router served it but nothing published it.
- A worktree a killed container left root-owned is taken back by the daemon, and a PR review's benchmark runner works again on configured models and public PRs.
- An unserved `/api` path returns 404 instead of the dashboard page, and an input group's select no longer loses clicks to its addon.
### archie-agent
- The runtime reaches model providers through the egress proxy, so provider keys stay out of the container.
- Agent steps resolve model aliases and their own org's models and credentials, failing on an unknown alias instead of falling back.
- A switch step runs the case matching its reference.
- A call waiting on another step is reconciled from durable steps, so it resumes after a daemon restart rather than stranding.
## [1.52.0] - 2026-10-09
### archied
+15 -1
View File
@@ -11,7 +11,7 @@ agent or tool in an isolated container, and the result lands somewhere for
review or action. A repository is an optional input to a workflow, not the point
of the product.
<temporary - this statement must be removed on 2026-10-10"> The product is
<temporary - this statement must be removed on 2026-10-20"> The product is
unfinished. Many features are half-built, and the codebase carries far more
code, binaries, docs and tooling than it delivers. The job right now is to
finish features and shrink the codebase, not to extend it. </temporary>
@@ -27,6 +27,20 @@ When sources disagree, the higher one wins:
5. Everything else (memories, comments, old docs) is a hint. Verify it against
the code before acting on it.
## Working with the maintainer
- **A clear direction is the go-ahead.** Do it in the same turn. Do not restate
the plan, offer options, or ask "shall I proceed?". Stop only for destructive
or outward-facing actions (deleting data, pushing, touching a live host).
- **Post outside the repo only when told.** Chat, tickets and any external
service get a message only on the maintainer's instruction.
- **Report, don't narrate.** Final replies say what changed, what was verified,
and anything blocked, in a few lines. No preamble, no recap of the request, no
closing offers of more help. Leave out what the maintainer can see or infer:
a clean tree, unpushed commits, a passing gate (commits imply it), and steps
skipped that the task never asked for. Name a skipped step only when the
task required it.
## How to decide
- **Broken beats new.** A bug that blocks work gets fixed now, with the smallest
+5 -5
View File
@@ -125,11 +125,6 @@ tasks:
cmds:
- golangci-lint run ./...
lint:playbooks:
desc: Validate playbook bindings
cmds:
- go run ./cmd/archied playbooks lint -dir ./playbooks
vuln:
desc: Scan for known vulnerabilities (requires network access)
cmds:
@@ -175,6 +170,11 @@ tasks:
cmds:
- bash tools/webhook-send.sh {{.CLI_ARGS}}
bench:prreview:
desc: Run the PR review benchmark on the configured models and write docs/benchmarks/pr-review-*.md (-- -model ALIAS -judge-model REF; --help for flags)
cmds:
- go run ./cmd/archied prbench {{.CLI_ARGS}}
dev:
desc: Run development services and Vite at http://localhost:5173
deps: [dev:daemon, dev:gateway, dev:messaging, dev:state-store, dev:dashboard, dev:ui]
+3 -4
View File
@@ -21,7 +21,6 @@ steps:
- id: plan
type: agent.run
settings:
role: planner
read_only: true
mission: |
Plan this change: {{ task.prompt }}
@@ -135,7 +134,7 @@ _runs without a repository._
| `protect_tests` | bool | Write-block the repository test files. |
| `read_only` | bool | Restrict the agent to read-only tools. |
| `result` | mapping | A JSON Schema object the agent must return; later steps read it as steps.<id>.result. |
| `role` | text | Which configured model role runs the agent, e.g. builder or planner. Empty means builder. |
| `model` | text | The model alias that runs the agent, e.g. default or fast. Empty means default. |
### `command.run`
@@ -247,8 +246,8 @@ _runs without a repository._
| Setting | Type | Meaning |
|---|---|---|
| `inputs` | mapping | |
| `outputs` | mapping | |
| `inputs` | mapping | Callee input names mapped to scalar literals or `inputs.<name>` references to caller inputs. |
| `outputs` | mapping | Callee output names mapped to `outputs.<name>` references to caller outputs; requires `wait: true`. |
| `wait` | bool | Wait for the child run to finish before continuing. |
| `workflow` | text | The workflow to start as a child run. |
+4
View File
@@ -15,6 +15,7 @@ import (
"syscall"
"github.com/samcharles93/archie-core/internal/app/agentworker"
"github.com/samcharles93/archie-core/internal/infrastructure/egress"
)
type workerRunner func(context.Context, agentworker.Settings, *slog.Logger) error
@@ -90,6 +91,9 @@ func runCommand(args []string, getenv func(string) string, stderr io.Writer, run
NATSToken: natsToken,
StateStoreTarget: stateStoreURL,
StateStoreToken: stateStoreToken,
ModelProxy: getenv(egress.ModelProxyEnv),
ModelProxyHosts: egress.ParseHosts(getenv(egress.ModelProxyHostsEnv)),
ModelProxyCA: getenv(egress.ModelProxyCAEnv),
}, log); err != nil {
return 1
}
+4
View File
@@ -6,6 +6,7 @@ import (
"github.com/samcharles93/archie-core/internal/app/archied"
"github.com/samcharles93/archie-core/internal/app/archieplaybooks"
"github.com/samcharles93/archie-core/internal/app/prbench"
)
func main() {
@@ -28,6 +29,9 @@ func main() {
if len(args) > 0 && args[0] == "status" {
os.Exit(archied.RunStatus(args[1:], os.Stdout, os.Stderr))
}
if len(args) > 0 && args[0] == "prbench" {
os.Exit(prbench.Main(args[1:], os.Stdout, os.Stderr))
}
if len(args) > 0 && args[0] == "playbooks" {
os.Exit(archieplaybooks.Run(args[1:], os.Stderr))
}
+8
View File
@@ -76,6 +76,14 @@ A non-loopback `[web].listen` requires a dashboard token (`-token`, or
binding-dispatch loop in `archied` consumes what arrives from the shared
State Store.
The dashboard speaks plain HTTP. A non-loopback bind therefore carries the
session cookie and the provider token in clear over the network, and
archie-ui warns about exactly that at startup. Terminate TLS in front of it
(a reverse proxy or the tunnel itself) and set
`[web].trust_forwarded_headers = true` to declare the proxy and silence the
warning; leaving it off while exposing the port directly is the case the
warning names.
Auth note: unlike `archied`, archie-ui has no secret registry --
`GATEWAY_TOKEN`/`STATE_STORE_TOKEN` resolve from its process environment
only (or from `[services.*].target_token` in `config.toml`), so whatever
+24
View File
@@ -0,0 +1,24 @@
# PR review benchmark
```
task bench:prreview
```
Reviews the 38 runnable Martian pull requests once each at `deep` depth with
the production pipeline, judges the posted comments against the golden
comments in `internal/app/prbench/martian.json`, and writes
`docs/benchmarks/pr-review-<timestamp>.md` (recall per PR, overall recall,
precision, models, archie version). Commit that file with the run.
- Models come from the config at `~/.config/archie/config.toml`: the reviewer
and judge both default to its `default` alias. Override with
`task bench:prreview -- -model ALIAS_OR_PROVIDER/MODEL -judge-model ALIAS_OR_PROVIDER/MODEL`.
Pick a judge other than the reviewer's model for an independent judgement.
Aliases set only in the dashboard are not in the file; pass a
`provider/model` ref instead.
- Pull requests are public, so no credential is needed. `GH_TOKEN` or
`GITHUB_TOKEN`, if set, only raises the GitHub API rate limit.
- `-problems ID,ID` runs a partial smoke run; its report says it is not the
acceptance run. A full run reports PASS when nothing failed and golden
micro-recall is at least 0.70.
- It is not part of `task test` and posts nothing to any pull request.
+2
View File
@@ -78,6 +78,7 @@
"/news/archie/1.51.0/": "/news/1.51.0/",
"/news/archie/1.51.1/": "/news/1.51.1/",
"/news/archie/1.52.0/": "/news/1.52.0/",
"/news/archie/1.53.0/": "/news/1.53.0/",
"/news/archie/1.6.0/": "/news/1.6.0/",
"/news/archie/1.7.0/": "/news/1.7.0/",
"/news/archie/1.8.0/": "/news/1.8.0/",
@@ -172,6 +173,7 @@
"/news/archied/1.51.0/": "/news/1.51.0/",
"/news/archied/1.51.1/": "/news/1.51.1/",
"/news/archied/1.52.0/": "/news/1.52.0/",
"/news/archied/1.53.0/": "/news/1.53.0/",
"/news/archied/1.6.0/": "/news/1.6.0/",
"/news/archied/1.7.0/": "/news/1.7.0/",
"/news/archied/1.8.0/": "/news/1.8.0/",
+5
View File
@@ -1,4 +1,9 @@
[
{
"version": "1.53.0",
"date": "2026-10-11",
"body": "### archied\n\n- Every organisation's data is its own. Chats, tasks, events, transitions, steps, tool calls, run statistics, event sources, event types, mappings, bindings and agent memory are read and written in the org that owns them, and a request can no longer write into another org. Rows written before orgs existed move to their owner's org on upgrade.\n- An instance admin decides what each org may reach: the models it can use, and whether it may register providers under its own credentials. Org Settings gains a Models tab and instance Models gains Org access. The system org is never restricted; another org uses nothing until it is granted.\n- Models are addressed by alias. Aliases replace model roles, one resolver serves every purpose (agent steps, implement, PR review, MCP sampling, chat, embedding, transcription), and an unknown alias fails instead of falling back to a default. The step editor and the chat picker offer aliases, and a workflow naming one that does not exist is refused when saved.\n- Chat sessions belong to one org; a chat resolves its models in the caller's org and is billed to it, and a task run from chat posts its outcome back to that chat.\n- Add a member by provider subject, or by email as a placeholder that the first sign-in the provider vouches for claims.\n- The workflow canvas authors parallel branches as lanes, adds a switch step that runs the case matching a reference, and edits a call's interface. A step shows its role, its number and its run trace.\n- The workflows list shows each workflow's running count and latest run result, the canvas reveals its data-edge handles on hover, and restoring a version asks for confirmation.\n- Store packages can auto-update: a package given the policy is moved to the catalogue's newest digest by a State Store ticker. An update asking for more authority than was accepted waits as a pending update with the running version untouched; approving it accepts the new authority, and a rollback restores the previous version with the authority accepted against it.\n- Model provider keys never enter an agent container: the runtime reaches its providers through the egress proxy, which holds the keys. A task's own credential can no longer replace the workflow definition its run is pinned to, and a package workflow's triggers, tools and forge calls are bounded by the authority accepted for the package.\n- The dashboard warns when it is reachable from off-host over plain HTTP, and the wildcard listen address counts as off-host rather than loopback. Instance health, logs, config and identities, the setup checklist and the skill catalogue are system-org only, and API requests are refused until the access chain has loaded.\n- `/approve` reaches Telegram; the router served it but nothing published it.\n- A worktree a killed container left root-owned is taken back by the daemon, and a PR review's benchmark runner works again on configured models and public PRs.\n- An unserved `/api` path returns 404 instead of the dashboard page, and an input group's select no longer loses clicks to its addon.\n\n### archie-agent\n\n- The runtime reaches model providers through the egress proxy, so provider keys stay out of the container.\n- Agent steps resolve model aliases and their own org's models and credentials, failing on an unknown alias instead of falling back.\n- A switch step runs the case matching its reference.\n- A call waiting on another step is reconciled from durable steps, so it resumes after a daemon restart rather than stranding."
},
{
"version": "1.52.0",
"date": "2026-10-09",
+1 -1
View File
@@ -95,7 +95,7 @@ require (
github.com/BurntSushi/toml v1.6.0
github.com/cedar-policy/cedar-go v1.8.0
github.com/coreos/go-oidc/v3 v3.21.0
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.6
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.8
github.com/go-git/go-billy/v6 v6.0.0-beta.1
github.com/go-git/go-git/v6 v6.0.0-beta.1
github.com/go-telegram/bot v1.27.0
+2 -2
View File
@@ -54,8 +54,8 @@ github.com/docker/go-connections v0.8.2 h1:9d9DEokx5W7qlr0shkFXIxEA7+TTD+llwzcyk
github.com/docker/go-connections v0.8.2/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.6 h1:co7vn6GgImYEGO/KPlhPRJT9i5EPWgOKcB9q7bNJlFQ=
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.6/go.mod h1:MrWO4PN3/VYC5xF8eyI8U82Nv/agTHcN4IUzheqA90c=
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.8 h1:ay2XEj0yW3uzsNwN3KXS/c4nlneHONFKaHlrVhu2WCQ=
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.8/go.mod h1:4G6VgFAp9eaQkwy7DbZbOfWRD/Mjvc/TnEtQBRRMEsA=
github.com/dustin/go-humanize v1.1.0 h1:dbKTrvD0klcbBV/h4AWJdMuZogJACoMlvWIWZ5b2xWg=
github.com/dustin/go-humanize v1.1.0/go.mod h1:hc1CvRkJMsgxqjmjMQF3QNRAZBwY8AXBAzKYoSX9sFI=
github.com/ebitengine/purego v0.11.1 h1:2zpWRSQNVKN4eKsKO9eM1ILDgWfYMY9GwqRmK6XeQ/0=
+3 -3
View File
@@ -31,7 +31,7 @@ type LoopRunner struct {
// Limits bounds the size of tool results fed back into the stage. The
// zero value applies no limits.
Limits ToolLimits
// AllowTools is the profile's allowlist over archie-added tools. Empty allows
// AllowTools is the allowlist over archie-added tools. Nil allows
// all. Capture tools and agentloop's built-in file tools are never
// filtered.
AllowTools []string
@@ -187,9 +187,9 @@ func makeCaptureHandler(spec agentrun.CaptureTool, captures map[string][]json.Ra
}
}
// allowedTools keeps the tools allow names; an empty allow keeps them all.
// allowedTools keeps the tools allow names; a nil allow keeps them all.
func allowedTools(set core.ToolSet, allow []string) core.ToolSet {
if len(allow) == 0 {
if allow == nil {
return set
}
kept := core.ToolSet{}
+30 -11
View File
@@ -4,31 +4,31 @@ import (
"context"
"errors"
"fmt"
"strings"
"log/slog"
"time"
"github.com/samcharles93/ai-sdk/chat"
"github.com/samcharles93/ai-sdk/core"
"github.com/samcharles93/ai-sdk/runtime"
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/usage"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/tools/mcp"
)
// taskSamplingModel returns the task's builder model, or "".
func taskSamplingModel(cfg config.TaskConfig) string {
return strings.TrimSpace(cfg.Models["builder"])
}
// taskSamplingHandler answers MCP sampling requests with the task's model.
// Without a runtime or model the request is refused.
func taskSamplingHandler(llm *runtime.Runtime, cfg config.TaskConfig) mcp.SamplingHandler {
// Without a runtime or model the request is refused. record receives each
// call's model ref and token usage.
func taskSamplingHandler(llm *runtime.Runtime, cfg config.TaskConfig, record func(context.Context, string, chat.Usage)) mcp.SamplingHandler {
return func(ctx context.Context, req mcp.SamplingRequest) (mcp.SamplingResult, error) {
if llm == nil {
return mcp.SamplingResult{}, errors.New("mcp sampling: no model runtime is configured")
}
model := taskSamplingModel(cfg)
if model == "" {
return mcp.SamplingResult{}, errors.New("mcp sampling: no builder model is configured")
model, err := config.ResolveModel(cfg.Models, "", config.PurposeAgent)
if err != nil {
return mcp.SamplingResult{}, fmt.Errorf("mcp sampling: %w", err)
}
messages, err := modelloop.SamplingMessages(req)
if err != nil {
@@ -45,6 +45,7 @@ func taskSamplingHandler(llm *runtime.Runtime, cfg config.TaskConfig) mcp.Sampli
options.Temperature = float32(*req.Temperature)
}
result, err := llm.Chat(ctx, model, options)
record(ctx, model, result.TotalUsage)
if err != nil {
return mcp.SamplingResult{}, fmt.Errorf("mcp sampling: %w", err)
}
@@ -56,3 +57,21 @@ func taskSamplingHandler(llm *runtime.Runtime, cfg config.TaskConfig) mcp.Sampli
}, nil
}
}
// samplingUsageRecorder records a task's MCP sampling calls against the task.
// A failed write is logged, never fails the call.
func samplingUsageRecorder(store workflow.Store, t *workflow.Task, log *slog.Logger) func(context.Context, string, chat.Usage) {
return func(ctx context.Context, model string, used chat.Usage) {
if store == nil || t == nil {
return
}
record := usage.Record{
Source: usage.SourceTask, TaskID: t.ID, Attempt: t.Attempt, Workflow: t.Workflow, Step: "mcp-sampling",
InputTokens: int64(used.PromptTokens), OutputTokens: int64(used.CompletionTokens),
CachedTokens: int64(used.CachedTokens), At: time.Now().UTC(),
}.WithRef(model)
if err := store.RecordUsage(ctx, record); err != nil && log != nil {
log.Warn("record model usage failed", "model", model, "err", err)
}
}
}
+1 -3
View File
@@ -198,8 +198,6 @@ func stageRunners(req taskrun.Request, mcpSet *mcpProviderSet, newRunner runnerF
// routeTask remains available for routing-only callers. Production execution
// compiles the request's pinned YAML directly.
func routeTask(req taskrun.Request, registry workflow.Registry) workflow.Workflow {
workflow.SetKindWorkflows(req.KindWorkflows)
workflow.SetLabelWorkflows(req.LabelWorkflows)
return workflow.Route(req.Task, registry)
}
@@ -233,7 +231,7 @@ func runTask(ctx context.Context, req taskrun.Request, dependencies taskDependen
if len(req.MCPServers) > 0 {
llm = modelloop.NewRuntime(req.Providers, req.Cfg.ModelLimits)
}
mcpSet, mcpErr := startMCPProviders(ctx, req.MCPServers, taskSamplingHandler(llm, req.Cfg), log)
mcpSet, mcpErr := startMCPProviders(ctx, req.MCPServers, taskSamplingHandler(llm, req.Cfg, samplingUsageRecorder(dependencies.store, req.Task, log)), log)
if mcpSet != nil {
defer mcpSet.cleanup(ctx, log)
}
+18
View File
@@ -5,6 +5,7 @@ import (
"context"
"errors"
"log/slog"
"net/http"
"strconv"
"time"
@@ -17,6 +18,7 @@ import (
"github.com/samcharles93/archie-core/internal/infrastructure/agentboot"
"github.com/samcharles93/archie-core/internal/infrastructure/agentgit"
agentnats "github.com/samcharles93/archie-core/internal/infrastructure/agenttransport/nats"
"github.com/samcharles93/archie-core/internal/infrastructure/egress"
"github.com/samcharles93/archie-core/internal/infrastructure/readiness"
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
"github.com/samcharles93/archie-core/internal/installtype"
@@ -48,6 +50,13 @@ type Settings struct {
// NATS storerpc path is deleted.
StateStoreTarget string
StateStoreToken string
// ModelProxy, ModelProxyHosts and ModelProxyCA route model provider
// requests through the daemon's egress proxy, which holds the keys.
// Empty ModelProxy sends them directly.
ModelProxy string
ModelProxyHosts []string
ModelProxyCA string
}
type workerTransport interface {
@@ -115,6 +124,15 @@ func run(ctx context.Context, settings Settings, log *slog.Logger, dependencies
workDir = storage.WorktreeMountDir
}
if settings.ModelProxy != "" {
transport, err := egress.ModelTransport(settings.ModelProxy, settings.ModelProxyHosts, settings.ModelProxyCA)
if err != nil {
return &StartupError{Operation: "model proxy", Err: err}
}
// The model SDK builds its clients on the default transport and takes
// no client of its own, so the process-wide default is the seam.
http.DefaultTransport = transport
}
dependencies.markSafe(ctx, workDir, log)
// The task is known before connecting: without a broker token the
+15 -6
View File
@@ -96,6 +96,7 @@ type boot struct {
// [services.state].target. It is set by openStateStoreAdapter, which
// requires [services.state].target to be set.
stateStore storecontract.TaskStore
taskModels daemon.TaskModels
// accessChain is the daemon's policy engine, built once from the stored
// policies (openAccessChain); accessProblems is what the readiness
// surface reports. Both are nil when no policy store is wired.
@@ -161,6 +162,9 @@ type boot struct {
// kitLauncher starts Kit profile tasks; nil when the egress path cannot
// be built on this host.
kitLauncher *kitrun.Launcher
// modelEgress gives native tasks model access through the egress proxy;
// nil when the proxy cannot start.
modelEgress daemon.ModelEgress
// sessionGrants records live setup sessions' granted credential services
// for the egress proxy's resolver.
sessionGrants *sessionGrants
@@ -266,6 +270,7 @@ func (b *boot) openStateStoreAdapter(ctx context.Context) error {
return err
}
b.stateStore = client
b.taskModels = client
b.controlPlaneRPC = client.ControlPlane()
b.controlPlane = controlplane.NewRPCClient(b.controlPlaneRPC)
b.applyStatus = applystatus.New(b.processName, client, b.log)
@@ -459,7 +464,7 @@ func (b *boot) setupContainers(ctx context.Context) func() {
// setupGatewayClient dials the Gateway, which owns every model call. The daemon
// only enqueues the chat tasks scheduled workflows create.
func (b *boot) setupGatewayClient() error {
func (b *boot) setupGatewayClient(ctx context.Context) error {
b.setupChatTasks(b.cfg)
contract, cleanup, err := composeChatContract(b.cfg.Services, b.secrets)
if err != nil {
@@ -473,6 +478,7 @@ func (b *boot) setupGatewayClient() error {
}
b.addCleanup(closeMessaging)
b.messaging = messaging
go postOutcomes(ctx, b.stateStore, messaging.Deliver, b.log)
b.setupReadinessProbes()
return nil
}
@@ -522,7 +528,7 @@ func (b *boot) registerNATSRPC() error {
log.Error("nats connection unavailable for task RPC", "err", err)
return err
}
unsubscribe, err := registerTaskRPCServers(coreConn, b.forgeClient, b.trees, b.identityRunners, b.stateStoreGrants.Client, log)
unsubscribe, err := registerTaskRPCServers(coreConn, b.forgeClient, b.trees, b.identityRunners, b.stateStoreGrants.Client, installedAuthorities{b.stateStoreGrants.Client}, log)
if err != nil {
log.Error("task RPC server registration failed", "err", err)
return err
@@ -567,6 +573,7 @@ func (b *boot) buildDaemon() {
Bus: b.bus,
Forge: b.forgeClient,
Trees: b.trees,
PackageAuthorities: installedAuthorities{b.stateStoreGrants.Client},
Storage: b.storeBackend,
Log: log,
Tasks: b.natsClient,
@@ -574,6 +581,8 @@ func (b *boot) buildDaemon() {
RunCredentials: b.stateStoreGrants,
ContainerPool: b.containerPool,
KitLauncher: b.kitLauncher,
ModelEgress: b.modelEgress,
TaskModels: b.taskModels,
Identities: b.identityRunners,
RootIdentityID: identity.StableID(servicekit.IdentityNames(b.cfg)[0]),
TaskLogs: b.taskLogs,
@@ -675,7 +684,7 @@ func (b *boot) publishConfig(ctx context.Context, cfg config.Config) {
}
func (b *boot) configViewInput() webui.ConfigViewInput {
catalog, _ := b.catalogState()
catalog := b.catalogState()
in := webui.ConfigViewInput{
Config: b.cfgHolder.Get(),
Catalog: catalogView(catalog),
@@ -799,7 +808,7 @@ func (b *boot) addCleanup(fn func()) {
b.cleanups = append(b.cleanups, fn)
}
func (b *boot) catalogState() (modelcatalog.Snapshot, []string) { return b.catalog.State() }
func (b *boot) catalogState() modelcatalog.Snapshot { return b.catalog.State() }
// loadCatalog layers the model catalog under the file config. A catalog that
// cannot be read leaves the configured providers and models in effect.
@@ -811,7 +820,7 @@ func (b *boot) loadCatalog(ctx context.Context) {
}
models := modelcatalog.Apply(&b.cfg, snapshot)
b.cfgHolder.Set(b.cfg.Clone())
b.catalog.Set(snapshot, models)
b.catalog.Set(snapshot)
b.log.Info("model catalog loaded", "providers", len(snapshot.Providers), "models", len(models))
}
@@ -826,7 +835,7 @@ func (b *boot) refreshModelCatalog(ctx context.Context) error {
if err != nil {
return err
}
b.catalog.Set(snapshot, models)
b.catalog.Set(snapshot)
b.publishConfig(ctx, cfg)
b.log.Info("model catalog refreshed", "providers", len(snapshot.Providers), "models", len(models))
return nil
+3 -3
View File
@@ -52,7 +52,7 @@ func (b *boot) reloadConfig(ctx context.Context, doc *configuration.Document) er
// drop them from the running config even though the file is
// unchanged. Re-apply the same merge (idempotent: a catalog that
// failed to load merges to identity).
catalog, _ := b.catalogState()
catalog := b.catalogState()
modelcatalog.Apply(&doc.Config, catalog)
// Bounded: this runs on the signal loop, which handles nothing else
// while it waits, and a State Store that has stopped answering must
@@ -264,7 +264,7 @@ func (b *boot) startLiveSettings(ctx context.Context) error {
var runtimeResourceKinds = []string{
controlplane.TaskPolicyKind,
controlplane.ProviderSettingsKind,
controlplane.ModelRoleAssignmentsKind,
controlplane.ModelAliasesKind,
controlplane.RepositoryPoliciesKind,
controlplane.SchedulingPolicyKind,
controlplane.ToolSettingsKind,
@@ -370,7 +370,7 @@ func (b *boot) applyRuntimeResourceUpdate(ctx context.Context, kind string, upda
// this read returns can lack the catalog's model limits and discovered
// providers. The merge is idempotent, and reloadConfig re-applies it
// before layering for the same reason.
catalog, _ := b.catalogState()
catalog := b.catalogState()
modelcatalog.Apply(&base, catalog)
cfg, _, err := b.runtimeConfig(ctx, base)
if err != nil {
@@ -0,0 +1,21 @@
package archied
import (
"context"
"github.com/samcharles93/archie-core/internal/domain/storepkg"
)
// installedAuthorities answers a package's accepted authority from the State
// Store. A package the operator has not accepted has none.
type installedAuthorities struct {
packages storepkg.Installations
}
func (a installedAuthorities) AcceptedAuthority(ctx context.Context, name string) (storepkg.Authority, error) {
installed, err := a.packages.GetInstalled(ctx, name)
if err != nil || installed.AcceptedAuthority == nil {
return storepkg.Authority{}, err
}
return *installed.AcceptedAuthority, nil
}
+10 -6
View File
@@ -42,11 +42,6 @@ func (b *boot) setupKitLauncher(ctx context.Context) {
log.Warn("kit harness runs disabled: egress CA", "err", err)
return
}
fetcher, err := fetch.New(fetch.WithDockerCredentials())
if err != nil {
log.Warn("kit harness runs disabled: kit registry client", "err", err)
return
}
ln, err := (&net.ListenConfig{}).Listen(ctx, "tcp", net.JoinHostPort(pool.HostGateway(), "0"))
if err != nil {
log.Warn("kit harness runs disabled: egress proxy listener", "err", err)
@@ -63,7 +58,7 @@ func (b *boot) setupKitLauncher(ctx context.Context) {
// container logs in with, against the live grants and bindings; a setup
// session's token resolves through the session grants instead.
b.sessionGrants = &sessionGrants{}
resolver := runCredentialResolver{runs: b.stateStoreGrants.Client, sessions: b.sessionGrants, config: b.cfgHolder, secrets: b.secrets}
resolver := runCredentialResolver{runs: b.stateStoreGrants.Client, sessions: b.sessionGrants, models: b.taskModels, config: b.cfgHolder, secrets: b.secrets}
// Without a harness secret store, Register refuses a Kit with a
// required OAuth credential.
oauthStore, _ := b.stateStore.(egress.OAuthStore)
@@ -74,6 +69,15 @@ func (b *boot) setupKitLauncher(ctx context.Context) {
log.Error("egress proxy stopped", "err", err)
}
}()
// Native tasks need only the proxy for model access, so it serves them
// even when a Kit-only piece below fails.
b.modelEgress = modelEgress{proxy: proxy, addr: ln.Addr().String(), caFile: egress.CACertPath(caDir), config: b.cfgHolder}
fetcher, err := fetch.New(fetch.WithDockerCredentials())
if err != nil {
b.addCleanup(func() { _ = srv.Close() })
log.Warn("kit harness runs disabled: kit registry client", "err", err)
return
}
networks := egress.NewNetworks(pool.Client(), egress.RelaySpec{
Name: relayName, Image: b.cfg.Containers.Image,
Entrypoint: []string{"archie-agent"}, Cmd: cmd, Network: pool.NetworkName(),
+3 -3
View File
@@ -185,7 +185,7 @@ func Run() int { //nolint:cyclop,funlen // the composition root's setup sequence
return 1
}
if err := b.setupGatewayClient(); err != nil {
if err := b.setupGatewayClient(ctx); err != nil {
return 1
}
if ps, ok := b.stateStore.(storecontract.PresenceStore); ok {
@@ -356,7 +356,7 @@ func subscribeAgentEvents(nc *natsio.Conn, bus *events.Bus, log *slog.Logger) (u
// tasks, and one server pair per identity answers that identity's scoped
// subjects so a container-mode task owned by a non-root identity has its RPC
// calls served by its own forge client and worktree manager.
func registerTaskRPCServers(nc *natsio.Conn, forgeClient forge.Forge, trees *worktree.Manager, identities []*daemon.IdentityRunner, runs worktreerpc.Runs, log *slog.Logger) (unsubscribe func(), err error) {
func registerTaskRPCServers(nc *natsio.Conn, forgeClient forge.Forge, trees *worktree.Manager, identities []*daemon.IdentityRunner, runs worktreerpc.Runs, authorities forgerpc.Authorities, log *slog.Logger) (unsubscribe func(), err error) {
unsubs := make([]func(), 0, 2+2*len(identities))
unsubAll := func() {
for _, u := range unsubs {
@@ -365,7 +365,7 @@ func registerTaskRPCServers(nc *natsio.Conn, forgeClient forge.Forge, trees *wor
}
registerForge := func(fg forge.Forge, identity string) error {
srv := &forgerpc.Server{Forge: fg, Runs: runs, Log: log.With("rpc_identity", identity)}
srv := &forgerpc.Server{Forge: fg, Runs: runs, Authorities: authorities, Log: log.With("rpc_identity", identity)}
u, err := srv.RegisterFor(nc, identity)
if err != nil {
return fmt.Errorf("register forgerpc%s: %w", identitySuffix(identity), err)
+71
View File
@@ -0,0 +1,71 @@
package archied
import (
"fmt"
"maps"
"slices"
"github.com/docker/sandbox-kit-spec/v3/spec"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/daemon"
"github.com/samcharles93/archie-core/internal/infrastructure/egress"
"github.com/samcharles93/archie-core/internal/storage"
)
const modelProxyCAPath = "/opt/archie/egress-ca.pem"
// modelEgress opens native task containers' model sessions on the egress
// proxy. Each keyed provider's key is the org's credential binding named
// after the provider.
type modelEgress struct {
proxy *egress.Proxy
addr string
caFile string
config *config.Holder
}
func (m modelEgress) Open(token, org string, providers map[string]config.Provider) (daemon.ModelSession, error) {
var keyed []string
for _, name := range slices.Sorted(maps.Keys(providers)) {
if providers[name].APIKeyEnv != "" {
keyed = append(keyed, name)
}
}
if len(keyed) == 0 {
return daemon.ModelSession{}, nil
}
bound := m.config.Get().Containers.BoundCredentials(org, keyed, keyed)
var (
subs []egress.Substitution
hosts []string
env []string
)
for _, name := range keyed {
p := providers[name]
if _, ok := bound[name]; !ok {
return daemon.ModelSession{}, fmt.Errorf("provider %q has no credential binding named %q in org %s", name, name, org)
}
host, err := egress.ProviderHost(p.Class, p.BaseURL)
if err != nil {
return daemon.ModelSession{}, fmt.Errorf("provider %q: %w", name, err)
}
subs = append(subs, egress.Substitution{Service: name, Host: host})
hosts = append(hosts, host)
env = append(env, p.APIKeyEnv+"="+egress.SentinelFor(name))
}
session, err := m.proxy.Register(egress.SessionOptions{
Token: token, Org: org, Substitutions: subs,
Network: &spec.PhasedNetwork{Runtime: &spec.NetworkRules{Allow: hosts}},
})
if err != nil {
return daemon.ModelSession{}, err
}
session.EnterRuntime()
return daemon.ModelSession{
Env: append(env, egress.ModelProxyEnvFor(m.addr, token, hosts, modelProxyCAPath)...),
Mounts: []storage.Mount{{Type: storage.MountTypeBind, Source: m.caFile, Destination: modelProxyCAPath}},
}, nil
}
func (m modelEgress) Close(token string) { m.proxy.Revoke(token) }
+72
View File
@@ -0,0 +1,72 @@
package archied
import (
"context"
"fmt"
"log/slog"
"strings"
"time"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
)
const outcomePostInterval = 15 * time.Second
type outcomeDeliverer func(ctx context.Context, channel, chatID, text string) error
// postOutcomes reports each finished chat task to the conversation that
// spawned it. A task stays unposted until delivery succeeds, so a Messaging
// Service that is down or restarting only delays the report.
func postOutcomes(ctx context.Context, tasks storecontract.OutcomePoster, deliver outcomeDeliverer, log *slog.Logger) {
ticker := time.NewTicker(outcomePostInterval)
defer ticker.Stop()
for {
postPendingOutcomes(ctx, tasks, deliver, log)
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}
func postPendingOutcomes(ctx context.Context, tasks storecontract.OutcomePoster, deliver outcomeDeliverer, log *slog.Logger) {
pending, err := tasks.UnpostedOutcomes(ctx)
if err != nil {
log.Warn("list unposted task outcomes", "err", err)
return
}
for _, t := range pending {
// An origin that names no chat can never be delivered; marking it
// posted keeps it from being retried forever.
if channel, chatID, ok := splitOrigin(t.Origin); ok {
if err := deliver(ctx, channel, chatID, outcomeText(t)); err != nil {
log.Warn("post task outcome", "task", t.ID, "err", err)
continue
}
}
if err := tasks.MarkOutcomePosted(ctx, t.ID); err != nil {
log.Warn("mark task outcome posted", "task", t.ID, "err", err)
}
}
}
// splitOrigin reads "<channel>:<chat>[/<thread>]", the form messaging.Origin
// writes. Delivery addresses the chat; threads are not addressable.
func splitOrigin(origin string) (channel, chatID string, ok bool) {
channel, conversation, found := strings.Cut(origin, ":")
chatID, _, _ = strings.Cut(conversation, "/")
return channel, chatID, found && channel != "" && chatID != ""
}
func outcomeText(t *task.Task) string {
text := fmt.Sprintf("Task %d (%s) finished: %s", t.ID, t.Title, t.Status)
if t.PRNumber > 0 {
text += fmt.Sprintf(", PR #%d", t.PRNumber)
}
if t.ParkReason != "" {
text += "\n" + t.ParkReason
}
return text
}
@@ -2,8 +2,10 @@ package archied
import (
"context"
"slices"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/daemon"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/infrastructure/egress"
)
@@ -24,6 +26,9 @@ type runCredentialResolver struct {
sessions interface {
GrantedFor(token string) (org string, services []string, ok bool)
}
// models answers a task's org's own providers. Nil grants only the
// instance providers.
models daemon.TaskModels
config *config.Holder
secrets interface {
Resolve(config.SecretRef) (string, error)
@@ -41,7 +46,13 @@ func (r runCredentialResolver) Resolve(ctx context.Context, credential, service
return "", egress.ErrUnbound
}
cfg := r.config.Get()
org, granted := cfg.CredentialAccess(task.Identity)
_, granted := cfg.CredentialAccess(task.Identity)
org := task.OrgOrDefault()
// A configured model provider's key is granted to every run of the org:
// any agent stage needs model access.
if r.isModelProvider(ctx, cfg, task, service) {
granted = append(slices.Clone(granted), service)
}
return r.resolveBinding(org, granted, service)
}
@@ -62,3 +73,20 @@ func (r runCredentialResolver) resolveBinding(org string, granted []string, serv
}
return r.secrets.Resolve(binding.Secret)
}
// isModelProvider reports whether service is a provider the task's models
// run on: an instance provider, or one of its org's own.
func (r runCredentialResolver) isModelProvider(ctx context.Context, cfg config.Config, task *workflow.Task, service string) bool {
if _, ok := cfg.Providers[service]; ok {
return true
}
if r.models == nil {
return false
}
models, err := r.models.TaskModels(ctx, task.ID)
if err != nil {
return false
}
_, ok := models.Providers[service]
return ok
}
@@ -29,17 +29,21 @@ func TestEgressResolvesOnlyGrantedCredentials(t *testing.T) {
own := []string{"anthropic"}
cfg := config.Config{
GrantedCredentials: []string{"github"},
Providers: map[string]config.Provider{"openai": {Class: "openai"}, "deepseek": {Class: "deepseek"}},
Identities: []config.IdentityConfig{{Name: "acme-bot", Org: "acme", GrantedCredentials: &own}},
Containers: config.ContainerConfig{Credentials: []config.CredentialBinding{
{Service: "github", Secret: config.SecretRef{Engine: "env", Key: "GH"}},
{Service: "anthropic", Secret: config.SecretRef{Engine: "env", Key: "ROOT_AI"}},
{Service: "anthropic", Org: "acme", Secret: config.SecretRef{Engine: "env", Key: "ACME_AI"}},
{Service: "openai", Secret: config.SecretRef{Engine: "env", Key: "ROOT_OPENAI"}},
{Service: "openai", Org: "acme", Secret: config.SecretRef{Engine: "env", Key: "ACME_OPENAI"}},
{Service: "deepseek", Secret: config.SecretRef{Engine: "env", Key: "ROOT_DEEPSEEK"}},
}},
}
r := runCredentialResolver{
runs: runsByToken{"root-run": {ID: 1}, "acme-run": {ID: 2, Identity: "acme-bot"}},
runs: runsByToken{"root-run": {ID: 1}, "acme-run": {ID: 2, Identity: "acme-bot", Org: "acme"}, "acme-row-run": {ID: 3, Org: "acme"}},
config: config.NewHolder(cfg),
secrets: secretValues{"GH": "gh-secret", "ROOT_AI": "root-ai", "ACME_AI": "acme-ai"},
secrets: secretValues{"GH": "gh-secret", "ROOT_AI": "root-ai", "ACME_AI": "acme-ai", "ROOT_OPENAI": "root-openai", "ACME_OPENAI": "acme-openai", "ROOT_DEEPSEEK": "root-deepseek"},
}
tests := []struct {
name, credential, service, want string
@@ -49,6 +53,10 @@ func TestEgressResolvesOnlyGrantedCredentials(t *testing.T) {
{"identity run gets its own org's binding", "acme-run", "anthropic", "acme-ai"},
{"identity grants replace the root's", "acme-run", "github", ""},
{"unknown credential", "gone", "github", ""},
{"a model provider key needs no grant", "root-run", "openai", "root-openai"},
{"a model provider key is the run's own org's", "acme-run", "openai", "acme-openai"},
{"another org's model provider key is unbound", "acme-run", "deepseek", ""},
{"the task's own org decides, not the identity mapping", "acme-row-run", "openai", "acme-openai"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
@@ -79,7 +87,7 @@ func TestEgressResolvesASessionGrant(t *testing.T) {
sessions.Grant("acme-session", "acme", []string{"github", "linear"})
sessions.Grant("other-session", "other", []string{"github"})
r := runCredentialResolver{
runs: runsByToken{"acme-run": {ID: 1, Identity: "acme-bot"}},
runs: runsByToken{"acme-run": {ID: 1, Identity: "acme-bot", Org: "acme"}},
sessions: sessions,
config: config.NewHolder(cfg),
secrets: secretValues{"ACME_GH": "acme-gh"},
+63 -143
View File
@@ -4,116 +4,51 @@ import (
"context"
"fmt"
"slices"
"strings"
"sync"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/org"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/modelcatalog"
)
// chatModelManager owns the process-local model selected for interactive chat.
// The configured role-to-model map supplies the available catalog; switching
// chat models does not rewrite daemon configuration or affect workflow roles.
// chatModelManager owns the process-local model alias selected for
// interactive chat. Switching it does not rewrite configuration or affect
// workflow steps.
type chatModelManager struct {
mu sync.RWMutex
models []string
active string
// configured, explicit and catalog are the sources the offered set is
// derived from: the role assignments (live via SetConfigured), the
// [chat].models catalog (fixed for the process), and the model catalog's
// own refs (replaced wholesale by SetModelCatalog).
configured map[string]string
explicit []string
catalog []string
mu sync.RWMutex
aliases map[string]string
active string
details map[string]gateway.ModelDetails
providerNames map[string]string
// orgAliases answers the aliases the principal ctx carries may use. Nil
// serves every caller the instance aliases.
orgAliases func(ctx context.Context) (map[string]string, error)
}
// pickChatDefault is the model the manager starts on: the chat role's
// assignment, falling back to the builder role's.
func pickChatDefault(configured map[string]string) string {
if active := strings.TrimSpace(configured["chat"]); active != "" {
return active
}
return strings.TrimSpace(configured["builder"])
}
// mergeModelRefs is the model set the manager offers: the configured role
// assignments plus every catalog, unique and sorted.
func mergeModelRefs(configured map[string]string, catalogs ...[]string) []string {
unique := make(map[string]struct{}, len(configured))
for _, ref := range configured {
ref = strings.TrimSpace(ref)
if ref != "" {
unique[ref] = struct{}{}
}
}
for _, catalog := range catalogs {
for _, ref := range catalog {
ref = strings.TrimSpace(ref)
if ref != "" {
unique[ref] = struct{}{}
}
}
}
models := make([]string, 0, len(unique))
for ref := range unique {
models = append(models, ref)
}
slices.Sort(models)
return models
}
func newChatModelManager(configured map[string]string, catalogs ...[]string) *chatModelManager {
explicit := make([]string, 0, len(catalogs))
for _, catalog := range catalogs {
explicit = append(explicit, catalog...)
}
m := &chatModelManager{
configured: configured,
explicit: explicit,
details: make(map[string]gateway.ModelDetails),
providerNames: make(map[string]string),
}
m.active = pickChatDefault(configured)
m.setModelsLocked()
func newChatModelManager(aliases map[string]string) *chatModelManager {
m := &chatModelManager{details: make(map[string]gateway.ModelDetails), providerNames: make(map[string]string)}
m.SetConfigured(aliases)
return m
}
// setModelsLocked re-derives the offered set from the three sources and keeps
// the active selection when the new set still carries it, falling back the way
// construction does when it does not.
func (m *chatModelManager) setModelsLocked() {
m.models = mergeModelRefs(m.configured, m.explicit, m.catalog)
if !slices.Contains(m.models, m.active) {
m.active = pickChatDefault(m.configured)
// SetConfigured replaces the alias table after a live change to model-aliases.
// The active alias survives when it still exists; otherwise chat returns to
// the default alias.
func (m *chatModelManager) SetConfigured(aliases map[string]string) {
m.mu.Lock()
defer m.mu.Unlock()
m.aliases = aliases
if !slices.Contains(config.ChatAliases(aliases), m.active) {
m.active = config.PurposeChat.DefaultAlias()
}
}
// SetConfigured re-derives the model set the manager offers after a live
// change to model-role-assignments or provider-settings: the list is built
// from the configured roles at construction, so a stored change would leave
// the chat surfaces offering refs the runtime can no longer resolve. The
// operator's active selection survives when the new roles still carry it; a
// removed one falls back the way construction does.
func (m *chatModelManager) SetConfigured(configured map[string]string) {
// SetModelCatalog replaces the provider display names and per-model details
// the catalog publishes.
func (m *chatModelManager) SetModelCatalog(snapshot modelcatalog.Snapshot) {
m.mu.Lock()
defer m.mu.Unlock()
m.configured = configured
m.setModelsLocked()
}
// SetModelCatalog replaces the model catalog's contribution to the manager:
// the references it offers, the provider display names and the per-model
// details. The previous snapshot's entries are dropped rather than merged --
// a model or provider the catalog no longer publishes must stop being offered
// and described, and these maps are the manager's own state rather than a
// second catalogue to fall back on.
func (m *chatModelManager) SetModelCatalog(snapshot modelcatalog.Snapshot, models []string) {
m.mu.Lock()
defer m.mu.Unlock()
m.catalog = slices.Clone(models)
m.details = make(map[string]gateway.ModelDetails)
m.providerNames = make(map[string]string)
for _, provider := range snapshot.Providers {
@@ -129,7 +64,6 @@ func (m *chatModelManager) SetModelCatalog(snapshot modelcatalog.Snapshot, model
}
}
}
m.setModelsLocked()
}
func (m *chatModelManager) ProviderDisplayName(provider string) string {
@@ -145,80 +79,66 @@ func (m *chatModelManager) ModelDetails(ref string) (gateway.ModelDetails, bool)
return details, ok
}
// Models returns the aliases chat may switch between.
func (m *chatModelManager) Models() []string {
m.mu.RLock()
defer m.mu.RUnlock()
return slices.Clone(m.models)
return config.ChatAliases(m.aliases)
}
func (m *chatModelManager) ActiveModel() string {
func (m *chatModelManager) ActiveAlias() string {
m.mu.RLock()
defer m.mu.RUnlock()
return m.active
}
func (m *chatModelManager) SetActiveModel(ctx context.Context, ref string) error {
// ActiveModel returns the active alias's model ref, or "" when it is unset.
func (m *chatModelManager) ActiveModel() string {
m.mu.RLock()
defer m.mu.RUnlock()
ref, _ := config.ResolveModel(m.aliases, m.active, config.PurposeChat)
return ref
}
func (m *chatModelManager) SetActiveModel(ctx context.Context, alias string) error {
if err := ctx.Err(); err != nil {
return err
}
m.mu.Lock()
defer m.mu.Unlock()
if !slices.Contains(m.models, ref) {
return fmt.Errorf("model %q is not configured", ref)
if !slices.Contains(config.ChatAliases(m.aliases), alias) {
return fmt.Errorf("model alias %q is not configured", alias)
}
m.active = ref
m.active = alias
return nil
}
func (m *chatModelManager) Providers() []string {
m.mu.RLock()
defer m.mu.RUnlock()
seen := make(map[string]bool)
var providers []string
for _, model := range m.models {
provider, _, ok := strings.Cut(model, "/")
if ok && provider != "" && !seen[provider] {
seen[provider] = true
providers = append(providers, provider)
}
// aliasesFor is the alias table ctx's org resolves against: the instance's
// for the system org, the org's own answer otherwise.
func (m *chatModelManager) aliasesFor(ctx context.Context) (map[string]string, error) {
if m.orgAliases == nil || org.OrgFromContext(ctx) == org.DefaultOrgID {
m.mu.RLock()
defer m.mu.RUnlock()
return m.aliases, nil
}
slices.Sort(providers)
return providers
return m.orgAliases(ctx)
}
func (m *chatModelManager) ActiveProvider() string {
m.mu.RLock()
defer m.mu.RUnlock()
provider, _, _ := strings.Cut(m.active, "/")
return provider
// ModelsFor returns the aliases ctx's org may switch between.
func (m *chatModelManager) ModelsFor(ctx context.Context) ([]string, error) {
aliases, err := m.aliasesFor(ctx)
if err != nil {
return nil, err
}
return config.ChatAliases(aliases), nil
}
func (m *chatModelManager) ModelsForProvider(provider string) []string {
m.mu.RLock()
defer m.mu.RUnlock()
prefix := provider + "/"
var models []string
for _, model := range m.models {
if strings.HasPrefix(model, prefix) {
models = append(models, model)
}
// ModelFor resolves the active alias in ctx's org. An org that cannot use it
// gets an error, never another model.
func (m *chatModelManager) ModelFor(ctx context.Context) (string, error) {
aliases, err := m.aliasesFor(ctx)
if err != nil {
return "", err
}
return models
}
func (m *chatModelManager) SetActiveProvider(ctx context.Context, provider string) error {
if err := ctx.Err(); err != nil {
return err
}
m.mu.Lock()
defer m.mu.Unlock()
prefix := provider + "/"
for _, model := range m.models {
if strings.HasPrefix(model, prefix) {
m.active = model
return nil
}
}
return fmt.Errorf("provider %q is not configured", provider)
return config.ResolveModel(aliases, m.ActiveAlias(), config.PurposeChat)
}
+7 -7
View File
@@ -57,9 +57,9 @@ func (b *server) setupTranscriber(cfg config.Config, log *slog.Logger) {
})
if ok {
b.transcriber = client
log.Info("voice transcription enabled", "role", transcription.Role)
} else if cfg.Models[transcription.Role] != "" {
log.Warn("voice transcription configured but unavailable; capability disabled", "role", transcription.Role)
log.Info("voice transcription enabled")
} else if _, err := config.ResolveModel(cfg.Models, "", config.PurposeTranscription); err == nil {
log.Warn("voice transcription configured but unavailable; capability disabled")
}
}
@@ -84,18 +84,18 @@ func (b *server) startRateLimiter(ctx context.Context, cfg config.RateLimitConfi
}
// rebuildChatModelRuntime re-derives the gateway chat runtime's provider set
// after a live change to provider-settings or model-role-assignments. The
// after a live change to provider-settings or model-aliases. The
// turn runner reads the runtime through server.chatLLM, so the swap reaches its
// next turn without rebuilding the runner, and the model manager re-derives
// the references it offers from the new role assignments and the catalog.
// the aliases it offers.
func (b *server) rebuildChatModelRuntime(cfg config.Config) {
if b.chatModels == nil {
return // this process serves no chat turns
}
b.setLLM(modelloop.NewRuntime(executionProviders(cfg), cfg.ModelLimits))
b.chatModels.SetConfigured(cfg.Models)
catalog, models := b.catalogState()
b.chatModels.SetModelCatalog(catalog, models)
catalog := b.catalogState()
b.chatModels.SetModelCatalog(catalog)
b.log.Info("chat model runtime rebuilt", "providers", len(cfg.Providers), "models", len(cfg.Models))
}
+26 -15
View File
@@ -15,6 +15,7 @@ import (
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/domain/usage"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
@@ -36,11 +37,13 @@ type chatSetup struct {
// than the value captured when the runner was built.
ToolLimits func() modelloop.ToolLimits
// LLM resolves the provider runtime at each use: a live provider-settings
// or model-role-assignments update swaps the runtime wholesale (ai-sdk's
// or model-aliases update swaps the runtime wholesale (ai-sdk's
// Runtime caches the provider instances it built), so the pointer cannot
// be captured at construction. nil means no model runtime is configured.
LLM func() *runtime.Runtime
ChatModels gateway.ModelManager
LLM func() *runtime.Runtime
ChatModels gateway.ModelManager
// Usage records each model call; nil records nothing.
Usage *usageSink
ToolReg *tools.Registry
Soul gateway.SoulSource
ChatTasks gateway.TaskCreator
@@ -129,7 +132,7 @@ func newChatTurnRunner(
Sessions: sessionStore,
Models: s.ChatModels,
Soul: s.Soul,
Model: newChatTurnModel(s.LLM, s.ToolReg, cfg.Chat.MaxSteps, s.ToolLimits, s.ProviderOutcomes),
Model: newChatTurnModel(s.LLM, s.ToolReg, cfg.Chat.MaxSteps, s.ToolLimits, s.ProviderOutcomes, s.Usage),
TaskLister: s.ChatTaskLister,
Tasks: s.ChatTasks,
TaskLogs: s.ChatTaskLogs,
@@ -191,31 +194,38 @@ func chatRepoEnv(cfg config.Config, identity string) []gateway.RepoEnv {
// and records at its own adapter (curatorLLMRunner). Anything that reaches the
// runtime without recording leaves /status reporting a stale last-known
// outcome, which is a /status that lies about the provider.
func sendChatTurn(ctx context.Context, llm *runtime.Runtime, chatModel string, options core.GenerateOptions, turn gateway.TurnStream, outcomes *providerOutcomeRecorder, icons map[string]string) (string, error) {
text, err := runChatTurn(ctx, llm, chatModel, options, turn, icons)
func sendChatTurn(ctx context.Context, llm *runtime.Runtime, chatModel string, options core.GenerateOptions, turn gateway.TurnStream, outcomes *providerOutcomeRecorder, usageRecords *usageSink, icons map[string]string) (string, error) {
text, used, err := runChatTurn(ctx, llm, chatModel, options, turn, icons)
usageRecords.record(ctx, usage.SourceChat, chatModel, used)
// record tolerates a nil recorder: a setup built without one (tests, a
// deployment whose health surface is unwired) still makes its calls.
outcomes.record(chatModel, err)
return text, err
}
func runChatTurn(ctx context.Context, llm *runtime.Runtime, chatModel string, options core.GenerateOptions, turn gateway.TurnStream, icons map[string]string) (string, error) {
// runChatTurn returns the reply and the tokens the call used, which a
// failed call may still report.
func runChatTurn(ctx context.Context, llm *runtime.Runtime, chatModel string, options core.GenerateOptions, turn gateway.TurnStream, icons map[string]string) (string, chat.Usage, error) {
if turn == nil {
result, err := llm.Chat(ctx, chatModel, options)
if err != nil {
return "", fmt.Errorf("llm chat: %w", err)
return "", result.TotalUsage, fmt.Errorf("llm chat: %w", err)
}
return result.Text, nil
return result.Text, result.TotalUsage, nil
}
stream, err := llm.ChatStream(ctx, chatModel, options)
if err != nil {
return "", fmt.Errorf("llm chat stream: %w", err)
return "", chat.Usage{}, fmt.Errorf("llm chat stream: %w", err)
}
text := drainChatStream(stream.FullStream, turn, icons)
if _, err := stream.FinishReason(); err != nil {
return "", fmt.Errorf("llm chat stream: %w", err)
var used chat.Usage
if stream.Usage != nil {
used, _ = stream.Usage()
}
return text, nil
if _, err := stream.FinishReason(); err != nil {
return "", used, fmt.Errorf("llm chat stream: %w", err)
}
return text, used, nil
}
// drainChatStream consumes a model stream to close, reporting assistant text
@@ -324,6 +334,7 @@ type chatTitleGenerator struct {
log *slog.Logger
llm func() *runtime.Runtime
chatModels gateway.ModelManager
usage *usageSink
}
// newChatTitleGenerator wires an LLM-backed title generator for a chat
@@ -333,7 +344,7 @@ func newChatTitleGenerator(s chatSetup) gateway.TitleGenerator {
if s.LLM == nil || s.ChatModels == nil {
return nil
}
return &chatTitleGenerator{log: s.Log, llm: s.LLM, chatModels: s.ChatModels}
return &chatTitleGenerator{log: s.Log, llm: s.LLM, chatModels: s.ChatModels, usage: s.Usage}
}
func (g *chatTitleGenerator) GenerateTitle(ctx context.Context, sessionID, firstMessage string) (string, error) {
@@ -357,7 +368,7 @@ func (g *chatTitleGenerator) GenerateTitle(ctx context.Context, sessionID, first
return "", fmt.Errorf("no model runtime is configured")
}
text, err := sendChatTurn(ctx, llm, g.chatModels.ActiveModel(),
core.GenerateOptions{Messages: messages, MaxSteps: 1}, nil, nil, nil)
core.GenerateOptions{Messages: messages, MaxSteps: 1}, nil, nil, g.usage, nil)
if err != nil {
if g.log != nil {
g.log.Error("session title generation failed", "session", sessionID, "err", err)
@@ -29,6 +29,7 @@ type chatTurnModel struct {
limits func() modelloop.ToolLimits
// outcomes records each call's result for /status (see sendChatTurn).
outcomes *providerOutcomeRecorder
usage *usageSink
}
func newChatTurnModel(
@@ -37,6 +38,7 @@ func newChatTurnModel(
maxSteps int,
limits func() modelloop.ToolLimits,
outcomes *providerOutcomeRecorder,
usageRecords *usageSink,
) gateway.TurnModel {
return &chatTurnModel{
llm: llm,
@@ -44,6 +46,7 @@ func newChatTurnModel(
maxSteps: maxSteps,
limits: limits,
outcomes: outcomes,
usage: usageRecords,
}
}
@@ -70,6 +73,7 @@ func (m *chatTurnModel) Prepare(
toolInfo: toolSummaries(options.Tools),
toolTokens: gateway.EstimateTokens(string(toolSchema)),
outcomes: m.outcomes,
usage: m.usage,
icons: toolIcons(m.registry),
}, nil
}
@@ -81,6 +85,7 @@ type preparedChatTurnModel struct {
toolInfo []gateway.ToolSummary
toolTokens int
outcomes *providerOutcomeRecorder
usage *usageSink
// icons maps tool name to the icon that tool registered, for chat
// surfaces to render. A nil map yields "" for every lookup, which is
// the no-icon rendering.
@@ -202,5 +207,5 @@ func (m *preparedChatTurnModel) Generate(
if m.llm == nil {
return "", fmt.Errorf("llm chat: no model runtime is configured")
}
return sendChatTurn(ctx, m.llm, m.model, options, stream, m.outcomes, m.icons)
return sendChatTurn(ctx, m.llm, m.model, options, stream, m.outcomes, m.usage, m.icons)
}
+4 -1
View File
@@ -14,6 +14,7 @@ import (
"github.com/samcharles93/archie-core/internal/app/controlplane"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/curator"
"github.com/samcharles93/archie-core/internal/domain/usage"
"github.com/samcharles93/archie-core/internal/events"
infraMemory "github.com/samcharles93/archie-core/internal/infrastructure/memory"
"github.com/samcharles93/archie-core/internal/infrastructure/sessioncurator"
@@ -34,7 +35,7 @@ func (b *server) setupCurators(ctx context.Context) {
MemoryEngines: b.memEngines,
Skills: skillcurator.NewStore(skillsRoot),
Conversations: sessioncurator.NewAdapter(b.chatSessionStore, b.cfg.BotUser),
LLM: curatorLLMRunner{llm: b.chatLLM, outcomes: b.providerOutcomes},
LLM: curatorLLMRunner{llm: b.chatLLM, outcomes: b.providerOutcomes, usage: b.usage},
Model: b.chatModels.ActiveModel(),
})
@@ -107,6 +108,7 @@ type curatorLLMRunner struct {
// provider is demonstrably reachable -- or worse, keeps showing a much
// older chat outcome as current.
outcomes *providerOutcomeRecorder
usage *usageSink
}
func (r curatorLLMRunner) Chat(ctx context.Context, req curator.ChatRequest) (curator.ChatResult, error) {
@@ -136,6 +138,7 @@ func (r curatorLLMRunner) Chat(ctx context.Context, req curator.ChatRequest) (cu
MaxSteps: max(req.MaxSteps, 1),
})
r.outcomes.record(req.Model, err)
r.usage.record(ctx, usage.SourceCurator, req.Model, res.TotalUsage)
if err != nil {
return curator.ChatResult{}, err
}
+13 -4
View File
@@ -30,15 +30,15 @@ func (b *server) setupChatRuntime(ctx context.Context, cfg config.Config, actor
// non-command message processing.
providers := executionProviders(cfg)
b.setLLM(modelloop.NewRuntime(providers, cfg.ModelLimits))
// Transcription is a model-role capability, like the chat models it sits
// Transcription is a model-alias capability, like the chat models it sits
// beside: it is built here, on the model-owning side, from this process's
// own [models]/[providers]. The Messaging Service carries a voice note's
// bytes to the Gateway and never holds the credential.
b.setupTranscriber(cfg, b.log)
b.toolReg = tools.NewRegistry()
chatModels := newChatModelManager(cfg.Models, cfg.Chat.Models)
catalog, catalogModels := b.catalogState()
chatModels.SetModelCatalog(catalog, catalogModels)
chatModels := newChatModelManager(cfg.Models)
catalog := b.catalogState()
chatModels.SetModelCatalog(catalog)
b.chatModels = chatModels
// ── Persona registry ─────────────────────────────────────────────
@@ -74,6 +74,14 @@ func (b *server) setupChatRuntime(ctx context.Context, cfg config.Config, actor
// later carries both: the recorder is written by sendChatTurn, the source
// is read by /status.
b.providerOutcomes = newProviderOutcomeRecorder()
b.usage = &usageSink{models: b.chatModels, log: b.log}
if b.stateStore != nil {
b.usage.store = b.stateStore
b.chatModels.orgAliases = func(ctx context.Context) (map[string]string, error) {
_, aliases, err := b.stateStore.CallerModels(ctx)
return aliases, err
}
}
b.statusHealth = newStatusHealth(b)
b.setupChatTasks(cfg)
@@ -253,6 +261,7 @@ func (b *server) setupGatewayChat(ctx context.Context, actor gateway.ChatTaskAct
// The Gateway executes the web chat's turns, so it is the process
// that must record their outcomes for /status.
ProviderOutcomes: b.providerOutcomes,
Usage: b.usage,
}
router.LLM, router.LLMStream = makeChatLLMResponder(ctx, "web", setup, b.chatSessionStore, router)
router.Titles = newChatTitleGenerator(setup)
+2 -5
View File
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"maps"
"reflect"
"slices"
"strings"
@@ -135,11 +136,7 @@ func (b *server) disconnectAbsentMCPServers(ctx context.Context, desired map[str
// applyDesiredMCPServers connects, reconnects or restarts the servers the
// stored document names, skipping the ones already running unchanged.
func (b *server) applyDesiredMCPServers(ctx context.Context, desired map[string]config.MCPServer, state mcpRegistryState) []error {
names := make([]string, 0, len(desired))
for name := range desired {
names = append(names, name)
}
slices.Sort(names)
names := slices.Sorted(maps.Keys(desired))
var problems []error
for _, name := range names {
@@ -8,6 +8,7 @@ import (
"github.com/samcharles93/ai-sdk/core"
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
"github.com/samcharles93/archie-core/internal/domain/usage"
"github.com/samcharles93/archie-core/internal/tools/mcp"
)
@@ -48,6 +49,7 @@ func (b *server) mcpSamplingHandler() mcp.SamplingHandler {
// recorded for /status like every other one; a nil recorder records
// nothing.
b.providerOutcomes.record(model, err)
b.usage.record(ctx, usage.SourceChat, model, result.TotalUsage)
if err != nil {
return mcp.SamplingResult{}, fmt.Errorf("mcp sampling: %w", err)
}
+9 -8
View File
@@ -92,6 +92,7 @@ type server struct {
curatorRegistry *curator.Registry
curatorRuntime *curator.Runtime
providerOutcomes *providerOutcomeRecorder
usage *usageSink
statusHealth gateway.HealthSource
updateService *releaseupdate.Service
@@ -260,7 +261,7 @@ func (b *server) waitForNATSEndpoint(ctx context.Context) (servicekit.NATSEndpoi
}
}
func (b *server) catalogState() (modelcatalog.Snapshot, []string) { return b.catalog.State() }
func (b *server) catalogState() modelcatalog.Snapshot { return b.catalog.State() }
// loadCatalog layers the model catalog under the running config. A catalog
// that cannot be read leaves the configured providers and models in effect.
@@ -270,9 +271,9 @@ func (b *server) loadCatalog(ctx context.Context) {
b.log.Warn("model catalog unavailable; using configured providers and models", "err", err)
return
}
models := modelcatalog.Apply(&b.cfg, snapshot)
modelcatalog.Apply(&b.cfg, snapshot)
b.cfgHolder.Set(b.cfg.Clone())
b.catalog.Set(snapshot, models)
b.catalog.Set(snapshot)
}
func (b *server) startModelCatalogRefresh(ctx context.Context) {
@@ -283,8 +284,8 @@ func (b *server) startModelCatalogRefresh(ctx context.Context) {
return
}
base := b.cfgHolder.Get().Clone()
models := modelcatalog.Apply(&base, snapshot)
b.catalog.Set(snapshot, models)
modelcatalog.Apply(&base, snapshot)
b.catalog.Set(snapshot)
if err := b.relayer(ctx, base, ""); err != nil {
b.log.Warn("model catalog refresh rejected with the settings; the running settings stay", "err", err)
}
@@ -295,7 +296,7 @@ func (b *server) startModelCatalogRefresh(ctx context.Context) {
// chat runtime and tool providers are built from.
var gatewayRuntimeKinds = []string{
controlplane.ProviderSettingsKind,
controlplane.ModelRoleAssignmentsKind,
controlplane.ModelAliasesKind,
controlplane.ToolSettingsKind,
}
@@ -304,7 +305,7 @@ func (b *server) startRuntimeWatches(ctx context.Context) error {
// the providers resolve again.
b.engines.OnChange(func() {
base := b.cfgHolder.Get()
catalog, _ := b.catalogState()
catalog := b.catalogState()
modelcatalog.Apply(&base, catalog)
_ = b.relayer(ctx, base, controlplane.ProviderSettingsKind)
})
@@ -342,7 +343,7 @@ func (b *server) startRuntimeWatches(ctx context.Context) error {
return
}
base := b.cfgHolder.Get()
catalog, _ := b.catalogState()
catalog := b.catalogState()
modelcatalog.Apply(&base, catalog)
lastApplyErr = b.relayer(ctx, base, kind)
},
+44
View File
@@ -0,0 +1,44 @@
package archiegateway
import (
"context"
"log/slog"
"time"
"github.com/samcharles93/ai-sdk/chat"
"github.com/samcharles93/archie-core/internal/domain/usage"
"github.com/samcharles93/archie-core/internal/gateway"
)
const usageWriteTimeout = 5 * time.Second
// usageSink records the gateway's model calls. The State Store bills each to
// the org of the principal the call was made for. A nil sink records nothing.
type usageSink struct {
store usage.Recorder
models gateway.ModelManager
log *slog.Logger
}
// record appends one call's usage. It never fails the call: the write is
// detached from the caller's cancellation and a failure is only logged.
func (u *usageSink) record(ctx context.Context, source usage.Source, ref string, used chat.Usage) {
if u == nil || u.store == nil || ref == "" {
return
}
alias := ""
if u.models != nil && u.models.ActiveModel() == ref {
alias = u.models.ActiveAlias()
}
record := usage.Record{
Source: source, Alias: alias,
InputTokens: int64(used.PromptTokens), OutputTokens: int64(used.CompletionTokens),
CachedTokens: int64(used.CachedTokens), At: time.Now().UTC(),
}.WithRef(ref)
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), usageWriteTimeout)
defer cancel()
if err := u.store.RecordUsage(ctx, record); err != nil && u.log != nil {
u.log.Warn("record model usage failed", "source", source, "model", ref, "err", err)
}
}
+5 -26
View File
@@ -36,32 +36,21 @@ func Run(args []string, stderr io.Writer) int {
return cmd(args[1:], stderr)
}
// runLint validates routing binding directories (-dir) and an EDA playbook
// directory (-eda-dir) against the loaders the daemon runs at startup.
// Exit codes: 0 clean, 1 findings, 2 usage error.
// runLint validates an EDA playbook directory against the compiler the daemon
// runs. Exit codes: 0 clean, 1 findings, 2 usage error.
func runLint(args []string, stderr io.Writer) int {
flags := flag.NewFlagSet("archied playbooks lint", flag.ContinueOnError)
flags.SetOutput(stderr)
var dirs multiFlag
flags.Var(&dirs, "dir", "routing binding directory to lint (repeatable)")
edaDir := flags.String("eda-dir", "", "EDA playbook directory to lint")
if err := flags.Parse(args); err != nil {
return 2 // flag.ContinueOnError already printed the message
}
if len(dirs) == 0 && *edaDir == "" {
fmt.Fprintln(stderr, "lint: at least one -dir or an -eda-dir is required")
if *edaDir == "" {
fmt.Fprintln(stderr, "lint: -eda-dir is required")
flags.Usage()
return 2
}
code := 0
if len(dirs) > 0 {
code = max(code, Lint(dirs, stderr).ExitCode)
}
if *edaDir != "" {
code = max(code, LintEDA(*edaDir, stderr).ExitCode)
}
return code
return LintEDA(*edaDir, stderr).ExitCode
}
// runServe runs the language server on stdin/stdout until the editor
@@ -87,13 +76,3 @@ type stdio struct{}
func (stdio) Read(p []byte) (int, error) { return os.Stdin.Read(p) }
func (stdio) Write(p []byte) (int, error) { return os.Stdout.Write(p) }
func (stdio) Close() error { return os.Stdin.Close() }
// multiFlag collects repeated -dir flags.
type multiFlag []string
func (m *multiFlag) String() string { return fmt.Sprint([]string(*m)) }
func (m *multiFlag) Set(v string) error {
*m = append(*m, v)
return nil
}
-20
View File
@@ -8,7 +8,6 @@ import (
"github.com/samcharles93/archie-core/internal/domain/eda/module"
"github.com/samcharles93/archie-core/internal/domain/eda/playbook"
"github.com/samcharles93/archie-core/internal/domain/workflow"
)
// Result is the outcome of one lint run: the exit code for the process and
@@ -18,25 +17,6 @@ type Result struct {
Findings []string
}
// Lint validates playbook directories with workflow.LoadPlaybookDirs and
// reports every finding. It returns 0 when clean, 1 otherwise.
func Lint(dirs []string, stderr io.Writer) Result {
var findings []string
_, _, err := workflow.LoadPlaybookDirs(dirs)
if err != nil {
findings = append(findings, err.Error())
}
if len(findings) > 0 {
for _, f := range findings {
fmt.Fprintln(stderr, f)
}
return Result{ExitCode: 1, Findings: findings}
}
return Result{ExitCode: 0, Findings: nil}
}
// LintEDA validates a directory of EDA playbook documents with the compiler
// the eda-playbooks resource validates against.
func LintEDA(dir string, stderr io.Writer) Result {
+1 -24
View File
@@ -1,10 +1,8 @@
package archieplaybooks
import (
"bytes"
"context"
"io"
"os"
"path/filepath"
"regexp"
"strconv"
@@ -14,7 +12,6 @@ import (
"go.lsp.dev/jsonrpc2"
"go.lsp.dev/protocol"
"go.lsp.dev/uri"
"gopkg.in/yaml.v3"
)
// Serve runs the language server over rwc until the client disconnects or ctx
@@ -87,12 +84,7 @@ func diagnose(u uri.URI) *protocol.PublishDiagnosticsParams {
return out
}
path := u.FsPath()
var result Result
if isEDAPlaybook(path) {
result = LintEDA(filepath.Dir(path), io.Discard)
} else {
result = Lint([]string{filepath.Dir(path)}, io.Discard)
}
result := LintEDA(filepath.Dir(path), io.Discard)
for _, finding := range result.Findings {
line, ok := findingLine(finding, path)
if !ok {
@@ -108,21 +100,6 @@ func diagnose(u uri.URI) *protocol.PublishDiagnosticsParams {
return out
}
// isEDAPlaybook reports whether the file is an EDA playbook document (it has
// a top-level `trigger` key) rather than a routing binding file.
func isEDAPlaybook(path string) bool {
data, err := os.ReadFile(path)
if err != nil {
return false
}
var doc map[string]yaml.Node
if yaml.NewDecoder(bytes.NewReader(data)).Decode(&doc) != nil {
return false
}
_, ok := doc["trigger"]
return ok
}
// findingLine places a finding on path. A compiler-style `path:line:` prefix
// gives the 0-based line; a finding naming path without a line goes on line
// 0; a finding naming another file is not path's.
+8 -14
View File
@@ -3,6 +3,7 @@ package archieui
import (
"context"
"log/slog"
"strings"
"time"
"github.com/samcharles93/archie-core/internal/app/servicekit"
@@ -62,6 +63,10 @@ func compose(ctx context.Context, d deps) *webui.Server {
Health: d.Health,
ControlPlane: d.ControlPlane,
Identities: d.Identities,
Issuer: strings.TrimSpace(d.Options.OidcIssuer),
}
if subjects, ok := d.Identities.(identity.SubjectBinding); ok {
srv.Subjects = subjects
}
if snapshots, ok := d.Store.(storecontract.ConfigSnapshotStore); ok {
srv.ConfigSource = webui.RemoteConfigView(snapshots)
@@ -116,10 +121,8 @@ func compose(ctx context.Context, d deps) *webui.Server {
wireCatalog(d, srv)
// The policy chain: wired together or not at all.
srv.Refusals = refusalStore(d.Store)
if d.Access != nil {
srv.Access = d.Access
srv.ReloadAccess = d.Access.Reload
}
srv.Access = d.Access
srv.ReloadAccess = d.Access.Reload
srv.Principals = d.Principals
srv.Denials = d.Denials
wireTaskLogs(d, srv)
@@ -195,7 +198,7 @@ func wireCaptureSurfaces(d deps, srv *webui.Server) {
MaxEvents: capture.MaxEvents,
MaxBodyBytes: int64(capture.MaxBodyBytes),
Publish: captureArrivalPublisher(d),
Delivery: deliveryAuthorizer(d.Access),
Delivery: d.Access,
Refusals: refusalStore(d.Store),
Log: d.Log,
}
@@ -221,15 +224,6 @@ func captureArrivalPublisher(d deps) func(context.Context, events.Event) {
// a hung State Store must not hold the sender's webhook past this.
const captureEventTimeout = 5 * time.Second
// deliveryAuthorizer keeps a missing chain a nil authorizer, which admits
// every sender.
func deliveryAuthorizer(chain *infraaccess.Live) access.DeliveryAuthorizer {
if chain == nil {
return nil
}
return chain
}
func refusalStore(store storecontract.TaskStore) storecontract.CaptureRefusalStore {
refusals, _ := store.(storecontract.CaptureRefusalStore)
return refusals
+1 -1
View File
@@ -176,7 +176,7 @@ func (o Options) validate() error {
}
if !webui.IsLoopback(o.Listen) && o.Token == "" {
return fmt.Errorf(
"ui listen address %q is non-loopback; a reachable dashboard requires a token (-token) or a token file to mint one (-token-file) (docs/prds/ui-service-boundary.md, listen and authentication)",
"ui listen address %q is non-loopback; a reachable dashboard requires a token (-token) or a token file to mint one (-token-file) (deployments/README.md, running the dashboard)",
o.Listen,
)
}
+1 -10
View File
@@ -18,7 +18,7 @@ import (
// Store and, when one is configured, the sign-in provider.
func newReadinessRegistry(o Options, tasks storecontract.TaskStore, chat messaging.ChatContract, chain *infraaccess.Live, provider *oidc.Provider) *health.Registry {
probes := []health.Probe{
readiness.NewProblemProbe("access_policies", problemSource(chain)),
readiness.NewProblemProbe("access_policies", chain),
captureRefusalProbe{store: refusalStore(tasks)},
readiness.NewContractProbe("state_db", o.DependencyTimeout, func(ctx context.Context) error {
_, err := tasks.StatusCounts(ctx)
@@ -35,15 +35,6 @@ func newReadinessRegistry(o Options, tasks storecontract.TaskStore, chat messagi
return health.NewRegistry(probes...)
}
// problemSource keeps a missing chain a nil source, which the probe reports
// as not wired.
func problemSource(chain *infraaccess.Live) readiness.PolicyProblems {
if chain == nil {
return nil
}
return chain
}
// captureRefusalProbe degrades when any source refused more events than it
// accepted over the last hour: a sender being blocked, or someone probing.
type captureRefusalProbe struct {
+22 -19
View File
@@ -12,9 +12,6 @@ import (
"slices"
"time"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"github.com/samcharles93/archie-core/internal/app/servicekit"
"github.com/samcharles93/archie-core/internal/domain/applystatus"
"github.com/samcharles93/archie-core/internal/domain/presence"
@@ -27,19 +24,14 @@ import (
"github.com/samcharles93/archie-core/internal/webui"
)
// buildAccessChain builds the live access engine over the stored policies.
// An unreachable State Store returns nil and logs a warning.
func buildAccessChain(ctx context.Context, store *staterpc.Client, log *slog.Logger) (*infraaccess.Live, error) {
live, err := infraaccess.NewLive(ctx, store.ListPolicies, log)
if status.Code(err) == codes.Unavailable {
log.Warn("access chain unavailable; the credential check is the gate", "err", err)
return nil, nil
}
if err != nil {
return nil, err
}
// buildAccessChain starts the live access engine over the stored policies.
// It refuses every request until Run's first load succeeds and keeps
// retrying, so a State Store or database that is down or slow at start
// closes the dashboard rather than opening it or stopping it.
func buildAccessChain(ctx context.Context, store *staterpc.Client, log *slog.Logger) *infraaccess.Live {
live := infraaccess.NewPending(store.ListPolicies, log)
go live.Run(ctx, applystatus.RestampInterval)
return live, nil
return live
}
// Run serves the dashboard until ctx is cancelled. It resolves its own
@@ -53,6 +45,7 @@ func Run(ctx context.Context, options Options) error {
if err != nil {
return err
}
warnPlainHTTP(log, opts)
var cleanups []func()
defer func() {
for _, cleanup := range slices.Backward(cleanups) {
@@ -69,10 +62,7 @@ func Run(ctx context.Context, options Options) error {
return err
}
cleanups = append(cleanups, closeGateway)
chain, err := buildAccessChain(ctx, tasks, log)
if err != nil {
return err
}
chain := buildAccessChain(ctx, tasks, log)
provider, err := signInProvider(opts)
if err != nil {
return err
@@ -131,6 +121,19 @@ func Run(ctx context.Context, options Options) error {
return serve(ctx, listener, srv.Handler(), opts)
}
// warnPlainHTTP names the exposure of a dashboard published off-host. The
// process serves plain HTTP, so the session cookie and the provider token
// cross the network in clear unless a TLS-terminating proxy sits in front of
// it -- which [web].trust_forwarded_headers declares. A loopback bind is
// reached over a tunnel or on the host, so it needs no warning.
func warnPlainHTTP(log *slog.Logger, opts Options) {
if webui.IsLoopback(opts.Listen) || opts.trustForwardedHeaders() {
return
}
log.Warn("dashboard is reachable off-host over plain HTTP; session and provider tokens travel in clear unless TLS terminates in front of it (set [web].trust_forwarded_headers behind a TLS proxy)",
"listen", opts.Listen)
}
// wireHarness attaches the setup terminal to the daemon's session contract
// when the operator configured one, and registers its cleanup.
func wireHarness(opts Options, srv *webui.Server, cleanups *[]func()) error {
@@ -23,7 +23,7 @@ const ChannelSettingsKind = controlplanerpc.ChannelSettingsKind
func seedChannels(cfg config.Config) any {
chat := cfg.Chat
return channelSettings{
Operator: chat.Operator, Workspace: chat.Workspace, UnrestrictedFilesystem: chat.UnrestrictedFilesystem, ShowToolCalls: chat.ShowToolCalls, MaxSteps: chat.MaxSteps, Models: chat.Models,
Operator: chat.Operator, Workspace: chat.Workspace, UnrestrictedFilesystem: chat.UnrestrictedFilesystem, ShowToolCalls: chat.ShowToolCalls, MaxSteps: chat.MaxSteps,
Telegram: telegramSettings{AllowedUserIDs: chat.Telegram.AllowedUserIDs, Token: chat.Telegram.Token, CredentialConfigured: chat.Telegram.Token != (config.SecretRef{})},
RateLimit: rateLimitSettings{Window: channelDuration(chat.RateLimit.Window), MaxRequests: chat.RateLimit.MaxRequests},
}
+24 -1
View File
@@ -3,7 +3,9 @@ package controlplane
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
@@ -50,6 +52,7 @@ type Server struct {
store ResourceStore
definitions map[string]Definition
ordered []Definition
steps workflow.StepRegistry
}
// NewServer builds the control plane server the State Store serves. steps is
@@ -66,7 +69,7 @@ func NewServer(resources ResourceStore, steps *workflow.Manager) (*Server, error
for _, definition := range definitions {
byKind[definition.Kind] = definition
}
return &Server{store: resources, definitions: byKind, ordered: definitions}, nil
return &Server{store: resources, definitions: byKind, ordered: definitions, steps: registry}, nil
}
func (s *Server) Catalog(context.Context, *pb.CatalogRequest) (*pb.CatalogResponse, error) {
@@ -169,6 +172,9 @@ func (s *Server) Command(ctx context.Context, request *pb.CommandRequest) (*pb.C
if err != nil {
return nil, mapError(err)
}
if err := s.checkOrgWrite(ctx, requestOrg(ctx), request.Kind, value); err != nil {
return nil, mapError(err)
}
resource, err := s.store.PutResource(ctx, storecontract.ResourceWrite{OrgID: requestOrg(ctx), Kind: request.Kind, Value: value, Actor: request.Actor, Source: request.Source, RequestID: request.RequestId, ExpectedVersion: request.ExpectedVersion, At: time.Now().UTC()})
if err != nil {
return nil, mapError(err)
@@ -247,3 +253,20 @@ func mapError(err error) error {
return status.Error(codes.Unavailable, err.Error())
}
}
// checkModelAliases refuses workflow definitions whose agent steps name a
// model alias the writing org cannot resolve.
func (s *Server) checkModelAliases(ctx context.Context, orgID string, value []byte) error {
models, err := s.ModelsFor(ctx, orgID)
if err != nil {
return err
}
var collection workflow.WorkflowDefinitionCollection
if err := json.Unmarshal(value, &collection); err != nil {
return err
}
if err := workflow.CheckModelAliases(collection, s.steps, models.Aliases); err != nil {
return fmt.Errorf("%w: %w", ErrValidation, err)
}
return nil
}
+1 -1
View File
@@ -8,7 +8,7 @@ import (
// Schemas are not declared here: each Definition names its document type, and
// Definition.Descriptor derives the JSON Schema from it (internal/app/controlplane/schema.go).
func builtinDefinitions(steps workflow.StepRegistry, settings map[string]any) []Definition {
definitions := []Definition{taskPolicyDefinition(), workflowDefinition(), workflowDefinitionsDefinition(steps, settings), workflowEnablementDefinition(), personaDefinition(), curatorDefinition(), soulDefinition(), scheduleDefinition(), agentProfileDefinition(), playbooksDefinition(), skillsDefinition(), credentialBindingsDefinition(), identityGrantsDefinition(), reviewSettingsDefinition()}
definitions := []Definition{taskPolicyDefinition(), workflowDefinition(), workflowDefinitionsDefinition(steps, settings), workflowEnablementDefinition(), personaDefinition(), curatorDefinition(), soulDefinition(), scheduleDefinition(), agentProfileDefinition(), playbooksDefinition(), skillsDefinition(), credentialBindingsDefinition(), identityGrantsDefinition(), reviewSettingsDefinition(), orgModelPolicyDefinition()}
definitions = append(definitions, modelDefinitions()...)
definitions = append(definitions, operationalDefinitions()...)
return definitions
+2 -11
View File
@@ -2,7 +2,6 @@ package controlplane
import (
"context"
"encoding/json"
"fmt"
"strings"
@@ -75,21 +74,13 @@ func validServices(owner string, services []string) error {
// each is its own resource, applies without a restart, and a stored value
// replaces the file's outright.
func layerCredentials(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, out *config.Config) error {
if err := layerResource(ctx, reader, versions, CredentialBindingsKind, func(value []byte) error {
var bindings []credentialBinding
if err := json.Unmarshal(value, &bindings); err != nil {
return err
}
if err := layerResource(ctx, reader, versions, CredentialBindingsKind, func(bindings []credentialBinding) error {
out.Containers.Credentials = credentialBindingsSettings(bindings)
return nil
}); err != nil {
return err
}
return layerResource(ctx, reader, versions, IdentityGrantsKind, func(value []byte) error {
var doc identityGrants
if err := json.Unmarshal(value, &doc); err != nil {
return err
}
return layerResource(ctx, reader, versions, IdentityGrantsKind, func(doc identityGrants) error {
applyIdentityGrants(out, doc)
return nil
})
+12 -7
View File
@@ -10,8 +10,8 @@ import (
)
const (
ProviderSettingsKind = "provider-settings"
ModelRoleAssignmentsKind = "model-role-assignments"
ProviderSettingsKind = "provider-settings"
ModelAliasesKind = "model-aliases"
)
var bootDerivedProviderEnv = regexp.MustCompile(`^ARCHIE_PROVIDER_[0-9A-F]{16}_API_KEY$`)
@@ -27,7 +27,7 @@ type providerDocument struct {
func modelDefinitions() []Definition {
return []Definition{
{Kind: ProviderSettingsKind, Title: "Providers", ApplyMode: "live", Document: map[string]providerDocument{}, Seed: seedProviders, Validate: validateProviders},
{Kind: ModelRoleAssignmentsKind, Title: "Model role assignments", ApplyMode: "live", Document: map[string]string{}, Seed: func(cfg config.Config) any { return cfg.Models }, Validate: validateModelRoles},
{Kind: ModelAliasesKind, Title: "Model aliases", ApplyMode: "live", Document: map[string]string{}, Seed: func(cfg config.Config) any { return cfg.Models }, Validate: validateModelAliases},
}
}
@@ -66,11 +66,16 @@ func rejectBootDerivedProviderEnv(name string, provider providerDocument) error
return nil
}
func validateModelRoles(input []byte) error {
var modelAliasName = regexp.MustCompile(`^[a-z0-9][a-z0-9._-]*$`)
func validateModelAliases(input []byte) error {
return validateAs(input, func(models map[string]string) error {
for role, model := range models {
if strings.TrimSpace(role) == "" || !strings.Contains(model, "/") {
return fmt.Errorf("model role %q must reference provider/model", role)
for alias, model := range models {
if !modelAliasName.MatchString(alias) {
return fmt.Errorf("model alias %q must be lowercase letters, digits, '.', '_' or '-'", alias)
}
if provider, name, ok := strings.Cut(model, "/"); !ok || provider == "" || name == "" {
return fmt.Errorf("model alias %q must reference provider/model", alias)
}
}
return nil
+234
View File
@@ -0,0 +1,234 @@
package controlplane
import (
"context"
"encoding/json"
"errors"
"fmt"
"maps"
"slices"
"strings"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
)
// OrgModelPolicyKind is the instance's per-org model policy. Only the
// system org writes it.
const OrgModelPolicyKind = "org-model-policy"
// orgModelPolicy is what one org may use. An org with no entry may use
// nothing until the instance admin allows it; the system org is never
// restricted.
type orgModelPolicy struct {
// Allowed lists the "provider/model" refs the org may use; "provider/*"
// allows every model of an instance provider.
Allowed []string `json:"allowed"`
// OwnProviders lets the org add providers under its own credentials.
OwnProviders bool `json:"own_providers"`
}
func (p *orgModelPolicy) allows(ref string) bool {
if p == nil {
return false
}
provider, _, _ := strings.Cut(ref, "/")
return slices.Contains(p.Allowed, ref) || slices.Contains(p.Allowed, provider+"/*")
}
func orgModelPolicyDefinition() Definition {
return Definition{
Kind: OrgModelPolicyKind, Title: "Org model policy", ApplyMode: "live",
Document: map[string]orgModelPolicy{},
Seed: func(config.Config) any { return map[string]orgModelPolicy{} },
Validate: func(input []byte) error {
return validateAs(input, func(policies map[string]orgModelPolicy) error {
for orgID, policy := range policies {
if orgID == storecontract.DefaultOrgID {
return fmt.Errorf("the system org uses every instance model; it takes no policy")
}
for _, ref := range policy.Allowed {
if provider, model, ok := strings.Cut(ref, "/"); !ok || provider == "" || model == "" {
return fmt.Errorf("org %q allows %q; use provider/model or provider/*", orgID, ref)
}
}
}
return nil
})
},
}
}
// ModelsFor resolves orgID's effective aliases and its own providers: the
// instance aliases it may use, with its own aliases over them. An alias
// whose model the org may not use is left out, so it fails as unknown.
func (s *Server) ModelsFor(ctx context.Context, orgID string) (config.OrgModels, error) {
instance, err := s.modelAliases(ctx, storecontract.DefaultOrgID)
if err != nil || orgID == "" || orgID == storecontract.DefaultOrgID {
return config.OrgModels{Org: orgID, Aliases: instance}, err
}
policy, err := s.orgPolicy(ctx, orgID)
if err != nil {
return config.OrgModels{}, err
}
providers, err := s.orgProviders(ctx, orgID, policy)
if err != nil {
return config.OrgModels{}, err
}
own, err := s.modelAliases(ctx, orgID)
if err != nil {
return config.OrgModels{}, err
}
aliases := make(map[string]string, len(instance)+len(own))
for alias, ref := range instance {
if policy.allows(ref) {
aliases[alias] = ref
}
}
for alias, ref := range own {
if usable(ref, policy, providers) {
aliases[alias] = ref
} else {
delete(aliases, alias)
}
}
return config.OrgModels{Org: orgID, Aliases: aliases, Providers: providers}, nil
}
func usable(ref string, policy *orgModelPolicy, own map[string]config.Provider) bool {
provider, _, _ := strings.Cut(ref, "/")
if _, ok := own[provider]; ok {
return true
}
return policy.allows(ref)
}
func (s *Server) orgPolicy(ctx context.Context, orgID string) (*orgModelPolicy, error) {
var policies map[string]orgModelPolicy
if err := s.readResource(ctx, storecontract.DefaultOrgID, OrgModelPolicyKind, &policies); err != nil {
return nil, err
}
policy, ok := policies[orgID]
if !ok {
return nil, nil
}
return &policy, nil
}
// orgProviders is orgID's own providers, empty unless its policy allows them.
func (s *Server) orgProviders(ctx context.Context, orgID string, policy *orgModelPolicy) (map[string]config.Provider, error) {
out := map[string]config.Provider{}
if policy == nil || !policy.OwnProviders {
return out, nil
}
var docs map[string]providerDocument
if err := s.readResource(ctx, orgID, ProviderSettingsKind, &docs); err != nil {
return nil, err
}
for name, doc := range docs {
out[name] = config.Provider{Class: doc.Class, BaseURL: doc.BaseURL, APIKeyEnv: doc.APIKeyEnv}
}
return out, nil
}
func (s *Server) modelAliases(ctx context.Context, orgID string) (map[string]string, error) {
aliases := map[string]string{}
return aliases, s.readResource(ctx, orgID, ModelAliasesKind, &aliases)
}
// readResource decodes orgID's kind into into, leaving it untouched when the
// org has none.
func (s *Server) readResource(ctx context.Context, orgID, kind string, into any) error {
resource, err := s.store.Resource(ctx, orgID, kind)
if errors.Is(err, storecontract.ErrResourceNotFound) {
return nil
}
if err != nil {
return err
}
if err := json.Unmarshal(resource.Value, into); err != nil {
return fmt.Errorf("read %s %s: %w", orgID, kind, err)
}
return nil
}
// checkOrgWrite applies the rules a resource's static validation cannot: the
// policy is the system org's alone, an org's providers need its policy's
// leave and its own credentials, and every alias must name a model the
// writing org may use.
func (s *Server) checkOrgWrite(ctx context.Context, orgID, kind string, value []byte) error {
switch kind {
case OrgModelPolicyKind:
if orgID != storecontract.DefaultOrgID {
return fmt.Errorf("%w: only the system org sets org model policy", ErrValidation)
}
case ProviderSettingsKind:
if orgID != storecontract.DefaultOrgID {
return s.checkOrgProviders(ctx, orgID, value)
}
case ModelAliasesKind:
return s.checkAliasWrite(ctx, orgID, value)
case WorkflowDefinitionsKind:
return s.checkModelAliases(ctx, orgID, value)
}
return nil
}
func (s *Server) checkOrgProviders(ctx context.Context, orgID string, value []byte) error {
var docs map[string]providerDocument
if err := json.Unmarshal(value, &docs); err != nil {
return err
}
if len(docs) == 0 {
return nil
}
policy, err := s.orgPolicy(ctx, orgID)
if err != nil {
return err
}
if policy == nil || !policy.OwnProviders {
return fmt.Errorf("%w: org %s may not add its own providers (instance Settings > Models > Org access)", ErrValidation, orgID)
}
var instance map[string]providerDocument
if err := s.readResource(ctx, storecontract.DefaultOrgID, ProviderSettingsKind, &instance); err != nil {
return err
}
for name, doc := range docs {
if _, taken := instance[name]; taken {
return fmt.Errorf("%w: provider %q is an instance provider; name yours differently", ErrValidation, name)
}
// The key comes from the org's own credential binding named after
// the provider, never a secret this org could point at elsewhere.
if doc.APIKey != (config.SecretRef{}) {
return fmt.Errorf("%w: provider %q: an org provider takes its key from the org's credential binding named %q, not a secret reference", ErrValidation, name, name)
}
}
return nil
}
func (s *Server) checkAliasWrite(ctx context.Context, orgID string, value []byte) error {
var aliases map[string]string
if err := json.Unmarshal(value, &aliases); err != nil {
return err
}
if orgID == storecontract.DefaultOrgID {
if _, ok := aliases[config.DefaultModelAlias]; !ok && len(aliases) > 0 {
return fmt.Errorf("%w: model alias %q is required", ErrValidation, config.DefaultModelAlias)
}
return nil
}
policy, err := s.orgPolicy(ctx, orgID)
if err != nil {
return err
}
providers, err := s.orgProviders(ctx, orgID, policy)
if err != nil {
return err
}
for _, alias := range slices.Sorted(maps.Keys(aliases)) {
if !usable(aliases[alias], policy, providers) {
return fmt.Errorf("%w: model alias %q: org %s may not use %q", ErrValidation, alias, orgID, aliases[alias])
}
}
return nil
}
@@ -0,0 +1,102 @@
package controlplane_test
import (
"context"
"maps"
"slices"
"strings"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/app/controlplane"
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/identity"
"github.com/samcharles93/archie-core/internal/domain/org"
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
)
// An org reaches only the models the instance admin allows it and the
// providers it was allowed to add; an org with no policy reaches nothing.
func TestOrgModelPolicy(t *testing.T) {
ctx := t.Context()
db := pgstore.Open(t)
if err := db.BootstrapIdentities(ctx, nil); err != nil {
t.Fatal(err)
}
if err := db.UpgradeDefaultOrg(ctx); err != nil {
t.Fatal(err)
}
steps, err := workflowsteps.NewManager()
if err != nil {
t.Fatal(err)
}
server, err := controlplane.NewServer(db, steps)
if err != nil {
t.Fatal(err)
}
owner, err := db.Create(ctx, identity.Identity{ID: "owner", Kind: identity.KindUser, DisplayName: "Owner", Lifecycle: identity.LifecycleActive}, identity.Audit{ActorID: identity.SystemID, Source: "test", RequestID: "owner", At: time.Now()})
if err != nil {
t.Fatal(err)
}
for _, id := range []org.OrgID{"acme", "newco"} {
if _, err := db.CreateOrg(ctx, org.Org{ID: id, Name: string(id)}, owner.ID); err != nil {
t.Fatal(err)
}
}
sys := org.WithOrg(ctx, org.DefaultOrgID)
acme := org.WithOrg(ctx, "acme")
put := func(ctx context.Context, kind, value string) error {
_, err := server.Command(ctx, &pb.CommandRequest{Kind: kind, Command: "replace", ValueJson: []byte(value), Actor: "test", Source: "test", RequestId: kind + value})
return err
}
for _, w := range [][2]string{
{controlplane.ProviderSettingsKind, `{"openai":{"class":"openai","api_key_ref":{}},"deepseek":{"class":"deepseek","api_key_ref":{}}}`},
{controlplane.ModelAliasesKind, `{"default":"openai/gpt-6-luna","cheap":"deepseek/chat"}`},
{controlplane.OrgModelPolicyKind, `{"acme":{"allowed":["openai/*"],"own_providers":true}}`},
} {
if err := put(sys, w[0], w[1]); err != nil {
t.Fatalf("seed %s: %v", w[0], err)
}
}
for _, tt := range []struct {
name, kind, value string
ok bool
}{
{"an alias to a model the org may not use", controlplane.ModelAliasesKind, `{"fast":"deepseek/chat"}`, false},
{"an org provider named like an instance one", controlplane.ProviderSettingsKind, `{"openai":{"class":"openai","api_key_ref":{}}}`, false},
{"an org provider pointing at a secret", controlplane.ProviderSettingsKind, `{"mine":{"class":"openai","api_key_ref":{"engine":"env","key":"X"}}}`, false},
{"an org writing the instance policy", controlplane.OrgModelPolicyKind, `{"acme":{"allowed":["deepseek/*"]}}`, false},
{"an org provider of its own", controlplane.ProviderSettingsKind, `{"mine":{"class":"openai","api_key_ref":{}}}`, true},
{"aliases to allowed and own models", controlplane.ModelAliasesKind, `{"fast":"openai/gpt-6-mini","own":"mine/m1"}`, true},
} {
t.Run(tt.name, func(t *testing.T) {
if err := put(acme, tt.kind, tt.value); (err == nil) != tt.ok {
t.Fatalf("err = %v, want ok %v", err, tt.ok)
}
})
}
for _, tt := range []struct {
org string
aliases, prov []string
}{
{"acme", []string{"default", "fast", "own"}, []string{"mine"}},
{"newco", nil, nil},
{string(org.DefaultOrgID), []string{"cheap", "default"}, nil},
} {
t.Run("resolve "+tt.org, func(t *testing.T) {
models, err := server.ModelsFor(ctx, tt.org)
if err != nil {
t.Fatal(err)
}
if got := slices.Sorted(maps.Keys(models.Aliases)); strings.Join(got, ",") != strings.Join(tt.aliases, ",") {
t.Fatalf("aliases = %v, want %v", got, tt.aliases)
}
if got := slices.Sorted(maps.Keys(models.Providers)); strings.Join(got, ",") != strings.Join(tt.prov, ",") {
t.Fatalf("providers = %v, want %v", got, tt.prov)
}
})
}
}
+29 -49
View File
@@ -68,11 +68,7 @@ func (r storeReader) Query(ctx context.Context, kind string, decode func([]byte)
func runtimeConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReader, base config.Config) (config.Config, map[string]int64, error) {
out := base.Clone()
versions := map[string]int64{}
if err := layerResource(ctx, reader, versions, ProviderSettingsKind, func(value []byte) error {
var providers map[string]providerDocument
if err := json.Unmarshal(value, &providers); err != nil {
return err
}
if err := layerResource(ctx, reader, versions, ProviderSettingsKind, func(providers map[string]providerDocument) error {
out.Providers = make(map[string]config.Provider, len(providers))
for name, provider := range providers {
if err := rejectBootDerivedProviderEnv(name, provider); err != nil {
@@ -84,13 +80,9 @@ func runtimeConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReade
}); err != nil {
return config.Config{}, nil, err
}
// Stored role assignments replace cfg.Models outright.
if err := layerResource(ctx, reader, versions, ModelRoleAssignmentsKind, func(value []byte) error {
var roles map[string]string
if err := json.Unmarshal(value, &roles); err != nil {
return err
}
out.Models = roles
// Stored aliases replace cfg.Models outright.
if err := layerResource(ctx, reader, versions, ModelAliasesKind, func(aliases map[string]string) error {
out.Models = aliases
return nil
}); err != nil {
return config.Config{}, nil, err
@@ -103,11 +95,7 @@ func runtimeConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReade
return config.Config{}, nil, err
}
out.Chat, versions[ChannelSettingsKind] = chat, chatVersion
if err := layerResource(ctx, reader, versions, SchedulingPolicyKind, func(value []byte) error {
var policy schedulingPolicy
if err := json.Unmarshal(value, &policy); err != nil {
return err
}
if err := layerResource(ctx, reader, versions, SchedulingPolicyKind, func(policy schedulingPolicy) error {
interval, err := time.ParseDuration(policy.PollInterval)
if err != nil {
return err
@@ -125,11 +113,7 @@ func runtimeConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReade
// The two fields are assigned individually, not `out.Review = ...`: a field
// added to config.Review later keeps the file's value until the stored
// document carries it.
if err := layerResource(ctx, reader, versions, ReviewSettingsKind, func(value []byte) error {
var settings reviewSettings
if err := json.Unmarshal(value, &settings); err != nil {
return err
}
if err := layerResource(ctx, reader, versions, ReviewSettingsKind, func(settings reviewSettings) error {
out.Review.PrecisionGate = settings.PrecisionGate
out.Review.ApproveBeforePost = settings.ApproveBeforePost
return nil
@@ -147,11 +131,7 @@ func runtimeToolConfigFrom(ctx context.Context, reader controlplanerpc.ResourceR
out.DiffCapLines = new(policy.DiffCapLines)
out.Notify.Webhook = policy.NotifyWebhook
if err := layerResource(ctx, reader, versions, ToolSettingsKind, func(value []byte) error {
var settings toolSettings
if err := json.Unmarshal(value, &settings); err != nil {
return err
}
if err := layerResource(ctx, reader, versions, ToolSettingsKind, func(settings toolSettings) error {
headers := make(map[string]map[string]string, len(out.Tools.MCPServers))
fileParallelToolCalls := make(map[string]bool, len(out.Tools.MCPServers))
for _, server := range out.Tools.MCPServers {
@@ -184,21 +164,13 @@ func runtimeToolConfigFrom(ctx context.Context, reader controlplanerpc.ResourceR
}); err != nil {
return config.Config{}, nil, err
}
if err := layerResource(ctx, reader, versions, PluginSettingsKind, func(value []byte) error {
var settings pluginSettings
if err := json.Unmarshal(value, &settings); err != nil {
return err
}
if err := layerResource(ctx, reader, versions, PluginSettingsKind, func(settings pluginSettings) error {
out.SkillsDir = settings.SkillsDir
return nil
}); err != nil {
return config.Config{}, nil, err
}
if err := layerResource(ctx, reader, versions, ContainerRuntimePoliciesKind, func(value []byte) error {
var policies containerRuntimePolicies
if err := json.Unmarshal(value, &policies); err != nil {
return err
}
if err := layerResource(ctx, reader, versions, ContainerRuntimePoliciesKind, func(policies containerRuntimePolicies) error {
// Keep the file-owned Profiles and RegistryAuth across the replacement.
profiles := out.Containers.Profiles
registryAuth := out.Containers.RegistryAuth
@@ -210,11 +182,7 @@ func runtimeToolConfigFrom(ctx context.Context, reader controlplanerpc.ResourceR
return config.Config{}, nil, err
}
// Stored profiles replace the file's outright.
if err := layerResource(ctx, reader, versions, AgentProfileKind, func(value []byte) error {
var profiles map[string]agentProfile
if err := json.Unmarshal(value, &profiles); err != nil {
return err
}
if err := layerResource(ctx, reader, versions, AgentProfileKind, func(profiles map[string]agentProfile) error {
out.Containers.Profiles = agentProfilesSettings(profiles)
return nil
}); err != nil {
@@ -224,9 +192,25 @@ func runtimeToolConfigFrom(ctx context.Context, reader controlplanerpc.ResourceR
return out, versions, err
}
// layerResource decodes a stored resource and records its version. A kind
// with no stored value is skipped.
func layerResource(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, decode func([]byte) error) error {
// layerResource decodes a stored JSON resource into a T, applies it and
// records its version. A kind with no stored value is skipped.
func layerResource[T any](ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, apply func(T) error) error {
return layerRaw(ctx, reader, versions, kind, func(value []byte) error {
var v T
if err := json.Unmarshal(value, &v); err != nil {
return err
}
return apply(v)
})
}
// layerResourceJSON decodes a stored resource over target, keeping fields
// the stored document omits.
func layerResourceJSON(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, target any) error {
return layerRaw(ctx, reader, versions, kind, func(value []byte) error { return json.Unmarshal(value, target) })
}
func layerRaw(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, decode func([]byte) error) error {
version, found, err := reader.Query(ctx, kind, decode)
if err != nil {
return err
@@ -237,10 +221,6 @@ func layerResource(ctx context.Context, reader controlplanerpc.ResourceReader, v
return nil
}
func layerResourceJSON(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, target any) error {
return layerResource(ctx, reader, versions, kind, func(value []byte) error { return json.Unmarshal(value, target) })
}
// query returns the version of the resource it decoded, so callers that layer
// a resource in can report the version they applied. A kind the store holds no
// value for is reported as not found rather than as an error: the caller leaves
+86 -27
View File
@@ -1,40 +1,68 @@
{
"description": "Shared accept/reject fixtures for the control-plane validators the dashboard hand-mirrors in ui/src/settings/validation.ts. The control plane is the authority: every case's `valid` is what internal/app/controlplane accepts or refuses. The Go test in this package runs the real server validators over these cases; the node test ui/test/control-plane-validator-parity.test.ts runs the real dashboard validators over the same cases. A one-sided change to either rule makes one of the two tests fail, which is the tie that stops the two implementations drifting apart (archie-core-ui-dashboard-3). Add a case here, never only to one side.",
"cases": {
"model-role-assignments": [
"model-aliases": [
{
"why": "a plain provider/model value",
"value": { "builder": "openai/gpt-4o" },
"value": {
"default": "openai/gpt-4o"
},
"valid": true
},
{
"why": "a trailing space: the server only asks for a slash, so the dashboard must not refuse it",
"value": { "builder": "openai/gpt-4o " },
"valid": true
},
{
"why": "a leading space: the server only asks for a slash, so the dashboard must not refuse it",
"value": { "builder": " openai/gpt-4o" },
"why": "a trailing space: the server only splits on the slash, so the dashboard must not refuse it",
"value": {
"default": "openai/gpt-4o "
},
"valid": true
},
{
"why": "a model id with a colon (a real ollama tag), which a stricter shape would wrongly refuse",
"value": { "planner": "ollama/qwen2.5-coder:7b" },
"value": {
"default": "openai/gpt-4o",
"fast": "ollama/qwen2.5-coder:7b"
},
"valid": true
},
{
"why": "an empty table: nothing configured yet",
"value": {},
"valid": true
},
{
"why": "no slash at all",
"value": { "builder": "gpt-4o" },
"value": {
"default": "gpt-4o"
},
"valid": false
},
{
"why": "an empty model half",
"value": {
"default": "openai/"
},
"valid": false
},
{
"why": "an empty model: the server refuses it, so the dashboard must too (SystemModelsPage deletes the key rather than storing one)",
"value": { "builder": "" },
"value": {
"default": ""
},
"valid": false
},
{
"why": "a blank role key",
"value": { "": "openai/gpt-4o" },
"why": "aliases without the default alias: an org's own table inherits the instance default, so only the instance save requires it",
"value": {
"fast": "openai/gpt-4o"
},
"valid": true
},
{
"why": "an alias name with capitals or spaces",
"value": {
"default": "openai/gpt-4o",
"Fast model": "openai/gpt-4o"
},
"valid": false
}
],
@@ -45,7 +73,9 @@
"poll_interval": "1m0s",
"max_retries": 0,
"label": "archie",
"dispatch": { "trigger": "assignee" }
"dispatch": {
"trigger": "assignee"
}
},
"valid": true
},
@@ -54,7 +84,9 @@
"value": {
"poll_interval": "30s",
"max_retries": 3,
"dispatch": { "trigger": "assignee" }
"dispatch": {
"trigger": "assignee"
}
},
"valid": true
},
@@ -63,7 +95,9 @@
"value": {
"poll_interval": "1h30m",
"label": "archie",
"dispatch": { "trigger": "either" }
"dispatch": {
"trigger": "either"
}
},
"valid": true
},
@@ -71,7 +105,9 @@
"why": "a fractional duration the server's parser accepts",
"value": {
"poll_interval": "1.5h",
"dispatch": { "trigger": "assignee" }
"dispatch": {
"trigger": "assignee"
}
},
"valid": true
},
@@ -80,7 +116,9 @@
"value": {
"poll_interval": "0s",
"label": "archie",
"dispatch": { "trigger": "assignee" }
"dispatch": {
"trigger": "assignee"
}
},
"valid": false
},
@@ -89,7 +127,9 @@
"value": {
"poll_interval": "nonsense",
"label": "archie",
"dispatch": { "trigger": "assignee" }
"dispatch": {
"trigger": "assignee"
}
},
"valid": false
},
@@ -98,7 +138,9 @@
"value": {
"poll_interval": "30s",
"max_retries": 3,
"dispatch": { "trigger": "label" }
"dispatch": {
"trigger": "label"
}
},
"valid": false
},
@@ -108,7 +150,9 @@
"poll_interval": "30s",
"max_retries": 0,
"label": "",
"dispatch": { "trigger": "either" }
"dispatch": {
"trigger": "either"
}
},
"valid": false
}
@@ -120,7 +164,10 @@
{
"id": "daily",
"kind": "workflow",
"schedule": { "kind": "interval", "interval": "24h0m0s" }
"schedule": {
"kind": "interval",
"interval": "24h0m0s"
}
}
],
"valid": true
@@ -132,7 +179,10 @@
"id": "nightly",
"kind": "workflow",
"pool": "parallel",
"schedule": { "kind": "cron", "cron": "0 3 * * *" }
"schedule": {
"kind": "cron",
"cron": "0 3 * * *"
}
}
],
"valid": true
@@ -143,7 +193,10 @@
{
"id": "",
"kind": "workflow",
"schedule": { "kind": "interval", "interval": "24h0m0s" }
"schedule": {
"kind": "interval",
"interval": "24h0m0s"
}
}
],
"valid": false
@@ -154,12 +207,18 @@
{
"id": "daily",
"kind": "workflow",
"schedule": { "kind": "interval", "interval": "24h0m0s" }
"schedule": {
"kind": "interval",
"interval": "24h0m0s"
}
},
{
"id": "daily",
"kind": "workflow",
"schedule": { "kind": "interval", "interval": "1h0m0s" }
"schedule": {
"kind": "interval",
"interval": "1h0m0s"
}
}
],
"valid": false
@@ -169,10 +169,7 @@ func checkWorkflowDashboard(t *testing.T, db *pgstore.TaskDB, client *staterpc.C
t.Helper()
ctx := t.Context()
policies := append(access.ShippedOrgPolicies("acme"), access.ShippedOrgPolicies(org.DefaultOrgID)...)
chain, err := infraaccess.New(policies)
if err != nil {
t.Fatal(err)
}
chain := infraaccess.New(policies)
srv := &webui.Server{Store: client, ControlPlane: client.ControlPlane(), Access: chain, Principals: client, Authenticate: func(ctx context.Context, token string) (identity.Identity, error) {
return db.Get(ctx, identity.IdentityID(token))
}}
@@ -56,7 +56,10 @@ func (p *WorkflowProjector) Apply(ctx context.Context, orgID, name, digest strin
if err != nil {
return fmt.Errorf("contributed workflow %q: %w", contentName, err)
}
entries = append(entries, workflow.WorkflowDefinitionEntry{ID: id, YAML: string(content)})
if workflow.PackageOf(string(content)) != "" {
return fmt.Errorf("contributed workflow %q: package is set by the installer, not the workflow", contentName)
}
entries = append(entries, workflow.WorkflowDefinitionEntry{ID: id, YAML: "package: " + strconv.Quote(name) + "\n" + string(content)})
recorded = append(recorded, storepkg.ProjectionEntry{Family: storepkg.FamilyWorkflows, EntryID: id})
}
// One apply, one request: a retried PutResource reuses the same request ID
+90
View File
@@ -0,0 +1,90 @@
package prbench
import (
"context"
"fmt"
"strings"
"github.com/samcharles93/ai-sdk/agent"
"github.com/samcharles93/ai-sdk/core"
"github.com/samcharles93/ai-sdk/runtime"
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview"
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview/bench"
)
// modelJudge asks one model, independent of the reviewer's, which posted
// comment catches each golden.
type modelJudge struct {
runtime *runtime.Runtime
modelRef string
}
func (j *modelJudge) Judge(ctx context.Context, p bench.Problem, comments []prreview.ScoredFinding) ([]bench.Verdict, error) {
provider, model, err := j.runtime.ChatProvider(ctx, j.modelRef)
if err != nil {
return nil, fmt.Errorf("resolve judge model %q: %w", j.modelRef, err)
}
var verdicts []bench.Verdict
sub := agent.Subagent{
Provider: provider,
Model: model,
System: judgeSystemPrompt,
Tools: core.ToolSet{"record_verdict": recordVerdictTool(len(p.Goldens), len(comments), &verdicts)},
MaxSteps: 2*len(p.Goldens) + 3,
}
// The verdicts are the judgement, not the closing reply: a run that recorded
// some and then ran out of steps is checked for completeness by the caller.
if _, err := sub.Run(ctx, judgePrompt(p, comments)); err != nil && len(verdicts) == 0 {
return nil, err
}
return verdicts, nil
}
const judgeSystemPrompt = `You score a code reviewer against known defects. You are given the pull request's known defects (goldens) and the numbered comments the reviewer posted.
For each golden, decide whether any posted comment identifies the same underlying defect: the same location and the same root cause. A comment about the same file but a different problem is not a match, and neither is a vague comment that would only be right by coincidence. Wording does not need to match.
Call record_verdict exactly once per golden. Pass the index of the matching comment, or -1 when no comment matches, and a one-sentence reason. When several comments match, pass the one that states the defect most directly.`
func judgePrompt(p bench.Problem, comments []prreview.ScoredFinding) string {
var b strings.Builder
fmt.Fprintf(&b, "Pull request: %s\n\n<goldens>\n", p.PRURL)
for i, g := range p.Goldens {
fmt.Fprintf(&b, "[%d] (%s) %s\n", i, g.Severity, g.Comment)
}
b.WriteString("</goldens>\n\n<comments>\n")
for i, c := range comments {
fmt.Fprintf(&b, "[%d] %s:%d-%d (%s) %s\n%s\n\n", i, c.File, c.LineStart, c.LineEnd, c.Severity, c.Title, c.Body)
}
b.WriteString("</comments>\n")
return b.String()
}
type verdictInput struct {
Golden int `json:"golden" jsonschema:"description=Index of the golden this verdict is for."`
Comment int `json:"comment" jsonschema:"description=Index of the comment that identifies the same defect, or -1 when none does."`
Reason string `json:"reason" jsonschema:"description=One sentence on why the comment does or does not match."`
}
func recordVerdictTool(goldens, comments int, verdicts *[]bench.Verdict) *core.Tool {
return core.NewTypedTool(
"record_verdict",
"Record whether a posted comment catches one golden. Call once per golden.",
func(_ context.Context, in verdictInput) (string, error) {
if in.Golden < 0 || in.Golden >= goldens {
return fmt.Sprintf("record_verdict rejected: golden must be 0..%d", goldens-1), nil
}
if in.Comment < bench.NoComment || in.Comment >= comments {
return fmt.Sprintf("record_verdict rejected: comment must be -1..%d", comments-1), nil
}
for _, v := range *verdicts {
if v.Golden == in.Golden {
return fmt.Sprintf("record_verdict rejected: golden %d already has a verdict", in.Golden), nil
}
}
*verdicts = append(*verdicts, bench.Verdict{Golden: in.Golden, Comment: in.Comment, Reason: in.Reason})
return "verdict recorded", nil
},
)
}
+150
View File
@@ -0,0 +1,150 @@
package prbench
import (
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"sync"
"sync/atomic"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/agentrun"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview"
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview/bench"
"github.com/samcharles93/archie-core/internal/infrastructure/prsource"
)
// liveReviewer runs the production pr-review decision phases on one problem.
type liveReviewer struct {
cfg config.Config
agent agentrun.Runner
source workflow.PRSource
nextID atomic.Int64
}
func (r *liveReviewer) Review(ctx context.Context, p bench.Problem) (bench.Review, error) {
owner, repo, number, err := parsePRURL(p.PRURL)
if err != nil {
return bench.Review{}, err
}
slog.Info("benchmark review started", "problem", p.ID)
tc := &workflow.TaskContext{
Task: &workflow.Task{ID: r.nextID.Add(1), Attempt: 1, Owner: owner, Repo: repo, PRNumber: number, Workflow: "pr-review", Inputs: map[string]any{"depth": "deep"}},
Cfg: r.cfg, Agent: r.agent, PRSource: r.source, Log: slog.Default(),
}
decision, err := workflow.ReviewPullRequest(ctx, tc)
if err != nil {
return bench.Review{}, err
}
// Non-nil empty comments means a completed pipeline found nothing.
if decision.Comments == nil {
decision.Comments = []prreview.ScoredFinding{}
}
slog.Info("benchmark review completed", "problem", p.ID, "comments", len(decision.Comments), "skipped_phases", decision.SkippedPhases)
return bench.Review{Comments: decision.Comments, SkippedPhases: decision.SkippedPhases}, nil
}
func parsePRURL(raw string) (string, string, int, error) {
u, err := url.Parse(raw)
if err != nil {
return "", "", 0, fmt.Errorf("invalid pull request URL: %w", err)
}
parts := strings.Split(strings.Trim(u.Path, "/"), "/")
if u.Scheme != "https" || u.Host != "github.com" || len(parts) != 4 || parts[2] != "pull" {
return "", "", 0, fmt.Errorf("benchmark requires a GitHub pull request URL")
}
n, err := strconv.Atoi(parts[3])
if err != nil || n <= 0 {
return "", "", 0, fmt.Errorf("invalid pull request URL")
}
return parts[0], parts[1], n, nil
}
// publicGitHub reads the benchmark's public pull requests anonymously. The
// diff and archive come from hosts outside the API rate limit, so a full run
// makes one API call per pull request. GH_TOKEN or GITHUB_TOKEN, when set,
// only lifts that API limit.
type publicGitHub struct {
token string
mu sync.Mutex
prs map[string]prsource.PullRequest
}
func newPublicGitHub() *publicGitHub {
token := os.Getenv("GH_TOKEN")
if token == "" {
token = os.Getenv("GITHUB_TOKEN")
}
return &publicGitHub{token: token, prs: map[string]prsource.PullRequest{}}
}
func (g *publicGitHub) get(ctx context.Context, rawURL string, api bool) (io.ReadCloser, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
if err != nil {
return nil, err
}
if api && g.token != "" {
req.Header.Set("Authorization", "Bearer "+g.token)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
_ = resp.Body.Close()
return nil, fmt.Errorf("GET %s: %s", rawURL, resp.Status)
}
return resp.Body, nil
}
func (g *publicGitHub) GetPullRequest(ctx context.Context, owner, repo string, number int) (prsource.PullRequest, error) {
key := fmt.Sprintf("%s/%s#%d", owner, repo, number)
g.mu.Lock()
pr, ok := g.prs[key]
g.mu.Unlock()
if ok {
return pr, nil
}
body, err := g.get(ctx, fmt.Sprintf("https://api.github.com/repos/%s/%s/pulls/%d", owner, repo, number), true)
if err != nil {
return prsource.PullRequest{}, err
}
defer body.Close()
var raw struct {
Title string `json:"title"`
Body string `json:"body"`
Head struct {
SHA string `json:"sha"`
} `json:"head"`
}
if err := json.NewDecoder(body).Decode(&raw); err != nil {
return prsource.PullRequest{}, err
}
pr = prsource.PullRequest{Title: raw.Title, Body: raw.Body, HeadSHA: raw.Head.SHA}
g.mu.Lock()
g.prs[key] = pr
g.mu.Unlock()
return pr, nil
}
func (g *publicGitHub) GetPullRequestDiff(ctx context.Context, owner, repo string, number int) (string, error) {
body, err := g.get(ctx, fmt.Sprintf("https://github.com/%s/%s/pull/%d.diff", owner, repo, number), false)
if err != nil {
return "", err
}
defer body.Close()
data, err := io.ReadAll(body)
return string(data), err
}
func (g *publicGitHub) GetRepoArchive(ctx context.Context, owner, repo, ref string) (io.ReadCloser, error) {
return g.get(ctx, fmt.Sprintf("https://codeload.github.com/%s/%s/tar.gz/%s", owner, repo, ref), false)
}
+638
View File
@@ -0,0 +1,638 @@
[
{
"id": "cal_dot_com#10600",
"pr_url": "https://github.com/calcom/cal.com/pull/10600",
"goldens": [
{
"comment": "The exported function TwoFactor handles backup codes and is in BackupCode.tsx. Inconsistent naming.",
"severity": "Low"
},
{
"comment": "Error message mentions 'backup code login' but this is a disable endpoint, not login",
"severity": "Low"
},
{
"comment": "Backup code validation is case-sensitive due to the use of indexOf(). This causes validation to fail if a user enters uppercase hex characters, as backup codes should be case-insensitive for a better user experience.",
"severity": "Medium"
},
{
"comment": "Because backupCodes are decrypted and mutated in memory before being written back, two concurrent login requests using the same backupCode could both pass this check and update, so a single backup code may effectively be accepted more than once if used concurrently, weakening the intended one-time-use semantics.",
"severity": "High"
}
]
},
{
"id": "cal_dot_com#10967",
"pr_url": "https://github.com/calcom/cal.com/pull/10967",
"goldens": [
{
"comment": "Potential null reference if mainHostDestinationCalendar is undefined if evt.destinationCalendar is null or an empty array ",
"severity": "High"
},
{
"comment": "The optional chaining on mainHostDestinationCalendar?.integration is redundant since you already check mainHostDestinationCalendar in the ternary condition.",
"severity": "Low"
},
{
"comment": "Logic error: when externalCalendarId is provided, you're searching for a calendar where externalId === externalCalendarId, but this will always fail since you're looking for a calendar that matches itself. Should likely find by credentialId or use different logic.",
"severity": "High"
},
{
"comment": "Logic inversion in organization creation: The slug property is now conditionally set when IS_TEAM_BILLING_ENABLED is true, instead of when it's false as originally intended. This change, combined with requestedSlug still being set when IS_TEAM_BILLING_ENABLED is true, results in both properties being set when billing is enabled, and neither when disabled",
"severity": "Medium"
},
{
"comment": "The Calendar interface now requires createEvent(event, credentialId), but some implementations (e.g., Lark/Office365) still declare createEvent(event) only—this breaks the interface contract (also applies to other locations in the PR).",
"severity": "Low"
}
]
},
{
"id": "cal_dot_com#11059",
"pr_url": "https://github.com/calcom/cal.com/pull/11059",
"goldens": [
{
"comment": "The parseRefreshTokenResponse function incorrectly sets refresh_token to the hardcoded string 'refresh_token' when it's missing from the OAuth refresh token response. This invalidates the token, breaking subsequent token refreshes and causing authentication failures.",
"severity": "High"
},
{
"comment": "Invalid Zod schema syntax. Computed property keys like [z.string().toString()] are not valid in Zod object schemas and will cause runtime errors. ",
"severity": "High"
},
{
"comment": "parseRefreshTokenResponse returns a Zod safeParse result ({ success, data, error }), not the credential key object. Persisting that as key stores the wrapper instead of the token payload; we should store the parsed data or use schema parse.",
"severity": "High"
},
{
"comment": "When APP_CREDENTIAL_SHARING_ENABLED and CALCOM_CREDENTIAL_SYNC_ENDPOINT are set, the refreshFunction helper returns the fetch Response, but several callers (for example GoogleCalendarService.refreshAccessToken expecting res.data, and HubspotCalendarService.refreshAccessToken expecting a HubspotToken) assume it returns the integration-specific token object. That mismatch will cause runtime errors in the sync-enabled path unless the return type or those call sites are adjusted.",
"severity": "High"
},
{
"comment": "When the sync endpoint path is used, res is a fetch Response and has no .data; res?.data will be undefined and token.access_token will throw at runtime. This relies on a consistent return shape from refreshOAuthTokens, which isn’t guaranteed currently.",
"severity": "High"
}
]
},
{
"id": "cal_dot_com#14740",
"pr_url": "https://github.com/calcom/cal.com/pull/14740",
"goldens": [
{
"comment": "Case sensitivity bypass in email blacklist",
"severity": "High"
},
{
"comment": "The logic for checking team admin/owner permissions is incorrect. This condition uses AND (&&) which requires both isTeamAdmin AND isTeamOwner to be true, but it should use OR (||) since a user needs to be either an admin OR an owner to have permission.",
"severity": "Critical"
},
{
"comment": "This calls the email sender with the original guests, so existing attendees included in the input will be treated as new when sending notifications, leading to incorrect emails.",
"severity": "Medium"
},
{
"comment": "uniqueGuests filters out existing attendees and blacklisted emails but does not deduplicate duplicates within the input; createMany can insert duplicate attendee rows if the client submits repeated emails.",
"severity": "Medium"
},
{
"comment": "Starting with an array containing an empty string may cause validation issues. Consider starting with an empty array [] and handling the empty state in the MultiEmail component instead.",
"severity": "Low"
}
]
},
{
"id": "cal_dot_com#14943",
"pr_url": "https://github.com/calcom/cal.com/pull/14943",
"goldens": [
{
"comment": "Using retryCount: reminder.retryCount + 1 reads a possibly stale value and can lose increments under concurrency; consider an atomic increment via Prisma (increment: 1) to avoid race conditions (also applies to the similar update in the catch block).",
"severity": "High"
},
{
"comment": "The deletion logic in scheduleSMSReminders.ts incorrectly deletes non-SMS workflow reminders (e.g., Email, WhatsApp) that have retryCount > 1. This occurs because the retryCount condition within the OR clause for deletion lacks a method: WorkflowMethods.SMS filter, causing it to apply to all reminder types instead of only SMS reminders, which is the intended scope of this function.",
"severity": "High"
}
]
},
{
"id": "cal_dot_com#22345",
"pr_url": "https://github.com/calcom/cal.com/pull/22345",
"goldens": [
{
"comment": "In getBaseConditions(), the else if (filterConditions) and final else branches are unreachable. This is because getAuthorizationConditions() always returns a non-null Prisma.Sql object, making authConditions always truthy, which means only the first two if/else if conditions are ever evaluated.",
"severity": "Low"
},
{
"comment": "Fetching userIdsFromOrg only when teamsFromOrg.length > 0 can exclude org-level members for orgs without child teams; consider deriving from teamIds (which includes orgId) or removing the guard so org-only orgs still include member user bookings.",
"severity": "Medium"
}
]
},
{
"id": "cal_dot_com#22532",
"pr_url": "https://github.com/calcom/cal.com/pull/22532",
"goldens": [
{
"comment": "The updateManyByCredentialId call uses an empty data object, which prevents Prisma's @updatedAt decorator from updating the updatedAt timestamp. This results in inaccurate cache status tracking, as the timestamp isn't updated when the cache is refreshed. To fix this, explicitly set the updatedAt field.",
"severity": "Medium"
},
{
"comment": "logic: macOS-specific sed syntax with empty string after -i flag will fail on Linux systems",
"severity": "Low"
}
]
},
{
"id": "cal_dot_com#7232",
"pr_url": "https://github.com/calcom/cal.com/pull/7232",
"goldens": [
{
"comment": "Asynchronous functions deleteScheduledEmailReminder and deleteScheduledSMSReminder are called without await inside forEach loops. This occurs during booking rescheduling/cancellation, and workflow/workflow step deletion/updates. Consequently, scheduled workflow reminders may not be reliably cancelled, potentially leaving them active.",
"severity": "Medium"
},
{
"comment": "When immediateDelete is true, the deleteScheduledEmailReminder function cancels the SendGrid email but fails to delete the corresponding WorkflowReminder record from the database. This creates orphaned database entries and is inconsistent with the immediateDelete: false path, which marks the record as cancelled. The SendGrid DELETE API call is also omitted in this path.",
"severity": "High"
}
]
},
{
"id": "cal_dot_com#8087",
"pr_url": "https://github.com/calcom/cal.com/pull/8087",
"goldens": [
{
"comment": "Consider adding try-catch around the await to handle import failures gracefully",
"severity": "Low"
},
{
"comment": "The code uses forEach with async callbacks, which causes asynchronous operations (e.g., calendar/video event deletions, payment refunds) to run concurrently without being awaited. This 'fire-and-forget' behavior leads to unhandled promise rejections, race conditions, and incomplete cleanup, as surrounding try-catch blocks cannot properly handle errors from these unawaited promises. Replace forEach with for...of loops or Promise.all() with map() to ensure proper sequential execution and error handling.",
"severity": "Critical"
}
]
},
{
"id": "cal_dot_com#8330",
"pr_url": "https://github.com/calcom/cal.com/pull/8330",
"goldens": [
{
"comment": "Incorrect end time calculation using slotStartTime instead of slotEndTime",
"severity": "Medium"
},
{
"comment": "Using === for dayjs object comparison will always return false as it compares object references, not values. Use .isSame() method instead: dayjs(date.start).add(utcOffset, 'minutes').isSame(dayjs(date.end).add(utcOffset, minutes))",
"severity": "Medium"
}
]
},
{
"id": "grafana#103633",
"pr_url": "https://github.com/grafana/grafana/pull/103633",
"goldens": [
{
"comment": "The Check operation exhibits asymmetric cache trust logic: cached permission grants are trusted and returned immediately, but cached denials from the same permission cache are ignored, leading to a fresh database lookup. This allows stale cached grants to provide access to revoked resources, posing a security risk. ",
"severity": "High"
},
{
"comment": "The test comment says the cached permissions 'allow access', but the map stores false for dashboards:uid:dash1, so checkPermission will still treat this scope as not allowed.",
"severity": "Low"
}
]
},
{
"id": "grafana#106778",
"pr_url": "https://github.com/grafana/grafana/pull/106778",
"goldens": [
{
"comment": "The rendered GrafanaRuleListItem is missing the required key prop for React list items. This can cause rendering issues when the list order changes.",
"severity": "Medium"
},
{
"comment": "RuleActionsButtons is invoked with only promRule, but SilenceGrafanaRuleDrawer inside RuleActionsButtons still depends on a Grafana Ruler rule being present, so for Grafana rules coming from list views the 'Silence notifications' menu entry (now driven by Grafana Prom abilities) will toggle showSilenceDrawer without ever rendering the drawer. This means clicking 'Silence notifications' for these rules has no visible effect, even when abilities indicate silencing is allowed.",
"severity": "High"
}
]
},
{
"id": "grafana#107534",
"pr_url": "https://github.com/grafana/grafana/pull/107534",
"goldens": [
{
"comment": "The applyTemplateVariables method is called with request.filters as the third parameter, but this parameter is not used in the corresponding test setup.",
"severity": "Low"
}
]
},
{
"id": "grafana#76186",
"pr_url": "https://github.com/grafana/grafana/pull/76186",
"goldens": [
{
"comment": "The ContextualLoggerMiddleware methods (QueryData, CallResource, CheckHealth, CollectMetrics) panic when a nil request is received. This occurs because they directly access req.PluginContext (via the instrumentContext function) without first checking if req is nil. This is a regression, as previous middleware layers gracefully handled nil requests.",
"severity": "High"
},
{
"comment": "The traceID is no longer logged for plugin requests. During a refactoring, the tracing import and the logic to extract and add traceID from the context to log parameters were removed from the LoggerMiddleware. The newly introduced ContextualLoggerMiddleware does not add this information, resulting in missing traceID in plugin request logs and impacting debugging and request tracing capabilities.",
"severity": "Low"
}
]
},
{
"id": "grafana#79265",
"pr_url": "https://github.com/grafana/grafana/pull/79265",
"goldens": [
{
"comment": "Race condition: Multiple concurrent requests could pass the device count check simultaneously and create devices beyond the limit. Consider using a database transaction or lock.",
"severity": "High"
},
{
"comment": "Anonymous authentication now fails entirely if anonDeviceService.TagDevice returns ErrDeviceLimitReached. Previously, device tagging was asynchronous and non-blocking. This change prevents anonymous users from authenticating when the device limit is reached.",
"severity": "Medium"
},
{
"comment": "This call won’t compile: dbSession.Exec(args...) is given a []interface{} where the first element is the query, but Exec’s signature requires a first parameter of type string (not an interface{} splat).",
"severity": "Medium"
},
{
"comment": "Returning ErrDeviceLimitReached when no rows were updated is misleading; the device might not exist.",
"severity": "Low"
},
{
"comment": "Time window calculation inconsistency: Using device.UpdatedAt.UTC().Add(-anonymousDeviceExpiration) as the lower bound but device.UpdatedAt as the current time may not match the intended logic. Consider using time.Now().UTC() consistently.",
"severity": "Low"
}
]
},
{
"id": "grafana#80329",
"pr_url": "https://github.com/grafana/grafana/pull/80329",
"goldens": [
{
"comment": "The code uses Error log level for what appears to be debugging information. This will pollute error logs in production. Consider using Debug or Info level instead.",
"severity": "Low"
}
]
},
{
"id": "grafana#90045",
"pr_url": "https://github.com/grafana/grafana/pull/90045",
"goldens": [
{
"comment": "The context is being created with d.Log instead of the log variable that was initialized with additional context values (name, kind, method). This means those values won't be propagated to the logging context.",
"severity": "Medium"
},
{
"comment": "Bug: calling recordLegacyDuration when storage operation fails should be recordStorageDuration.",
"severity": "High"
},
{
"comment": "Inconsistency: using name instead of options.Kind for metrics recording differs from other methods.",
"severity": "Medium"
}
]
},
{
"id": "grafana#90939",
"pr_url": "https://github.com/grafana/grafana/pull/90939",
"goldens": [
{
"comment": "The GetWebAssets function implements an incomplete double-checked locking pattern for caching web assets. The function first checks if the cache is populated using a read lock (RLock), and if the cache is empty, it acquires a write lock to populate it. However, it fails to re-check whether the cache was populated by another goroutine while waiting to acquire the write lock.",
"severity": "Medium"
},
{
"comment": "In addition to the missing double-check, the function has a critical flaw in its error handling: it unconditionally assigns the fetch result to the cache (line 69: entryPointAssetsCache = result) regardless of whether the fetch succeeded or failed. When an error occurs during asset fetching, result is nil, and this nil value overwrites any previously valid cache entry.",
"severity": "High"
}
]
},
{
"id": "grafana#94942",
"pr_url": "https://github.com/grafana/grafana/pull/94942",
"goldens": [
{
"comment": "The enableSqlExpressions function has flawed logic that always returns false, effectively disabling SQL expressions unconditionally:",
"severity": "Critical"
},
{
"comment": "Several methods such as NewInMemoryDB().RunCommands and db.QueryFramesInto return 'not implemented'.",
"severity": "High"
}
]
},
{
"id": "grafana#97529",
"pr_url": "https://github.com/grafana/grafana/pull/97529",
"goldens": [
{
"comment": "A race condition in BuildIndex allows multiple goroutines to concurrently build the same expensive index for the same key. This is caused by moving the b.cacheMu lock from protecting the entire function to only protecting the final cache assignment. ",
"severity": "High"
},
{
"comment": "Calling s.search.TotalDocs() here may race with concurrent index creation: TotalDocs iterates b.cache without synchronization, and the event watcher goroutine started just above could trigger BuildIndex writes concurrently, potentially causing a concurrent map read/write panic.",
"severity": "High"
}
]
},
{
"id": "keycloak#32918",
"pr_url": "https://github.com/keycloak/keycloak/pull/32918",
"goldens": [
{
"comment": "Recursive caching call using session instead of delegate",
"severity": "Critical"
},
{
"comment": "Cleanup reference uses incorrect alias - should be 'idp-alias-' + i instead of 'alias'.",
"severity": "Medium"
}
]
},
{
"id": "keycloak#33832",
"pr_url": "https://github.com/keycloak/keycloak/pull/33832",
"goldens": [
{
"comment": "Returns wrong provider (default keystore instead of BouncyCastle)",
"severity": "High"
},
{
"comment": "Dead code exists where ASN1Encoder instances are created and written to, but their results are immediately discarded. The actual encoding is performed by new ASN1Encoder instances created in the subsequent return statement, rendering the earlier operations useless.",
"severity": "Low"
}
]
},
{
"id": "keycloak#36880",
"pr_url": "https://github.com/keycloak/keycloak/pull/36880",
"goldens": [
{
"comment": "Inconsistent feature flag bug causing orphaned permissions. The AdminPermissions event listener, responsible for cleaning up permissions upon role, client, or group removal, is incorrectly guarded by the ADMIN_FINE_GRAINED_AUTHZ (V1) feature flag. This is inconsistent with other methods in the class that use ADMIN_FINE_GRAINED_AUTHZ_V2. Consequently, if ADMIN_FINE_GRAINED_AUTHZ_V2 is enabled but V1 is not, the permission cleanup logic will not execute, leading to orphaned permission data. Cleanup should occur regardless of which fine-grained authorization version is enabled.",
"severity": "High"
},
{
"comment": "In hasPermission(ClientModel client, String scope), the resource lookup uses findByName(server, client.getId(), server.getId()), but AdminPermissionsSchema.getOrCreateResource creates per-client resources with the owner set to resourceServer.getClientId(), so this lookup will never find those resources and will always fall back to the 'all-clients' resource, effectively ignoring client-specific permissions.",
"severity": "High"
},
{
"comment": "In getClientsWithPermission(String scope), iterating resourceStore.findByType(server, AdminPermissionsSchema.CLIENTS_RESOURCE_TYPE) and returning resource.getName() will only ever consider the type-level 'Clients' resource (per-client resources have no type) and return its name, while AvailableRoleMappingResource#getRoleIdsWithPermissions expects actual client IDs to pass to realm.getClientById, which can lead to incorrect behavior or a null client and subsequent failures.",
"severity": "High"
}
]
},
{
"id": "keycloak#36882",
"pr_url": "https://github.com/keycloak/keycloak/pull/36882",
"goldens": [
{
"comment": "Incorrect method call for exit codes. The picocli.exit() method calls System.exit() directly, which is problematic:",
"severity": "Medium"
}
]
},
{
"id": "keycloak#37038",
"pr_url": "https://github.com/keycloak/keycloak/pull/37038",
"goldens": [
{
"comment": "Incorrect permission check in canManage() method",
"severity": "High"
},
{
"comment": "In getGroupIdsWithViewPermission, hasPermission is called with groupResource.getId() and the same groupResource.getId() is added to granted, but hasPermission resolves resources by name (treating the argument as a group id) and the GroupPermissionEvaluator contract says this method returns group IDs that are later used as UserModel.GROUPS and in getUsersCount group filters. This mismatch means per-group VIEW_MEMBERS/MANAGE_MEMBERS permissions may not yield the expected group IDs for filtering and counts, and evaluation may effectively only look at the type-level 'all-groups' resource; consider revisiting whether this should operate on the underlying group ids (resource names) instead so it aligns with the JPA queries and the interface contract.",
"severity": "High"
}
]
},
{
"id": "keycloak#37429",
"pr_url": "https://github.com/keycloak/keycloak/pull/37429",
"goldens": [
{
"comment": "The translation is in Italian instead of Lithuanian. This should be translated to Lithuanian to match the file's locale (messages_lt.properties).",
"severity": "Medium"
},
{
"comment": "The totpStep1 value uses Traditional Chinese terms in the Simplified Chinese file (zh_CN), which is likely incorrect for this locale. Please verify the locale‑appropriate translation.",
"severity": "Medium"
},
{
"comment": "The anchor sanitization logic has a potential issue where it consumes English matcher groups without proper validation. If the translated text has more anchor tags than the English text, this could lead to incorrect validation results.",
"severity": "Low"
},
{
"comment": "The method name 'santizeAnchors' should be 'sanitizeAnchors' (missing 'i').",
"severity": "Low"
}
]
},
{
"id": "keycloak#37634",
"pr_url": "https://github.com/keycloak/keycloak/pull/37634",
"goldens": [
{
"comment": "Wrong parameter in null check (grantType vs. rawTokenId)",
"severity": "Critical"
},
{
"comment": "In isAccessTokenId, the substring for the grant shortcut and the equality check look inverted: the grant shortcut occupies indices 4–5 (substring(4,6)), and a match should return true (combined with UUID check), not false.",
"severity": "High"
},
{
"comment": "Javadoc mentions \"usually like 3-letters shortcut\" but some implementations use 2-letter shortcuts (\"ac\", \"cc\", \"rt\", \"te\", \"pc\", \"ci\", \"ro\"). Consider updating documentation to reflect actual usage pattern.",
"severity": "Low"
},
{
"comment": " Catching generic RuntimeException is too broad. The implementation throws IllegalArgumentException specifically - catch that instead for more precise testing.",
"severity": "Low"
}
]
},
{
"id": "keycloak#38446",
"pr_url": "https://github.com/keycloak/keycloak/pull/38446",
"goldens": [
{
"comment": "Unsafe raw List deserialization without type safety. Calling Optional.get() directly on the Optional returned by RecoveryAuthnCodesUtils.getCredential(user) without checking isPresent() can lead to a NoSuchElementException if the Optional is empty.",
"severity": "Medium"
},
{
"comment": "After creating the RecoveryAuthnCodesCredentialModel, consider setting its id from the stored credential (e.g., myUser.recoveryCodes.getId()); otherwise getId() will be null and downstream removal by id (e.g., removeStoredCredentialById in the authenticator flow) may not work.",
"severity": "Low"
}
]
},
{
"id": "keycloak#40940",
"pr_url": "https://github.com/keycloak/keycloak/pull/40940",
"goldens": [
{
"comment": "Returning null from getSubGroupsCount() violates the GroupModel contract (Javadoc says it never returns null) and may lead to NPEs in callers that expect a non-null count.",
"severity": "Critical"
},
{
"comment": "The reader thread isn’t waited for; flipping deletedAll to true and asserting immediately can race and miss exceptions added just after the flag change, making this test flaky.",
"severity": "Medium"
}
]
},
{
"id": "keycloak#41249",
"pr_url": "https://github.com/keycloak/keycloak/pull/41249",
"goldens": [
{
"comment": "ConditionalPasskeysEnabled() called without UserModel parameter",
"severity": "Medium"
},
{
"comment": "With isConditionalPasskeysEnabled(UserModel user) requiring user != null, authenticate(...) will not call webauthnAuth.fillContextForm(context) on the initial login page where context.getUser() is still null, so conditional passkey UI will not be set up for first-time passkey login. Consider whether this should also be enabled when no user has been selected yet so ID-less passkey authentication on the initial login form continues to work.",
"severity": "Medium"
}
]
},
{
"id": "sentry#67876",
"pr_url": "https://github.com/getsentry/sentry/pull/67876",
"goldens": [
{
"comment": "Null reference if github_authenticated_user state is missing",
"severity": "Medium"
},
{
"comment": "OAuth state uses pipeline.signature (static) instead of a per-request random value",
"severity": "Medium"
},
{
"comment": "The code attempts to access integration.metadata[sender][login] without checking for the existence of the sender key. This causes a KeyError for integrations where the sender metadata was not set during creation",
"severity": "High"
}
]
},
{
"id": "sentry#77754",
"pr_url": "https://github.com/getsentry/sentry/pull/77754",
"goldens": [
{
"comment": "Shared mutable default in dataclass timestamp",
"severity": "Medium"
},
{
"comment": "The method name has a typo: test_from_dict_inalid_data should be test_from_dict_invalid_data.",
"severity": "Low"
},
{
"comment": "Method name says 'empty_array' but tests empty dict - consider renaming to 'test_from_dict_empty_dict' for clarity.",
"severity": "Low"
},
{
"comment": "to_dict() returns a datetime for queued; if this dict is passed in task kwargs (e.g., via apply_async), JSON serialization may fail depending on the serializer, which can cause enqueue errors.",
"severity": "Medium"
}
]
},
{
"id": "sentry#80168",
"pr_url": "https://github.com/getsentry/sentry/pull/80168",
"goldens": [
{
"comment": "MetricAlertDetectorHandler inherits from StatefulDetectorHandler but only contains pass, failing to implement its required abstract methods: counter_names (property), get_dedupe_value(), get_group_key_values(), and build_occurrence_and_event_data(). This will cause a TypeError at runtime when the class is instantiated.",
"severity": "High"
},
{
"comment": "Docstring says this returns a list of DetectorEvaluationResult, but the method now returns a dict keyed by DetectorGroupKey. Consider updating the docstring to match the new return type.",
"severity": "Low"
}
]
},
{
"id": "sentry#80528",
"pr_url": "https://github.com/getsentry/sentry/pull/80528",
"goldens": [
{
"comment": "The function modifies the config variable to include display values but then returns the original monitor.config instead of the modified version.",
"severity": "High"
},
{
"comment": "The code fetches MonitorCheckIn objects by ID when the required data already exists in previous_checkins. This creates an unnecessary database query.",
"severity": "Low"
}
]
},
{
"id": "sentry#92393",
"pr_url": "https://github.com/getsentry/sentry/pull/92393",
"goldens": [
{
"comment": "OptimizedCursorPaginator negative-offset branch slices QuerySet with a negative start index",
"severity": "Critical"
},
{
"comment": "BasePaginator negative-offset branch slices QuerySet with a negative start index",
"severity": "High"
},
{
"comment": "OptimizedCursorPaginator.get_item_key uses floor/ceil on a datetime key (order_by='-datetime'), causing TypeError.",
"severity": "High"
}
]
},
{
"id": "sentry#93824",
"pr_url": "https://github.com/getsentry/sentry/pull/93824",
"goldens": [
{
"comment": "Inconsistent metric tagging with 'shard' and 'shards'",
"severity": "Medium"
},
{
"comment": "Fixed sleep in tests can be flaky; wait on condition instead",
"severity": "Low"
},
{
"comment": "Because flusher processes are created via multiprocessing.get_context('spawn').Process, they are instances of multiprocessing.context.SpawnProcess, which on POSIX is not a subclass of multiprocessing.Process, so this isinstance check will always be false and hung processes won't be killed here.",
"severity": "High"
},
{
"comment": "Sleep in test_consumer.py won’t actually wait because time.sleep was monkeypatched above; consider restoring sleep or using a different sync to ensure the flusher has time to process.",
"severity": "Medium"
},
{
"comment": "Breaking out of the loop when the deadline has elapsed can skip terminating remaining flusher processes, potentially leaving them running after shutdown; consider ensuring termination is attempted even if the deadline is exceeded.",
"severity": "Medium"
}
]
},
{
"id": "sentry#94376",
"pr_url": "https://github.com/getsentry/sentry/pull/94376",
"goldens": [
{
"comment": "sample_rate = 0.0 is falsy and skipped",
"severity": "Low"
},
{
"comment": "Using Python’s built-in hash() to build cache keys is non-deterministic across processes (hash randomization), so keys won’t match across workers and invalidate_upsampling_cache may fail to delete them. Use a deterministic serialization of project_ids for the cache key.",
"severity": "Low"
},
{
"comment": "The upsampling eligibility check passes the outer dataset instead of the actual dataset used by scoped_dataset. In paths where the query ultimately runs against discover (e.g., dashboard split) while the original dataset is metrics, upsampling may be skipped even when all projects are allowlisted.",
"severity": "Medium"
}
]
},
{
"id": "sentry#95633",
"pr_url": "https://github.com/getsentry/sentry/pull/95633",
"goldens": [
{
"comment": "The queue.shutdown() method with 'immediate=False' parameter may not exist in the standard Python queue module. This could cause AttributeError at runtime. Verify the correct API or implement a custom shutdown mechanism.",
"severity": "High"
},
{
"comment": "The magic number 50 for max_wait is used repeatedly throughout the tests. Consider extracting this as a named constant to improve maintainability.",
"severity": "Low"
},
{
"comment": "The test test_thread_queue_parallel_error_handling has a docstring that doesn't match the test implementation.",
"severity": "Low"
}
]
}
]
+199
View File
@@ -0,0 +1,199 @@
// Package prbench runs the PR reviewer benchmark: the Martian Code-Review-Bench
// problems, the production review pipeline, and a judge model, all on the
// providers and model aliases the instance already configures.
package prbench
import (
"context"
_ "embed"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"log/slog"
"os"
"os/signal"
"path/filepath"
"slices"
"strings"
"syscall"
"time"
"github.com/samcharles93/archie-core/internal/agentexec"
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
"github.com/samcharles93/archie-core/internal/app/servicekit"
"github.com/samcharles93/archie-core/internal/buildinfo"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview/bench"
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
"github.com/samcharles93/archie-core/internal/infrastructure/prsource"
)
// acceptanceRecall is the PRD's golden micro-recall bar for a full run.
const acceptanceRecall = 0.70
// martian holds the 38 runnable problems of the Martian Code-Review-Bench
// offline set: those whose pull request still exists upstream.
//
//go:embed martian.json
var martian []byte
// Problems returns the benchmark problems, optionally narrowed to ids.
func Problems(ids []string) ([]bench.Problem, error) {
var all []bench.Problem
if err := json.Unmarshal(martian, &all); err != nil {
return nil, fmt.Errorf("decode benchmark problems: %w", err)
}
if len(ids) == 0 {
return all, nil
}
var out []bench.Problem
for _, id := range ids {
i := slices.IndexFunc(all, func(p bench.Problem) bool { return p.ID == id })
if i < 0 {
return nil, fmt.Errorf("no benchmark problem %q", id)
}
out = append(out, all[i])
}
return out, nil
}
// Options configures one benchmark run. Model and Judge each name a model
// alias of the config or a provider/model ref; empty means the "default" alias.
type Options struct {
Config string
Model string
Judge string
OutDir string
Problems []string
Concurrency int
}
// Run reviews and judges the selected problems, writes the report into
// Options.OutDir, and prints it to w.
func Run(ctx context.Context, opts Options, w io.Writer) (bench.Summary, error) {
problems, err := Problems(opts.Problems)
if err != nil {
return bench.Summary{}, err
}
doc, err := configuration.New(nil).File(opts.Config)
if err != nil {
return bench.Summary{}, err
}
cfg := doc.Config
if err := servicekit.ResolveProviders(&cfg, slog.Default()); err != nil {
return bench.Summary{}, err
}
reviewRef, err := resolveRef(cfg.Models, opts.Model)
if err != nil {
return bench.Summary{}, err
}
judgeRef, err := resolveRef(cfg.Models, opts.Judge)
if err != nil {
return bench.Summary{}, err
}
rt := modelloop.NewRuntime(agentexec.ProvidersFromConfig(cfg.Providers), cfg.ModelLimits)
if rt == nil {
return bench.Summary{}, fmt.Errorf("%s configures no providers", opts.Config)
}
// The pipeline resolves its model through the default alias.
cfg.Models = map[string]string{config.DefaultModelAlias: reviewRef}
cfg.Review.ApproveBeforePost = false
cfg.Review.PrecisionGate = false
reviewer := &liveReviewer{cfg: cfg, agent: modelloop.NewLoopRunner(rt, slog.Default()), source: prsource.New(newPublicGitHub())}
judge := &modelJudge{runtime: rt, modelRef: judgeRef}
results := bench.Run(ctx, problems, reviewer, judge, opts.Concurrency)
summary := bench.Summarize(results)
report := render(results, summary, reviewRef, judgeRef, len(opts.Problems) == 0)
path, err := writeReport(opts.OutDir, report)
if err != nil {
return summary, err
}
fmt.Fprint(w, report)
fmt.Fprintf(w, "\nreport written to %s\n", path)
if summary.Failed != 0 {
return summary, fmt.Errorf("%d benchmark problem(s) failed; results are incomplete", summary.Failed)
}
return summary, nil
}
// Main is the `archied prbench` subcommand.
func Main(args []string, stdout, stderr io.Writer) int {
var opts Options
var problems string
flags := flag.NewFlagSet("archied prbench", flag.ContinueOnError)
flags.SetOutput(stderr)
flags.StringVar(&opts.Config, "config", configuration.DefaultConfigPath(), "config file whose providers and model aliases serve the run")
flags.StringVar(&opts.Model, "model", "", "model alias or provider/model for the reviewer (default: the default alias)")
flags.StringVar(&opts.Judge, "judge-model", "", "model alias or provider/model for the judge (default: the default alias; use a different model from the reviewer's for an independent judge)")
flags.StringVar(&opts.OutDir, "out", "docs/benchmarks", "directory the report is written to")
flags.StringVar(&problems, "problems", "", "comma-separated problem ids for a partial run (default: all 38)")
flags.IntVar(&opts.Concurrency, "concurrency", 4, "problems reviewed at once")
if err := flags.Parse(args); err != nil {
if errors.Is(err, flag.ErrHelp) {
return 0
}
return 2
}
if problems != "" {
opts.Problems = strings.Split(problems, ",")
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
if _, err := Run(ctx, opts, stdout); err != nil {
fmt.Fprintln(stderr, "archied prbench:", err)
return 1
}
return 0
}
// resolveRef accepts a provider/model ref as is and otherwise looks the alias up.
func resolveRef(aliases map[string]string, v string) (string, error) {
v = strings.TrimSpace(v)
if strings.Contains(v, "/") {
return v, nil
}
return config.ResolveModel(aliases, v, config.PurposeAgent)
}
func render(results []bench.Result, s bench.Summary, reviewRef, judgeRef string, full bool) string {
var b strings.Builder
scope, verdict := "partial run (not the acceptance run)", ""
if full {
scope = "full run"
verdict = fmt.Sprintf("\nAcceptance (recall >= %.2f): %s\n", acceptanceRecall, passFail(s.Failed == 0 && s.Recall >= acceptanceRecall))
}
fmt.Fprintf(&b, "# PR review benchmark\n\nDate: %s\nArchie version: %s\nReview model: %s\nJudge model: %s\nScope: %s\n\n",
time.Now().UTC().Format("2006-01-02"), buildinfo.Version, reviewRef, judgeRef, scope)
fmt.Fprintf(&b, "Problems %d (failed %d) recall %.3f (%d/%d) golden-only precision %.3f (%d/%d)\n%s\n",
s.Problems, s.Failed, s.Recall, s.Hits, s.Goldens, s.Precision, s.Matched, s.Comments, verdict)
b.WriteString("| Problem | Recall | Comments | Notes |\n| --- | --- | --- | --- |\n")
for _, r := range results {
recall, notes := fmt.Sprintf("%d/%d", r.Hits(), len(r.Problem.Goldens)), ""
switch {
case r.Err != "":
recall, notes = "-", "failed: "+strings.ReplaceAll(r.Err, "|", "/")
case len(r.SkippedPhases) != 0:
notes = "skipped phases: " + strings.Join(r.SkippedPhases, ", ")
}
fmt.Fprintf(&b, "| %s | %s | %d | %s |\n", r.Problem.ID, recall, len(r.Comments), notes)
}
return b.String()
}
func passFail(ok bool) string {
if ok {
return "PASS"
}
return "FAIL"
}
func writeReport(dir, report string) (string, error) {
if err := os.MkdirAll(dir, 0o755); err != nil {
return "", err
}
path := filepath.Join(dir, "pr-review-"+time.Now().UTC().Format("20060102-150405")+".md")
return path, os.WriteFile(path, []byte(report), 0o644)
}
@@ -0,0 +1,32 @@
package statestore
import (
"context"
"time"
"github.com/samcharles93/archie-core/internal/domain/storepkg"
)
// packageUpdateInterval is how often auto packages are checked against the
// catalogue. Catalogue releases are infrequent, so hours is enough.
const packageUpdateInterval = 6 * time.Hour
// autoUpdatePackages updates auto packages now and then every interval until
// ctx ends. A failed tick is logged and retried on the next one.
func (b *server) autoUpdatePackages(ctx context.Context, packages storepkg.Manager) {
if packages == nil {
return
}
ticker := time.NewTicker(packageUpdateInterval)
defer ticker.Stop()
for {
if err := packages.UpdateAuto(ctx); err != nil && ctx.Err() == nil {
b.log.Error("auto-update packages", "err", err)
}
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}
+2 -9
View File
@@ -26,11 +26,7 @@ type guardedPolicies struct {
}
func (g guardedPolicies) PutPolicy(ctx context.Context, p access.Policy) (int64, error) {
validator, err := accessengine.New(nil)
if err != nil {
return 0, err
}
if err := validator.Validate(p); err != nil {
if err := accessengine.New(nil).Validate(p); err != nil {
return 0, err
}
if err := g.checkLockout(ctx, func(current []access.Policy) []access.Policy {
@@ -55,10 +51,7 @@ func (g guardedPolicies) checkLockout(ctx context.Context, change func([]access.
if err != nil {
return err
}
engine, err := accessengine.New(change(current))
if err != nil {
return err
}
engine := accessengine.New(change(current))
orgs, err := g.owners.ListOrgs(ctx)
if err != nil {
return err
+17 -5
View File
@@ -26,6 +26,7 @@ import (
"github.com/samcharles93/archie-core/internal/domain/presence"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/storepkg"
"github.com/samcharles93/archie-core/internal/domain/usage"
infraaccess "github.com/samcharles93/archie-core/internal/infrastructure/access"
"github.com/samcharles93/archie-core/internal/infrastructure/postgres"
"github.com/samcharles93/archie-core/internal/infrastructure/readiness"
@@ -142,7 +143,8 @@ func Run(ctx context.Context, options Options) error { //nolint:cyclop // the co
b.announceReady()
deps := b.stateStoreDeps(grants)
deps.ControlPlane = control
controlPlaneDeps(&deps, control)
go b.autoUpdatePackages(ctx, deps.Packages)
return serveStateStore(ctx, listener, deps, opts)
}
@@ -180,10 +182,7 @@ func (b *server) seedAndValidatePolicies(ctx context.Context) error {
if err != nil {
return fmt.Errorf("load stored policies: %w", err)
}
engine, err := infraaccess.New(stored)
if err != nil {
return fmt.Errorf("validate stored policies: %w", err)
}
engine := infraaccess.New(stored)
for _, problem := range engine.Problems() {
b.log.Error("stored access policy is invalid and denies its level",
"policy", problem.Policy.ID, "level", problem.Policy.Level, "err", problem.Err)
@@ -375,6 +374,9 @@ func (b *server) executionDeps(deps *staterpc.Deps) {
if ec, ok := b.st.(storecontract.ExecutionCanceller); ok {
deps.Canceller = ec
}
if ur, ok := b.st.(usage.Recorder); ok {
deps.Usage = ur
}
if sr, ok := b.st.(storecontract.StepReader); ok {
deps.StepReader = sr
}
@@ -581,3 +583,13 @@ func newPackageProjections(pool *pgxpool.Pool) (map[string]storepkg.FamilyProjec
storepkg.FamilyProfiles: profiles,
}, nil
}
// controlPlaneDeps serves the control plane and the per-org model answers
// built on it.
func controlPlaneDeps(deps *staterpc.Deps, control *controlplane.Server) {
deps.ControlPlane = control
deps.OrgModels = func(ctx context.Context, orgID string) (map[string]string, map[string]config.Provider, error) {
models, err := control.ModelsFor(ctx, orgID)
return models.Aliases, models.Providers, err
}
}
+1
View File
@@ -27,6 +27,7 @@ var gatewayCommandSpecs = []commandSpec{
{Command: "personality", Description: "Choose a communication style", Usage: "/personality [name]"},
{Command: "spawn", Description: "Create a tracked task", Usage: "/spawn [identity=name] [repo=owner/name] [workflow=name] <title>"},
{Command: "settings", Description: "Inspect or change runtime settings", Usage: "/settings [list|get <kind>|set <kind> <field>=<value> ...]"},
{Command: "approve", Description: "Approve a waiting task", Usage: "/approve [identity=name] <task-id>"},
{Command: "cancel", Description: "Cancel a queued or waiting task", Usage: "/cancel [identity=name] <task-id>"},
{Command: "start", Description: "Confirm that Archie is running", Usage: "/start"},
{Command: "new", Description: "Start a fresh session, clearing conversation history", Usage: "/new [title]"},
+4 -11
View File
@@ -5,7 +5,8 @@ import (
"crypto/rand"
"encoding/hex"
"fmt"
"sort"
"maps"
"slices"
"strings"
"time"
@@ -236,11 +237,7 @@ func formatInstallFailure(err error) string {
}
func formatVersionChanges(result releaseupdate.Result) string {
ids := make([]string, 0, len(result.Installed))
for id := range result.Installed {
ids = append(ids, id)
}
sort.Strings(ids)
ids := slices.Sorted(maps.Keys(result.Installed))
parts := make([]string, 0, len(ids))
for _, id := range ids {
installed := result.Installed[id]
@@ -266,11 +263,7 @@ func formatPendingReport(report releaseupdate.Report, running map[string]string)
return formatHealthyReport(report, report.Verify(running))
}
ids := make([]string, 0, len(report.Previous))
for id := range report.Previous {
ids = append(ids, id)
}
sort.Strings(ids)
ids := slices.Sorted(maps.Keys(report.Previous))
versions := make([]string, 0, len(ids))
for _, id := range ids {
versions = append(versions, fmt.Sprintf("%s %s", id, report.Previous[id]))
+3 -8
View File
@@ -348,8 +348,8 @@ type Config struct {
Dispatch Dispatch `toml:"dispatch" yaml:"dispatch"`
Review Review `toml:"review" yaml:"review"`
// Models maps a role ("planner", "builder", "triage") to a runtime
// model ref ("provider/model").
// Models maps a model alias ("default", "fast", "embedding") to a
// runtime model ref ("provider/model"). Resolve it with ResolveModel.
Models map[string]string `toml:"models" yaml:"models"`
Providers map[string]Provider `toml:"providers" yaml:"providers"`
@@ -513,7 +513,6 @@ func (c Config) Clone() Config {
c.Providers = maps.Clone(c.Providers)
c.Repos = cloneRepos(c.Repos)
c.Identities = cloneIdentities(c.Identities)
c.Chat.Models = append([]string(nil), c.Chat.Models...)
c.Chat.Telegram.AllowedUserIDs = append([]int64(nil), c.Chat.Telegram.AllowedUserIDs...)
c.Bindings.PreviousEncryptionKeys = append([]SecretRef(nil), c.Bindings.PreviousEncryptionKeys...)
c.Tools.MCPServers = cloneMCPServers(c.Tools.MCPServers)
@@ -864,11 +863,7 @@ type ChatConfig struct {
// channel. Off by default.
ShowToolCalls bool `toml:"show_tool_calls" yaml:"show_tool_calls"`
// MaxSteps caps model/tool round-trips per chat turn. Zero uses the default.
MaxSteps int `toml:"max_steps" yaml:"max_steps"`
// Models is the optional interactive-chat model catalog. When empty,
// chat falls back to the distinct model references assigned to workflow
// roles in the top-level [models] table.
Models []string `toml:"models" yaml:"models"`
MaxSteps int `toml:"max_steps" yaml:"max_steps"`
Telegram TelegramConfig `toml:"telegram" yaml:"telegram"`
// RateLimit budgets inbound messages per channel and sender. Off unless set.
// Seeds the channel-settings resource.
+82
View File
@@ -0,0 +1,82 @@
package config
import (
"fmt"
"maps"
"slices"
"strings"
)
// ModelPurpose is what a model is used for. Each purpose has a default alias
// that an empty alias resolves to.
type ModelPurpose string
const (
PurposeAgent ModelPurpose = "agent"
PurposeChat ModelPurpose = "chat"
PurposeEmbedding ModelPurpose = "embedding"
PurposeTranscription ModelPurpose = "transcription"
)
// DefaultModelAlias is the alias agent steps and chat use when none is named.
const DefaultModelAlias = "default"
// DefaultAlias is the alias an empty alias means for purpose.
func (p ModelPurpose) DefaultAlias() string {
switch p {
case PurposeEmbedding, PurposeTranscription:
return string(p)
default:
return DefaultModelAlias
}
}
// ResolveModel returns the "provider/model" ref aliases maps alias to, or
// purpose's default alias when alias is empty. An unknown alias is an error,
// never a substitute.
func ResolveModel(aliases map[string]string, alias string, purpose ModelPurpose) (string, error) {
alias = strings.TrimSpace(alias)
if alias == "" {
alias = purpose.DefaultAlias()
}
ref := strings.TrimSpace(aliases[alias])
if ref == "" {
return "", fmt.Errorf("no model alias %q is configured (Settings > Models)", alias)
}
return ref, nil
}
// ChatAliases are the aliases chat may switch between: every alias but the
// embedding and transcription ones, sorted. Never nil.
func ChatAliases(aliases map[string]string) []string {
out := make([]string, 0, len(aliases))
for _, alias := range slices.Sorted(maps.Keys(aliases)) {
if alias != string(PurposeEmbedding) && alias != string(PurposeTranscription) {
out = append(out, alias)
}
}
return out
}
// OrgModels is what one org resolves models against: the aliases it may use
// and its own providers, which are never the instance's.
type OrgModels struct {
Org string
Aliases map[string]string
Providers map[string]Provider
}
// WithOrgModels is c for a run in models.Org: its aliases, and the instance
// providers with the org's own beside them.
func (c Config) WithOrgModels(models OrgModels) Config {
c.Models = models.Aliases
if len(models.Providers) > 0 {
providers := maps.Clone(c.Providers)
if providers == nil {
providers = map[string]Provider{}
}
maps.Copy(providers, models.Providers)
c.Providers = providers
}
return c
}
+85 -5
View File
@@ -9,6 +9,7 @@ import (
"path"
"strings"
"github.com/docker/sandbox-kit-spec/v3/spec"
"github.com/moby/moby/api/pkg/stdcopy"
"github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/mount"
@@ -81,10 +82,8 @@ func StartKit(ctx context.Context, cli *client.Client, s KitSpec) (string, error
if _, err := cli.ContainerStart(ctx, created.ID, client.ContainerStartOptions{}); err != nil {
return fail(fmt.Errorf("start kit container: %w", err))
}
for i, h := range s.Launch.Install {
if err := runHook(ctx, cli, created.ID, h); err != nil {
return fail(fmt.Errorf("install hook %d: %w", i+1, err))
}
if err := installKit(ctx, cli, created.ID, s.Launch); err != nil {
return fail(err)
}
if s.InstallDone != nil {
s.InstallDone()
@@ -95,10 +94,15 @@ func StartKit(ctx context.Context, cli *client.Client, s KitSpec) (string, error
}
}
for _, f := range s.Launch.Context {
if err := writeKitFile(ctx, cli, created.ID, "0", f.Path, f.Content, f.Mode, nil); err != nil {
if err := writeKitContext(ctx, cli, created.ID, f); err != nil {
return fail(fmt.Errorf("write agent context %s: %w", f.Path, err))
}
}
for _, skill := range s.Launch.Skills {
if err := stageKitSkill(ctx, cli, created.ID, s.Launch.Harness.User, skill); err != nil {
return fail(err)
}
}
for i, h := range s.Launch.Startup {
if err := runHook(ctx, cli, created.ID, h); err != nil {
return fail(fmt.Errorf("startup hook %d: %w", i+1, err))
@@ -107,6 +111,82 @@ func StartKit(ctx context.Context, cli *client.Client, s KitSpec) (string, error
return created.ID, nil
}
func installKit(ctx context.Context, cli *client.Client, id string, launch kit.Launch) error {
for _, f := range launch.InstallFiles {
if err := writeKitFile(ctx, cli, id, launch.Harness.User, f.Path, f.Content, f.Mode, f.Overwrite); err != nil {
return fmt.Errorf("write install credential file %s: %w", f.Path, err)
}
}
for i, h := range launch.Install {
if err := runHook(ctx, cli, id, h); err != nil {
return fmt.Errorf("install hook %d: %w", i+1, err)
}
}
for _, f := range launch.InstallFiles {
code, out, err := ExecIn(ctx, cli, id, launch.Harness.User, []string{"KIT_PATH=" + f.Path}, []string{"sh", "-c", `rm -f -- "$KIT_PATH"`})
if err != nil {
return fmt.Errorf("remove install credential file %s: %w", f.Path, err)
}
if code != 0 {
return fmt.Errorf("remove install credential file %s: exit %d: %s", f.Path, code, out)
}
}
return nil
}
func stageKitSkill(ctx context.Context, cli *client.Client, id, user string, skill kit.Skill) error {
code, out, err := ExecIn(ctx, cli, id, "0", []string{"SKILL_SOURCE=" + skill.Source, "SKILL_TARGET=" + skill.Target, "SKILL_USER=" + user}, []string{"sh", "-ec", `
[ -d "$SKILL_SOURCE" ] && [ -f "$SKILL_SOURCE/SKILL.md" ] || exit 1
[ "$SKILL_SOURCE" != "$SKILL_TARGET" ] || exit 0
at="$SKILL_TARGET"
while [ "$at" != / ]; do
[ ! -L "$at" ] || exit 1
at=$(dirname "$at")
done
[ ! -e "$SKILL_TARGET" ] || exit 0
mkdir -p "$(dirname "$SKILL_TARGET")"
cp -R "$SKILL_SOURCE" "$SKILL_TARGET"
chown -R "$SKILL_USER" "$SKILL_TARGET"
`})
if err != nil {
return fmt.Errorf("stage bundled skill %s: %w", skill.Source, err)
}
if code != 0 {
return fmt.Errorf("stage bundled skill %s: exit %d: %s", skill.Source, code, out)
}
return nil
}
// Replace only Archie's managed guidance, retaining the image's instructions.
func writeKitContext(ctx context.Context, cli *client.Client, id string, f spec.File) error {
code, out, err := ExecIn(ctx, cli, id, "0", []string{"KIT_PATH=" + f.Path, "KIT_CONTENT=" + base64.StdEncoding.EncodeToString([]byte(f.Content))}, []string{"sh", "-ec", `
at="$KIT_PATH"
while [ "$at" != / ]; do
[ ! -L "$at" ] || exit 1
at=$(dirname "$at")
done
[ ! -e "$KIT_PATH" ] || [ -f "$KIT_PATH" ] || exit 1
mkdir -p "$(dirname "$KIT_PATH")"
tmp=$(mktemp "$KIT_PATH.XXXXXX")
trap 'rm -f "$tmp"' 0
if [ -f "$KIT_PATH" ]; then
awk '/^<!-- BEGIN ARCHIE -->$/ { managed=1; next } /^<!-- END ARCHIE -->$/ { managed=0; next } !managed { print }' "$KIT_PATH" > "$tmp"
fi
printf '%s\n' '<!-- BEGIN ARCHIE -->' >> "$tmp"
printf %s "$KIT_CONTENT" | base64 -d >> "$tmp"
printf '\n%s\n' '<!-- END ARCHIE -->' >> "$tmp"
chmod 0644 "$tmp"
mv "$tmp" "$KIT_PATH"
`})
if err != nil {
return err
}
if code != 0 {
return fmt.Errorf("exited %d: %s", code, out)
}
return nil
}
// RemoveKitVolumes removes an execution's Kit volumes once the execution
// has ended; a retry of the same execution reuses them until then.
func RemoveKitVolumes(ctx context.Context, cli *client.Client, volumes []kit.Volume) error {
+82
View File
@@ -0,0 +1,82 @@
package container
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/docker/sandbox-kit-spec/v3/spec"
"github.com/moby/moby/client"
"github.com/samcharles93/archie-core/internal/domain/agentrun"
"github.com/samcharles93/archie-core/internal/infrastructure/kit"
)
func TestKitRuntimeDiscovery(t *testing.T) {
if testing.Short() {
t.Skip("requires Docker")
}
cli, err := client.New(client.FromEnv)
if err != nil {
t.Fatal(err)
}
defer cli.Close()
shared := t.TempDir()
if err := os.Chmod(shared, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(shared, "SKILL.md"), []byte("shared body"), 0o644); err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
name string
symlink bool
}{
{name: "preserves and replaces managed guidance"},
{name: "refuses symlink guidance", symlink: true},
} {
t.Run(tc.name, func(t *testing.T) {
script := `test -f /tmp/install.json; mkdir -p /tmp/discovery /tmp/bundle; printf 'SKILL body' > /tmp/bundle/SKILL.md; printf 'image instructions\n' > /tmp/discovery/AGENTS.md`
if tc.symlink {
script += `; mv /tmp/discovery/AGENTS.md /tmp/other; ln -s /tmp/other /tmp/discovery/AGENTS.md`
}
launch := kit.Launch{
Harness: agentrun.HarnessSpec{User: "0"}, Install: []kit.Hook{{User: "0", Argv: []string{"sh", "-ec", script}}}, InstallFiles: []spec.File{{Path: "/tmp/install.json", Content: "install sentinel"}},
Files: []spec.File{{Path: "/tmp/runtime.json", Content: "runtime sentinel"}},
Startup: []kit.Hook{{User: "0", Argv: []string{"sh", "-ec", "test ! -e /tmp/install.json; test -f /tmp/runtime.json"}}},
Context: []spec.File{{Path: "/tmp/discovery/AGENTS.md", Content: "first guidance"}}, Skills: []kit.Skill{{Source: "/tmp/bundle", Target: "/tmp/skills/bundle"}},
}
id, err := StartKit(t.Context(), cli, KitSpec{Image: "alpine:3", Worker: []string{"tail", "-f", "/dev/null"}, Binds: []string{shared + ":/tmp/skills/shared:ro,z"}, Launch: launch})
if tc.symlink {
if err == nil {
_, _ = cli.ContainerRemove(context.Background(), id, client.ContainerRemoveOptions{Force: true})
t.Fatal("symlink was followed")
}
return
}
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
_, _ = cli.ContainerRemove(context.Background(), id, client.ContainerRemoveOptions{Force: true})
})
if err := writeKitContext(t.Context(), cli, id, spec.File{Path: "/tmp/discovery/AGENTS.md", Content: "replacement guidance"}); err != nil {
t.Fatal(err)
}
code, out, err := ExecIn(t.Context(), cli, id, "0", nil, []string{"sh", "-ec", "cat /tmp/discovery/AGENTS.md; cat /tmp/skills/bundle/SKILL.md; cat /tmp/skills/shared/SKILL.md; if echo overwrite > /tmp/skills/shared/SKILL.md 2>/dev/null; then exit 1; fi"})
if err != nil || code != 0 {
t.Fatalf("read discovery: %d %q %v", code, out, err)
}
for _, want := range []string{"image instructions", "replacement guidance", "SKILL body", "shared body"} {
if !strings.Contains(out, want) {
t.Errorf("missing %q in %q", want, out)
}
}
if strings.Contains(out, "first guidance") || strings.Count(out, "<!-- BEGIN ARCHIE -->") != 1 {
t.Fatalf("managed guidance was not replaced: %q", out)
}
})
}
}
+53
View File
@@ -0,0 +1,53 @@
package container
import (
"context"
"fmt"
"strconv"
"github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/mount"
"github.com/moby/moby/client"
)
// Reown hands dir back to uid:gid with a throwaway root container. A task
// container that was SIGKILLed never restores ownership of the files it
// created as root, and the non-root daemon has no privilege to do it itself.
func (p *Pool) Reown(ctx context.Context, dir string, uid, gid int) error {
cfg := p.current()
if err := p.ensureImage(ctx, cfg, cfg.Image); err != nil {
return err
}
resp, err := p.cli.ContainerCreate(ctx, client.ContainerCreateOptions{
Config: &container.Config{
Image: cfg.Image,
Entrypoint: []string{"chown", "-R", strconv.Itoa(uid) + ":" + strconv.Itoa(gid), "/w"},
Labels: map[string]string{"archie-daemon": "true"},
},
HostConfig: &container.HostConfig{
NetworkMode: "none",
Mounts: []mount.Mount{{Type: mount.TypeBind, Source: dir, Target: "/w"}},
},
})
if err != nil {
return fmt.Errorf("create reown container: %w", err)
}
defer func() {
_, _ = p.cli.ContainerRemove(context.WithoutCancel(ctx), resp.ID, client.ContainerRemoveOptions{Force: true})
}()
wait := p.cli.ContainerWait(ctx, resp.ID, client.ContainerWaitOptions{Condition: container.WaitConditionNextExit})
if _, err := p.cli.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {
return fmt.Errorf("start reown container: %w", err)
}
select {
case res := <-wait.Result:
if res.StatusCode != 0 {
return fmt.Errorf("reown %s: chown exited %d", dir, res.StatusCode)
}
return nil
case err := <-wait.Error:
return fmt.Errorf("wait for reown container: %w", err)
case <-ctx.Done():
return ctx.Err()
}
}
File diff suppressed because it is too large. Load diff
@@ -27,6 +27,10 @@ const (
StateStoreService_ListInstalledPackages_FullMethodName = "/state.v1.StateStoreService/ListInstalledPackages"
StateStoreService_AcceptPackageAuthority_FullMethodName = "/state.v1.StateStoreService/AcceptPackageAuthority"
StateStoreService_RemoveInstalledPackage_FullMethodName = "/state.v1.StateStoreService/RemoveInstalledPackage"
StateStoreService_UpdatePackage_FullMethodName = "/state.v1.StateStoreService/UpdatePackage"
StateStoreService_ApprovePackageUpdate_FullMethodName = "/state.v1.StateStoreService/ApprovePackageUpdate"
StateStoreService_RollbackPackage_FullMethodName = "/state.v1.StateStoreService/RollbackPackage"
StateStoreService_SetPackageUpdatePolicy_FullMethodName = "/state.v1.StateStoreService/SetPackageUpdatePolicy"
StateStoreService_RegisterTaskGrant_FullMethodName = "/state.v1.StateStoreService/RegisterTaskGrant"
StateStoreService_RevokeTaskGrant_FullMethodName = "/state.v1.StateStoreService/RevokeTaskGrant"
StateStoreService_ResolveTaskGrant_FullMethodName = "/state.v1.StateStoreService/ResolveTaskGrant"
@@ -39,17 +43,23 @@ const (
StateStoreService_RetireIdentity_FullMethodName = "/state.v1.StateStoreService/RetireIdentity"
StateStoreService_ResolveIdentitySubject_FullMethodName = "/state.v1.StateStoreService/ResolveIdentitySubject"
StateStoreService_BindIdentitySubject_FullMethodName = "/state.v1.StateStoreService/BindIdentitySubject"
StateStoreService_ClaimIdentitySubject_FullMethodName = "/state.v1.StateStoreService/ClaimIdentitySubject"
StateStoreService_AddPersonalToken_FullMethodName = "/state.v1.StateStoreService/AddPersonalToken"
StateStoreService_ListPersonalTokens_FullMethodName = "/state.v1.StateStoreService/ListPersonalTokens"
StateStoreService_RevokePersonalToken_FullMethodName = "/state.v1.StateStoreService/RevokePersonalToken"
StateStoreService_EnqueueIssue_FullMethodName = "/state.v1.StateStoreService/EnqueueIssue"
StateStoreService_EnqueueChatTask_FullMethodName = "/state.v1.StateStoreService/EnqueueChatTask"
StateStoreService_ActiveTasksByOrigin_FullMethodName = "/state.v1.StateStoreService/ActiveTasksByOrigin"
StateStoreService_UnpostedOutcomes_FullMethodName = "/state.v1.StateStoreService/UnpostedOutcomes"
StateStoreService_MarkOutcomePosted_FullMethodName = "/state.v1.StateStoreService/MarkOutcomePosted"
StateStoreService_ClaimNext_FullMethodName = "/state.v1.StateStoreService/ClaimNext"
StateStoreService_ClaimByIssue_FullMethodName = "/state.v1.StateStoreService/ClaimByIssue"
StateStoreService_Transition_FullMethodName = "/state.v1.StateStoreService/Transition"
StateStoreService_Update_FullMethodName = "/state.v1.StateStoreService/Update"
StateStoreService_StartStep_FullMethodName = "/state.v1.StateStoreService/StartStep"
StateStoreService_RecordUsage_FullMethodName = "/state.v1.StateStoreService/RecordUsage"
StateStoreService_TaskModels_FullMethodName = "/state.v1.StateStoreService/TaskModels"
StateStoreService_CallerModels_FullMethodName = "/state.v1.StateStoreService/CallerModels"
StateStoreService_FinishStep_FullMethodName = "/state.v1.StateStoreService/FinishStep"
StateStoreService_CancelExecution_FullMethodName = "/state.v1.StateStoreService/CancelExecution"
StateStoreService_ListSteps_FullMethodName = "/state.v1.StateStoreService/ListSteps"
@@ -171,6 +181,16 @@ type StateStoreServiceClient interface {
// Administrative until org-scoped principals land.
AcceptPackageAuthority(ctx context.Context, in *AcceptPackageAuthorityRequest, opts ...grpc.CallOption) (*AcceptPackageAuthorityResponse, error)
RemoveInstalledPackage(ctx context.Context, in *RemoveInstalledPackageRequest, opts ...grpc.CallOption) (*RemoveInstalledPackageResponse, error)
// UpdatePackage moves a package to the digest the verified catalogue lists:
// a non-widening update applies, a widening one waits as pending.
// ApprovePackageUpdate applies the pending update and accepts its authority;
// RollbackPackage restores the previous digest and its accepted authority.
UpdatePackage(ctx context.Context, in *UpdatePackageRequest, opts ...grpc.CallOption) (*UpdatePackageResponse, error)
ApprovePackageUpdate(ctx context.Context, in *ApprovePackageUpdateRequest, opts ...grpc.CallOption) (*ApprovePackageUpdateResponse, error)
RollbackPackage(ctx context.Context, in *RollbackPackageRequest, opts ...grpc.CallOption) (*RollbackPackageResponse, error)
// SetPackageUpdatePolicy sets "manual" or "auto"; the State Store updates
// auto packages on a timer.
SetPackageUpdatePolicy(ctx context.Context, in *SetPackageUpdatePolicyRequest, opts ...grpc.CallOption) (*SetPackageUpdatePolicyResponse, error)
// Administrative task credential lifecycle. Worker credentials cannot call these.
RegisterTaskGrant(ctx context.Context, in *RegisterTaskGrantRequest, opts ...grpc.CallOption) (*RegisterTaskGrantResponse, error)
RevokeTaskGrant(ctx context.Context, in *RevokeTaskGrantRequest, opts ...grpc.CallOption) (*RevokeTaskGrantResponse, error)
@@ -188,6 +208,7 @@ type StateStoreServiceClient interface {
// identity store.
ResolveIdentitySubject(ctx context.Context, in *ResolveIdentitySubjectRequest, opts ...grpc.CallOption) (*ResolveIdentitySubjectResponse, error)
BindIdentitySubject(ctx context.Context, in *BindIdentitySubjectRequest, opts ...grpc.CallOption) (*BindIdentitySubjectResponse, error)
ClaimIdentitySubject(ctx context.Context, in *ClaimIdentitySubjectRequest, opts ...grpc.CallOption) (*ClaimIdentitySubjectResponse, error)
// Personal API tokens act on the caller's own principal: the State Store
// refuses a call that carries none.
AddPersonalToken(ctx context.Context, in *AddPersonalTokenRequest, opts ...grpc.CallOption) (*AddPersonalTokenResponse, error)
@@ -197,6 +218,10 @@ type StateStoreServiceClient interface {
EnqueueIssue(ctx context.Context, in *EnqueueIssueRequest, opts ...grpc.CallOption) (*EnqueueIssueResponse, error)
EnqueueChatTask(ctx context.Context, in *EnqueueChatTaskRequest, opts ...grpc.CallOption) (*EnqueueChatTaskResponse, error)
ActiveTasksByOrigin(ctx context.Context, in *ActiveTasksByOriginRequest, opts ...grpc.CallOption) (*ActiveTasksByOriginResponse, error)
// UnpostedOutcomes lists finished chat tasks whose outcome has not yet been
// posted to their conversation; MarkOutcomePosted records that it was.
UnpostedOutcomes(ctx context.Context, in *UnpostedOutcomesRequest, opts ...grpc.CallOption) (*UnpostedOutcomesResponse, error)
MarkOutcomePosted(ctx context.Context, in *MarkOutcomePostedRequest, opts ...grpc.CallOption) (*MarkOutcomePostedResponse, error)
ClaimNext(ctx context.Context, in *ClaimNextRequest, opts ...grpc.CallOption) (*ClaimNextResponse, error)
ClaimByIssue(ctx context.Context, in *ClaimByIssueRequest, opts ...grpc.CallOption) (*ClaimByIssueResponse, error)
Transition(ctx context.Context, in *TransitionRequest, opts ...grpc.CallOption) (*TransitionResponse, error)
@@ -204,6 +229,16 @@ type StateStoreServiceClient interface {
// Step executions (docs/prds/execution-tree-state-machine.md). The run
// credential authorises both on its own execution only.
StartStep(ctx context.Context, in *StartStepRequest, opts ...grpc.CallOption) (*StartStepResponse, error)
// RecordUsage appends one model call's usage record. A run credential may
// record usage only for its own task.
RecordUsage(ctx context.Context, in *RecordUsageRequest, opts ...grpc.CallOption) (*RecordUsageResponse, error)
// TaskModels answers what a task's models resolve against: the aliases its
// org may use and the org's own providers. The org is the task's, never
// the caller's.
TaskModels(ctx context.Context, in *TaskModelsRequest, opts ...grpc.CallOption) (*TaskModelsResponse, error)
// CallerModels answers the model aliases the caller's org may use. The org
// is the calling principal's; a call with none is the system org's.
CallerModels(ctx context.Context, in *CallerModelsRequest, opts ...grpc.CallOption) (*CallerModelsResponse, error)
FinishStep(ctx context.Context, in *FinishStepRequest, opts ...grpc.CallOption) (*FinishStepResponse, error)
// Cancellation. Deliberately administrative (deny-by-default for a
// task-scoped grant): a cancel is a dashboard, API and dispatch action
@@ -468,6 +503,46 @@ func (c *stateStoreServiceClient) RemoveInstalledPackage(ctx context.Context, in
return out, nil
}
func (c *stateStoreServiceClient) UpdatePackage(ctx context.Context, in *UpdatePackageRequest, opts ...grpc.CallOption) (*UpdatePackageResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(UpdatePackageResponse)
err := c.cc.Invoke(ctx, StateStoreService_UpdatePackage_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) ApprovePackageUpdate(ctx context.Context, in *ApprovePackageUpdateRequest, opts ...grpc.CallOption) (*ApprovePackageUpdateResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ApprovePackageUpdateResponse)
err := c.cc.Invoke(ctx, StateStoreService_ApprovePackageUpdate_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) RollbackPackage(ctx context.Context, in *RollbackPackageRequest, opts ...grpc.CallOption) (*RollbackPackageResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(RollbackPackageResponse)
err := c.cc.Invoke(ctx, StateStoreService_RollbackPackage_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) SetPackageUpdatePolicy(ctx context.Context, in *SetPackageUpdatePolicyRequest, opts ...grpc.CallOption) (*SetPackageUpdatePolicyResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(SetPackageUpdatePolicyResponse)
err := c.cc.Invoke(ctx, StateStoreService_SetPackageUpdatePolicy_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) RegisterTaskGrant(ctx context.Context, in *RegisterTaskGrantRequest, opts ...grpc.CallOption) (*RegisterTaskGrantResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(RegisterTaskGrantResponse)
@@ -588,6 +663,16 @@ func (c *stateStoreServiceClient) BindIdentitySubject(ctx context.Context, in *B
return out, nil
}
func (c *stateStoreServiceClient) ClaimIdentitySubject(ctx context.Context, in *ClaimIdentitySubjectRequest, opts ...grpc.CallOption) (*ClaimIdentitySubjectResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ClaimIdentitySubjectResponse)
err := c.cc.Invoke(ctx, StateStoreService_ClaimIdentitySubject_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) AddPersonalToken(ctx context.Context, in *AddPersonalTokenRequest, opts ...grpc.CallOption) (*AddPersonalTokenResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(AddPersonalTokenResponse)
@@ -648,6 +733,26 @@ func (c *stateStoreServiceClient) ActiveTasksByOrigin(ctx context.Context, in *A
return out, nil
}
func (c *stateStoreServiceClient) UnpostedOutcomes(ctx context.Context, in *UnpostedOutcomesRequest, opts ...grpc.CallOption) (*UnpostedOutcomesResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(UnpostedOutcomesResponse)
err := c.cc.Invoke(ctx, StateStoreService_UnpostedOutcomes_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) MarkOutcomePosted(ctx context.Context, in *MarkOutcomePostedRequest, opts ...grpc.CallOption) (*MarkOutcomePostedResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(MarkOutcomePostedResponse)
err := c.cc.Invoke(ctx, StateStoreService_MarkOutcomePosted_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) ClaimNext(ctx context.Context, in *ClaimNextRequest, opts ...grpc.CallOption) (*ClaimNextResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ClaimNextResponse)
@@ -698,6 +803,36 @@ func (c *stateStoreServiceClient) StartStep(ctx context.Context, in *StartStepRe
return out, nil
}
func (c *stateStoreServiceClient) RecordUsage(ctx context.Context, in *RecordUsageRequest, opts ...grpc.CallOption) (*RecordUsageResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(RecordUsageResponse)
err := c.cc.Invoke(ctx, StateStoreService_RecordUsage_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) TaskModels(ctx context.Context, in *TaskModelsRequest, opts ...grpc.CallOption) (*TaskModelsResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(TaskModelsResponse)
err := c.cc.Invoke(ctx, StateStoreService_TaskModels_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) CallerModels(ctx context.Context, in *CallerModelsRequest, opts ...grpc.CallOption) (*CallerModelsResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(CallerModelsResponse)
err := c.cc.Invoke(ctx, StateStoreService_CallerModels_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) FinishStep(ctx context.Context, in *FinishStepRequest, opts ...grpc.CallOption) (*FinishStepResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(FinishStepResponse)
@@ -1681,6 +1816,16 @@ type StateStoreServiceServer interface {
// Administrative until org-scoped principals land.
AcceptPackageAuthority(context.Context, *AcceptPackageAuthorityRequest) (*AcceptPackageAuthorityResponse, error)
RemoveInstalledPackage(context.Context, *RemoveInstalledPackageRequest) (*RemoveInstalledPackageResponse, error)
// UpdatePackage moves a package to the digest the verified catalogue lists:
// a non-widening update applies, a widening one waits as pending.
// ApprovePackageUpdate applies the pending update and accepts its authority;
// RollbackPackage restores the previous digest and its accepted authority.
UpdatePackage(context.Context, *UpdatePackageRequest) (*UpdatePackageResponse, error)
ApprovePackageUpdate(context.Context, *ApprovePackageUpdateRequest) (*ApprovePackageUpdateResponse, error)
RollbackPackage(context.Context, *RollbackPackageRequest) (*RollbackPackageResponse, error)
// SetPackageUpdatePolicy sets "manual" or "auto"; the State Store updates
// auto packages on a timer.
SetPackageUpdatePolicy(context.Context, *SetPackageUpdatePolicyRequest) (*SetPackageUpdatePolicyResponse, error)
// Administrative task credential lifecycle. Worker credentials cannot call these.
RegisterTaskGrant(context.Context, *RegisterTaskGrantRequest) (*RegisterTaskGrantResponse, error)
RevokeTaskGrant(context.Context, *RevokeTaskGrantRequest) (*RevokeTaskGrantResponse, error)
@@ -1698,6 +1843,7 @@ type StateStoreServiceServer interface {
// identity store.
ResolveIdentitySubject(context.Context, *ResolveIdentitySubjectRequest) (*ResolveIdentitySubjectResponse, error)
BindIdentitySubject(context.Context, *BindIdentitySubjectRequest) (*BindIdentitySubjectResponse, error)
ClaimIdentitySubject(context.Context, *ClaimIdentitySubjectRequest) (*ClaimIdentitySubjectResponse, error)
// Personal API tokens act on the caller's own principal: the State Store
// refuses a call that carries none.
AddPersonalToken(context.Context, *AddPersonalTokenRequest) (*AddPersonalTokenResponse, error)
@@ -1707,6 +1853,10 @@ type StateStoreServiceServer interface {
EnqueueIssue(context.Context, *EnqueueIssueRequest) (*EnqueueIssueResponse, error)
EnqueueChatTask(context.Context, *EnqueueChatTaskRequest) (*EnqueueChatTaskResponse, error)
ActiveTasksByOrigin(context.Context, *ActiveTasksByOriginRequest) (*ActiveTasksByOriginResponse, error)
// UnpostedOutcomes lists finished chat tasks whose outcome has not yet been
// posted to their conversation; MarkOutcomePosted records that it was.
UnpostedOutcomes(context.Context, *UnpostedOutcomesRequest) (*UnpostedOutcomesResponse, error)
MarkOutcomePosted(context.Context, *MarkOutcomePostedRequest) (*MarkOutcomePostedResponse, error)
ClaimNext(context.Context, *ClaimNextRequest) (*ClaimNextResponse, error)
ClaimByIssue(context.Context, *ClaimByIssueRequest) (*ClaimByIssueResponse, error)
Transition(context.Context, *TransitionRequest) (*TransitionResponse, error)
@@ -1714,6 +1864,16 @@ type StateStoreServiceServer interface {
// Step executions (docs/prds/execution-tree-state-machine.md). The run
// credential authorises both on its own execution only.
StartStep(context.Context, *StartStepRequest) (*StartStepResponse, error)
// RecordUsage appends one model call's usage record. A run credential may
// record usage only for its own task.
RecordUsage(context.Context, *RecordUsageRequest) (*RecordUsageResponse, error)
// TaskModels answers what a task's models resolve against: the aliases its
// org may use and the org's own providers. The org is the task's, never
// the caller's.
TaskModels(context.Context, *TaskModelsRequest) (*TaskModelsResponse, error)
// CallerModels answers the model aliases the caller's org may use. The org
// is the calling principal's; a call with none is the system org's.
CallerModels(context.Context, *CallerModelsRequest) (*CallerModelsResponse, error)
FinishStep(context.Context, *FinishStepRequest) (*FinishStepResponse, error)
// Cancellation. Deliberately administrative (deny-by-default for a
// task-scoped grant): a cancel is a dashboard, API and dispatch action
@@ -1922,6 +2082,18 @@ func (UnimplementedStateStoreServiceServer) AcceptPackageAuthority(context.Conte
func (UnimplementedStateStoreServiceServer) RemoveInstalledPackage(context.Context, *RemoveInstalledPackageRequest) (*RemoveInstalledPackageResponse, error) {
return nil, status.Error(codes.Unimplemented, "method RemoveInstalledPackage not implemented")
}
func (UnimplementedStateStoreServiceServer) UpdatePackage(context.Context, *UpdatePackageRequest) (*UpdatePackageResponse, error) {
return nil, status.Error(codes.Unimplemented, "method UpdatePackage not implemented")
}
func (UnimplementedStateStoreServiceServer) ApprovePackageUpdate(context.Context, *ApprovePackageUpdateRequest) (*ApprovePackageUpdateResponse, error) {
return nil, status.Error(codes.Unimplemented, "method ApprovePackageUpdate not implemented")
}
func (UnimplementedStateStoreServiceServer) RollbackPackage(context.Context, *RollbackPackageRequest) (*RollbackPackageResponse, error) {
return nil, status.Error(codes.Unimplemented, "method RollbackPackage not implemented")
}
func (UnimplementedStateStoreServiceServer) SetPackageUpdatePolicy(context.Context, *SetPackageUpdatePolicyRequest) (*SetPackageUpdatePolicyResponse, error) {
return nil, status.Error(codes.Unimplemented, "method SetPackageUpdatePolicy not implemented")
}
func (UnimplementedStateStoreServiceServer) RegisterTaskGrant(context.Context, *RegisterTaskGrantRequest) (*RegisterTaskGrantResponse, error) {
return nil, status.Error(codes.Unimplemented, "method RegisterTaskGrant not implemented")
}
@@ -1958,6 +2130,9 @@ func (UnimplementedStateStoreServiceServer) ResolveIdentitySubject(context.Conte
func (UnimplementedStateStoreServiceServer) BindIdentitySubject(context.Context, *BindIdentitySubjectRequest) (*BindIdentitySubjectResponse, error) {
return nil, status.Error(codes.Unimplemented, "method BindIdentitySubject not implemented")
}
func (UnimplementedStateStoreServiceServer) ClaimIdentitySubject(context.Context, *ClaimIdentitySubjectRequest) (*ClaimIdentitySubjectResponse, error) {
return nil, status.Error(codes.Unimplemented, "method ClaimIdentitySubject not implemented")
}
func (UnimplementedStateStoreServiceServer) AddPersonalToken(context.Context, *AddPersonalTokenRequest) (*AddPersonalTokenResponse, error) {
return nil, status.Error(codes.Unimplemented, "method AddPersonalToken not implemented")
}
@@ -1976,6 +2151,12 @@ func (UnimplementedStateStoreServiceServer) EnqueueChatTask(context.Context, *En
func (UnimplementedStateStoreServiceServer) ActiveTasksByOrigin(context.Context, *ActiveTasksByOriginRequest) (*ActiveTasksByOriginResponse, error) {
return nil, status.Error(codes.Unimplemented, "method ActiveTasksByOrigin not implemented")
}
func (UnimplementedStateStoreServiceServer) UnpostedOutcomes(context.Context, *UnpostedOutcomesRequest) (*UnpostedOutcomesResponse, error) {
return nil, status.Error(codes.Unimplemented, "method UnpostedOutcomes not implemented")
}
func (UnimplementedStateStoreServiceServer) MarkOutcomePosted(context.Context, *MarkOutcomePostedRequest) (*MarkOutcomePostedResponse, error) {
return nil, status.Error(codes.Unimplemented, "method MarkOutcomePosted not implemented")
}
func (UnimplementedStateStoreServiceServer) ClaimNext(context.Context, *ClaimNextRequest) (*ClaimNextResponse, error) {
return nil, status.Error(codes.Unimplemented, "method ClaimNext not implemented")
}
@@ -1991,6 +2172,15 @@ func (UnimplementedStateStoreServiceServer) Update(context.Context, *UpdateReque
func (UnimplementedStateStoreServiceServer) StartStep(context.Context, *StartStepRequest) (*StartStepResponse, error) {
return nil, status.Error(codes.Unimplemented, "method StartStep not implemented")
}
func (UnimplementedStateStoreServiceServer) RecordUsage(context.Context, *RecordUsageRequest) (*RecordUsageResponse, error) {
return nil, status.Error(codes.Unimplemented, "method RecordUsage not implemented")
}
func (UnimplementedStateStoreServiceServer) TaskModels(context.Context, *TaskModelsRequest) (*TaskModelsResponse, error) {
return nil, status.Error(codes.Unimplemented, "method TaskModels not implemented")
}
func (UnimplementedStateStoreServiceServer) CallerModels(context.Context, *CallerModelsRequest) (*CallerModelsResponse, error) {
return nil, status.Error(codes.Unimplemented, "method CallerModels not implemented")
}
func (UnimplementedStateStoreServiceServer) FinishStep(context.Context, *FinishStepRequest) (*FinishStepResponse, error) {
return nil, status.Error(codes.Unimplemented, "method FinishStep not implemented")
}
@@ -2435,6 +2625,78 @@ func _StateStoreService_RemoveInstalledPackage_Handler(srv interface{}, ctx cont
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_UpdatePackage_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(UpdatePackageRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).UpdatePackage(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_UpdatePackage_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).UpdatePackage(ctx, req.(*UpdatePackageRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_ApprovePackageUpdate_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ApprovePackageUpdateRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).ApprovePackageUpdate(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_ApprovePackageUpdate_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).ApprovePackageUpdate(ctx, req.(*ApprovePackageUpdateRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_RollbackPackage_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(RollbackPackageRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).RollbackPackage(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_RollbackPackage_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).RollbackPackage(ctx, req.(*RollbackPackageRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_SetPackageUpdatePolicy_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(SetPackageUpdatePolicyRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).SetPackageUpdatePolicy(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_SetPackageUpdatePolicy_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).SetPackageUpdatePolicy(ctx, req.(*SetPackageUpdatePolicyRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_RegisterTaskGrant_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(RegisterTaskGrantRequest)
if err := dec(in); err != nil {
@@ -2651,6 +2913,24 @@ func _StateStoreService_BindIdentitySubject_Handler(srv interface{}, ctx context
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_ClaimIdentitySubject_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ClaimIdentitySubjectRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).ClaimIdentitySubject(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_ClaimIdentitySubject_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).ClaimIdentitySubject(ctx, req.(*ClaimIdentitySubjectRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_AddPersonalToken_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(AddPersonalTokenRequest)
if err := dec(in); err != nil {
@@ -2759,6 +3039,42 @@ func _StateStoreService_ActiveTasksByOrigin_Handler(srv interface{}, ctx context
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_UnpostedOutcomes_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(UnpostedOutcomesRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).UnpostedOutcomes(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_UnpostedOutcomes_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).UnpostedOutcomes(ctx, req.(*UnpostedOutcomesRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_MarkOutcomePosted_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(MarkOutcomePostedRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).MarkOutcomePosted(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_MarkOutcomePosted_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).MarkOutcomePosted(ctx, req.(*MarkOutcomePostedRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_ClaimNext_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ClaimNextRequest)
if err := dec(in); err != nil {
@@ -2849,6 +3165,60 @@ func _StateStoreService_StartStep_Handler(srv interface{}, ctx context.Context,
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_RecordUsage_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(RecordUsageRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).RecordUsage(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_RecordUsage_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).RecordUsage(ctx, req.(*RecordUsageRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_TaskModels_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(TaskModelsRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).TaskModels(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_TaskModels_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).TaskModels(ctx, req.(*TaskModelsRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_CallerModels_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(CallerModelsRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).CallerModels(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_CallerModels_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).CallerModels(ctx, req.(*CallerModelsRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_FinishStep_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(FinishStepRequest)
if err := dec(in); err != nil {
@@ -4541,6 +4911,22 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
MethodName: "RemoveInstalledPackage",
Handler: _StateStoreService_RemoveInstalledPackage_Handler,
},
{
MethodName: "UpdatePackage",
Handler: _StateStoreService_UpdatePackage_Handler,
},
{
MethodName: "ApprovePackageUpdate",
Handler: _StateStoreService_ApprovePackageUpdate_Handler,
},
{
MethodName: "RollbackPackage",
Handler: _StateStoreService_RollbackPackage_Handler,
},
{
MethodName: "SetPackageUpdatePolicy",
Handler: _StateStoreService_SetPackageUpdatePolicy_Handler,
},
{
MethodName: "RegisterTaskGrant",
Handler: _StateStoreService_RegisterTaskGrant_Handler,
@@ -4589,6 +4975,10 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
MethodName: "BindIdentitySubject",
Handler: _StateStoreService_BindIdentitySubject_Handler,
},
{
MethodName: "ClaimIdentitySubject",
Handler: _StateStoreService_ClaimIdentitySubject_Handler,
},
{
MethodName: "AddPersonalToken",
Handler: _StateStoreService_AddPersonalToken_Handler,
@@ -4613,6 +5003,14 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
MethodName: "ActiveTasksByOrigin",
Handler: _StateStoreService_ActiveTasksByOrigin_Handler,
},
{
MethodName: "UnpostedOutcomes",
Handler: _StateStoreService_UnpostedOutcomes_Handler,
},
{
MethodName: "MarkOutcomePosted",
Handler: _StateStoreService_MarkOutcomePosted_Handler,
},
{
MethodName: "ClaimNext",
Handler: _StateStoreService_ClaimNext_Handler,
@@ -4633,6 +5031,18 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
MethodName: "StartStep",
Handler: _StateStoreService_StartStep_Handler,
},
{
MethodName: "RecordUsage",
Handler: _StateStoreService_RecordUsage_Handler,
},
{
MethodName: "TaskModels",
Handler: _StateStoreService_TaskModels_Handler,
},
{
MethodName: "CallerModels",
Handler: _StateStoreService_CallerModels_Handler,
},
{
MethodName: "FinishStep",
Handler: _StateStoreService_FinishStep_Handler,
+35 -13
View File
@@ -2,8 +2,12 @@ package daemon
import (
"context"
"time"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/taskstate"
)
// callWaitBuffer is the watcher's subscription buffer. The bus carries the
@@ -16,13 +20,16 @@ const callWaitBuffer = 1024
// daemon's own bus, republished from the container running the workflow, and
// carry the caller's task id.
func (d *Daemon) watchCallWaits(ctx context.Context) {
if d.Bus == nil {
reader, ok := d.Store.(storecontract.StepReader)
if d.Bus == nil || !ok {
return
}
sub := d.Bus.Subscribe(callWaitBuffer)
dispatcher := d.taskDispatcher()
go func() {
defer sub.Close()
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
@@ -31,23 +38,38 @@ func (d *Daemon) watchCallWaits(ctx context.Context) {
if !ok {
return
}
applyCallWait(dispatcher, e)
if e.Kind == events.KindWorkflowCallStarted || e.Kind == events.KindWorkflowCallFinished || e.Kind == events.KindStageStart || e.Kind == events.KindStageFinish {
d.reconcileCallWaits(ctx, dispatcher, reader)
}
case <-ticker.C:
d.reconcileCallWaits(ctx, dispatcher, reader)
}
}
}()
}
// applyCallWait suspends the run a call start names and resumes it when the
// call finishes. It stays non-blocking: a full subscriber buffer drops events,
// and a dropped start leaves a waiting run holding the capacity its callee
// needs, so this must never wait on anything.
func applyCallWait(dispatcher *taskDispatcher, e events.Event) {
switch e.Kind {
case events.KindWorkflowCallStarted:
if waiting, _ := e.Data["wait"].(bool); waiting {
dispatcher.Suspend(e.TaskID)
// Events only prompt a read; periodic reads recover dropped starts and finishes.
func (d *Daemon) reconcileCallWaits(ctx context.Context, dispatcher *taskDispatcher, reader storecontract.StepReader) {
dispatcher.mu.Lock()
holds := make([]*taskHold, 0, len(dispatcher.holds))
for _, hold := range dispatcher.holds {
holds = append(holds, hold)
}
dispatcher.mu.Unlock()
for _, hold := range holds {
readCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
steps, err := reader.ListSteps(readCtx, hold.id, hold.attempt)
cancel()
if err != nil {
d.Log.Warn("read call waits", "task", hold.id, "err", err)
continue
}
case events.KindWorkflowCallFinished:
dispatcher.Resume(e.TaskID)
waits := 0
for _, step := range steps {
if step.Kind == task.StepKindCall && step.CalledExecutionID != 0 && (step.Status == taskstate.StepPending || step.Status == taskstate.StepRunning) {
waits++
}
}
dispatcher.setCallWaits(hold, waits)
}
}
+120 -103
View File
@@ -14,7 +14,6 @@ import (
"os"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/samcharles93/archie-core/internal/agentexec"
@@ -152,6 +151,12 @@ type Daemon struct {
RunCredentials RunCredentialIssuer
// KitLauncher starts tasks whose agent profile is a Kit. Nil parks them.
KitLauncher KitLauncher
// ModelEgress gives native task containers model access without provider
// keys. Nil parks a task whose providers need a key.
ModelEgress ModelEgress
// TaskModels answers the model aliases a task's org resolves against.
// Nil runs every task on the instance aliases.
TaskModels TaskModels
// TaskRunReadyTimeout bounds how long runViaAgent retries a taskrun request
// while the new container has not subscribed yet. Zero uses
// defaultTaskRunReadyTimeout.
@@ -185,10 +190,9 @@ type Daemon struct {
IdentityRepository identity.Repository
RootIdentityID identity.IdentityID
// KindWorkflows and LabelWorkflows are the routing bindings loaded at
// startup, sent to the agent. Nil means defaults.
KindWorkflows workflow.KindWorkflows
LabelWorkflows workflow.LabelWorkflows
// PackageAuthorities bounds the tools a package workflow's run is offered.
PackageAuthorities PackageAuthorities
// Playbooks are the loaded EDA playbooks, consulted before kind/label
// bindings when pinning a workflow. Nil means none.
Playbooks interface {
@@ -224,11 +228,10 @@ type Daemon struct {
// to use.
running runningTasks
// lastPollAt is when the most recent poll pass began, in Unix
// nanoseconds (zero = no pass has started yet), read by
// LastPollAt. Atomic because runIdentities runs one poll goroutine per
// identity and they all stamp this one "is the poller alive" reading.
lastPollAt atomic.Int64
// lastPollAt maps each poll loop (an identity ID, or "" for the root
// loop) to when its latest pass began, as Unix nanoseconds. LastPollAt
// reports the oldest, so one wedged loop is not hidden by a healthy one.
lastPollAt sync.Map
}
// IdentityRunner bundles identity-specific state for a single agent
@@ -434,10 +437,10 @@ func (d *Daemon) runIdentities(ctx context.Context) error {
// drains or reconciles -- those are store-wide and run in the shared
// maintainAndDrain loop.
func (d *Daemon) pollForIdentity(ctx context.Context, id *IdentityRunner) {
d.markPoll(string(id.ID))
if !d.identityActive(ctx, id.ID) {
return
}
d.markPoll()
cfg := configForIdentity(d.Cfg.Get(), id.Cfg)
for _, repo := range identityRepositories(cfg.Repos, id.Repos) {
issues, complete := d.pollIssuesWithConfig(ctx, id.Forge, cfg, repo)
@@ -1026,25 +1029,27 @@ type taskHold struct {
baton chan struct{}
// slot is true while the run counts against the global limit.
slot bool
// waits counts the callees the run is waiting on: the first Suspend gives
// capacity back and the last Resume takes it again.
// waits counts the callees the run is waiting on: an open call gives capacity
// back and the last closed call takes it again.
waits int
// finished is true once the run's process returned.
finished bool
// id is the task this hold belongs to, named by the run once it knows the
// task's id.
id int64
id int64
attempt int
}
// name records which task's capacity this hold is, so a wait on a callee can
// give it back by task id. Zero names nothing: a task arriving on the NATS
// stream has no id until the store claims its row inside the run.
func (h *taskHold) name(id int64) {
func (h *taskHold) name(id int64, attempt int) {
if id == 0 {
return
}
h.dispatcher.mu.Lock()
h.id = id
h.attempt = attempt
h.dispatcher.holds[id] = h
h.dispatcher.mu.Unlock()
}
@@ -1125,47 +1130,25 @@ func (d *taskDispatcher) finishRun(hold *taskHold) {
d.slotsFree.Broadcast()
}
// Suspend gives a run's capacity back while it waits on a callee, so the
// callee can start. Calls are refcounted per task: a run waiting on callees
// from parallel branches releases once. An unknown task, or one whose run has
// already returned, is a no-op.
func (d *taskDispatcher) Suspend(id int64) {
// setCallWaits applies a durable snapshot without accumulating duplicate events.
func (d *taskDispatcher) setCallWaits(hold *taskHold, waits int) {
d.mu.Lock()
hold := d.holds[id]
if hold == nil || hold.finished {
if d.holds[hold.id] != hold || hold.finished {
d.mu.Unlock()
return
}
hold.waits++
if hold.waits == 1 {
previous := hold.waits
hold.waits = waits
if waits > 0 {
d.giveBackLocked(hold)
}
d.mu.Unlock()
d.slotsFree.Broadcast()
}
// Resume takes a run's capacity back when one of its waits ends. Only the last
// wait does: a run waiting on callees from parallel branches keeps nothing
// until all of them are done. An unknown task, one whose run has returned, and
// one that never suspended are all no-ops.
func (d *taskDispatcher) Resume(id int64) {
d.mu.Lock()
hold := d.holds[id]
if hold == nil || hold.finished || hold.waits == 0 {
d.mu.Unlock()
return
if waits > 0 {
d.slotsFree.Broadcast()
} else if previous > 0 {
// Reacquiring may block behind another task; the caller keeps running.
go d.reacquire(hold)
}
hold.waits--
if hold.waits > 0 {
d.mu.Unlock()
return
}
d.mu.Unlock()
// The two waits below can block, and neither may hold up the bus
// subscriber calling this or the resumed run, which is already executing
// inside its container: the execution cap can be exceeded by at most the
// number of simultaneous resumes, and re-converges as tasks finish.
go d.reacquire(hold)
}
// reacquire returns a resumed run's capacity: its place in its repo's chain
@@ -1176,7 +1159,7 @@ func (d *taskDispatcher) Resume(id int64) {
func (d *taskDispatcher) reacquire(hold *taskHold) {
d.mu.Lock()
if hold.finished || hold.waits > 0 {
// The run returned, or started waiting again, after the Resume that
// The run returned, or started waiting again, after the reconciliation that
// queued this: it needs no capacity.
d.mu.Unlock()
return
@@ -1265,10 +1248,10 @@ func (d *taskDispatcher) Wait() {
}
func (d *Daemon) poll(ctx context.Context) {
d.markPoll("")
if !d.identityActive(ctx, d.RootIdentityID) {
return
}
d.markPoll()
for _, repo := range d.Cfg.Get().Repos {
issues, complete := d.pollIssues(ctx, repo)
if complete {
@@ -1593,7 +1576,7 @@ func (d *Daemon) reconcilePRs(ctx context.Context) {
func (d *Daemon) process(ctx context.Context, task *workflow.Task, hold *taskHold) {
// A task off the NATS stream only has an id once the store has claimed its
// row, inside this call, so its capacity is named here.
hold.name(task.ID)
hold.name(task.ID, task.Attempt)
// The workflow may name the identity the run acts as, which everything
// below is chosen by, so it is settled before anything else.
if !d.adoptWorkflowIdentity(ctx, task) {
@@ -1666,14 +1649,17 @@ func (d *Daemon) process(ctx context.Context, task *workflow.Task, hold *taskHol
// teardownStorage runs after the workflow completes. The Docker backend is
// a no-op; future backends (temp volumes, NFS leases) use this hook.
func (d *Daemon) teardownStorage(ctx context.Context, task *workflow.Task, repo config.Repo, workDir string) {
if d.Storage != nil {
_ = d.Storage.Teardown(ctx, storage.TaskRef{
WorktreeDir: workDir,
Ecosystem: repo.Ecosystem,
PersistentStorage: repo.PersistentStorage,
Owner: task.Owner,
Repo: task.Repo,
})
if d.Storage == nil {
return
}
if err := d.Storage.Teardown(ctx, storage.TaskRef{
WorktreeDir: workDir,
Ecosystem: repo.Ecosystem,
PersistentStorage: repo.PersistentStorage,
Owner: task.Owner,
Repo: task.Repo,
}); err != nil {
d.Log.Warn("storage teardown failed", "task", task.ID, "err", err)
}
}
@@ -1705,7 +1691,13 @@ func (d *Daemon) prepareWorkspace(ctx context.Context, task *workflow.Task, tree
target = worktree.Target(task.Branch)
}
// Every task gets an independent full clone.
dir, branch, err := trees.Prepare(ctx, task.Owner, task.Repo, repo.BaseBranch(), task.IssueNumber, task.Title, task.Body, task.Labels, target)
prepare := func() (string, string, error) {
return trees.Prepare(ctx, task.Owner, task.Repo, repo.BaseBranch(), task.IssueNumber, task.Title, task.Body, task.Labels, target)
}
dir, branch, err := prepare()
if d.reownDenied(ctx, trees.Dir(task.Owner, task.Repo, task.IssueNumber), err) {
dir, branch, err = prepare()
}
if err != nil {
reason := "worktree prepare failed: " + err.Error()
if target != worktree.Fresh {
@@ -1748,7 +1740,7 @@ func (d *Daemon) cleanupTerminalTaskWorktree(ctx context.Context, task *workflow
if d.worktreeHoldsUncapturedWork(cleanupCtx, trees, task) {
return
}
if err := trees.Cleanup(task.Owner, task.Repo, task.IssueNumber); err != nil {
if err := d.cleanupWorktree(cleanupCtx, trees, task.Owner, task.Repo, task.IssueNumber); err != nil {
d.Log.Warn("terminal worktree cleanup failed", "task", task.ID, "err", err)
}
}
@@ -1848,35 +1840,29 @@ func (d *Daemon) acquireTaskContainer(
credential, revokeCredential, err := d.runCredential(task)
if err != nil {
if terr := d.Storage.Teardown(ctx, storage.TaskRef{
WorktreeDir: workDir,
Ecosystem: repo.Ecosystem,
PersistentStorage: repo.PersistentStorage,
Owner: task.Owner,
Repo: task.Repo,
}); terr != nil {
d.Log.Warn("storage teardown after run credential failure failed", "err", terr)
}
d.teardownStorage(ctx, task, repo, workDir)
park("run credential failed", err)
return nil, "", nil, false
}
ctr, err := d.ContainerPool.Acquire(ctx, image, mounts, d.containerEnv(task, credential))
model, closeModel, err := d.openModelEgress(ctx, task, credential)
if err != nil {
revokeCredential()
d.teardownStorage(ctx, task, repo, workDir)
park("model egress failed", err)
return nil, "", nil, false
}
revokeRun := revokeCredential
revokeCredential = func() { closeModel(); revokeRun() }
ctr, err := d.ContainerPool.Acquire(ctx, image, append(mounts, model.Mounts...), append(d.containerEnv(task, credential), model.Env...))
if err != nil {
revokeCredential()
// Roll back the storage we just set up: the mounts were created
// for this task but no container will use them. Leaking them until
// a later TTL sweep is not acceptable on a backend that allocates
// real resources (volumes, NFS leases).
if terr := d.Storage.Teardown(ctx, storage.TaskRef{
WorktreeDir: workDir,
Ecosystem: repo.Ecosystem,
PersistentStorage: repo.PersistentStorage,
Owner: task.Owner,
Repo: task.Repo,
}); terr != nil {
d.Log.Warn("storage teardown after acquire failure failed", "err", terr)
}
d.teardownStorage(ctx, task, repo, workDir)
park("container acquire failed", err)
return nil, "", nil, false
}
@@ -1960,7 +1946,11 @@ func (d *Daemon) recordPark(ctx context.Context, taskID int64, reason string) {
// credential is the task's run credential; the agent presents it to push the
// task's branch.
func (d *Daemon) runViaAgent(ctx context.Context, task *workflow.Task, repo config.Repo, profile config.AgentProfile, harness *agentrun.HarnessSpec, credential string) {
cfg := d.configFor(task)
cfg, err := d.taskConfig(ctx, task)
if err != nil {
d.parkRunningTask(ctx, task.ID, err.Error(), taskstate.ParkTransient)
return
}
taskCfg := cfg.ForTask()
d.captureAttemptConfig(ctx, task, taskCfg)
req := taskrun.Request{
@@ -1970,10 +1960,8 @@ func (d *Daemon) runViaAgent(ctx context.Context, task *workflow.Task, repo conf
Providers: agentexec.ProvidersFromConfig(cfg.Providers),
MCPServers: cfg.Tools.MCPServers,
RunCredential: credential,
KindWorkflows: d.KindWorkflows,
LabelWorkflows: d.LabelWorkflows,
WorkflowDefinition: task.WorkflowDefinitionYAML,
Tools: profile.Tools,
Tools: d.toolAllowlist(ctx, task, profile.Tools),
Harness: harness,
}
data, err := json.Marshal(req)
@@ -2076,6 +2064,9 @@ func (d *Daemon) pinWorkflowDefinition(ctx context.Context, task *workflow.Task)
if !enablement.Enabled(task.Org, task.Workflow) {
return pinFailure{taskstate.ParkNeedsHuman, fmt.Errorf("workflow %s is disabled", task.Workflow)}
}
if err := d.checkPackageTrigger(ctx, task); err != nil {
return pinFailure{taskstate.ParkNeedsHuman, err}
}
return nil
}
@@ -2149,14 +2140,14 @@ func (d *Daemon) runActionPlaybooks(ctx context.Context, task *workflow.Task) {
}
// resolveWorkflowID picks the definition to pin: a matching playbook, then
// the kind/label bindings. A task already naming a workflow keeps it.
// the kind defaults. A task already naming a workflow keeps it.
func (d *Daemon) resolveWorkflowID(task *workflow.Task, available map[string]struct{}) (string, error) {
if d.Playbooks == nil || task.Workflow != "" {
return workflow.ResolveWorkflowID(task, available, d.KindWorkflows, d.LabelWorkflows)
return workflow.ResolveWorkflowID(task, available)
}
decision, matched := d.Playbooks.Dispatch(playbookInput(task))
if !matched {
return workflow.ResolveWorkflowID(task, available, d.KindWorkflows, d.LabelWorkflows)
return workflow.ResolveWorkflowID(task, available)
}
if _, defined := available[decision.Workflow]; !defined {
// Reported, not silently downgraded to the binding's choice: the
@@ -2290,13 +2281,6 @@ func (d *Daemon) containerEnv(task *workflow.Task, stateStoreToken string) []str
// back before the daemon reads it to push.
env = append(env, fmt.Sprintf("WORKTREE_UID=%d", os.Getuid()))
env = append(env, fmt.Sprintf("WORKTREE_GID=%d", os.Getgid()))
for _, p := range d.configFor(task).Providers {
if p.APIKeyEnv != "" {
if v := os.Getenv(p.APIKeyEnv); v != "" {
env = append(env, p.APIKeyEnv+"="+v)
}
}
}
return env
}
@@ -2468,19 +2452,26 @@ func (d *Daemon) allowConcurrentForTask(task *workflow.Task) bool {
return ok && repo.AllowConcurrent
}
// LastPollAt reports when the daemon last began a poll pass, or the zero
// time when no pass has started yet.
// LastPollAt reports when the least recently polled loop last began a pass,
// or the zero time when no pass has started yet.
func (d *Daemon) LastPollAt() time.Time {
ns := d.lastPollAt.Load()
if ns == 0 {
var oldest int64
d.lastPollAt.Range(func(_, v any) bool {
if ns, ok := v.(int64); ok && (oldest == 0 || ns < oldest) {
oldest = ns
}
return true
})
if oldest == 0 {
return time.Time{}
}
return time.Unix(0, ns).UTC()
return time.Unix(0, oldest).UTC()
}
// markPoll stamps the start of a poll pass, so a hung pass shows as stale.
func (d *Daemon) markPoll() {
d.lastPollAt.Store(time.Now().UnixNano())
// markPoll stamps the start of a poll pass for one loop, so a hung pass shows
// as stale.
func (d *Daemon) markPoll(loop string) {
d.lastPollAt.Store(loop, time.Now().UnixNano())
}
// errContainerExited is the cancellation cause when a task's agent container
@@ -2523,7 +2514,33 @@ func (d *Daemon) RemoveWorktree(owner, repo, identity string, issue int) error {
if trees == nil {
return nil
}
return trees.Cleanup(owner, repo, issue)
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
return d.cleanupWorktree(ctx, trees, owner, repo, issue)
}
// cleanupWorktree removes a task's clone, first handing back any root-owned
// files a killed container left in it.
func (d *Daemon) cleanupWorktree(ctx context.Context, trees *worktree.Manager, owner, repo string, issue int) error {
err := trees.Cleanup(owner, repo, issue)
if d.reownDenied(ctx, trees.Dir(owner, repo, issue), err) {
return trees.Cleanup(owner, repo, issue)
}
return err
}
// reownDenied repairs dir when err says the daemon was refused access to it,
// and reports whether the caller should retry. A SIGKILLed task container
// leaves root-owned files that only a container can chown back.
func (d *Daemon) reownDenied(ctx context.Context, dir string, err error) bool {
if err == nil || !errors.Is(err, fs.ErrPermission) || d.ContainerPool == nil {
return false
}
if rerr := d.ContainerPool.Reown(ctx, dir, os.Getuid(), os.Getgid()); rerr != nil {
d.Log.Warn("worktree ownership repair failed", "dir", dir, "err", rerr)
return false
}
return true
}
// InFlight counts the same submitted work that graceful shutdown waits for,
+92 -61
View File
@@ -2,7 +2,6 @@ package daemon
import (
"context"
"encoding/json"
"log/slog"
"runtime"
"sync"
@@ -12,8 +11,10 @@ import (
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
"github.com/samcharles93/archie-core/internal/eventbus"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/taskstate"
)
// runDeadline bounds a wait for something that must happen. Only a broken
@@ -139,12 +140,25 @@ func slotHolders(d *taskDispatcher) int {
return held
}
func changeCallWaits(d *taskDispatcher, delta int, id int64) {
d.mu.Lock()
hold := d.holds[id]
waits := 0
if hold != nil {
waits = max(0, hold.waits+delta)
}
d.mu.Unlock()
if hold != nil {
d.setCallWaits(hold, waits)
}
}
// submitRun queues a task whose process reports its start on started and then
// blocks until released. Naming the run's capacity is what the daemon's own
// process does, where the task's id first exists.
func submitRun(d *taskDispatcher, id int64, repo string, started chan<- int64, released <-chan struct{}) {
d.Submit(context.Background(), &workflow.Task{ID: id, Owner: "acme", Repo: repo}, func(_ context.Context, task *workflow.Task, hold *taskHold) {
hold.name(task.ID)
hold.name(task.ID, task.Attempt)
started <- task.ID
<-released
})
@@ -164,11 +178,11 @@ func TestSuspendReleasesCapacityForTheTaskBehindIt(t *testing.T) {
submitRun(d, 2, "app", started, b)
awaitNoStart(t, started, "while the waiting run held the only slot and the repo's place")
d.Suspend(1)
changeCallWaits(d, 1, 1)
awaitStart(t, started, 2)
close(b)
d.Resume(1)
changeCallWaits(d, -1, 1)
awaitState(t, "task 1 did not take its slot and its place back", func() bool { return holdsCapacity(d, 1) })
submitRun(d, 3, "app", started, c)
@@ -193,11 +207,11 @@ func TestResumedRunKeepsItsPlaceInTheReposChain(t *testing.T) {
submitRun(d, 2, "app", started, b)
awaitNoStart(t, started, "while the repo's place was another run's")
d.Suspend(1)
changeCallWaits(d, 1, 1)
awaitStart(t, started, 2)
close(b)
d.Resume(1)
changeCallWaits(d, -1, 1)
awaitState(t, "task 1 did not take its slot and its place back", func() bool { return holdsCapacity(d, 1) })
submitRun(d, 3, "app", started, c)
awaitNoStart(t, started, "while the resumed run held the repo's place, with slots free")
@@ -217,8 +231,8 @@ func TestResumeTakesCapacityBackOnlyAfterTheLastWait(t *testing.T) {
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
d.Suspend(1)
d.Suspend(1)
changeCallWaits(d, 1, 1)
changeCallWaits(d, 1, 1)
// The callee queued behind it runs, and returns.
submitRun(d, 2, "app", started, b)
@@ -228,11 +242,11 @@ func TestResumeTakesCapacityBackOnlyAfterTheLastWait(t *testing.T) {
// One wait outstanding: the run keeps no capacity while its other callee
// runs.
d.Resume(1)
changeCallWaits(d, -1, 1)
awaitNoState(t, "task 1 took capacity back with a callee still running", func() bool { return holdsCapacity(d, 1) })
// With the last wait done it takes capacity back.
d.Resume(1)
changeCallWaits(d, -1, 1)
awaitState(t, "task 1 did not take its capacity back", func() bool { return holdsCapacity(d, 1) })
close(a)
@@ -253,13 +267,13 @@ func TestResumeTwiceDoesNotCountASecondSlot(t *testing.T) {
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
d.Suspend(1)
changeCallWaits(d, 1, 1)
// X takes the repo's place while A waits on its callee, so A's first
// re-acquire queues its next place behind X.
submitRun(d, 2, "app", started, x)
awaitStart(t, started, 2)
d.Resume(1)
changeCallWaits(d, -1, 1)
awaitState(t, "task 1 did not queue its place back", func() bool {
_, place, _ := capacity(d, 1)
return place
@@ -268,8 +282,8 @@ func TestResumeTwiceDoesNotCountASecondSlot(t *testing.T) {
// That callee ends and the run starts its next call at once: the place the
// first re-acquire claimed is given back with it, and the second re-acquire
// takes a slot immediately.
d.Suspend(1)
d.Resume(1)
changeCallWaits(d, 1, 1)
changeCallWaits(d, -1, 1)
awaitState(t, "task 1 did not take its slot back", func() bool { return holdsCapacity(d, 1) })
// X leaves, so the first re-acquire can take a slot as well.
@@ -308,13 +322,13 @@ func TestSecondReacquireDoesNotQueueASecondPlace(t *testing.T) {
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
d.Suspend(1)
changeCallWaits(d, 1, 1)
// X holds the repo's place, so the run's re-acquire queues behind it and
// stays there.
submitRun(d, 2, "app", started, x)
awaitStart(t, started, 2)
d.Resume(1)
changeCallWaits(d, -1, 1)
awaitState(t, "task 1 did not queue its place back", func() bool {
_, place, _ := capacity(d, 1)
return place
@@ -363,14 +377,14 @@ func TestResumeAfterTheRunReturnedLeavesNoCapacity(t *testing.T) {
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
d.Suspend(1)
changeCallWaits(d, 1, 1)
submitRun(d, 2, "app", started, b)
awaitStart(t, started, 2)
// The resume is queued for a slot and a place in the chain, and overtaken by
// the run returning before either comes free.
d.Resume(1)
changeCallWaits(d, -1, 1)
awaitState(t, "task 1 did not take its place in the chain back", func() bool {
_, place, _ := capacity(d, 1)
return place
@@ -394,49 +408,7 @@ func TestResumeAfterTheRunReturnedLeavesNoCapacity(t *testing.T) {
close(c)
// A Resume arriving after the run returned is a no-op.
d.Resume(1)
d.Wait()
}
// callEvent builds one call step's event as the daemon receives it: the event
// the container's workflow engine published, JSON-encoded and decoded again by
// the agent-event bridge, so the wait flag arrives as a decoded bool.
func callEvent(t *testing.T, kind string, taskID int64, wait bool) events.Event {
t.Helper()
raw, err := json.Marshal(events.Event{Kind: kind, TaskID: taskID, Data: map[string]any{"wait": wait}})
if err != nil {
t.Fatal(err)
}
var e events.Event
if err := json.Unmarshal(raw, &e); err != nil {
t.Fatal(err)
}
return e
}
// Only a call the caller waits on releases its capacity, and the call
// finishing takes it back.
func TestCallWaitEventsReleaseCapacity(t *testing.T) {
d := newTaskDispatcher(1, nil)
started := make(chan int64, 4)
a, b := make(chan struct{}), make(chan struct{})
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
submitRun(d, 2, "app", started, b)
// A call that does not wait is not waited on: nothing is released.
applyCallWait(d, callEvent(t, events.KindWorkflowCallStarted, 1, false))
awaitNoStart(t, started, "while the call that started does not wait")
applyCallWait(d, callEvent(t, events.KindWorkflowCallStarted, 1, true))
awaitStart(t, started, 2)
close(b)
applyCallWait(d, callEvent(t, events.KindWorkflowCallFinished, 1, false))
awaitState(t, "task 1 did not take its capacity back", func() bool { return holdsCapacity(d, 1) })
close(a)
changeCallWaits(d, -1, 1)
d.Wait()
}
@@ -447,6 +419,65 @@ func TestCallWaitsWithoutABus(t *testing.T) {
d.WaitForTasks()
}
type callWaitStore struct {
storecontract.TaskStore
mu sync.Mutex
steps []task.StepExecution
}
func (s *callWaitStore) ListSteps(_ context.Context, id int64, _ int) ([]task.StepExecution, error) {
if id != 1 {
return nil, nil
}
s.mu.Lock()
defer s.mu.Unlock()
return append([]task.StepExecution(nil), s.steps...), nil
}
func TestCallWaitsRecoverDroppedStart(t *testing.T) {
store := &callWaitStore{steps: []task.StepExecution{
{Kind: task.StepKindCall, Status: taskstate.StepRunning, CalledExecutionID: 2},
{Kind: task.StepKindCall, Status: taskstate.StepRunning, CalledExecutionID: 3},
}}
cfg := config.Config{}
cfg.Containers.MaxConcurrency = 1
d := &Daemon{Cfg: config.NewHolder(cfg), Store: store, Bus: events.NewBus(), Log: slog.New(slog.DiscardHandler)}
dispatcher := d.taskDispatcher()
started := make(chan int64, 2)
a, b := make(chan struct{}), make(chan struct{})
defer func() { close(a); dispatcher.Wait() }()
defer func() {
if b != nil {
close(b)
}
}()
submitRun(dispatcher, 1, "app", started, a)
awaitStart(t, started, 1)
submitRun(dispatcher, 2, "app", started, b)
// The live start event was dropped; only the recorded call remains.
d.watchCallWaits(t.Context())
awaitStart(t, started, 2)
close(b)
b = nil
// Repeated reads must not accumulate waits; a partial finish stays suspended.
store.mu.Lock()
store.steps[0].Status = taskstate.StepSucceeded
store.mu.Unlock()
awaitState(t, "remaining call was not reconciled", func() bool {
dispatcher.mu.Lock()
defer dispatcher.mu.Unlock()
return dispatcher.holds[1].waits == 1
})
if holdsCapacity(dispatcher, 1) {
t.Fatal("parent reacquired capacity with a call still open")
}
// The finish event is also missing; the durable close restores capacity.
store.mu.Lock()
store.steps[1].Status = taskstate.StepSucceeded
store.mu.Unlock()
awaitState(t, "parent did not recover its capacity", func() bool { return holdsCapacity(dispatcher, 1) })
}
// stubTasks is a task bus with an empty stream, so a drain pass falls through
// to the store's ClaimNext.
type stubTasks struct{}
+59
View File
@@ -11,6 +11,7 @@ import (
workflowtask "github.com/samcharles93/archie-core/internal/domain/workflow/task"
"github.com/samcharles93/archie-core/internal/infrastructure/kit"
"github.com/samcharles93/archie-core/internal/infrastructure/kitrun"
"github.com/samcharles93/archie-core/internal/storage"
"github.com/samcharles93/archie-core/internal/taskstate"
)
@@ -112,3 +113,61 @@ func (d *Daemon) removeEndedKitVolumes(ctx context.Context, taskID int64, volume
d.Log.Warn("kit volume removal failed", "task", taskID, "err", err)
}
}
// ModelEgress opens a native task's model proxy session. The container holds
// a sentinel under each provider's key variable and reaches the providers
// through the egress proxy, which swaps in the org's bound key.
type ModelEgress interface {
Open(token, org string, providers map[string]config.Provider) (ModelSession, error)
Close(token string)
}
// TaskModels answers what a task's org resolves models against.
type TaskModels interface {
TaskModels(ctx context.Context, taskID int64) (config.OrgModels, error)
}
// taskConfig is task's configuration with its org's model aliases and own
// providers.
func (d *Daemon) taskConfig(ctx context.Context, task *workflow.Task) (config.Config, error) {
cfg := d.configFor(task)
if d.TaskModels == nil {
return cfg, nil
}
models, err := d.TaskModels.TaskModels(ctx, task.ID)
if err != nil {
return config.Config{}, fmt.Errorf("resolve the task's models: %w", err)
}
return cfg.WithOrgModels(models), nil
}
// ModelSession is what a native container needs for its model proxy session.
type ModelSession struct {
Env []string
Mounts []storage.Mount
}
// openModelEgress opens task's model session, or refuses: provider keys
// never reach a container, so without the proxy no model call can run.
func (d *Daemon) openModelEgress(ctx context.Context, task *workflow.Task, credential string) (ModelSession, func(), error) {
cfg, err := d.taskConfig(ctx, task)
if err != nil {
return ModelSession{}, nil, err
}
if d.ModelEgress == nil {
for _, p := range cfg.Providers {
if p.APIKeyEnv != "" {
return ModelSession{}, nil, fmt.Errorf("model egress proxy is unavailable on this daemon, and provider keys never enter a container")
}
}
return ModelSession{}, func() {}, nil
}
if credential == "" {
return ModelSession{}, nil, fmt.Errorf("model egress needs a run credential, and no State Store issues one")
}
session, err := d.ModelEgress.Open(credential, task.OrgOrDefault(), cfg.Providers)
if err != nil {
return ModelSession{}, nil, err
}
return session, func() { d.ModelEgress.Close(credential) }, nil
}
+45
View File
@@ -0,0 +1,45 @@
package daemon
import (
"context"
"slices"
"github.com/samcharles93/archie-core/internal/domain/storepkg"
"github.com/samcharles93/archie-core/internal/domain/workflow"
)
// PackageAuthorities answers what the operator accepted for an installed
// package. A package that is not installed, or not accepted, has none.
type PackageAuthorities interface {
AcceptedAuthority(ctx context.Context, name string) (storepkg.Authority, error)
}
// toolAllowlist is the tool allowlist a task runs with. nil offers every tool;
// a non-nil list offers exactly its members. A run of a package's workflow is
// bounded by the tools accepted for that package, so a missing, unaccepted or
// unreadable package offers none.
func (d *Daemon) toolAllowlist(ctx context.Context, task *workflow.Task, profileTools []string) []string {
name := workflow.PackageOf(task.WorkflowDefinitionYAML)
if name == "" {
if len(profileTools) == 0 {
return nil
}
return profileTools
}
var accepted []string
if d.PackageAuthorities != nil {
authority, err := d.PackageAuthorities.AcceptedAuthority(ctx, name)
if err != nil {
d.Log.Warn("package authority unavailable; offering no tools", "task", task.ID, "package", name, "err", err)
} else {
accepted = authority.Tools
}
}
allowed := []string{}
for _, tool := range accepted {
if len(profileTools) == 0 || slices.Contains(profileTools, tool) {
allowed = append(allowed, tool)
}
}
return allowed
}
+53
View File
@@ -0,0 +1,53 @@
package daemon
import (
"context"
"errors"
"log/slog"
"reflect"
"testing"
"github.com/samcharles93/archie-core/internal/domain/storepkg"
"github.com/samcharles93/archie-core/internal/domain/workflow"
)
type acceptedTools map[string]storepkg.Authority
func (a acceptedTools) AcceptedAuthority(_ context.Context, name string) (storepkg.Authority, error) {
if authority, ok := a[name]; ok {
return authority, nil
}
return storepkg.Authority{}, errors.New("not installed")
}
// TestToolAllowlistFollowsPackageAuthority pins that a package workflow is
// offered only the tools accepted for its package, and none when the package
// is missing or unaccepted, while an operator workflow keeps its profile's list.
func TestToolAllowlistFollowsPackageAuthority(t *testing.T) {
d := &Daemon{Log: slog.Default(), PackageAuthorities: acceptedTools{
"triage": {Tools: []string{"search", "fetch"}},
"unsigned": {},
}}
tests := []struct {
name string
yaml string
profile []string
want []string
}{
{"operator workflow without a profile list", "id: a\n", nil, nil},
{"operator workflow keeps its profile list", "id: a\n", []string{"x"}, []string{"x"}},
{"package workflow gets its accepted tools", "package: triage\nid: a\n", nil, []string{"search", "fetch"}},
{"profile narrows the accepted tools", "package: triage\nid: a\n", []string{"fetch", "shell"}, []string{"fetch"}},
{"profile cannot widen the accepted tools", "package: triage\nid: a\n", []string{"shell"}, []string{}},
{"package with nothing accepted", "package: unsigned\nid: a\n", nil, []string{}},
{"package that is gone", "package: gone\nid: a\n", nil, []string{}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := d.toolAllowlist(context.Background(), &workflow.Task{WorkflowDefinitionYAML: tt.yaml}, tt.profile)
if !reflect.DeepEqual(got, tt.want) {
t.Fatalf("got %#v, want %#v", got, tt.want)
}
})
}
}
+59
View File
@@ -0,0 +1,59 @@
package daemon
import (
"context"
"fmt"
"slices"
"strings"
"github.com/samcharles93/archie-core/internal/domain/workflow"
)
// startTriggers names how a task was started, in the vocabulary a package's
// accepted authority lists: "source:<source>" for where the task came from,
// "call" for a workflow.call, "binding:<id>" for a binding dispatch and
// "label:<name>" for each issue label.
func startTriggers(task *workflow.Task) []string {
source := task.Source
if source == "" {
source = workflow.SourceForge
}
triggers := []string{"source:" + source}
if task.CallParentTaskID != 0 {
triggers = append(triggers, "call")
}
if task.BindingID != "" {
triggers = append(triggers, "binding:"+task.BindingID)
}
for label := range strings.SplitSeq(task.Labels, ",") {
if label = strings.TrimSpace(label); label != "" {
triggers = append(triggers, "label:"+label)
}
}
return triggers
}
// checkPackageTrigger refuses to run a package's workflow unless one of the
// ways the task started is a trigger accepted for that package. A missing,
// unaccepted or unreadable package accepts none. Operator workflows are not
// bounded here.
func (d *Daemon) checkPackageTrigger(ctx context.Context, task *workflow.Task) error {
name := workflow.PackageOf(task.WorkflowDefinitionYAML)
if name == "" {
return nil
}
if d.PackageAuthorities == nil {
return fmt.Errorf("package %s: no accepted authority is available", name)
}
authority, err := d.PackageAuthorities.AcceptedAuthority(ctx, name)
if err != nil {
return fmt.Errorf("package %s: accepted authority unavailable: %w", name, err)
}
started := startTriggers(task)
for _, accepted := range authority.Triggers {
if slices.Contains(started, accepted) {
return nil
}
}
return fmt.Errorf("package %s: none of the triggers that started this task (%s) is accepted", name, strings.Join(started, ", "))
}
+44
View File
@@ -0,0 +1,44 @@
package daemon
import (
"context"
"testing"
"github.com/samcharles93/archie-core/internal/domain/storepkg"
"github.com/samcharles93/archie-core/internal/domain/workflow"
)
// TestCheckPackageTrigger pins that a package workflow starts only by a
// trigger accepted for its package, failing closed when none is accepted or
// the package is unknown, while operator workflows are unbounded.
func TestCheckPackageTrigger(t *testing.T) {
d := &Daemon{PackageAuthorities: acceptedTools{
"triage": storepkg.Authority{Triggers: []string{"label:triage", "binding:b1", "call"}},
"chatty": storepkg.Authority{Triggers: []string{"source:chat"}},
"unsigned": storepkg.Authority{},
}}
tests := []struct {
name string
task workflow.Task
wantErr bool
}{
{"operator workflow", workflow.Task{WorkflowDefinitionYAML: "id: a\n"}, false},
{"accepted label", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", Labels: "bug, triage"}, false},
{"accepted binding", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", BindingID: "b1"}, false},
{"accepted call", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", CallParentTaskID: 7}, false},
{"accepted source", workflow.Task{WorkflowDefinitionYAML: "package: chatty\nid: a\n", Source: "chat"}, false},
{"unaccepted label", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", Labels: "bug"}, true},
{"unaccepted binding", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", BindingID: "b2"}, true},
{"forge source not accepted", workflow.Task{WorkflowDefinitionYAML: "package: chatty\nid: a\n"}, true},
{"nothing accepted", workflow.Task{WorkflowDefinitionYAML: "package: unsigned\nid: a\n", Labels: "triage"}, true},
{"package that is gone", workflow.Task{WorkflowDefinitionYAML: "package: gone\nid: a\n", Labels: "triage"}, true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := d.checkPackageTrigger(context.Background(), &tt.task)
if (err != nil) != tt.wantErr {
t.Fatalf("err = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
+4
View File
@@ -181,4 +181,8 @@ var (
// ErrPolicyLockout is returned when a change would leave an org with no
// owner who may manage its policies.
ErrPolicyLockout = errors.New("access: change leaves no owner able to manage policies")
// ErrChainUnavailable reports that no policy engine has loaded yet: the
// request was refused without evaluating any policy, so it is an
// availability failure, not a denial.
ErrChainUnavailable = errors.New("access: policy chain not loaded")
)
+6
View File
@@ -131,6 +131,12 @@ type Decision struct {
// Allowed is the shared permit decision: no level denied.
func Allowed() Decision { return Decision{Allowed: true} }
// Unavailable is the refusal of a chain that has not loaded: nothing was
// evaluated, so it is an availability failure rather than a denial.
func Unavailable() Decision {
return Decision{Level: LevelInstance, Err: ErrChainUnavailable}
}
// DeniedAt records a denial decided at a level by the given policies.
func DeniedAt(level Level, policies []string) Decision {
return Decision{Level: level, Policies: policies}
+2 -2
View File
@@ -30,10 +30,10 @@ func Processes() []string { return []string{Daemon, Gateway, Messaging} }
// Applicability names the consumers of each resource, including boot-only settings.
func Applicability() map[string][]string {
kinds := map[string][]string{}
for _, kind := range []string{"task-policy", "provider-settings", "model-role-assignments", "repository-policies", "scheduling-policy", "tool-settings", "plugin-settings", "review-settings", "channel-settings", "container-runtime-policies", "credential-bindings", "identity-grants", "agent-profiles", "workflow-execution-settings", "extension-settings", "workflow-definitions", "workflow-enablement", "schedules", "eda-playbooks"} {
for _, kind := range []string{"task-policy", "provider-settings", "model-aliases", "repository-policies", "scheduling-policy", "tool-settings", "plugin-settings", "review-settings", "channel-settings", "container-runtime-policies", "credential-bindings", "identity-grants", "agent-profiles", "workflow-execution-settings", "extension-settings", "workflow-definitions", "workflow-enablement", "schedules", "eda-playbooks"} {
kinds[kind] = []string{Daemon}
}
for _, kind := range []string{"provider-settings", "model-role-assignments", "tool-settings", "channel-settings", "plugin-settings", "extension-settings", "personas", "soul", "skills", "curators"} {
for _, kind := range []string{"provider-settings", "model-aliases", "tool-settings", "channel-settings", "plugin-settings", "extension-settings", "personas", "soul", "skills", "curators"} {
kinds[kind] = append(kinds[kind], Gateway)
}
for _, kind := range []string{"channel-settings", "extension-settings"} {
+3 -7
View File
@@ -2,7 +2,8 @@ package binding
import (
"fmt"
"sort"
"maps"
"slices"
"strings"
"github.com/samcharles93/archie-core/internal/domain/mapping"
@@ -113,10 +114,5 @@ func (b Binding) RepositoryFrom(values map[string]any) (owner, repo string, ok b
}
func sortedKeys[V any](m map[string]V) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
return keys
return slices.Sorted(maps.Keys(m))
}
+9 -10
View File
@@ -7,22 +7,21 @@ import (
"errors"
"strings"
"time"
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
)
// HealthStatus is a curator's point-in-time health.
type HealthStatus string
// HealthStatus is a point-in-time health status.
type HealthStatus = lifecycle.HealthStatus
const (
HealthHealthy HealthStatus = "healthy"
HealthDegraded HealthStatus = "degraded"
HealthUnhealthy HealthStatus = "unhealthy"
HealthHealthy = lifecycle.HealthHealthy
HealthDegraded = lifecycle.HealthDegraded
HealthUnhealthy = lifecycle.HealthUnhealthy
)
// Health is a point-in-time curator health report.
type Health struct {
Status HealthStatus
Message string
}
// Health is a point-in-time health report.
type Health = lifecycle.Health
// Manifest is a curator's declared shape: what input wakes it, how often it
// runs, which tools it may reach, and which memory engine it uses. The
+19 -223
View File
@@ -1,50 +1,26 @@
package curator
import (
"context"
"errors"
"fmt"
"reflect"
"slices"
"sync"
"time"
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
)
// Registry errors.
var (
// ErrDuplicate is returned when a curator name is already registered.
ErrDuplicate = errors.New("duplicate curator")
// ErrStarted is returned when registration or lifecycle is attempted
// after the registry has started.
ErrStarted = errors.New("curator registry already started")
ErrDuplicate = lifecycle.ErrDuplicate
ErrStarted = lifecycle.ErrStarted
)
type registryState uint8
const (
stateIdle registryState = iota
stateRunning
stateStopped
)
type curatorStatus struct {
started bool
startErr error
}
// Registry owns curator registration and lifecycle. A curator's failure
// affects only that curator.
type Registry struct {
mu sync.Mutex
host Registrar
curators map[string]CuratorEngine
status map[string]curatorStatus
order []string
state registryState
*lifecycle.Registry[CuratorEngine, Registrar]
host Registrar
// activity tracks recent per-curator run history for observability.
// It has its own mutex and is safe to
// use independently of r.mu.
// It has its own mutex.
activity *activityTracker
}
@@ -56,9 +32,8 @@ func NewRegistry(host Registrar) *Registry {
host.Clock = systemClock{}
}
return &Registry{
Registry: lifecycle.New[CuratorEngine, Registrar]("curator", host.Log),
host: host,
curators: make(map[string]CuratorEngine),
status: make(map[string]curatorStatus),
activity: newActivityTracker(),
}
}
@@ -77,7 +52,7 @@ func (r *Registry) Activity(name string) (Activity, bool) {
// Register validates the curator's manifest and binds it to a filtered host
// view. Failures affect only that curator.
func (r *Registry) Register(c CuratorEngine) error {
if isNilEngine(c) {
if lifecycle.IsNil(c) {
return errors.New("curator: nil engine")
}
m := c.Manifest()
@@ -87,21 +62,13 @@ func (r *Registry) Register(c CuratorEngine) error {
if err := r.validateDeclared(m); err != nil {
return err
}
return r.Add(c, r.filter(m))
}
r.mu.Lock()
defer r.mu.Unlock()
if r.state != stateIdle {
return ErrStarted
}
if _, dup := r.curators[c.Name()]; dup {
return fmt.Errorf("%w: %s", ErrDuplicate, c.Name())
}
if err := bindSafely(c, r.filter(m)); err != nil {
return err
}
r.curators[c.Name()] = c
r.order = append(r.order, c.Name())
return nil
// Host returns the raw registrar bundle the registry was built with. The
// daemon itself is never part of it.
func (r *Registry) Host() Registrar {
return r.host
}
// validateDeclared enforces "only the host services a curator's declared
@@ -152,180 +119,9 @@ func agentic(m Manifest) bool {
return len(m.Tools) > 0 || m.Skills || m.MemoryEngine != "" || m.Conversations
}
// bindSafely isolates a panicking Bind: a curator that panics while
// receiving its host access is refused without taking the registry down.
func bindSafely(c CuratorEngine, host Registrar) (err error) {
defer func() {
if p := recover(); p != nil {
err = fmt.Errorf("curator %s: bind panic: %v", c.Name(), p)
}
}()
c.Bind(host)
return nil
}
// Host returns the raw registrar bundle the registry was built with — the
// shared services (events, clock, model) the runtime and future surfaces
// use. It is narrow typed; the daemon itself is never part of it.
func (r *Registry) Host() Registrar {
r.mu.Lock()
defer r.mu.Unlock()
return r.host
}
// Get returns the registered curator by name.
func (r *Registry) Get(name string) (CuratorEngine, bool) {
r.mu.Lock()
defer r.mu.Unlock()
c, ok := r.curators[name]
return c, ok
}
// Names returns the registered curator names in sorted order.
func (r *Registry) Names() []string {
r.mu.Lock()
defer r.mu.Unlock()
names := make([]string, 0, len(r.curators))
for name := range r.curators {
names = append(names, name)
}
slices.Sort(names)
return names
}
// Start starts every registered curator in registration order. A curator
// whose Start fails (error or panic) is marked unhealthy and skipped; the
// rest start regardless, and all failures are joined into the returned
// error. This is the family's failure-isolation boundary for lifecycle.
func (r *Registry) Start(ctx context.Context) error {
r.mu.Lock()
if r.state == stateRunning {
r.mu.Unlock()
return ErrStarted
}
r.state = stateRunning
order := slices.Clone(r.order)
r.mu.Unlock()
var errs []error
for _, name := range order {
c := r.curators[name]
err := startSafely(ctx, c)
r.mu.Lock()
if err != nil {
r.status[name] = curatorStatus{startErr: err}
if r.host.Log != nil {
r.host.Log.Error("curator lifecycle failed", "curator", name, "phase", "start", "err", err)
}
errs = append(errs, fmt.Errorf("curator %s: %w", name, err))
} else {
r.status[name] = curatorStatus{started: true}
}
r.mu.Unlock()
}
return errors.Join(errs...)
}
func startSafely(ctx context.Context, c CuratorEngine) (err error) {
defer func() {
if p := recover(); p != nil {
err = fmt.Errorf("start panic: %v", p)
}
}()
return c.Start(ctx)
}
// Health reports every registered curator's health. A curator whose Start
// failed is reported unhealthy with the failure; otherwise the curator's own
// Health is used. A panicking Health is reported unhealthy, not propagated.
func (r *Registry) Health(ctx context.Context) map[string]Health {
r.mu.Lock()
defer r.mu.Unlock()
out := make(map[string]Health, len(r.curators))
for name, c := range r.curators {
if st, ok := r.status[name]; ok && st.startErr != nil {
out[name] = Health{Status: HealthUnhealthy, Message: st.startErr.Error()}
continue
}
out[name] = healthSafely(ctx, c)
}
return out
}
func healthSafely(ctx context.Context, c CuratorEngine) (h Health) {
defer func() {
if p := recover(); p != nil {
h = Health{Status: HealthUnhealthy, Message: fmt.Sprintf("health panic: %v", p)}
}
}()
return c.Health(ctx)
}
// Stop stops started curators in reverse order, bounded by ctx, and joins
// their errors.
func (r *Registry) Stop(ctx context.Context) error {
r.mu.Lock()
if r.state == stateIdle || r.state == stateStopped {
r.mu.Unlock()
return nil
}
r.state = stateStopped
order := slices.Clone(r.order)
r.mu.Unlock()
var errs []error
for _, name := range slices.Backward(order) {
r.mu.Lock()
st, ok := r.status[name]
r.mu.Unlock()
if !ok || !st.started {
continue
}
c := r.curators[name]
if err := stopSafely(ctx, c); err != nil {
if r.host.Log != nil {
r.host.Log.Error("curator lifecycle failed", "curator", name, "phase", "stop", "err", err)
}
errs = append(errs, fmt.Errorf("curator %s: %w", name, err))
}
}
return errors.Join(errs...)
}
func stopSafely(ctx context.Context, c CuratorEngine) (err error) {
defer func() {
if p := recover(); p != nil {
err = fmt.Errorf("stop panic: %v", p)
}
}()
return c.Stop(ctx)
}
func isNilEngine(c CuratorEngine) bool {
v := reflect.ValueOf(c)
switch v.Kind() {
case reflect.Invalid:
return true
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return v.IsNil()
}
return false
}
func (r *Registry) replaceDefinitions(candidate *Registry) {
r.mu.Lock()
defer r.mu.Unlock()
r.order = slices.DeleteFunc(r.order, func(name string) bool {
if _, custom := r.curators[name].(*DefinitionEngine); !custom {
return false
}
delete(r.curators, name)
delete(r.status, name)
return true
})
for _, name := range candidate.order {
r.curators[name] = candidate.curators[name]
r.order = append(r.order, name)
}
r.Replace(func(c CuratorEngine) bool {
_, custom := c.(*DefinitionEngine)
return custom
}, candidate.Registry)
}
+3 -5
View File
@@ -3,7 +3,9 @@
package expr
import (
"maps"
"reflect"
"slices"
"sort"
"strings"
@@ -252,11 +254,7 @@ func actionReferences(ast *cel.Ast) ([]string, bool) {
}
})
celast.PreOrderVisit(checked.Expr(), visitor)
ids := make([]string, 0, len(seen))
for id := range seen {
ids = append(ids, id)
}
sort.Strings(ids)
ids := slices.Sorted(maps.Keys(seen))
return ids, identCount == staticCount
}
+3 -6
View File
@@ -8,8 +8,9 @@ import (
"context"
"fmt"
"log/slog"
"maps"
"reflect"
"sort"
"slices"
"strings"
"github.com/samcharles93/archie-core/internal/domain/eda/module/log"
@@ -164,11 +165,7 @@ func decodeResultStruct(kind string, t reflect.Type, raw map[string]any) (any, e
byName[name] = out.Field(i)
}
keys := make([]string, 0, len(raw))
for key := range raw {
keys = append(keys, key)
}
sort.Strings(keys)
keys := slices.Sorted(maps.Keys(raw))
for _, key := range keys {
if _, ok := byName[key]; !ok {
return nil, fmt.Errorf("module %s: unknown result field %q", kind, key)
+6 -17
View File
@@ -6,10 +6,11 @@ import (
"context"
"crypto/sha256"
"fmt"
"maps"
"os"
"path/filepath"
"reflect"
"sort"
"slices"
"strings"
"gopkg.in/yaml.v3"
@@ -235,7 +236,7 @@ func Load(dir string, modules Modules) (*Store, error) {
// Compile validates and compiles playbook documents in id order. Any invalid
// playbook fails the whole set: a partially valid set never runs.
func Compile(docs []Document, modules Modules) (*Store, error) {
sort.Slice(docs, func(i, j int) bool { return docs[i].ID < docs[j].ID })
slices.SortFunc(docs, func(a, b Document) int { return strings.Compare(a.ID, b.ID) })
store := &Store{modules: modules}
for _, doc := range docs {
pb, err := compileOne(doc, modules)
@@ -481,11 +482,7 @@ func compileArgs(path string, lines map[string]int, label, kind string, raw map[
return nil, err
}
}
keys := make([]string, 0, len(raw))
for key := range raw {
keys = append(keys, key)
}
sort.Strings(keys)
keys := slices.Sorted(maps.Keys(raw))
args := make(map[string]*expr.Program, len(raw))
for _, key := range keys {
prg, err := compileExpr(at(path, lines[key]), argsLabel(label, key), raw[key], env)
@@ -512,11 +509,7 @@ func validateArgsKeys(path string, lines map[string]int, label, kind string, arg
for field := range t.Fields() {
fields[strings.ToLower(field.Name)] = struct{}{}
}
keys := make([]string, 0, len(raw))
for key := range raw {
keys = append(keys, key)
}
sort.Strings(keys)
keys := slices.Sorted(maps.Keys(raw))
loc := kind + " kind"
if label != "" {
loc = label + " " + loc
@@ -671,11 +664,7 @@ func (a Action) whenHolds(ctx expr.Context) bool {
// the first evaluation error.
func (a Action) evalArgs(ctx expr.Context) (map[string]any, error) {
out := make(map[string]any, len(a.Args))
keys := make([]string, 0, len(a.Args))
for key := range a.Args {
keys = append(keys, key)
}
sort.Strings(keys)
keys := slices.Sorted(maps.Keys(a.Args))
for _, key := range keys {
if a.Args[key] == nil {
continue
+17
View File
@@ -17,6 +17,12 @@ type Subject struct {
Subject string `json:"subject"`
}
// EmailSubject is the placeholder binding of a person added by email: the
// issuer's verified address stands in until their first sign-in claims it.
func EmailSubject(issuer, email string) Subject {
return Subject{Issuer: issuer, Subject: "email:" + strings.ToLower(strings.TrimSpace(email))}
}
func (s Subject) Validate() error {
if strings.TrimSpace(s.Issuer) == "" || strings.TrimSpace(s.Subject) == "" {
return fmt.Errorf("%w: issuer and subject are required", ErrInvalid)
@@ -28,6 +34,9 @@ func (s Subject) Validate() error {
// verification, never an assertion from the caller.
type Credential struct {
Subject Subject
// Email is the address the provider vouched for (email_verified), empty
// otherwise. It resolves a person added by email before any subject is known.
Email string
Scopes []string
Expires time.Time
}
@@ -69,6 +78,9 @@ type SubjectResolver interface {
// configured with resolves to an identity, and a subject a provider asserts
// resolves to the same identity.
ResolveSubject(context.Context, Subject) (Identity, error)
// ClaimSubject moves a placeholder binding onto subject and returns its
// identity; a placeholder that is gone (already claimed) is ErrNotFound.
ClaimSubject(ctx context.Context, placeholder, subject Subject) (Identity, error)
}
// ProviderSession is what a completed authorization flow yields: the credential
@@ -126,6 +138,11 @@ func Resolve(ctx context.Context, subjects SubjectResolver, credential Credentia
return Identity{}, fmt.Errorf("%w: %w", ErrCredentialRejected, err)
}
value, err := subjects.ResolveSubject(ctx, credential.Subject)
if errors.Is(err, ErrNotFound) && credential.Email != "" {
// The first sign-in binds the provider's subject in place of the
// address, so later sign-ins rest on the subject alone.
value, err = subjects.ClaimSubject(ctx, EmailSubject(credential.Subject.Issuer, credential.Email), credential.Subject)
}
if err != nil {
if errors.Is(err, ErrNotFound) {
return Identity{}, fmt.Errorf("%w: %s", ErrSubjectUnbound, credential.Subject.Subject)
+5 -19
View File
@@ -3,9 +3,11 @@ package image
import (
"errors"
"fmt"
"reflect"
"maps"
"slices"
"sync"
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
)
// ErrDuplicate is returned when a provider name is already registered.
@@ -28,7 +30,7 @@ func NewRegistry() *Registry {
// Registration fails on a nil provider, an invalid Capability, or a
// duplicate name.
func (r *Registry) Register(p Provider) error {
if isNilProvider(p) {
if lifecycle.IsNil(p) {
return errors.New("image: nil provider")
}
if err := p.Capability().Validate(); err != nil {
@@ -56,12 +58,7 @@ func (r *Registry) Get(name string) (Provider, bool) {
func (r *Registry) Names() []string {
r.mu.RLock()
defer r.mu.RUnlock()
names := make([]string, 0, len(r.providers))
for name := range r.providers {
names = append(names, name)
}
slices.Sort(names)
return names
return slices.Sorted(maps.Keys(r.providers))
}
// ByClass returns the sorted names of registered providers matching class.
@@ -77,14 +74,3 @@ func (r *Registry) ByClass(class ProviderClass) []string {
slices.Sort(names)
return names
}
func isNilProvider(p Provider) bool {
v := reflect.ValueOf(p)
switch v.Kind() {
case reflect.Invalid:
return true
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return v.IsNil()
}
return false
}
+247
View File
@@ -0,0 +1,247 @@
// Package lifecycle is the registration and start/health/stop machinery
// shared by engine families such as curators and memory engines.
package lifecycle
import (
"context"
"errors"
"fmt"
"log/slog"
"maps"
"reflect"
"slices"
"sync"
)
// HealthStatus is an engine's point-in-time health.
type HealthStatus string
const (
HealthHealthy HealthStatus = "healthy"
HealthDegraded HealthStatus = "degraded"
HealthUnhealthy HealthStatus = "unhealthy"
)
// Health is a point-in-time engine health report.
type Health struct {
Status HealthStatus
Message string
}
// Registry errors.
var (
// ErrDuplicate is returned when an engine name is already registered.
ErrDuplicate = errors.New("duplicate engine")
// ErrStarted is returned when registration or lifecycle is attempted
// after the registry has started.
ErrStarted = errors.New("registry already started")
)
// Engine is a named, host-bound component with a lifecycle.
type Engine[H any] interface {
Name() string
Bind(host H)
Start(ctx context.Context) error
Health(ctx context.Context) Health
Stop(ctx context.Context) error
}
type state uint8
const (
stateIdle state = iota
stateRunning
stateStopped
)
type status struct {
started bool
startErr error
}
// Registry owns registration and lifecycle for one engine family. An
// engine's failure, including a panic, affects only that engine.
type Registry[E Engine[H], H any] struct {
kind string
log *slog.Logger
mu sync.Mutex
engines map[string]E
status map[string]status
order []string
state state
}
// New builds an empty registry. kind prefixes errors and log lines; a nil
// log disables lifecycle logging.
func New[E Engine[H], H any](kind string, log *slog.Logger) *Registry[E, H] {
return &Registry[E, H]{
kind: kind,
log: log,
engines: make(map[string]E),
status: make(map[string]status),
}
}
// IsNil reports whether v is nil or an interface holding a nil value.
func IsNil(v any) bool {
rv := reflect.ValueOf(v)
switch rv.Kind() {
case reflect.Invalid:
return true
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return rv.IsNil()
}
return false
}
// Add binds e to host and registers it. A panicking Bind refuses the engine.
func (r *Registry[E, H]) Add(e E, host H) error {
r.mu.Lock()
defer r.mu.Unlock()
if r.state != stateIdle {
return fmt.Errorf("%s: %w", r.kind, ErrStarted)
}
name := e.Name()
if _, dup := r.engines[name]; dup {
return fmt.Errorf("%w: %s %s", ErrDuplicate, r.kind, name)
}
if err := guard(func() error { e.Bind(host); return nil }); err != nil {
return fmt.Errorf("%s %s: bind %w", r.kind, name, err)
}
r.engines[name] = e
r.order = append(r.order, name)
return nil
}
// Get returns the registered engine by name.
func (r *Registry[E, H]) Get(name string) (E, bool) {
r.mu.Lock()
defer r.mu.Unlock()
e, ok := r.engines[name]
return e, ok
}
// Names returns the registered engine names in sorted order.
func (r *Registry[E, H]) Names() []string {
r.mu.Lock()
defer r.mu.Unlock()
return slices.Sorted(maps.Keys(r.engines))
}
// Start starts every engine in registration order. An engine whose Start
// fails is marked unhealthy and skipped; the rest start regardless, and all
// failures are joined into the returned error. A stopped registry may be
// started again.
func (r *Registry[E, H]) Start(ctx context.Context) error {
r.mu.Lock()
if r.state == stateRunning {
r.mu.Unlock()
return fmt.Errorf("%s: %w", r.kind, ErrStarted)
}
r.state = stateRunning
order := slices.Clone(r.order)
r.mu.Unlock()
var errs []error
for _, name := range order {
e, _ := r.Get(name)
err := guard(func() error { return e.Start(ctx) })
r.mu.Lock()
if err != nil {
r.status[name] = status{startErr: err}
errs = append(errs, r.fail(name, "start", err))
} else {
r.status[name] = status{started: true}
}
r.mu.Unlock()
}
return errors.Join(errs...)
}
// Health reports every engine's health. An engine whose Start failed, or
// whose Health panics, is reported unhealthy.
func (r *Registry[E, H]) Health(ctx context.Context) map[string]Health {
r.mu.Lock()
defer r.mu.Unlock()
out := make(map[string]Health, len(r.engines))
for name, e := range r.engines {
if st := r.status[name]; st.startErr != nil {
out[name] = Health{Status: HealthUnhealthy, Message: st.startErr.Error()}
continue
}
out[name] = healthSafely(ctx, e)
}
return out
}
// Stop stops started engines in reverse order, bounded by ctx, and joins
// their errors.
func (r *Registry[E, H]) Stop(ctx context.Context) error {
r.mu.Lock()
if r.state != stateRunning {
r.mu.Unlock()
return nil
}
r.state = stateStopped
order := slices.Clone(r.order)
r.mu.Unlock()
var errs []error
for _, name := range slices.Backward(order) {
r.mu.Lock()
e, st := r.engines[name], r.status[name]
r.mu.Unlock()
if !st.started {
continue
}
if err := guard(func() error { return e.Stop(ctx) }); err != nil {
errs = append(errs, r.fail(name, "stop", err))
}
}
return errors.Join(errs...)
}
// Replace removes the engines drop selects and appends every engine from
// src, keeping src's order. Call it only while stopped.
func (r *Registry[E, H]) Replace(drop func(E) bool, src *Registry[E, H]) {
r.mu.Lock()
defer r.mu.Unlock()
r.order = slices.DeleteFunc(r.order, func(name string) bool {
if !drop(r.engines[name]) {
return false
}
delete(r.engines, name)
delete(r.status, name)
return true
})
for _, name := range src.order {
r.engines[name] = src.engines[name]
r.order = append(r.order, name)
}
}
func (r *Registry[E, H]) fail(name, phase string, err error) error {
if r.log != nil {
r.log.Error(r.kind+" lifecycle failed", "name", name, "phase", phase, "err", err)
}
return fmt.Errorf("%s %s: %w", r.kind, name, err)
}
// guard runs fn, converting a panic into an error.
func guard(fn func() error) (err error) {
defer func() {
if p := recover(); p != nil {
err = fmt.Errorf("panic: %v", p)
}
}()
return fn()
}
func healthSafely[E Engine[H], H any](ctx context.Context, e E) (h Health) {
defer func() {
if p := recover(); p != nil {
h = Health{Status: HealthUnhealthy, Message: fmt.Sprintf("health panic: %v", p)}
}
}()
return e.Health(ctx)
}
+28 -14
View File
@@ -8,22 +8,22 @@ import (
"fmt"
"strconv"
"time"
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
"github.com/samcharles93/archie-core/internal/domain/org"
)
// HealthStatus is an engine's point-in-time health.
type HealthStatus string
// HealthStatus is a point-in-time health status.
type HealthStatus = lifecycle.HealthStatus
const (
HealthHealthy HealthStatus = "healthy"
HealthDegraded HealthStatus = "degraded"
HealthUnhealthy HealthStatus = "unhealthy"
HealthHealthy = lifecycle.HealthHealthy
HealthDegraded = lifecycle.HealthDegraded
HealthUnhealthy = lifecycle.HealthUnhealthy
)
// Health is a point-in-time engine health report.
type Health struct {
Status HealthStatus
Message string
}
// Health is a point-in-time health report.
type Health = lifecycle.Health
// Manifest is an engine's declared shape. RequiresNetwork marks an engine
// that calls an external service, so a caller can size timeouts and
@@ -78,6 +78,9 @@ type Scope struct {
Kind ScopeKind
Agent AgentID
User IdentityID
// Org owns the region. Every kind, global included, is per org: one
// org's facts never reach another org's turns. Empty is the system org.
Org org.OrgID
}
// Validate rejects an unknown kind, a missing component, and — just as
@@ -116,6 +119,14 @@ func (s Scope) Validate() error {
// Key returns the scope's length-prefixed storage key, or "" for an invalid
// scope.
func (s Scope) Key() string {
key := s.kindKey()
if key == "" || s.Org == "" || s.Org == org.DefaultOrgID {
return key
}
return "org:" + lengthPrefixed(string(s.Org)) + ":" + key
}
func (s Scope) kindKey() string {
switch s.Kind {
case ScopeGlobal:
return "global"
@@ -151,12 +162,15 @@ type Subject struct {
// carries no per-person identity (the dashboard, a webhook) — which
// means "show none of it", never "fall back to a wider scope".
UserID IdentityID
// Org is the org the turn runs in; every scope it reads or writes
// belongs to that org.
Org org.OrgID
}
// Scopes returns the subject's readable scopes, narrowest first: agent-user,
// agent, user, global.
func (s Subject) Scopes() []Scope {
return append(s.WritableScopes(), Scope{Kind: ScopeGlobal})
return append(s.WritableScopes(), Scope{Kind: ScopeGlobal, Org: s.Org})
}
// WritableScopes returns the scopes a write for this subject may name —
@@ -165,13 +179,13 @@ func (s Subject) Scopes() []Scope {
func (s Subject) WritableScopes() []Scope {
var scopes []Scope
if s.AgentID != "" && s.UserID != "" {
scopes = append(scopes, Scope{Kind: ScopeAgentUser, Agent: s.AgentID, User: s.UserID})
scopes = append(scopes, Scope{Kind: ScopeAgentUser, Agent: s.AgentID, User: s.UserID, Org: s.Org})
}
if s.AgentID != "" {
scopes = append(scopes, Scope{Kind: ScopeAgent, Agent: s.AgentID})
scopes = append(scopes, Scope{Kind: ScopeAgent, Agent: s.AgentID, Org: s.Org})
}
if s.UserID != "" {
scopes = append(scopes, Scope{Kind: ScopeUser, User: s.UserID})
scopes = append(scopes, Scope{Kind: ScopeUser, User: s.UserID, Org: s.Org})
}
return scopes
}
+11 -199
View File
@@ -1,45 +1,22 @@
package memory
import (
"context"
"errors"
"fmt"
"reflect"
"slices"
"sync"
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
)
// Registry errors.
var (
// ErrDuplicate is returned when an engine name is already registered.
ErrDuplicate = errors.New("duplicate memory engine")
// ErrStarted is returned when registration or lifecycle is attempted
// after the registry has started.
ErrStarted = errors.New("memory registry already started")
ErrDuplicate = lifecycle.ErrDuplicate
ErrStarted = lifecycle.ErrStarted
)
type registryState uint8
const (
stateIdle registryState = iota
stateRunning
stateStopped
)
type engineStatus struct {
started bool
startErr error
}
// Registry owns memory engine registration and lifecycle. An engine's
// failure affects only that engine.
type Registry struct {
mu sync.Mutex
host Registrar
engines map[string]MemoryEngine
status map[string]engineStatus
order []string
state registryState
*lifecycle.Registry[MemoryEngine, Registrar]
host Registrar
}
// NewRegistry builds the family registry. A nil Clock is replaced with the
@@ -48,182 +25,17 @@ func NewRegistry(host Registrar) *Registry {
if host.Clock == nil {
host.Clock = systemClock{}
}
return &Registry{
host: host,
engines: make(map[string]MemoryEngine),
status: make(map[string]engineStatus),
}
return &Registry{Registry: lifecycle.New[MemoryEngine, Registrar]("memory engine", nil), host: host}
}
// Register validates the engine's declared shape and adds it, binding the
// engine to the registrar. Registration fails on nil engines, invalid
// manifests, duplicate names, or a panicking Bind — and a failure affects
// only that engine.
// Register validates the engine's declared shape and binds it to the
// registrar. A failure affects only that engine.
func (r *Registry) Register(e MemoryEngine) error {
if isNilEngine(e) {
if lifecycle.IsNil(e) {
return errors.New("memory: nil engine")
}
if err := e.Manifest().Validate(); err != nil {
return err
}
r.mu.Lock()
defer r.mu.Unlock()
if r.state != stateIdle {
return ErrStarted
}
if _, dup := r.engines[e.Name()]; dup {
return fmt.Errorf("%w: %s", ErrDuplicate, e.Name())
}
if err := bindSafely(e, r.host); err != nil {
return err
}
r.engines[e.Name()] = e
r.order = append(r.order, e.Name())
return nil
}
// bindSafely isolates a panicking Bind: an engine that panics while
// receiving its host access is refused without taking the registry down.
func bindSafely(e MemoryEngine, host Registrar) (err error) {
defer func() {
if p := recover(); p != nil {
err = fmt.Errorf("memory engine %s: bind panic: %v", e.Name(), p)
}
}()
e.Bind(host)
return nil
}
// Get returns the registered engine by name.
func (r *Registry) Get(name string) (MemoryEngine, bool) {
r.mu.Lock()
defer r.mu.Unlock()
e, ok := r.engines[name]
return e, ok
}
// Names returns the registered engine names in sorted order.
func (r *Registry) Names() []string {
r.mu.Lock()
defer r.mu.Unlock()
names := make([]string, 0, len(r.engines))
for name := range r.engines {
names = append(names, name)
}
slices.Sort(names)
return names
}
// Start starts every registered engine in registration order. An engine
// whose Start fails (error or panic) is marked unhealthy and skipped; the
// rest start regardless, and all failures are joined into the returned
// error. This is the family's failure-isolation boundary for lifecycle.
func (r *Registry) Start(ctx context.Context) error {
r.mu.Lock()
if r.state != stateIdle {
r.mu.Unlock()
return ErrStarted
}
r.state = stateRunning
order := slices.Clone(r.order)
r.mu.Unlock()
var errs []error
for _, name := range order {
e := r.engines[name]
err := startSafely(ctx, e)
r.mu.Lock()
if err != nil {
r.status[name] = engineStatus{startErr: err}
errs = append(errs, fmt.Errorf("memory engine %s: %w", name, err))
} else {
r.status[name] = engineStatus{started: true}
}
r.mu.Unlock()
}
return errors.Join(errs...)
}
func startSafely(ctx context.Context, e MemoryEngine) (err error) {
defer func() {
if p := recover(); p != nil {
err = fmt.Errorf("start panic: %v", p)
}
}()
return e.Start(ctx)
}
// Health reports every registered engine's health. An engine whose Start
// failed is reported unhealthy with the failure; otherwise the engine's own
// Health is used. A panicking Health is reported unhealthy, not propagated.
func (r *Registry) Health(ctx context.Context) map[string]Health {
r.mu.Lock()
defer r.mu.Unlock()
out := make(map[string]Health, len(r.engines))
for name, e := range r.engines {
if st, ok := r.status[name]; ok && st.startErr != nil {
out[name] = Health{Status: HealthUnhealthy, Message: st.startErr.Error()}
continue
}
out[name] = healthSafely(ctx, e)
}
return out
}
func healthSafely(ctx context.Context, e MemoryEngine) (h Health) {
defer func() {
if p := recover(); p != nil {
h = Health{Status: HealthUnhealthy, Message: fmt.Sprintf("health panic: %v", p)}
}
}()
return e.Health(ctx)
}
// Stop stops started engines in reverse order, bounded by ctx, and joins
// their errors.
func (r *Registry) Stop(ctx context.Context) error {
r.mu.Lock()
if r.state == stateIdle || r.state == stateStopped {
r.mu.Unlock()
return nil
}
r.state = stateStopped
order := slices.Clone(r.order)
r.mu.Unlock()
var errs []error
for _, name := range slices.Backward(order) {
r.mu.Lock()
st, ok := r.status[name]
r.mu.Unlock()
if !ok || !st.started {
continue
}
e := r.engines[name]
if err := stopSafely(ctx, e); err != nil {
errs = append(errs, fmt.Errorf("memory engine %s: %w", name, err))
}
}
return errors.Join(errs...)
}
func stopSafely(ctx context.Context, e MemoryEngine) (err error) {
defer func() {
if p := recover(); p != nil {
err = fmt.Errorf("stop panic: %v", p)
}
}()
return e.Stop(ctx)
}
func isNilEngine(e MemoryEngine) bool {
v := reflect.ValueOf(e)
switch v.Kind() {
case reflect.Invalid:
return true
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return v.IsNil()
}
return false
return r.Add(e, r.host)
}
-1
View File
@@ -15,7 +15,6 @@ type ChatSnapshot struct {
// frontends display it rather than stamping their own build, so a
// partially upgraded deployment cannot report itself as matched.
Version string
RestartAvailable bool
CancellationAvailable bool
// VoiceAvailable reports that a voice clip is transcribed into the turn.
VoiceAvailable bool
+1 -2
View File
@@ -10,7 +10,7 @@ var localCommands = []string{
"/new", "/reset", "/topic", "/retry", "/undo",
"/title", "/branch", "/fork", "/compress", "/compact",
"/whoami", "/profile", "/sessions", "/resume", "/delete", "/agents",
"/personality", "/help", "/version", "/update", "/restart",
"/personality", "/help", "/version", "/update",
}
// CommandSpec describes a local command for adapter-provided discovery.
@@ -49,7 +49,6 @@ var localCommandSpecs = []CommandSpec{
{Command: "/help", Description: "See what Archie can do", Usage: "/help"},
{Command: "/version", Description: "Show installed Archie versions", Usage: "/version"},
{Command: "/update", Description: "Check for Archie updates", Usage: "/update"},
{Command: "/restart", Description: "Reload the chat adapter", Usage: "/restart"},
}
// LocalCommands returns the command names Route answers from local state.
+2 -5
View File
@@ -8,6 +8,7 @@ import (
"errors"
"fmt"
"maps"
"slices"
"sort"
"strconv"
"strings"
@@ -323,11 +324,7 @@ func renderSetting(resource SettingResource, descriptor SettingDescriptor) strin
}
return fmt.Sprintf("%s (version %d):\n%s", resource.Kind, resource.Version, value)
}
names := make([]string, 0, len(resource.Value))
for name := range resource.Value {
names = append(names, name)
}
sort.Strings(names)
names := slices.Sorted(maps.Keys(resource.Value))
lines := []string{fmt.Sprintf("%s (version %d):", resource.Kind, resource.Version)}
for _, name := range names {
field := descriptor.Fields[name]
+15
View File
@@ -22,3 +22,18 @@ func OrgFromContext(ctx context.Context) OrgID {
}
return DefaultOrgID
}
type scopeContextKey struct{}
// WithScope confines a request's reads to one org. A request without a scope
// is an internal service acting across orgs, such as the daemon running every
// org's work or webhook intake resolving a source by its path.
func WithScope(ctx context.Context, id OrgID) context.Context {
return context.WithValue(ctx, scopeContextKey{}, id)
}
// Scope returns the org WithScope confined ctx to.
func Scope(ctx context.Context) (OrgID, bool) {
id, ok := ctx.Value(scopeContextKey{}).(OrgID)
return id, ok && id != ""
}
@@ -81,6 +81,7 @@ type HarnessSecretStore interface {
// status guard. It is separate from the already broad lifecycle contract so
// consumers that only run tasks do not acquire an operator-only capability.
type TaskArchiver interface {
OutcomePoster
ArchiveTask(ctx context.Context, taskID int64, fromStatus string, audit events.Event) (eventID int64, err error)
ClearTerminalTasks(ctx context.Context) (int64, error)
}
@@ -120,6 +121,16 @@ type RemediationStarter interface {
SetReviewCursors(ctx context.Context, taskID, reviewCursor, commentCursor int64) error
}
// OutcomePoster tracks which finished chat tasks have reported back to their
// conversation.
type OutcomePoster interface {
// UnpostedOutcomes returns finished chat tasks whose outcome has not been
// posted to the conversation that created them.
UnpostedOutcomes(ctx context.Context) ([]*task.Task, error)
// MarkOutcomePosted records that a task's outcome was posted.
MarkOutcomePosted(ctx context.Context, taskID int64) error
}
// TaskQueries groups read-only task accessors.
type TaskQueries interface {
TaskByIssue(ctx context.Context, owner, repo string, number int) (*task.Task, error)
@@ -0,0 +1,65 @@
package storepkg
import (
"context"
"errors"
"slices"
"testing"
)
type fakeAuto struct {
refs []OrgPackage
listErr error
failing string
tried []string
}
func (f *fakeAuto) ListAutoUpdate(context.Context) ([]OrgPackage, error) { return f.refs, f.listErr }
func (f *fakeAuto) UpdatePackage(_ context.Context, _, name string) (Installed, error) {
f.tried = append(f.tried, name)
if name == f.failing {
return Installed{}, errors.New("catalogue unreachable")
}
return Installed{}, nil
}
func (f *fakeAuto) ApprovePackageUpdate(context.Context, string, string) (Installed, error) {
return Installed{}, nil
}
func (f *fakeAuto) RollbackPackage(context.Context, string, string) (Installed, error) {
return Installed{}, nil
}
func (f *fakeAuto) SetPackageUpdatePolicy(context.Context, string, string, string) (Installed, error) {
return Installed{}, nil
}
func (f *fakeAuto) UpdateAuto(context.Context) error { return nil }
// A tick updates every package the store lists as auto, and one failing
// package neither stops the rest nor hides its error.
func TestUpdateAuto(t *testing.T) {
tests := []struct {
name string
fake fakeAuto
tried []string
wantErr bool
}{
{name: "no auto packages", tried: nil},
{name: "every auto package is tried", fake: fakeAuto{refs: []OrgPackage{{"o", "a"}, {"o", "b"}}}, tried: []string{"a", "b"}},
{name: "a failure does not stop the rest", fake: fakeAuto{refs: []OrgPackage{{"o", "a"}, {"o", "b"}}, failing: "a"}, tried: []string{"a", "b"}, wantErr: true},
{name: "a failed listing is an error", fake: fakeAuto{listErr: errors.New("db down")}, wantErr: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := updateAuto(t.Context(), &tt.fake, &tt.fake)
if (err != nil) != tt.wantErr {
t.Fatalf("err = %v; wantErr %v", err, tt.wantErr)
}
if !slices.Equal(tt.fake.tried, tt.tried) {
t.Fatalf("tried %v; want %v", tt.fake.tried, tt.tried)
}
})
}
}
+53 -1
View File
@@ -35,6 +35,22 @@ type Installed struct {
// pinned digest. A nil Authority means the operator has not accepted;
// until then the package's authority at use time is nothing.
AcceptedAuthority *Authority
// Pending is an update waiting for an admin to approve widened authority.
// The live pin and AcceptedAuthority stay as they were until then.
Pending *Pin
// PendingAuthority is what the pending update declares.
PendingAuthority *Authority
// Previous is the pin an update or rollback replaced, with the authority
// accepted against it, so a rollback restores both.
Previous *Pin
// PreviousAccepted is the authority accepted against Previous.
PreviousAccepted *Authority
}
// Pin is an immutable package reference at a digest.
type Pin struct {
Reference string
Digest string
}
// Repository owns installed package records and dependency-safe deletion.
@@ -46,6 +62,22 @@ type Repository interface {
// Accept records the operator's accepted authority against the package's
// currently pinned digest.
Accept(context.Context, string, string, Authority) error
UpdateRecords
}
// UpdateRecords is the part of Repository the update lifecycle writes.
type UpdateRecords interface {
// SetPending records an update waiting for approval, leaving the live pin.
SetPending(ctx context.Context, orgID, name string, pending Pin, declared Authority) error
// Replace swaps the live pin, descriptor, layer and accepted authority for
// next's, keeping the replaced pin and its accepted authority as Previous
// and clearing Pending.
Replace(ctx context.Context, next Installed) error
// SetUpdatePolicy sets whether a package follows the catalogue on its own.
SetUpdatePolicy(ctx context.Context, orgID, name, policy string) error
// ListAutoUpdate names every installed package, in any org, whose policy
// is auto.
ListAutoUpdate(ctx context.Context) ([]OrgPackage, error)
}
// Registry reads a package by immutable OCI manifest digest.
@@ -62,6 +94,17 @@ type Manager interface {
AcceptPackageAuthority(ctx context.Context, orgID, name string, accepted Authority) (Installed, error)
ListCatalogue(ctx context.Context) (Catalogue, error)
InstallFromCatalogue(ctx context.Context, orgID, name string) (Installed, error)
OrgUpdates
}
// OrgUpdates is the update lifecycle of an installed package in one org.
type OrgUpdates interface {
UpdatePackage(ctx context.Context, orgID, name string) (Installed, error)
ApprovePackageUpdate(ctx context.Context, orgID, name string) (Installed, error)
RollbackPackage(ctx context.Context, orgID, name string) (Installed, error)
SetPackageUpdatePolicy(ctx context.Context, orgID, name, policy string) (Installed, error)
// UpdateAuto runs UpdatePackage for every auto package.
UpdateAuto(ctx context.Context) error
}
// Installations is Manager as a remote caller sees it: the State Store acts
@@ -74,6 +117,15 @@ type Installations interface {
AcceptPackageAuthority(ctx context.Context, name string, accepted Authority) (Installed, error)
ListCatalogue(ctx context.Context) (Catalogue, error)
InstallFromCatalogue(ctx context.Context, name string) (Installed, error)
Updates
}
// Updates is OrgUpdates as a remote caller sees it.
type Updates interface {
UpdatePackage(ctx context.Context, name string) (Installed, error)
ApprovePackageUpdate(ctx context.Context, name string) (Installed, error)
RollbackPackage(ctx context.Context, name string) (Installed, error)
SetPackageUpdatePolicy(ctx context.Context, name, policy string) (Installed, error)
}
// Service validates the registry content before persisting an installation.
@@ -157,7 +209,7 @@ func (s Service) Install(ctx context.Context, orgID, name, reference, digest str
}
installed := Installed{
OrgID: orgID, Name: name, Reference: reference, Digest: digest,
Descriptor: descriptor, Layer: layer, UpdatePolicy: "manual",
Descriptor: descriptor, Layer: layer, UpdatePolicy: PolicyManual,
}
contents, err := s.resolveFamilyContents(descriptor, layer)
if err != nil {
+190
View File
@@ -0,0 +1,190 @@
package storepkg
import (
"context"
"errors"
"fmt"
)
// Update policies. Manual is the default; an auto package follows the
// catalogue, with a widening update still waiting for approval.
const (
PolicyManual = "manual"
PolicyAuto = "auto"
)
// ErrInvalidPolicy: the update policy is neither manual nor auto.
var ErrInvalidPolicy = errors.New("store package update policy must be manual or auto")
// OrgPackage names one installed package.
type OrgPackage struct{ OrgID, Name string }
// SetPackageUpdatePolicy sets whether name follows the catalogue on its own.
func (s Service) SetPackageUpdatePolicy(ctx context.Context, orgID, name, policy string) (Installed, error) {
if policy != PolicyManual && policy != PolicyAuto {
return Installed{}, ErrInvalidPolicy
}
if err := s.Store.SetUpdatePolicy(ctx, orgID, name, policy); err != nil {
return Installed{}, err
}
return s.Store.Get(ctx, orgID, name)
}
// UpdateAuto runs UpdatePackage for every auto package.
func (s Service) UpdateAuto(ctx context.Context) error {
return updateAuto(ctx, s.Store, s)
}
// updateAuto tries every auto package: one that fails does not stop the rest.
func updateAuto(ctx context.Context, store interface {
ListAutoUpdate(context.Context) ([]OrgPackage, error)
}, updates OrgUpdates,
) error {
refs, err := store.ListAutoUpdate(ctx)
if err != nil {
return fmt.Errorf("list auto-update packages: %w", err)
}
var errs []error
for _, ref := range refs {
if _, err := updates.UpdatePackage(ctx, ref.OrgID, ref.Name); err != nil {
errs = append(errs, fmt.Errorf("update %s/%s: %w", ref.OrgID, ref.Name, err))
}
}
return errors.Join(errs...)
}
var (
// ErrNoPendingUpdate: there is no update waiting for approval.
ErrNoPendingUpdate = errors.New("store package has no pending update")
// ErrNoPrevious: the package has no earlier version to roll back to.
ErrNoPrevious = errors.New("store package has no previous version")
)
// Widens reports whether an update's declared authority asks for more than
// the operator accepted. A package never accepted holds no authority, so
// any grant it declares is wider.
func (i Installed) Widens(next Authority) bool {
var held Authority
if i.AcceptedAuthority != nil {
held = *i.AcceptedAuthority
}
return !held.Covers(next)
}
// UpdatePackage moves name to the digest the verified catalogue lists. An
// update that does not widen authority applies at once; a widening one waits
// as pending until ApprovePackageUpdate, with the live pin untouched.
func (s Service) UpdatePackage(ctx context.Context, orgID, name string) (Installed, error) {
installed, err := s.Store.Get(ctx, orgID, name)
if err != nil {
return Installed{}, fmt.Errorf("installed package %q: %w", name, err)
}
catalogue, err := s.ListCatalogue(ctx)
if err != nil {
return Installed{}, err
}
for _, entry := range catalogue.Packages {
if entry.Name != name {
continue
}
pin := Pin{Reference: entry.Reference, Digest: entry.Digest}
if pin.Digest == installed.Digest || (installed.Pending != nil && installed.Pending.Digest == pin.Digest) {
return installed, nil
}
next, contents, err := s.prepare(ctx, orgID, name, pin)
if err != nil {
return Installed{}, err
}
if installed.Widens(next.Descriptor.Authority) {
if err := s.Store.SetPending(ctx, orgID, name, pin, next.Descriptor.Authority); err != nil {
return Installed{}, err
}
return s.Store.Get(ctx, orgID, name)
}
next.AcceptedAuthority = installed.AcceptedAuthority
return s.swap(ctx, installed, next, contents)
}
return Installed{}, fmt.Errorf("catalogue package %q: %w", name, ErrNotFound)
}
// ApprovePackageUpdate applies the pending update and accepts the authority
// it declares.
func (s Service) ApprovePackageUpdate(ctx context.Context, orgID, name string) (Installed, error) {
installed, err := s.Store.Get(ctx, orgID, name)
if err != nil {
return Installed{}, fmt.Errorf("installed package %q: %w", name, err)
}
if installed.Pending == nil {
return Installed{}, fmt.Errorf("package %q: %w", name, ErrNoPendingUpdate)
}
next, contents, err := s.prepare(ctx, orgID, name, *installed.Pending)
if err != nil {
return Installed{}, err
}
accepted := next.Descriptor.Authority
next.AcceptedAuthority = &accepted
return s.swap(ctx, installed, next, contents)
}
// RollbackPackage reinstalls the previous digest with the authority accepted
// against it.
func (s Service) RollbackPackage(ctx context.Context, orgID, name string) (Installed, error) {
installed, err := s.Store.Get(ctx, orgID, name)
if err != nil {
return Installed{}, fmt.Errorf("installed package %q: %w", name, err)
}
if installed.Previous == nil {
return Installed{}, fmt.Errorf("package %q: %w", name, ErrNoPrevious)
}
next, contents, err := s.prepare(ctx, orgID, name, *installed.Previous)
if err != nil {
return Installed{}, err
}
next.AcceptedAuthority = installed.PreviousAccepted
return s.swap(ctx, installed, next, contents)
}
// prepare fetches and validates the package at pin and resolves its
// contributions before anything live changes.
func (s Service) prepare(ctx context.Context, orgID, name string, pin Pin) (Installed, map[string]map[string][]byte, error) {
descriptor, layer, err := s.Registry.Fetch(ctx, pin.Reference, pin.Digest)
if err != nil {
return Installed{}, nil, fmt.Errorf("fetch package: %w", err)
}
if err := descriptor.Validate(); err != nil {
return Installed{}, nil, fmt.Errorf("validate package: %w", err)
}
contents, err := s.resolveFamilyContents(descriptor, layer)
if err != nil {
return Installed{}, nil, err
}
return Installed{
OrgID: orgID, Name: name, Reference: pin.Reference, Digest: pin.Digest,
Descriptor: descriptor, Layer: layer,
}, contents, nil
}
// swap replaces installed with next: old contributions come out, new ones go
// in, then the record flips. A failure puts the old contributions back so
// the record, which still names the old pin, stays true.
func (s Service) swap(ctx context.Context, installed, next Installed, contents map[string]map[string][]byte) (Installed, error) {
if err := s.withdraw(ctx, installed); err != nil {
return Installed{}, err
}
err := s.project(ctx, next, contents)
if err == nil {
err = s.Store.Replace(ctx, next)
}
if err != nil {
return Installed{}, errors.Join(err, s.withdraw(ctx, next), s.restore(ctx, installed))
}
return s.Store.Get(ctx, installed.OrgID, installed.Name)
}
func (s Service) restore(ctx context.Context, installed Installed) error {
contents, err := s.resolveFamilyContents(installed.Descriptor, installed.Layer)
if err != nil {
return err
}
return s.project(ctx, installed, contents)
}
Loaded 100 of 301 files, more files were not shown because too many files have changed in this diff. Show more