Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e2e24f9d56 | ||
|
|
5aa5c2a6c8 | ||
|
|
41b2daed0c | ||
|
|
e0a79b3990 | ||
|
|
e2d6b75b10 | ||
|
|
b4be999777 | ||
|
|
5c3bf0b642 | ||
|
|
be2b897149 | ||
|
|
f5a39cfb9b | ||
|
|
f5a6fa5b32 | ||
|
|
93ec6d916b | ||
|
|
c36fb29683 | ||
|
|
cac0f43b58 | ||
|
|
805364f201 | ||
|
|
3b2a3a7630 | ||
|
|
bf29d3640e | ||
|
|
de15bffc2f | ||
|
|
1377e7ebf2 | ||
|
|
d53ebbda95 | ||
|
|
b227df3c3f | ||
|
|
a3e0e5c1bb | ||
|
|
116f8a10fb | ||
|
|
116b03534f | ||
|
|
b697a12b89 | ||
|
|
7b64699ff1 | ||
|
|
1598b3cc74 | ||
|
|
212742e880 | ||
|
|
6693e60f6f | ||
|
|
c097e50405 | ||
|
|
ccbc9c7844 | ||
|
|
948a8d34fd | ||
|
|
e078b5c4bc | ||
|
|
4a24c0b31f | ||
|
|
e988d3cc40 | ||
|
|
cd20aeadad | ||
|
|
1bb7ba1248 | ||
|
|
fdafec3d9d | ||
|
|
0fcf792db3 | ||
|
|
9cd52dd9aa | ||
|
|
5290dab199 | ||
|
|
e17c7895e4 | ||
|
|
e4ea2eaabc | ||
|
|
12b9a57404 | ||
|
|
514c6cf61c | ||
|
|
bef0ecea3a | ||
|
|
5bb5d9b660 | ||
|
|
f30405ea2b | ||
|
|
f9efae8a5e | ||
|
|
819cfc655b | ||
|
|
5afe5bcf99 | ||
|
|
0c25b2f4a6 | ||
|
|
252895e01c | ||
|
|
3ea49c7e07 | ||
|
|
66a27bd7ef | ||
|
|
b070ce4715 | ||
|
|
84b581fde6 | ||
|
|
af5e11a734 | ||
|
|
522ced45ff | ||
|
|
79d7f2fa68 | ||
|
|
eb3ee14a0e | ||
|
|
6ed6b28d02 | ||
|
|
2f4e9bbe14 | ||
|
|
3057f054b2 | ||
|
|
a4e7ea8cc3 | ||
|
|
c812192800 | ||
|
|
81121d03b0 | ||
|
|
e5f03587c9 | ||
|
|
c25e70a588 | ||
|
|
b97c79aa0b | ||
|
|
2d2c39f9e5 |
No files matched your search
@@ -103,6 +103,12 @@ jobs:
|
||||
- name: Allow unprivileged user namespaces
|
||||
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
|
||||
|
||||
# The gate's uncached `go test ./...` runs the Docker-backed tests that
|
||||
# -short skips, and those start a container from alpine:3. A fresh runner
|
||||
# has no such image, so the suite failed on "No such image: alpine:3".
|
||||
- name: Pull test container images
|
||||
run: docker pull alpine:3
|
||||
|
||||
- name: Quality gate
|
||||
run: |
|
||||
# The formatter sequence has one owner: task fmt, which drives go fix
|
||||
|
||||
@@ -5,6 +5,31 @@ release's per-component sections are labelled beneath its heading.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [1.53.0] - 2026-10-11
|
||||
|
||||
### archied
|
||||
|
||||
- Every organisation's data is its own. Chats, tasks, events, transitions, steps, tool calls, run statistics, event sources, event types, mappings, bindings and agent memory are read and written in the org that owns them, and a request can no longer write into another org. Rows written before orgs existed move to their owner's org on upgrade.
|
||||
- An instance admin decides what each org may reach: the models it can use, and whether it may register providers under its own credentials. Org Settings gains a Models tab and instance Models gains Org access. The system org is never restricted; another org uses nothing until it is granted.
|
||||
- Models are addressed by alias. Aliases replace model roles, one resolver serves every purpose (agent steps, implement, PR review, MCP sampling, chat, embedding, transcription), and an unknown alias fails instead of falling back to a default. The step editor and the chat picker offer aliases, and a workflow naming one that does not exist is refused when saved.
|
||||
- Chat sessions belong to one org; a chat resolves its models in the caller's org and is billed to it, and a task run from chat posts its outcome back to that chat.
|
||||
- Add a member by provider subject, or by email as a placeholder that the first sign-in the provider vouches for claims.
|
||||
- The workflow canvas authors parallel branches as lanes, adds a switch step that runs the case matching a reference, and edits a call's interface. A step shows its role, its number and its run trace.
|
||||
- The workflows list shows each workflow's running count and latest run result, the canvas reveals its data-edge handles on hover, and restoring a version asks for confirmation.
|
||||
- Store packages can auto-update: a package given the policy is moved to the catalogue's newest digest by a State Store ticker. An update asking for more authority than was accepted waits as a pending update with the running version untouched; approving it accepts the new authority, and a rollback restores the previous version with the authority accepted against it.
|
||||
- Model provider keys never enter an agent container: the runtime reaches its providers through the egress proxy, which holds the keys. A task's own credential can no longer replace the workflow definition its run is pinned to, and a package workflow's triggers, tools and forge calls are bounded by the authority accepted for the package.
|
||||
- The dashboard warns when it is reachable from off-host over plain HTTP, and the wildcard listen address counts as off-host rather than loopback. Instance health, logs, config and identities, the setup checklist and the skill catalogue are system-org only, and API requests are refused until the access chain has loaded.
|
||||
- `/approve` reaches Telegram; the router served it but nothing published it.
|
||||
- A worktree a killed container left root-owned is taken back by the daemon, and a PR review's benchmark runner works again on configured models and public PRs.
|
||||
- An unserved `/api` path returns 404 instead of the dashboard page, and an input group's select no longer loses clicks to its addon.
|
||||
|
||||
### archie-agent
|
||||
|
||||
- The runtime reaches model providers through the egress proxy, so provider keys stay out of the container.
|
||||
- Agent steps resolve model aliases and their own org's models and credentials, failing on an unknown alias instead of falling back.
|
||||
- A switch step runs the case matching its reference.
|
||||
- A call waiting on another step is reconciled from durable steps, so it resumes after a daemon restart rather than stranding.
|
||||
|
||||
## [1.52.0] - 2026-10-09
|
||||
|
||||
### archied
|
||||
|
||||
@@ -11,7 +11,7 @@ agent or tool in an isolated container, and the result lands somewhere for
|
||||
review or action. A repository is an optional input to a workflow, not the point
|
||||
of the product.
|
||||
|
||||
<temporary - this statement must be removed on 2026-10-10"> The product is
|
||||
<temporary - this statement must be removed on 2026-10-20"> The product is
|
||||
unfinished. Many features are half-built, and the codebase carries far more
|
||||
code, binaries, docs and tooling than it delivers. The job right now is to
|
||||
finish features and shrink the codebase, not to extend it. </temporary>
|
||||
@@ -27,6 +27,20 @@ When sources disagree, the higher one wins:
|
||||
5. Everything else (memories, comments, old docs) is a hint. Verify it against
|
||||
the code before acting on it.
|
||||
|
||||
## Working with the maintainer
|
||||
|
||||
- **A clear direction is the go-ahead.** Do it in the same turn. Do not restate
|
||||
the plan, offer options, or ask "shall I proceed?". Stop only for destructive
|
||||
or outward-facing actions (deleting data, pushing, touching a live host).
|
||||
- **Post outside the repo only when told.** Chat, tickets and any external
|
||||
service get a message only on the maintainer's instruction.
|
||||
- **Report, don't narrate.** Final replies say what changed, what was verified,
|
||||
and anything blocked, in a few lines. No preamble, no recap of the request, no
|
||||
closing offers of more help. Leave out what the maintainer can see or infer:
|
||||
a clean tree, unpushed commits, a passing gate (commits imply it), and steps
|
||||
skipped that the task never asked for. Name a skipped step only when the
|
||||
task required it.
|
||||
|
||||
## How to decide
|
||||
|
||||
- **Broken beats new.** A bug that blocks work gets fixed now, with the smallest
|
||||
|
||||
+5
-5
@@ -125,11 +125,6 @@ tasks:
|
||||
cmds:
|
||||
- golangci-lint run ./...
|
||||
|
||||
lint:playbooks:
|
||||
desc: Validate playbook bindings
|
||||
cmds:
|
||||
- go run ./cmd/archied playbooks lint -dir ./playbooks
|
||||
|
||||
vuln:
|
||||
desc: Scan for known vulnerabilities (requires network access)
|
||||
cmds:
|
||||
@@ -175,6 +170,11 @@ tasks:
|
||||
cmds:
|
||||
- bash tools/webhook-send.sh {{.CLI_ARGS}}
|
||||
|
||||
bench:prreview:
|
||||
desc: Run the PR review benchmark on the configured models and write docs/benchmarks/pr-review-*.md (-- -model ALIAS -judge-model REF; --help for flags)
|
||||
cmds:
|
||||
- go run ./cmd/archied prbench {{.CLI_ARGS}}
|
||||
|
||||
dev:
|
||||
desc: Run development services and Vite at http://localhost:5173
|
||||
deps: [dev:daemon, dev:gateway, dev:messaging, dev:state-store, dev:dashboard, dev:ui]
|
||||
|
||||
+3
-4
@@ -21,7 +21,6 @@ steps:
|
||||
- id: plan
|
||||
type: agent.run
|
||||
settings:
|
||||
role: planner
|
||||
read_only: true
|
||||
mission: |
|
||||
Plan this change: {{ task.prompt }}
|
||||
@@ -135,7 +134,7 @@ _runs without a repository._
|
||||
| `protect_tests` | bool | Write-block the repository test files. |
|
||||
| `read_only` | bool | Restrict the agent to read-only tools. |
|
||||
| `result` | mapping | A JSON Schema object the agent must return; later steps read it as steps.<id>.result. |
|
||||
| `role` | text | Which configured model role runs the agent, e.g. builder or planner. Empty means builder. |
|
||||
| `model` | text | The model alias that runs the agent, e.g. default or fast. Empty means default. |
|
||||
|
||||
### `command.run`
|
||||
|
||||
@@ -247,8 +246,8 @@ _runs without a repository._
|
||||
|
||||
| Setting | Type | Meaning |
|
||||
|---|---|---|
|
||||
| `inputs` | mapping | |
|
||||
| `outputs` | mapping | |
|
||||
| `inputs` | mapping | Callee input names mapped to scalar literals or `inputs.<name>` references to caller inputs. |
|
||||
| `outputs` | mapping | Callee output names mapped to `outputs.<name>` references to caller outputs; requires `wait: true`. |
|
||||
| `wait` | bool | Wait for the child run to finish before continuing. |
|
||||
| `workflow` | text | The workflow to start as a child run. |
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ import (
|
||||
"syscall"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/app/agentworker"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/egress"
|
||||
)
|
||||
|
||||
type workerRunner func(context.Context, agentworker.Settings, *slog.Logger) error
|
||||
@@ -90,6 +91,9 @@ func runCommand(args []string, getenv func(string) string, stderr io.Writer, run
|
||||
NATSToken: natsToken,
|
||||
StateStoreTarget: stateStoreURL,
|
||||
StateStoreToken: stateStoreToken,
|
||||
ModelProxy: getenv(egress.ModelProxyEnv),
|
||||
ModelProxyHosts: egress.ParseHosts(getenv(egress.ModelProxyHostsEnv)),
|
||||
ModelProxyCA: getenv(egress.ModelProxyCAEnv),
|
||||
}, log); err != nil {
|
||||
return 1
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/app/archied"
|
||||
"github.com/samcharles93/archie-core/internal/app/archieplaybooks"
|
||||
"github.com/samcharles93/archie-core/internal/app/prbench"
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -28,6 +29,9 @@ func main() {
|
||||
if len(args) > 0 && args[0] == "status" {
|
||||
os.Exit(archied.RunStatus(args[1:], os.Stdout, os.Stderr))
|
||||
}
|
||||
if len(args) > 0 && args[0] == "prbench" {
|
||||
os.Exit(prbench.Main(args[1:], os.Stdout, os.Stderr))
|
||||
}
|
||||
if len(args) > 0 && args[0] == "playbooks" {
|
||||
os.Exit(archieplaybooks.Run(args[1:], os.Stderr))
|
||||
}
|
||||
|
||||
@@ -76,6 +76,14 @@ A non-loopback `[web].listen` requires a dashboard token (`-token`, or
|
||||
binding-dispatch loop in `archied` consumes what arrives from the shared
|
||||
State Store.
|
||||
|
||||
The dashboard speaks plain HTTP. A non-loopback bind therefore carries the
|
||||
session cookie and the provider token in clear over the network, and
|
||||
archie-ui warns about exactly that at startup. Terminate TLS in front of it
|
||||
(a reverse proxy or the tunnel itself) and set
|
||||
`[web].trust_forwarded_headers = true` to declare the proxy and silence the
|
||||
warning; leaving it off while exposing the port directly is the case the
|
||||
warning names.
|
||||
|
||||
Auth note: unlike `archied`, archie-ui has no secret registry --
|
||||
`GATEWAY_TOKEN`/`STATE_STORE_TOKEN` resolve from its process environment
|
||||
only (or from `[services.*].target_token` in `config.toml`), so whatever
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
# PR review benchmark
|
||||
|
||||
```
|
||||
task bench:prreview
|
||||
```
|
||||
|
||||
Reviews the 38 runnable Martian pull requests once each at `deep` depth with
|
||||
the production pipeline, judges the posted comments against the golden
|
||||
comments in `internal/app/prbench/martian.json`, and writes
|
||||
`docs/benchmarks/pr-review-<timestamp>.md` (recall per PR, overall recall,
|
||||
precision, models, archie version). Commit that file with the run.
|
||||
|
||||
- Models come from the config at `~/.config/archie/config.toml`: the reviewer
|
||||
and judge both default to its `default` alias. Override with
|
||||
`task bench:prreview -- -model ALIAS_OR_PROVIDER/MODEL -judge-model ALIAS_OR_PROVIDER/MODEL`.
|
||||
Pick a judge other than the reviewer's model for an independent judgement.
|
||||
Aliases set only in the dashboard are not in the file; pass a
|
||||
`provider/model` ref instead.
|
||||
- Pull requests are public, so no credential is needed. `GH_TOKEN` or
|
||||
`GITHUB_TOKEN`, if set, only raises the GitHub API rate limit.
|
||||
- `-problems ID,ID` runs a partial smoke run; its report says it is not the
|
||||
acceptance run. A full run reports PASS when nothing failed and golden
|
||||
micro-recall is at least 0.70.
|
||||
- It is not part of `task test` and posts nothing to any pull request.
|
||||
@@ -78,6 +78,7 @@
|
||||
"/news/archie/1.51.0/": "/news/1.51.0/",
|
||||
"/news/archie/1.51.1/": "/news/1.51.1/",
|
||||
"/news/archie/1.52.0/": "/news/1.52.0/",
|
||||
"/news/archie/1.53.0/": "/news/1.53.0/",
|
||||
"/news/archie/1.6.0/": "/news/1.6.0/",
|
||||
"/news/archie/1.7.0/": "/news/1.7.0/",
|
||||
"/news/archie/1.8.0/": "/news/1.8.0/",
|
||||
@@ -172,6 +173,7 @@
|
||||
"/news/archied/1.51.0/": "/news/1.51.0/",
|
||||
"/news/archied/1.51.1/": "/news/1.51.1/",
|
||||
"/news/archied/1.52.0/": "/news/1.52.0/",
|
||||
"/news/archied/1.53.0/": "/news/1.53.0/",
|
||||
"/news/archied/1.6.0/": "/news/1.6.0/",
|
||||
"/news/archied/1.7.0/": "/news/1.7.0/",
|
||||
"/news/archied/1.8.0/": "/news/1.8.0/",
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
[
|
||||
{
|
||||
"version": "1.53.0",
|
||||
"date": "2026-10-11",
|
||||
"body": "### archied\n\n- Every organisation's data is its own. Chats, tasks, events, transitions, steps, tool calls, run statistics, event sources, event types, mappings, bindings and agent memory are read and written in the org that owns them, and a request can no longer write into another org. Rows written before orgs existed move to their owner's org on upgrade.\n- An instance admin decides what each org may reach: the models it can use, and whether it may register providers under its own credentials. Org Settings gains a Models tab and instance Models gains Org access. The system org is never restricted; another org uses nothing until it is granted.\n- Models are addressed by alias. Aliases replace model roles, one resolver serves every purpose (agent steps, implement, PR review, MCP sampling, chat, embedding, transcription), and an unknown alias fails instead of falling back to a default. The step editor and the chat picker offer aliases, and a workflow naming one that does not exist is refused when saved.\n- Chat sessions belong to one org; a chat resolves its models in the caller's org and is billed to it, and a task run from chat posts its outcome back to that chat.\n- Add a member by provider subject, or by email as a placeholder that the first sign-in the provider vouches for claims.\n- The workflow canvas authors parallel branches as lanes, adds a switch step that runs the case matching a reference, and edits a call's interface. A step shows its role, its number and its run trace.\n- The workflows list shows each workflow's running count and latest run result, the canvas reveals its data-edge handles on hover, and restoring a version asks for confirmation.\n- Store packages can auto-update: a package given the policy is moved to the catalogue's newest digest by a State Store ticker. An update asking for more authority than was accepted waits as a pending update with the running version untouched; approving it accepts the new authority, and a rollback restores the previous version with the authority accepted against it.\n- Model provider keys never enter an agent container: the runtime reaches its providers through the egress proxy, which holds the keys. A task's own credential can no longer replace the workflow definition its run is pinned to, and a package workflow's triggers, tools and forge calls are bounded by the authority accepted for the package.\n- The dashboard warns when it is reachable from off-host over plain HTTP, and the wildcard listen address counts as off-host rather than loopback. Instance health, logs, config and identities, the setup checklist and the skill catalogue are system-org only, and API requests are refused until the access chain has loaded.\n- `/approve` reaches Telegram; the router served it but nothing published it.\n- A worktree a killed container left root-owned is taken back by the daemon, and a PR review's benchmark runner works again on configured models and public PRs.\n- An unserved `/api` path returns 404 instead of the dashboard page, and an input group's select no longer loses clicks to its addon.\n\n### archie-agent\n\n- The runtime reaches model providers through the egress proxy, so provider keys stay out of the container.\n- Agent steps resolve model aliases and their own org's models and credentials, failing on an unknown alias instead of falling back.\n- A switch step runs the case matching its reference.\n- A call waiting on another step is reconciled from durable steps, so it resumes after a daemon restart rather than stranding."
|
||||
},
|
||||
{
|
||||
"version": "1.52.0",
|
||||
"date": "2026-10-09",
|
||||
|
||||
@@ -95,7 +95,7 @@ require (
|
||||
github.com/BurntSushi/toml v1.6.0
|
||||
github.com/cedar-policy/cedar-go v1.8.0
|
||||
github.com/coreos/go-oidc/v3 v3.21.0
|
||||
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.6
|
||||
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.8
|
||||
github.com/go-git/go-billy/v6 v6.0.0-beta.1
|
||||
github.com/go-git/go-git/v6 v6.0.0-beta.1
|
||||
github.com/go-telegram/bot v1.27.0
|
||||
|
||||
@@ -54,8 +54,8 @@ github.com/docker/go-connections v0.8.2 h1:9d9DEokx5W7qlr0shkFXIxEA7+TTD+llwzcyk
|
||||
github.com/docker/go-connections v0.8.2/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
|
||||
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
|
||||
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
|
||||
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.6 h1:co7vn6GgImYEGO/KPlhPRJT9i5EPWgOKcB9q7bNJlFQ=
|
||||
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.6/go.mod h1:MrWO4PN3/VYC5xF8eyI8U82Nv/agTHcN4IUzheqA90c=
|
||||
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.8 h1:ay2XEj0yW3uzsNwN3KXS/c4nlneHONFKaHlrVhu2WCQ=
|
||||
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.8/go.mod h1:4G6VgFAp9eaQkwy7DbZbOfWRD/Mjvc/TnEtQBRRMEsA=
|
||||
github.com/dustin/go-humanize v1.1.0 h1:dbKTrvD0klcbBV/h4AWJdMuZogJACoMlvWIWZ5b2xWg=
|
||||
github.com/dustin/go-humanize v1.1.0/go.mod h1:hc1CvRkJMsgxqjmjMQF3QNRAZBwY8AXBAzKYoSX9sFI=
|
||||
github.com/ebitengine/purego v0.11.1 h1:2zpWRSQNVKN4eKsKO9eM1ILDgWfYMY9GwqRmK6XeQ/0=
|
||||
|
||||
@@ -31,7 +31,7 @@ type LoopRunner struct {
|
||||
// Limits bounds the size of tool results fed back into the stage. The
|
||||
// zero value applies no limits.
|
||||
Limits ToolLimits
|
||||
// AllowTools is the profile's allowlist over archie-added tools. Empty allows
|
||||
// AllowTools is the allowlist over archie-added tools. Nil allows
|
||||
// all. Capture tools and agentloop's built-in file tools are never
|
||||
// filtered.
|
||||
AllowTools []string
|
||||
@@ -187,9 +187,9 @@ func makeCaptureHandler(spec agentrun.CaptureTool, captures map[string][]json.Ra
|
||||
}
|
||||
}
|
||||
|
||||
// allowedTools keeps the tools allow names; an empty allow keeps them all.
|
||||
// allowedTools keeps the tools allow names; a nil allow keeps them all.
|
||||
func allowedTools(set core.ToolSet, allow []string) core.ToolSet {
|
||||
if len(allow) == 0 {
|
||||
if allow == nil {
|
||||
return set
|
||||
}
|
||||
kept := core.ToolSet{}
|
||||
|
||||
@@ -4,31 +4,31 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"log/slog"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/ai-sdk/chat"
|
||||
"github.com/samcharles93/ai-sdk/core"
|
||||
"github.com/samcharles93/ai-sdk/runtime"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/usage"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
"github.com/samcharles93/archie-core/internal/tools/mcp"
|
||||
)
|
||||
|
||||
// taskSamplingModel returns the task's builder model, or "".
|
||||
func taskSamplingModel(cfg config.TaskConfig) string {
|
||||
return strings.TrimSpace(cfg.Models["builder"])
|
||||
}
|
||||
|
||||
// taskSamplingHandler answers MCP sampling requests with the task's model.
|
||||
// Without a runtime or model the request is refused.
|
||||
func taskSamplingHandler(llm *runtime.Runtime, cfg config.TaskConfig) mcp.SamplingHandler {
|
||||
// Without a runtime or model the request is refused. record receives each
|
||||
// call's model ref and token usage.
|
||||
func taskSamplingHandler(llm *runtime.Runtime, cfg config.TaskConfig, record func(context.Context, string, chat.Usage)) mcp.SamplingHandler {
|
||||
return func(ctx context.Context, req mcp.SamplingRequest) (mcp.SamplingResult, error) {
|
||||
if llm == nil {
|
||||
return mcp.SamplingResult{}, errors.New("mcp sampling: no model runtime is configured")
|
||||
}
|
||||
model := taskSamplingModel(cfg)
|
||||
if model == "" {
|
||||
return mcp.SamplingResult{}, errors.New("mcp sampling: no builder model is configured")
|
||||
model, err := config.ResolveModel(cfg.Models, "", config.PurposeAgent)
|
||||
if err != nil {
|
||||
return mcp.SamplingResult{}, fmt.Errorf("mcp sampling: %w", err)
|
||||
}
|
||||
messages, err := modelloop.SamplingMessages(req)
|
||||
if err != nil {
|
||||
@@ -45,6 +45,7 @@ func taskSamplingHandler(llm *runtime.Runtime, cfg config.TaskConfig) mcp.Sampli
|
||||
options.Temperature = float32(*req.Temperature)
|
||||
}
|
||||
result, err := llm.Chat(ctx, model, options)
|
||||
record(ctx, model, result.TotalUsage)
|
||||
if err != nil {
|
||||
return mcp.SamplingResult{}, fmt.Errorf("mcp sampling: %w", err)
|
||||
}
|
||||
@@ -56,3 +57,21 @@ func taskSamplingHandler(llm *runtime.Runtime, cfg config.TaskConfig) mcp.Sampli
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
|
||||
// samplingUsageRecorder records a task's MCP sampling calls against the task.
|
||||
// A failed write is logged, never fails the call.
|
||||
func samplingUsageRecorder(store workflow.Store, t *workflow.Task, log *slog.Logger) func(context.Context, string, chat.Usage) {
|
||||
return func(ctx context.Context, model string, used chat.Usage) {
|
||||
if store == nil || t == nil {
|
||||
return
|
||||
}
|
||||
record := usage.Record{
|
||||
Source: usage.SourceTask, TaskID: t.ID, Attempt: t.Attempt, Workflow: t.Workflow, Step: "mcp-sampling",
|
||||
InputTokens: int64(used.PromptTokens), OutputTokens: int64(used.CompletionTokens),
|
||||
CachedTokens: int64(used.CachedTokens), At: time.Now().UTC(),
|
||||
}.WithRef(model)
|
||||
if err := store.RecordUsage(ctx, record); err != nil && log != nil {
|
||||
log.Warn("record model usage failed", "model", model, "err", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -198,8 +198,6 @@ func stageRunners(req taskrun.Request, mcpSet *mcpProviderSet, newRunner runnerF
|
||||
// routeTask remains available for routing-only callers. Production execution
|
||||
// compiles the request's pinned YAML directly.
|
||||
func routeTask(req taskrun.Request, registry workflow.Registry) workflow.Workflow {
|
||||
workflow.SetKindWorkflows(req.KindWorkflows)
|
||||
workflow.SetLabelWorkflows(req.LabelWorkflows)
|
||||
return workflow.Route(req.Task, registry)
|
||||
}
|
||||
|
||||
@@ -233,7 +231,7 @@ func runTask(ctx context.Context, req taskrun.Request, dependencies taskDependen
|
||||
if len(req.MCPServers) > 0 {
|
||||
llm = modelloop.NewRuntime(req.Providers, req.Cfg.ModelLimits)
|
||||
}
|
||||
mcpSet, mcpErr := startMCPProviders(ctx, req.MCPServers, taskSamplingHandler(llm, req.Cfg), log)
|
||||
mcpSet, mcpErr := startMCPProviders(ctx, req.MCPServers, taskSamplingHandler(llm, req.Cfg, samplingUsageRecorder(dependencies.store, req.Task, log)), log)
|
||||
if mcpSet != nil {
|
||||
defer mcpSet.cleanup(ctx, log)
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
@@ -17,6 +18,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/agentboot"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/agentgit"
|
||||
agentnats "github.com/samcharles93/archie-core/internal/infrastructure/agenttransport/nats"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/egress"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/readiness"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
|
||||
"github.com/samcharles93/archie-core/internal/installtype"
|
||||
@@ -48,6 +50,13 @@ type Settings struct {
|
||||
// NATS storerpc path is deleted.
|
||||
StateStoreTarget string
|
||||
StateStoreToken string
|
||||
|
||||
// ModelProxy, ModelProxyHosts and ModelProxyCA route model provider
|
||||
// requests through the daemon's egress proxy, which holds the keys.
|
||||
// Empty ModelProxy sends them directly.
|
||||
ModelProxy string
|
||||
ModelProxyHosts []string
|
||||
ModelProxyCA string
|
||||
}
|
||||
|
||||
type workerTransport interface {
|
||||
@@ -115,6 +124,15 @@ func run(ctx context.Context, settings Settings, log *slog.Logger, dependencies
|
||||
workDir = storage.WorktreeMountDir
|
||||
}
|
||||
|
||||
if settings.ModelProxy != "" {
|
||||
transport, err := egress.ModelTransport(settings.ModelProxy, settings.ModelProxyHosts, settings.ModelProxyCA)
|
||||
if err != nil {
|
||||
return &StartupError{Operation: "model proxy", Err: err}
|
||||
}
|
||||
// The model SDK builds its clients on the default transport and takes
|
||||
// no client of its own, so the process-wide default is the seam.
|
||||
http.DefaultTransport = transport
|
||||
}
|
||||
dependencies.markSafe(ctx, workDir, log)
|
||||
|
||||
// The task is known before connecting: without a broker token the
|
||||
|
||||
@@ -96,6 +96,7 @@ type boot struct {
|
||||
// [services.state].target. It is set by openStateStoreAdapter, which
|
||||
// requires [services.state].target to be set.
|
||||
stateStore storecontract.TaskStore
|
||||
taskModels daemon.TaskModels
|
||||
// accessChain is the daemon's policy engine, built once from the stored
|
||||
// policies (openAccessChain); accessProblems is what the readiness
|
||||
// surface reports. Both are nil when no policy store is wired.
|
||||
@@ -161,6 +162,9 @@ type boot struct {
|
||||
// kitLauncher starts Kit profile tasks; nil when the egress path cannot
|
||||
// be built on this host.
|
||||
kitLauncher *kitrun.Launcher
|
||||
// modelEgress gives native tasks model access through the egress proxy;
|
||||
// nil when the proxy cannot start.
|
||||
modelEgress daemon.ModelEgress
|
||||
// sessionGrants records live setup sessions' granted credential services
|
||||
// for the egress proxy's resolver.
|
||||
sessionGrants *sessionGrants
|
||||
@@ -266,6 +270,7 @@ func (b *boot) openStateStoreAdapter(ctx context.Context) error {
|
||||
return err
|
||||
}
|
||||
b.stateStore = client
|
||||
b.taskModels = client
|
||||
b.controlPlaneRPC = client.ControlPlane()
|
||||
b.controlPlane = controlplane.NewRPCClient(b.controlPlaneRPC)
|
||||
b.applyStatus = applystatus.New(b.processName, client, b.log)
|
||||
@@ -459,7 +464,7 @@ func (b *boot) setupContainers(ctx context.Context) func() {
|
||||
|
||||
// setupGatewayClient dials the Gateway, which owns every model call. The daemon
|
||||
// only enqueues the chat tasks scheduled workflows create.
|
||||
func (b *boot) setupGatewayClient() error {
|
||||
func (b *boot) setupGatewayClient(ctx context.Context) error {
|
||||
b.setupChatTasks(b.cfg)
|
||||
contract, cleanup, err := composeChatContract(b.cfg.Services, b.secrets)
|
||||
if err != nil {
|
||||
@@ -473,6 +478,7 @@ func (b *boot) setupGatewayClient() error {
|
||||
}
|
||||
b.addCleanup(closeMessaging)
|
||||
b.messaging = messaging
|
||||
go postOutcomes(ctx, b.stateStore, messaging.Deliver, b.log)
|
||||
b.setupReadinessProbes()
|
||||
return nil
|
||||
}
|
||||
@@ -522,7 +528,7 @@ func (b *boot) registerNATSRPC() error {
|
||||
log.Error("nats connection unavailable for task RPC", "err", err)
|
||||
return err
|
||||
}
|
||||
unsubscribe, err := registerTaskRPCServers(coreConn, b.forgeClient, b.trees, b.identityRunners, b.stateStoreGrants.Client, log)
|
||||
unsubscribe, err := registerTaskRPCServers(coreConn, b.forgeClient, b.trees, b.identityRunners, b.stateStoreGrants.Client, installedAuthorities{b.stateStoreGrants.Client}, log)
|
||||
if err != nil {
|
||||
log.Error("task RPC server registration failed", "err", err)
|
||||
return err
|
||||
@@ -567,6 +573,7 @@ func (b *boot) buildDaemon() {
|
||||
Bus: b.bus,
|
||||
Forge: b.forgeClient,
|
||||
Trees: b.trees,
|
||||
PackageAuthorities: installedAuthorities{b.stateStoreGrants.Client},
|
||||
Storage: b.storeBackend,
|
||||
Log: log,
|
||||
Tasks: b.natsClient,
|
||||
@@ -574,6 +581,8 @@ func (b *boot) buildDaemon() {
|
||||
RunCredentials: b.stateStoreGrants,
|
||||
ContainerPool: b.containerPool,
|
||||
KitLauncher: b.kitLauncher,
|
||||
ModelEgress: b.modelEgress,
|
||||
TaskModels: b.taskModels,
|
||||
Identities: b.identityRunners,
|
||||
RootIdentityID: identity.StableID(servicekit.IdentityNames(b.cfg)[0]),
|
||||
TaskLogs: b.taskLogs,
|
||||
@@ -675,7 +684,7 @@ func (b *boot) publishConfig(ctx context.Context, cfg config.Config) {
|
||||
}
|
||||
|
||||
func (b *boot) configViewInput() webui.ConfigViewInput {
|
||||
catalog, _ := b.catalogState()
|
||||
catalog := b.catalogState()
|
||||
in := webui.ConfigViewInput{
|
||||
Config: b.cfgHolder.Get(),
|
||||
Catalog: catalogView(catalog),
|
||||
@@ -799,7 +808,7 @@ func (b *boot) addCleanup(fn func()) {
|
||||
b.cleanups = append(b.cleanups, fn)
|
||||
}
|
||||
|
||||
func (b *boot) catalogState() (modelcatalog.Snapshot, []string) { return b.catalog.State() }
|
||||
func (b *boot) catalogState() modelcatalog.Snapshot { return b.catalog.State() }
|
||||
|
||||
// loadCatalog layers the model catalog under the file config. A catalog that
|
||||
// cannot be read leaves the configured providers and models in effect.
|
||||
@@ -811,7 +820,7 @@ func (b *boot) loadCatalog(ctx context.Context) {
|
||||
}
|
||||
models := modelcatalog.Apply(&b.cfg, snapshot)
|
||||
b.cfgHolder.Set(b.cfg.Clone())
|
||||
b.catalog.Set(snapshot, models)
|
||||
b.catalog.Set(snapshot)
|
||||
b.log.Info("model catalog loaded", "providers", len(snapshot.Providers), "models", len(models))
|
||||
}
|
||||
|
||||
@@ -826,7 +835,7 @@ func (b *boot) refreshModelCatalog(ctx context.Context) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
b.catalog.Set(snapshot, models)
|
||||
b.catalog.Set(snapshot)
|
||||
b.publishConfig(ctx, cfg)
|
||||
b.log.Info("model catalog refreshed", "providers", len(snapshot.Providers), "models", len(models))
|
||||
return nil
|
||||
|
||||
@@ -52,7 +52,7 @@ func (b *boot) reloadConfig(ctx context.Context, doc *configuration.Document) er
|
||||
// drop them from the running config even though the file is
|
||||
// unchanged. Re-apply the same merge (idempotent: a catalog that
|
||||
// failed to load merges to identity).
|
||||
catalog, _ := b.catalogState()
|
||||
catalog := b.catalogState()
|
||||
modelcatalog.Apply(&doc.Config, catalog)
|
||||
// Bounded: this runs on the signal loop, which handles nothing else
|
||||
// while it waits, and a State Store that has stopped answering must
|
||||
@@ -264,7 +264,7 @@ func (b *boot) startLiveSettings(ctx context.Context) error {
|
||||
var runtimeResourceKinds = []string{
|
||||
controlplane.TaskPolicyKind,
|
||||
controlplane.ProviderSettingsKind,
|
||||
controlplane.ModelRoleAssignmentsKind,
|
||||
controlplane.ModelAliasesKind,
|
||||
controlplane.RepositoryPoliciesKind,
|
||||
controlplane.SchedulingPolicyKind,
|
||||
controlplane.ToolSettingsKind,
|
||||
@@ -370,7 +370,7 @@ func (b *boot) applyRuntimeResourceUpdate(ctx context.Context, kind string, upda
|
||||
// this read returns can lack the catalog's model limits and discovered
|
||||
// providers. The merge is idempotent, and reloadConfig re-applies it
|
||||
// before layering for the same reason.
|
||||
catalog, _ := b.catalogState()
|
||||
catalog := b.catalogState()
|
||||
modelcatalog.Apply(&base, catalog)
|
||||
cfg, _, err := b.runtimeConfig(ctx, base)
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
package archied
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
)
|
||||
|
||||
// installedAuthorities answers a package's accepted authority from the State
|
||||
// Store. A package the operator has not accepted has none.
|
||||
type installedAuthorities struct {
|
||||
packages storepkg.Installations
|
||||
}
|
||||
|
||||
func (a installedAuthorities) AcceptedAuthority(ctx context.Context, name string) (storepkg.Authority, error) {
|
||||
installed, err := a.packages.GetInstalled(ctx, name)
|
||||
if err != nil || installed.AcceptedAuthority == nil {
|
||||
return storepkg.Authority{}, err
|
||||
}
|
||||
return *installed.AcceptedAuthority, nil
|
||||
}
|
||||
@@ -42,11 +42,6 @@ func (b *boot) setupKitLauncher(ctx context.Context) {
|
||||
log.Warn("kit harness runs disabled: egress CA", "err", err)
|
||||
return
|
||||
}
|
||||
fetcher, err := fetch.New(fetch.WithDockerCredentials())
|
||||
if err != nil {
|
||||
log.Warn("kit harness runs disabled: kit registry client", "err", err)
|
||||
return
|
||||
}
|
||||
ln, err := (&net.ListenConfig{}).Listen(ctx, "tcp", net.JoinHostPort(pool.HostGateway(), "0"))
|
||||
if err != nil {
|
||||
log.Warn("kit harness runs disabled: egress proxy listener", "err", err)
|
||||
@@ -63,7 +58,7 @@ func (b *boot) setupKitLauncher(ctx context.Context) {
|
||||
// container logs in with, against the live grants and bindings; a setup
|
||||
// session's token resolves through the session grants instead.
|
||||
b.sessionGrants = &sessionGrants{}
|
||||
resolver := runCredentialResolver{runs: b.stateStoreGrants.Client, sessions: b.sessionGrants, config: b.cfgHolder, secrets: b.secrets}
|
||||
resolver := runCredentialResolver{runs: b.stateStoreGrants.Client, sessions: b.sessionGrants, models: b.taskModels, config: b.cfgHolder, secrets: b.secrets}
|
||||
// Without a harness secret store, Register refuses a Kit with a
|
||||
// required OAuth credential.
|
||||
oauthStore, _ := b.stateStore.(egress.OAuthStore)
|
||||
@@ -74,6 +69,15 @@ func (b *boot) setupKitLauncher(ctx context.Context) {
|
||||
log.Error("egress proxy stopped", "err", err)
|
||||
}
|
||||
}()
|
||||
// Native tasks need only the proxy for model access, so it serves them
|
||||
// even when a Kit-only piece below fails.
|
||||
b.modelEgress = modelEgress{proxy: proxy, addr: ln.Addr().String(), caFile: egress.CACertPath(caDir), config: b.cfgHolder}
|
||||
fetcher, err := fetch.New(fetch.WithDockerCredentials())
|
||||
if err != nil {
|
||||
b.addCleanup(func() { _ = srv.Close() })
|
||||
log.Warn("kit harness runs disabled: kit registry client", "err", err)
|
||||
return
|
||||
}
|
||||
networks := egress.NewNetworks(pool.Client(), egress.RelaySpec{
|
||||
Name: relayName, Image: b.cfg.Containers.Image,
|
||||
Entrypoint: []string{"archie-agent"}, Cmd: cmd, Network: pool.NetworkName(),
|
||||
|
||||
@@ -185,7 +185,7 @@ func Run() int { //nolint:cyclop,funlen // the composition root's setup sequence
|
||||
return 1
|
||||
}
|
||||
|
||||
if err := b.setupGatewayClient(); err != nil {
|
||||
if err := b.setupGatewayClient(ctx); err != nil {
|
||||
return 1
|
||||
}
|
||||
if ps, ok := b.stateStore.(storecontract.PresenceStore); ok {
|
||||
@@ -356,7 +356,7 @@ func subscribeAgentEvents(nc *natsio.Conn, bus *events.Bus, log *slog.Logger) (u
|
||||
// tasks, and one server pair per identity answers that identity's scoped
|
||||
// subjects so a container-mode task owned by a non-root identity has its RPC
|
||||
// calls served by its own forge client and worktree manager.
|
||||
func registerTaskRPCServers(nc *natsio.Conn, forgeClient forge.Forge, trees *worktree.Manager, identities []*daemon.IdentityRunner, runs worktreerpc.Runs, log *slog.Logger) (unsubscribe func(), err error) {
|
||||
func registerTaskRPCServers(nc *natsio.Conn, forgeClient forge.Forge, trees *worktree.Manager, identities []*daemon.IdentityRunner, runs worktreerpc.Runs, authorities forgerpc.Authorities, log *slog.Logger) (unsubscribe func(), err error) {
|
||||
unsubs := make([]func(), 0, 2+2*len(identities))
|
||||
unsubAll := func() {
|
||||
for _, u := range unsubs {
|
||||
@@ -365,7 +365,7 @@ func registerTaskRPCServers(nc *natsio.Conn, forgeClient forge.Forge, trees *wor
|
||||
}
|
||||
|
||||
registerForge := func(fg forge.Forge, identity string) error {
|
||||
srv := &forgerpc.Server{Forge: fg, Runs: runs, Log: log.With("rpc_identity", identity)}
|
||||
srv := &forgerpc.Server{Forge: fg, Runs: runs, Authorities: authorities, Log: log.With("rpc_identity", identity)}
|
||||
u, err := srv.RegisterFor(nc, identity)
|
||||
if err != nil {
|
||||
return fmt.Errorf("register forgerpc%s: %w", identitySuffix(identity), err)
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
package archied
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"maps"
|
||||
"slices"
|
||||
|
||||
"github.com/docker/sandbox-kit-spec/v3/spec"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/daemon"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/egress"
|
||||
"github.com/samcharles93/archie-core/internal/storage"
|
||||
)
|
||||
|
||||
const modelProxyCAPath = "/opt/archie/egress-ca.pem"
|
||||
|
||||
// modelEgress opens native task containers' model sessions on the egress
|
||||
// proxy. Each keyed provider's key is the org's credential binding named
|
||||
// after the provider.
|
||||
type modelEgress struct {
|
||||
proxy *egress.Proxy
|
||||
addr string
|
||||
caFile string
|
||||
config *config.Holder
|
||||
}
|
||||
|
||||
func (m modelEgress) Open(token, org string, providers map[string]config.Provider) (daemon.ModelSession, error) {
|
||||
var keyed []string
|
||||
for _, name := range slices.Sorted(maps.Keys(providers)) {
|
||||
if providers[name].APIKeyEnv != "" {
|
||||
keyed = append(keyed, name)
|
||||
}
|
||||
}
|
||||
if len(keyed) == 0 {
|
||||
return daemon.ModelSession{}, nil
|
||||
}
|
||||
bound := m.config.Get().Containers.BoundCredentials(org, keyed, keyed)
|
||||
var (
|
||||
subs []egress.Substitution
|
||||
hosts []string
|
||||
env []string
|
||||
)
|
||||
for _, name := range keyed {
|
||||
p := providers[name]
|
||||
if _, ok := bound[name]; !ok {
|
||||
return daemon.ModelSession{}, fmt.Errorf("provider %q has no credential binding named %q in org %s", name, name, org)
|
||||
}
|
||||
host, err := egress.ProviderHost(p.Class, p.BaseURL)
|
||||
if err != nil {
|
||||
return daemon.ModelSession{}, fmt.Errorf("provider %q: %w", name, err)
|
||||
}
|
||||
subs = append(subs, egress.Substitution{Service: name, Host: host})
|
||||
hosts = append(hosts, host)
|
||||
env = append(env, p.APIKeyEnv+"="+egress.SentinelFor(name))
|
||||
}
|
||||
session, err := m.proxy.Register(egress.SessionOptions{
|
||||
Token: token, Org: org, Substitutions: subs,
|
||||
Network: &spec.PhasedNetwork{Runtime: &spec.NetworkRules{Allow: hosts}},
|
||||
})
|
||||
if err != nil {
|
||||
return daemon.ModelSession{}, err
|
||||
}
|
||||
session.EnterRuntime()
|
||||
return daemon.ModelSession{
|
||||
Env: append(env, egress.ModelProxyEnvFor(m.addr, token, hosts, modelProxyCAPath)...),
|
||||
Mounts: []storage.Mount{{Type: storage.MountTypeBind, Source: m.caFile, Destination: modelProxyCAPath}},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (m modelEgress) Close(token string) { m.proxy.Revoke(token) }
|
||||
@@ -0,0 +1,72 @@
|
||||
package archied
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
)
|
||||
|
||||
const outcomePostInterval = 15 * time.Second
|
||||
|
||||
type outcomeDeliverer func(ctx context.Context, channel, chatID, text string) error
|
||||
|
||||
// postOutcomes reports each finished chat task to the conversation that
|
||||
// spawned it. A task stays unposted until delivery succeeds, so a Messaging
|
||||
// Service that is down or restarting only delays the report.
|
||||
func postOutcomes(ctx context.Context, tasks storecontract.OutcomePoster, deliver outcomeDeliverer, log *slog.Logger) {
|
||||
ticker := time.NewTicker(outcomePostInterval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
postPendingOutcomes(ctx, tasks, deliver, log)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func postPendingOutcomes(ctx context.Context, tasks storecontract.OutcomePoster, deliver outcomeDeliverer, log *slog.Logger) {
|
||||
pending, err := tasks.UnpostedOutcomes(ctx)
|
||||
if err != nil {
|
||||
log.Warn("list unposted task outcomes", "err", err)
|
||||
return
|
||||
}
|
||||
for _, t := range pending {
|
||||
// An origin that names no chat can never be delivered; marking it
|
||||
// posted keeps it from being retried forever.
|
||||
if channel, chatID, ok := splitOrigin(t.Origin); ok {
|
||||
if err := deliver(ctx, channel, chatID, outcomeText(t)); err != nil {
|
||||
log.Warn("post task outcome", "task", t.ID, "err", err)
|
||||
continue
|
||||
}
|
||||
}
|
||||
if err := tasks.MarkOutcomePosted(ctx, t.ID); err != nil {
|
||||
log.Warn("mark task outcome posted", "task", t.ID, "err", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// splitOrigin reads "<channel>:<chat>[/<thread>]", the form messaging.Origin
|
||||
// writes. Delivery addresses the chat; threads are not addressable.
|
||||
func splitOrigin(origin string) (channel, chatID string, ok bool) {
|
||||
channel, conversation, found := strings.Cut(origin, ":")
|
||||
chatID, _, _ = strings.Cut(conversation, "/")
|
||||
return channel, chatID, found && channel != "" && chatID != ""
|
||||
}
|
||||
|
||||
func outcomeText(t *task.Task) string {
|
||||
text := fmt.Sprintf("Task %d (%s) finished: %s", t.ID, t.Title, t.Status)
|
||||
if t.PRNumber > 0 {
|
||||
text += fmt.Sprintf(", PR #%d", t.PRNumber)
|
||||
}
|
||||
if t.ParkReason != "" {
|
||||
text += "\n" + t.ParkReason
|
||||
}
|
||||
return text
|
||||
}
|
||||
@@ -2,8 +2,10 @@ package archied
|
||||
|
||||
import (
|
||||
"context"
|
||||
"slices"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/daemon"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/egress"
|
||||
)
|
||||
@@ -24,6 +26,9 @@ type runCredentialResolver struct {
|
||||
sessions interface {
|
||||
GrantedFor(token string) (org string, services []string, ok bool)
|
||||
}
|
||||
// models answers a task's org's own providers. Nil grants only the
|
||||
// instance providers.
|
||||
models daemon.TaskModels
|
||||
config *config.Holder
|
||||
secrets interface {
|
||||
Resolve(config.SecretRef) (string, error)
|
||||
@@ -41,7 +46,13 @@ func (r runCredentialResolver) Resolve(ctx context.Context, credential, service
|
||||
return "", egress.ErrUnbound
|
||||
}
|
||||
cfg := r.config.Get()
|
||||
org, granted := cfg.CredentialAccess(task.Identity)
|
||||
_, granted := cfg.CredentialAccess(task.Identity)
|
||||
org := task.OrgOrDefault()
|
||||
// A configured model provider's key is granted to every run of the org:
|
||||
// any agent stage needs model access.
|
||||
if r.isModelProvider(ctx, cfg, task, service) {
|
||||
granted = append(slices.Clone(granted), service)
|
||||
}
|
||||
return r.resolveBinding(org, granted, service)
|
||||
}
|
||||
|
||||
@@ -62,3 +73,20 @@ func (r runCredentialResolver) resolveBinding(org string, granted []string, serv
|
||||
}
|
||||
return r.secrets.Resolve(binding.Secret)
|
||||
}
|
||||
|
||||
// isModelProvider reports whether service is a provider the task's models
|
||||
// run on: an instance provider, or one of its org's own.
|
||||
func (r runCredentialResolver) isModelProvider(ctx context.Context, cfg config.Config, task *workflow.Task, service string) bool {
|
||||
if _, ok := cfg.Providers[service]; ok {
|
||||
return true
|
||||
}
|
||||
if r.models == nil {
|
||||
return false
|
||||
}
|
||||
models, err := r.models.TaskModels(ctx, task.ID)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
_, ok := models.Providers[service]
|
||||
return ok
|
||||
}
|
||||
@@ -29,17 +29,21 @@ func TestEgressResolvesOnlyGrantedCredentials(t *testing.T) {
|
||||
own := []string{"anthropic"}
|
||||
cfg := config.Config{
|
||||
GrantedCredentials: []string{"github"},
|
||||
Providers: map[string]config.Provider{"openai": {Class: "openai"}, "deepseek": {Class: "deepseek"}},
|
||||
Identities: []config.IdentityConfig{{Name: "acme-bot", Org: "acme", GrantedCredentials: &own}},
|
||||
Containers: config.ContainerConfig{Credentials: []config.CredentialBinding{
|
||||
{Service: "github", Secret: config.SecretRef{Engine: "env", Key: "GH"}},
|
||||
{Service: "anthropic", Secret: config.SecretRef{Engine: "env", Key: "ROOT_AI"}},
|
||||
{Service: "anthropic", Org: "acme", Secret: config.SecretRef{Engine: "env", Key: "ACME_AI"}},
|
||||
{Service: "openai", Secret: config.SecretRef{Engine: "env", Key: "ROOT_OPENAI"}},
|
||||
{Service: "openai", Org: "acme", Secret: config.SecretRef{Engine: "env", Key: "ACME_OPENAI"}},
|
||||
{Service: "deepseek", Secret: config.SecretRef{Engine: "env", Key: "ROOT_DEEPSEEK"}},
|
||||
}},
|
||||
}
|
||||
r := runCredentialResolver{
|
||||
runs: runsByToken{"root-run": {ID: 1}, "acme-run": {ID: 2, Identity: "acme-bot"}},
|
||||
runs: runsByToken{"root-run": {ID: 1}, "acme-run": {ID: 2, Identity: "acme-bot", Org: "acme"}, "acme-row-run": {ID: 3, Org: "acme"}},
|
||||
config: config.NewHolder(cfg),
|
||||
secrets: secretValues{"GH": "gh-secret", "ROOT_AI": "root-ai", "ACME_AI": "acme-ai"},
|
||||
secrets: secretValues{"GH": "gh-secret", "ROOT_AI": "root-ai", "ACME_AI": "acme-ai", "ROOT_OPENAI": "root-openai", "ACME_OPENAI": "acme-openai", "ROOT_DEEPSEEK": "root-deepseek"},
|
||||
}
|
||||
tests := []struct {
|
||||
name, credential, service, want string
|
||||
@@ -49,6 +53,10 @@ func TestEgressResolvesOnlyGrantedCredentials(t *testing.T) {
|
||||
{"identity run gets its own org's binding", "acme-run", "anthropic", "acme-ai"},
|
||||
{"identity grants replace the root's", "acme-run", "github", ""},
|
||||
{"unknown credential", "gone", "github", ""},
|
||||
{"a model provider key needs no grant", "root-run", "openai", "root-openai"},
|
||||
{"a model provider key is the run's own org's", "acme-run", "openai", "acme-openai"},
|
||||
{"another org's model provider key is unbound", "acme-run", "deepseek", ""},
|
||||
{"the task's own org decides, not the identity mapping", "acme-row-run", "openai", "acme-openai"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
@@ -79,7 +87,7 @@ func TestEgressResolvesASessionGrant(t *testing.T) {
|
||||
sessions.Grant("acme-session", "acme", []string{"github", "linear"})
|
||||
sessions.Grant("other-session", "other", []string{"github"})
|
||||
r := runCredentialResolver{
|
||||
runs: runsByToken{"acme-run": {ID: 1, Identity: "acme-bot"}},
|
||||
runs: runsByToken{"acme-run": {ID: 1, Identity: "acme-bot", Org: "acme"}},
|
||||
sessions: sessions,
|
||||
config: config.NewHolder(cfg),
|
||||
secrets: secretValues{"ACME_GH": "acme-gh"},
|
||||
|
||||
@@ -4,116 +4,51 @@ import (
|
||||
"context"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/org"
|
||||
"github.com/samcharles93/archie-core/internal/gateway"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/modelcatalog"
|
||||
)
|
||||
|
||||
// chatModelManager owns the process-local model selected for interactive chat.
|
||||
// The configured role-to-model map supplies the available catalog; switching
|
||||
// chat models does not rewrite daemon configuration or affect workflow roles.
|
||||
// chatModelManager owns the process-local model alias selected for
|
||||
// interactive chat. Switching it does not rewrite configuration or affect
|
||||
// workflow steps.
|
||||
type chatModelManager struct {
|
||||
mu sync.RWMutex
|
||||
models []string
|
||||
active string
|
||||
// configured, explicit and catalog are the sources the offered set is
|
||||
// derived from: the role assignments (live via SetConfigured), the
|
||||
// [chat].models catalog (fixed for the process), and the model catalog's
|
||||
// own refs (replaced wholesale by SetModelCatalog).
|
||||
configured map[string]string
|
||||
explicit []string
|
||||
catalog []string
|
||||
mu sync.RWMutex
|
||||
aliases map[string]string
|
||||
active string
|
||||
details map[string]gateway.ModelDetails
|
||||
providerNames map[string]string
|
||||
// orgAliases answers the aliases the principal ctx carries may use. Nil
|
||||
// serves every caller the instance aliases.
|
||||
orgAliases func(ctx context.Context) (map[string]string, error)
|
||||
}
|
||||
|
||||
// pickChatDefault is the model the manager starts on: the chat role's
|
||||
// assignment, falling back to the builder role's.
|
||||
func pickChatDefault(configured map[string]string) string {
|
||||
if active := strings.TrimSpace(configured["chat"]); active != "" {
|
||||
return active
|
||||
}
|
||||
return strings.TrimSpace(configured["builder"])
|
||||
}
|
||||
|
||||
// mergeModelRefs is the model set the manager offers: the configured role
|
||||
// assignments plus every catalog, unique and sorted.
|
||||
func mergeModelRefs(configured map[string]string, catalogs ...[]string) []string {
|
||||
unique := make(map[string]struct{}, len(configured))
|
||||
for _, ref := range configured {
|
||||
ref = strings.TrimSpace(ref)
|
||||
if ref != "" {
|
||||
unique[ref] = struct{}{}
|
||||
}
|
||||
}
|
||||
for _, catalog := range catalogs {
|
||||
for _, ref := range catalog {
|
||||
ref = strings.TrimSpace(ref)
|
||||
if ref != "" {
|
||||
unique[ref] = struct{}{}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
models := make([]string, 0, len(unique))
|
||||
for ref := range unique {
|
||||
models = append(models, ref)
|
||||
}
|
||||
slices.Sort(models)
|
||||
return models
|
||||
}
|
||||
|
||||
func newChatModelManager(configured map[string]string, catalogs ...[]string) *chatModelManager {
|
||||
explicit := make([]string, 0, len(catalogs))
|
||||
for _, catalog := range catalogs {
|
||||
explicit = append(explicit, catalog...)
|
||||
}
|
||||
m := &chatModelManager{
|
||||
configured: configured,
|
||||
explicit: explicit,
|
||||
details: make(map[string]gateway.ModelDetails),
|
||||
providerNames: make(map[string]string),
|
||||
}
|
||||
m.active = pickChatDefault(configured)
|
||||
m.setModelsLocked()
|
||||
func newChatModelManager(aliases map[string]string) *chatModelManager {
|
||||
m := &chatModelManager{details: make(map[string]gateway.ModelDetails), providerNames: make(map[string]string)}
|
||||
m.SetConfigured(aliases)
|
||||
return m
|
||||
}
|
||||
|
||||
// setModelsLocked re-derives the offered set from the three sources and keeps
|
||||
// the active selection when the new set still carries it, falling back the way
|
||||
// construction does when it does not.
|
||||
func (m *chatModelManager) setModelsLocked() {
|
||||
m.models = mergeModelRefs(m.configured, m.explicit, m.catalog)
|
||||
if !slices.Contains(m.models, m.active) {
|
||||
m.active = pickChatDefault(m.configured)
|
||||
// SetConfigured replaces the alias table after a live change to model-aliases.
|
||||
// The active alias survives when it still exists; otherwise chat returns to
|
||||
// the default alias.
|
||||
func (m *chatModelManager) SetConfigured(aliases map[string]string) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.aliases = aliases
|
||||
if !slices.Contains(config.ChatAliases(aliases), m.active) {
|
||||
m.active = config.PurposeChat.DefaultAlias()
|
||||
}
|
||||
}
|
||||
|
||||
// SetConfigured re-derives the model set the manager offers after a live
|
||||
// change to model-role-assignments or provider-settings: the list is built
|
||||
// from the configured roles at construction, so a stored change would leave
|
||||
// the chat surfaces offering refs the runtime can no longer resolve. The
|
||||
// operator's active selection survives when the new roles still carry it; a
|
||||
// removed one falls back the way construction does.
|
||||
func (m *chatModelManager) SetConfigured(configured map[string]string) {
|
||||
// SetModelCatalog replaces the provider display names and per-model details
|
||||
// the catalog publishes.
|
||||
func (m *chatModelManager) SetModelCatalog(snapshot modelcatalog.Snapshot) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.configured = configured
|
||||
m.setModelsLocked()
|
||||
}
|
||||
|
||||
// SetModelCatalog replaces the model catalog's contribution to the manager:
|
||||
// the references it offers, the provider display names and the per-model
|
||||
// details. The previous snapshot's entries are dropped rather than merged --
|
||||
// a model or provider the catalog no longer publishes must stop being offered
|
||||
// and described, and these maps are the manager's own state rather than a
|
||||
// second catalogue to fall back on.
|
||||
func (m *chatModelManager) SetModelCatalog(snapshot modelcatalog.Snapshot, models []string) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.catalog = slices.Clone(models)
|
||||
m.details = make(map[string]gateway.ModelDetails)
|
||||
m.providerNames = make(map[string]string)
|
||||
for _, provider := range snapshot.Providers {
|
||||
@@ -129,7 +64,6 @@ func (m *chatModelManager) SetModelCatalog(snapshot modelcatalog.Snapshot, model
|
||||
}
|
||||
}
|
||||
}
|
||||
m.setModelsLocked()
|
||||
}
|
||||
|
||||
func (m *chatModelManager) ProviderDisplayName(provider string) string {
|
||||
@@ -145,80 +79,66 @@ func (m *chatModelManager) ModelDetails(ref string) (gateway.ModelDetails, bool)
|
||||
return details, ok
|
||||
}
|
||||
|
||||
// Models returns the aliases chat may switch between.
|
||||
func (m *chatModelManager) Models() []string {
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
return slices.Clone(m.models)
|
||||
return config.ChatAliases(m.aliases)
|
||||
}
|
||||
|
||||
func (m *chatModelManager) ActiveModel() string {
|
||||
func (m *chatModelManager) ActiveAlias() string {
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
return m.active
|
||||
}
|
||||
|
||||
func (m *chatModelManager) SetActiveModel(ctx context.Context, ref string) error {
|
||||
// ActiveModel returns the active alias's model ref, or "" when it is unset.
|
||||
func (m *chatModelManager) ActiveModel() string {
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
ref, _ := config.ResolveModel(m.aliases, m.active, config.PurposeChat)
|
||||
return ref
|
||||
}
|
||||
|
||||
func (m *chatModelManager) SetActiveModel(ctx context.Context, alias string) error {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if !slices.Contains(m.models, ref) {
|
||||
return fmt.Errorf("model %q is not configured", ref)
|
||||
if !slices.Contains(config.ChatAliases(m.aliases), alias) {
|
||||
return fmt.Errorf("model alias %q is not configured", alias)
|
||||
}
|
||||
m.active = ref
|
||||
m.active = alias
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *chatModelManager) Providers() []string {
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
seen := make(map[string]bool)
|
||||
var providers []string
|
||||
for _, model := range m.models {
|
||||
provider, _, ok := strings.Cut(model, "/")
|
||||
if ok && provider != "" && !seen[provider] {
|
||||
seen[provider] = true
|
||||
providers = append(providers, provider)
|
||||
}
|
||||
// aliasesFor is the alias table ctx's org resolves against: the instance's
|
||||
// for the system org, the org's own answer otherwise.
|
||||
func (m *chatModelManager) aliasesFor(ctx context.Context) (map[string]string, error) {
|
||||
if m.orgAliases == nil || org.OrgFromContext(ctx) == org.DefaultOrgID {
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
return m.aliases, nil
|
||||
}
|
||||
slices.Sort(providers)
|
||||
return providers
|
||||
return m.orgAliases(ctx)
|
||||
}
|
||||
|
||||
func (m *chatModelManager) ActiveProvider() string {
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
provider, _, _ := strings.Cut(m.active, "/")
|
||||
return provider
|
||||
// ModelsFor returns the aliases ctx's org may switch between.
|
||||
func (m *chatModelManager) ModelsFor(ctx context.Context) ([]string, error) {
|
||||
aliases, err := m.aliasesFor(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return config.ChatAliases(aliases), nil
|
||||
}
|
||||
|
||||
func (m *chatModelManager) ModelsForProvider(provider string) []string {
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
prefix := provider + "/"
|
||||
var models []string
|
||||
for _, model := range m.models {
|
||||
if strings.HasPrefix(model, prefix) {
|
||||
models = append(models, model)
|
||||
}
|
||||
// ModelFor resolves the active alias in ctx's org. An org that cannot use it
|
||||
// gets an error, never another model.
|
||||
func (m *chatModelManager) ModelFor(ctx context.Context) (string, error) {
|
||||
aliases, err := m.aliasesFor(ctx)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return models
|
||||
}
|
||||
|
||||
func (m *chatModelManager) SetActiveProvider(ctx context.Context, provider string) error {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
prefix := provider + "/"
|
||||
for _, model := range m.models {
|
||||
if strings.HasPrefix(model, prefix) {
|
||||
m.active = model
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("provider %q is not configured", provider)
|
||||
return config.ResolveModel(aliases, m.ActiveAlias(), config.PurposeChat)
|
||||
}
|
||||
@@ -57,9 +57,9 @@ func (b *server) setupTranscriber(cfg config.Config, log *slog.Logger) {
|
||||
})
|
||||
if ok {
|
||||
b.transcriber = client
|
||||
log.Info("voice transcription enabled", "role", transcription.Role)
|
||||
} else if cfg.Models[transcription.Role] != "" {
|
||||
log.Warn("voice transcription configured but unavailable; capability disabled", "role", transcription.Role)
|
||||
log.Info("voice transcription enabled")
|
||||
} else if _, err := config.ResolveModel(cfg.Models, "", config.PurposeTranscription); err == nil {
|
||||
log.Warn("voice transcription configured but unavailable; capability disabled")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -84,18 +84,18 @@ func (b *server) startRateLimiter(ctx context.Context, cfg config.RateLimitConfi
|
||||
}
|
||||
|
||||
// rebuildChatModelRuntime re-derives the gateway chat runtime's provider set
|
||||
// after a live change to provider-settings or model-role-assignments. The
|
||||
// after a live change to provider-settings or model-aliases. The
|
||||
// turn runner reads the runtime through server.chatLLM, so the swap reaches its
|
||||
// next turn without rebuilding the runner, and the model manager re-derives
|
||||
// the references it offers from the new role assignments and the catalog.
|
||||
// the aliases it offers.
|
||||
func (b *server) rebuildChatModelRuntime(cfg config.Config) {
|
||||
if b.chatModels == nil {
|
||||
return // this process serves no chat turns
|
||||
}
|
||||
b.setLLM(modelloop.NewRuntime(executionProviders(cfg), cfg.ModelLimits))
|
||||
b.chatModels.SetConfigured(cfg.Models)
|
||||
catalog, models := b.catalogState()
|
||||
b.chatModels.SetModelCatalog(catalog, models)
|
||||
catalog := b.catalogState()
|
||||
b.chatModels.SetModelCatalog(catalog)
|
||||
b.log.Info("chat model runtime rebuilt", "providers", len(cfg.Providers), "models", len(cfg.Models))
|
||||
}
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/messaging"
|
||||
"github.com/samcharles93/archie-core/internal/domain/usage"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
"github.com/samcharles93/archie-core/internal/gateway"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
|
||||
@@ -36,11 +37,13 @@ type chatSetup struct {
|
||||
// than the value captured when the runner was built.
|
||||
ToolLimits func() modelloop.ToolLimits
|
||||
// LLM resolves the provider runtime at each use: a live provider-settings
|
||||
// or model-role-assignments update swaps the runtime wholesale (ai-sdk's
|
||||
// or model-aliases update swaps the runtime wholesale (ai-sdk's
|
||||
// Runtime caches the provider instances it built), so the pointer cannot
|
||||
// be captured at construction. nil means no model runtime is configured.
|
||||
LLM func() *runtime.Runtime
|
||||
ChatModels gateway.ModelManager
|
||||
LLM func() *runtime.Runtime
|
||||
ChatModels gateway.ModelManager
|
||||
// Usage records each model call; nil records nothing.
|
||||
Usage *usageSink
|
||||
ToolReg *tools.Registry
|
||||
Soul gateway.SoulSource
|
||||
ChatTasks gateway.TaskCreator
|
||||
@@ -129,7 +132,7 @@ func newChatTurnRunner(
|
||||
Sessions: sessionStore,
|
||||
Models: s.ChatModels,
|
||||
Soul: s.Soul,
|
||||
Model: newChatTurnModel(s.LLM, s.ToolReg, cfg.Chat.MaxSteps, s.ToolLimits, s.ProviderOutcomes),
|
||||
Model: newChatTurnModel(s.LLM, s.ToolReg, cfg.Chat.MaxSteps, s.ToolLimits, s.ProviderOutcomes, s.Usage),
|
||||
TaskLister: s.ChatTaskLister,
|
||||
Tasks: s.ChatTasks,
|
||||
TaskLogs: s.ChatTaskLogs,
|
||||
@@ -191,31 +194,38 @@ func chatRepoEnv(cfg config.Config, identity string) []gateway.RepoEnv {
|
||||
// and records at its own adapter (curatorLLMRunner). Anything that reaches the
|
||||
// runtime without recording leaves /status reporting a stale last-known
|
||||
// outcome, which is a /status that lies about the provider.
|
||||
func sendChatTurn(ctx context.Context, llm *runtime.Runtime, chatModel string, options core.GenerateOptions, turn gateway.TurnStream, outcomes *providerOutcomeRecorder, icons map[string]string) (string, error) {
|
||||
text, err := runChatTurn(ctx, llm, chatModel, options, turn, icons)
|
||||
func sendChatTurn(ctx context.Context, llm *runtime.Runtime, chatModel string, options core.GenerateOptions, turn gateway.TurnStream, outcomes *providerOutcomeRecorder, usageRecords *usageSink, icons map[string]string) (string, error) {
|
||||
text, used, err := runChatTurn(ctx, llm, chatModel, options, turn, icons)
|
||||
usageRecords.record(ctx, usage.SourceChat, chatModel, used)
|
||||
// record tolerates a nil recorder: a setup built without one (tests, a
|
||||
// deployment whose health surface is unwired) still makes its calls.
|
||||
outcomes.record(chatModel, err)
|
||||
return text, err
|
||||
}
|
||||
|
||||
func runChatTurn(ctx context.Context, llm *runtime.Runtime, chatModel string, options core.GenerateOptions, turn gateway.TurnStream, icons map[string]string) (string, error) {
|
||||
// runChatTurn returns the reply and the tokens the call used, which a
|
||||
// failed call may still report.
|
||||
func runChatTurn(ctx context.Context, llm *runtime.Runtime, chatModel string, options core.GenerateOptions, turn gateway.TurnStream, icons map[string]string) (string, chat.Usage, error) {
|
||||
if turn == nil {
|
||||
result, err := llm.Chat(ctx, chatModel, options)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("llm chat: %w", err)
|
||||
return "", result.TotalUsage, fmt.Errorf("llm chat: %w", err)
|
||||
}
|
||||
return result.Text, nil
|
||||
return result.Text, result.TotalUsage, nil
|
||||
}
|
||||
stream, err := llm.ChatStream(ctx, chatModel, options)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("llm chat stream: %w", err)
|
||||
return "", chat.Usage{}, fmt.Errorf("llm chat stream: %w", err)
|
||||
}
|
||||
text := drainChatStream(stream.FullStream, turn, icons)
|
||||
if _, err := stream.FinishReason(); err != nil {
|
||||
return "", fmt.Errorf("llm chat stream: %w", err)
|
||||
var used chat.Usage
|
||||
if stream.Usage != nil {
|
||||
used, _ = stream.Usage()
|
||||
}
|
||||
return text, nil
|
||||
if _, err := stream.FinishReason(); err != nil {
|
||||
return "", used, fmt.Errorf("llm chat stream: %w", err)
|
||||
}
|
||||
return text, used, nil
|
||||
}
|
||||
|
||||
// drainChatStream consumes a model stream to close, reporting assistant text
|
||||
@@ -324,6 +334,7 @@ type chatTitleGenerator struct {
|
||||
log *slog.Logger
|
||||
llm func() *runtime.Runtime
|
||||
chatModels gateway.ModelManager
|
||||
usage *usageSink
|
||||
}
|
||||
|
||||
// newChatTitleGenerator wires an LLM-backed title generator for a chat
|
||||
@@ -333,7 +344,7 @@ func newChatTitleGenerator(s chatSetup) gateway.TitleGenerator {
|
||||
if s.LLM == nil || s.ChatModels == nil {
|
||||
return nil
|
||||
}
|
||||
return &chatTitleGenerator{log: s.Log, llm: s.LLM, chatModels: s.ChatModels}
|
||||
return &chatTitleGenerator{log: s.Log, llm: s.LLM, chatModels: s.ChatModels, usage: s.Usage}
|
||||
}
|
||||
|
||||
func (g *chatTitleGenerator) GenerateTitle(ctx context.Context, sessionID, firstMessage string) (string, error) {
|
||||
@@ -357,7 +368,7 @@ func (g *chatTitleGenerator) GenerateTitle(ctx context.Context, sessionID, first
|
||||
return "", fmt.Errorf("no model runtime is configured")
|
||||
}
|
||||
text, err := sendChatTurn(ctx, llm, g.chatModels.ActiveModel(),
|
||||
core.GenerateOptions{Messages: messages, MaxSteps: 1}, nil, nil, nil)
|
||||
core.GenerateOptions{Messages: messages, MaxSteps: 1}, nil, nil, g.usage, nil)
|
||||
if err != nil {
|
||||
if g.log != nil {
|
||||
g.log.Error("session title generation failed", "session", sessionID, "err", err)
|
||||
|
||||
@@ -29,6 +29,7 @@ type chatTurnModel struct {
|
||||
limits func() modelloop.ToolLimits
|
||||
// outcomes records each call's result for /status (see sendChatTurn).
|
||||
outcomes *providerOutcomeRecorder
|
||||
usage *usageSink
|
||||
}
|
||||
|
||||
func newChatTurnModel(
|
||||
@@ -37,6 +38,7 @@ func newChatTurnModel(
|
||||
maxSteps int,
|
||||
limits func() modelloop.ToolLimits,
|
||||
outcomes *providerOutcomeRecorder,
|
||||
usageRecords *usageSink,
|
||||
) gateway.TurnModel {
|
||||
return &chatTurnModel{
|
||||
llm: llm,
|
||||
@@ -44,6 +46,7 @@ func newChatTurnModel(
|
||||
maxSteps: maxSteps,
|
||||
limits: limits,
|
||||
outcomes: outcomes,
|
||||
usage: usageRecords,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -70,6 +73,7 @@ func (m *chatTurnModel) Prepare(
|
||||
toolInfo: toolSummaries(options.Tools),
|
||||
toolTokens: gateway.EstimateTokens(string(toolSchema)),
|
||||
outcomes: m.outcomes,
|
||||
usage: m.usage,
|
||||
icons: toolIcons(m.registry),
|
||||
}, nil
|
||||
}
|
||||
@@ -81,6 +85,7 @@ type preparedChatTurnModel struct {
|
||||
toolInfo []gateway.ToolSummary
|
||||
toolTokens int
|
||||
outcomes *providerOutcomeRecorder
|
||||
usage *usageSink
|
||||
// icons maps tool name to the icon that tool registered, for chat
|
||||
// surfaces to render. A nil map yields "" for every lookup, which is
|
||||
// the no-icon rendering.
|
||||
@@ -202,5 +207,5 @@ func (m *preparedChatTurnModel) Generate(
|
||||
if m.llm == nil {
|
||||
return "", fmt.Errorf("llm chat: no model runtime is configured")
|
||||
}
|
||||
return sendChatTurn(ctx, m.llm, m.model, options, stream, m.outcomes, m.icons)
|
||||
return sendChatTurn(ctx, m.llm, m.model, options, stream, m.outcomes, m.usage, m.icons)
|
||||
}
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/app/controlplane"
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/curator"
|
||||
"github.com/samcharles93/archie-core/internal/domain/usage"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
infraMemory "github.com/samcharles93/archie-core/internal/infrastructure/memory"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/sessioncurator"
|
||||
@@ -34,7 +35,7 @@ func (b *server) setupCurators(ctx context.Context) {
|
||||
MemoryEngines: b.memEngines,
|
||||
Skills: skillcurator.NewStore(skillsRoot),
|
||||
Conversations: sessioncurator.NewAdapter(b.chatSessionStore, b.cfg.BotUser),
|
||||
LLM: curatorLLMRunner{llm: b.chatLLM, outcomes: b.providerOutcomes},
|
||||
LLM: curatorLLMRunner{llm: b.chatLLM, outcomes: b.providerOutcomes, usage: b.usage},
|
||||
Model: b.chatModels.ActiveModel(),
|
||||
})
|
||||
|
||||
@@ -107,6 +108,7 @@ type curatorLLMRunner struct {
|
||||
// provider is demonstrably reachable -- or worse, keeps showing a much
|
||||
// older chat outcome as current.
|
||||
outcomes *providerOutcomeRecorder
|
||||
usage *usageSink
|
||||
}
|
||||
|
||||
func (r curatorLLMRunner) Chat(ctx context.Context, req curator.ChatRequest) (curator.ChatResult, error) {
|
||||
@@ -136,6 +138,7 @@ func (r curatorLLMRunner) Chat(ctx context.Context, req curator.ChatRequest) (cu
|
||||
MaxSteps: max(req.MaxSteps, 1),
|
||||
})
|
||||
r.outcomes.record(req.Model, err)
|
||||
r.usage.record(ctx, usage.SourceCurator, req.Model, res.TotalUsage)
|
||||
if err != nil {
|
||||
return curator.ChatResult{}, err
|
||||
}
|
||||
|
||||
@@ -30,15 +30,15 @@ func (b *server) setupChatRuntime(ctx context.Context, cfg config.Config, actor
|
||||
// non-command message processing.
|
||||
providers := executionProviders(cfg)
|
||||
b.setLLM(modelloop.NewRuntime(providers, cfg.ModelLimits))
|
||||
// Transcription is a model-role capability, like the chat models it sits
|
||||
// Transcription is a model-alias capability, like the chat models it sits
|
||||
// beside: it is built here, on the model-owning side, from this process's
|
||||
// own [models]/[providers]. The Messaging Service carries a voice note's
|
||||
// bytes to the Gateway and never holds the credential.
|
||||
b.setupTranscriber(cfg, b.log)
|
||||
b.toolReg = tools.NewRegistry()
|
||||
chatModels := newChatModelManager(cfg.Models, cfg.Chat.Models)
|
||||
catalog, catalogModels := b.catalogState()
|
||||
chatModels.SetModelCatalog(catalog, catalogModels)
|
||||
chatModels := newChatModelManager(cfg.Models)
|
||||
catalog := b.catalogState()
|
||||
chatModels.SetModelCatalog(catalog)
|
||||
b.chatModels = chatModels
|
||||
|
||||
// ── Persona registry ─────────────────────────────────────────────
|
||||
@@ -74,6 +74,14 @@ func (b *server) setupChatRuntime(ctx context.Context, cfg config.Config, actor
|
||||
// later carries both: the recorder is written by sendChatTurn, the source
|
||||
// is read by /status.
|
||||
b.providerOutcomes = newProviderOutcomeRecorder()
|
||||
b.usage = &usageSink{models: b.chatModels, log: b.log}
|
||||
if b.stateStore != nil {
|
||||
b.usage.store = b.stateStore
|
||||
b.chatModels.orgAliases = func(ctx context.Context) (map[string]string, error) {
|
||||
_, aliases, err := b.stateStore.CallerModels(ctx)
|
||||
return aliases, err
|
||||
}
|
||||
}
|
||||
b.statusHealth = newStatusHealth(b)
|
||||
|
||||
b.setupChatTasks(cfg)
|
||||
@@ -253,6 +261,7 @@ func (b *server) setupGatewayChat(ctx context.Context, actor gateway.ChatTaskAct
|
||||
// The Gateway executes the web chat's turns, so it is the process
|
||||
// that must record their outcomes for /status.
|
||||
ProviderOutcomes: b.providerOutcomes,
|
||||
Usage: b.usage,
|
||||
}
|
||||
router.LLM, router.LLMStream = makeChatLLMResponder(ctx, "web", setup, b.chatSessionStore, router)
|
||||
router.Titles = newChatTitleGenerator(setup)
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"maps"
|
||||
"reflect"
|
||||
"slices"
|
||||
"strings"
|
||||
@@ -135,11 +136,7 @@ func (b *server) disconnectAbsentMCPServers(ctx context.Context, desired map[str
|
||||
// applyDesiredMCPServers connects, reconnects or restarts the servers the
|
||||
// stored document names, skipping the ones already running unchanged.
|
||||
func (b *server) applyDesiredMCPServers(ctx context.Context, desired map[string]config.MCPServer, state mcpRegistryState) []error {
|
||||
names := make([]string, 0, len(desired))
|
||||
for name := range desired {
|
||||
names = append(names, name)
|
||||
}
|
||||
slices.Sort(names)
|
||||
names := slices.Sorted(maps.Keys(desired))
|
||||
|
||||
var problems []error
|
||||
for _, name := range names {
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"github.com/samcharles93/ai-sdk/core"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
|
||||
"github.com/samcharles93/archie-core/internal/domain/usage"
|
||||
"github.com/samcharles93/archie-core/internal/tools/mcp"
|
||||
)
|
||||
|
||||
@@ -48,6 +49,7 @@ func (b *server) mcpSamplingHandler() mcp.SamplingHandler {
|
||||
// recorded for /status like every other one; a nil recorder records
|
||||
// nothing.
|
||||
b.providerOutcomes.record(model, err)
|
||||
b.usage.record(ctx, usage.SourceChat, model, result.TotalUsage)
|
||||
if err != nil {
|
||||
return mcp.SamplingResult{}, fmt.Errorf("mcp sampling: %w", err)
|
||||
}
|
||||
|
||||
@@ -92,6 +92,7 @@ type server struct {
|
||||
curatorRegistry *curator.Registry
|
||||
curatorRuntime *curator.Runtime
|
||||
providerOutcomes *providerOutcomeRecorder
|
||||
usage *usageSink
|
||||
statusHealth gateway.HealthSource
|
||||
updateService *releaseupdate.Service
|
||||
|
||||
@@ -260,7 +261,7 @@ func (b *server) waitForNATSEndpoint(ctx context.Context) (servicekit.NATSEndpoi
|
||||
}
|
||||
}
|
||||
|
||||
func (b *server) catalogState() (modelcatalog.Snapshot, []string) { return b.catalog.State() }
|
||||
func (b *server) catalogState() modelcatalog.Snapshot { return b.catalog.State() }
|
||||
|
||||
// loadCatalog layers the model catalog under the running config. A catalog
|
||||
// that cannot be read leaves the configured providers and models in effect.
|
||||
@@ -270,9 +271,9 @@ func (b *server) loadCatalog(ctx context.Context) {
|
||||
b.log.Warn("model catalog unavailable; using configured providers and models", "err", err)
|
||||
return
|
||||
}
|
||||
models := modelcatalog.Apply(&b.cfg, snapshot)
|
||||
modelcatalog.Apply(&b.cfg, snapshot)
|
||||
b.cfgHolder.Set(b.cfg.Clone())
|
||||
b.catalog.Set(snapshot, models)
|
||||
b.catalog.Set(snapshot)
|
||||
}
|
||||
|
||||
func (b *server) startModelCatalogRefresh(ctx context.Context) {
|
||||
@@ -283,8 +284,8 @@ func (b *server) startModelCatalogRefresh(ctx context.Context) {
|
||||
return
|
||||
}
|
||||
base := b.cfgHolder.Get().Clone()
|
||||
models := modelcatalog.Apply(&base, snapshot)
|
||||
b.catalog.Set(snapshot, models)
|
||||
modelcatalog.Apply(&base, snapshot)
|
||||
b.catalog.Set(snapshot)
|
||||
if err := b.relayer(ctx, base, ""); err != nil {
|
||||
b.log.Warn("model catalog refresh rejected with the settings; the running settings stay", "err", err)
|
||||
}
|
||||
@@ -295,7 +296,7 @@ func (b *server) startModelCatalogRefresh(ctx context.Context) {
|
||||
// chat runtime and tool providers are built from.
|
||||
var gatewayRuntimeKinds = []string{
|
||||
controlplane.ProviderSettingsKind,
|
||||
controlplane.ModelRoleAssignmentsKind,
|
||||
controlplane.ModelAliasesKind,
|
||||
controlplane.ToolSettingsKind,
|
||||
}
|
||||
|
||||
@@ -304,7 +305,7 @@ func (b *server) startRuntimeWatches(ctx context.Context) error {
|
||||
// the providers resolve again.
|
||||
b.engines.OnChange(func() {
|
||||
base := b.cfgHolder.Get()
|
||||
catalog, _ := b.catalogState()
|
||||
catalog := b.catalogState()
|
||||
modelcatalog.Apply(&base, catalog)
|
||||
_ = b.relayer(ctx, base, controlplane.ProviderSettingsKind)
|
||||
})
|
||||
@@ -342,7 +343,7 @@ func (b *server) startRuntimeWatches(ctx context.Context) error {
|
||||
return
|
||||
}
|
||||
base := b.cfgHolder.Get()
|
||||
catalog, _ := b.catalogState()
|
||||
catalog := b.catalogState()
|
||||
modelcatalog.Apply(&base, catalog)
|
||||
lastApplyErr = b.relayer(ctx, base, kind)
|
||||
},
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
package archiegateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/ai-sdk/chat"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/usage"
|
||||
"github.com/samcharles93/archie-core/internal/gateway"
|
||||
)
|
||||
|
||||
const usageWriteTimeout = 5 * time.Second
|
||||
|
||||
// usageSink records the gateway's model calls. The State Store bills each to
|
||||
// the org of the principal the call was made for. A nil sink records nothing.
|
||||
type usageSink struct {
|
||||
store usage.Recorder
|
||||
models gateway.ModelManager
|
||||
log *slog.Logger
|
||||
}
|
||||
|
||||
// record appends one call's usage. It never fails the call: the write is
|
||||
// detached from the caller's cancellation and a failure is only logged.
|
||||
func (u *usageSink) record(ctx context.Context, source usage.Source, ref string, used chat.Usage) {
|
||||
if u == nil || u.store == nil || ref == "" {
|
||||
return
|
||||
}
|
||||
alias := ""
|
||||
if u.models != nil && u.models.ActiveModel() == ref {
|
||||
alias = u.models.ActiveAlias()
|
||||
}
|
||||
record := usage.Record{
|
||||
Source: source, Alias: alias,
|
||||
InputTokens: int64(used.PromptTokens), OutputTokens: int64(used.CompletionTokens),
|
||||
CachedTokens: int64(used.CachedTokens), At: time.Now().UTC(),
|
||||
}.WithRef(ref)
|
||||
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), usageWriteTimeout)
|
||||
defer cancel()
|
||||
if err := u.store.RecordUsage(ctx, record); err != nil && u.log != nil {
|
||||
u.log.Warn("record model usage failed", "source", source, "model", ref, "err", err)
|
||||
}
|
||||
}
|
||||
@@ -36,32 +36,21 @@ func Run(args []string, stderr io.Writer) int {
|
||||
return cmd(args[1:], stderr)
|
||||
}
|
||||
|
||||
// runLint validates routing binding directories (-dir) and an EDA playbook
|
||||
// directory (-eda-dir) against the loaders the daemon runs at startup.
|
||||
// Exit codes: 0 clean, 1 findings, 2 usage error.
|
||||
// runLint validates an EDA playbook directory against the compiler the daemon
|
||||
// runs. Exit codes: 0 clean, 1 findings, 2 usage error.
|
||||
func runLint(args []string, stderr io.Writer) int {
|
||||
flags := flag.NewFlagSet("archied playbooks lint", flag.ContinueOnError)
|
||||
flags.SetOutput(stderr)
|
||||
var dirs multiFlag
|
||||
flags.Var(&dirs, "dir", "routing binding directory to lint (repeatable)")
|
||||
edaDir := flags.String("eda-dir", "", "EDA playbook directory to lint")
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return 2 // flag.ContinueOnError already printed the message
|
||||
}
|
||||
if len(dirs) == 0 && *edaDir == "" {
|
||||
fmt.Fprintln(stderr, "lint: at least one -dir or an -eda-dir is required")
|
||||
if *edaDir == "" {
|
||||
fmt.Fprintln(stderr, "lint: -eda-dir is required")
|
||||
flags.Usage()
|
||||
return 2
|
||||
}
|
||||
|
||||
code := 0
|
||||
if len(dirs) > 0 {
|
||||
code = max(code, Lint(dirs, stderr).ExitCode)
|
||||
}
|
||||
if *edaDir != "" {
|
||||
code = max(code, LintEDA(*edaDir, stderr).ExitCode)
|
||||
}
|
||||
return code
|
||||
return LintEDA(*edaDir, stderr).ExitCode
|
||||
}
|
||||
|
||||
// runServe runs the language server on stdin/stdout until the editor
|
||||
@@ -87,13 +76,3 @@ type stdio struct{}
|
||||
func (stdio) Read(p []byte) (int, error) { return os.Stdin.Read(p) }
|
||||
func (stdio) Write(p []byte) (int, error) { return os.Stdout.Write(p) }
|
||||
func (stdio) Close() error { return os.Stdin.Close() }
|
||||
|
||||
// multiFlag collects repeated -dir flags.
|
||||
type multiFlag []string
|
||||
|
||||
func (m *multiFlag) String() string { return fmt.Sprint([]string(*m)) }
|
||||
|
||||
func (m *multiFlag) Set(v string) error {
|
||||
*m = append(*m, v)
|
||||
return nil
|
||||
}
|
||||
@@ -8,7 +8,6 @@ import (
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/eda/module"
|
||||
"github.com/samcharles93/archie-core/internal/domain/eda/playbook"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
)
|
||||
|
||||
// Result is the outcome of one lint run: the exit code for the process and
|
||||
@@ -18,25 +17,6 @@ type Result struct {
|
||||
Findings []string
|
||||
}
|
||||
|
||||
// Lint validates playbook directories with workflow.LoadPlaybookDirs and
|
||||
// reports every finding. It returns 0 when clean, 1 otherwise.
|
||||
func Lint(dirs []string, stderr io.Writer) Result {
|
||||
var findings []string
|
||||
|
||||
_, _, err := workflow.LoadPlaybookDirs(dirs)
|
||||
if err != nil {
|
||||
findings = append(findings, err.Error())
|
||||
}
|
||||
|
||||
if len(findings) > 0 {
|
||||
for _, f := range findings {
|
||||
fmt.Fprintln(stderr, f)
|
||||
}
|
||||
return Result{ExitCode: 1, Findings: findings}
|
||||
}
|
||||
return Result{ExitCode: 0, Findings: nil}
|
||||
}
|
||||
|
||||
// LintEDA validates a directory of EDA playbook documents with the compiler
|
||||
// the eda-playbooks resource validates against.
|
||||
func LintEDA(dir string, stderr io.Writer) Result {
|
||||
|
||||
@@ -1,10 +1,8 @@
|
||||
package archieplaybooks
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strconv"
|
||||
@@ -14,7 +12,6 @@ import (
|
||||
"go.lsp.dev/jsonrpc2"
|
||||
"go.lsp.dev/protocol"
|
||||
"go.lsp.dev/uri"
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// Serve runs the language server over rwc until the client disconnects or ctx
|
||||
@@ -87,12 +84,7 @@ func diagnose(u uri.URI) *protocol.PublishDiagnosticsParams {
|
||||
return out
|
||||
}
|
||||
path := u.FsPath()
|
||||
var result Result
|
||||
if isEDAPlaybook(path) {
|
||||
result = LintEDA(filepath.Dir(path), io.Discard)
|
||||
} else {
|
||||
result = Lint([]string{filepath.Dir(path)}, io.Discard)
|
||||
}
|
||||
result := LintEDA(filepath.Dir(path), io.Discard)
|
||||
for _, finding := range result.Findings {
|
||||
line, ok := findingLine(finding, path)
|
||||
if !ok {
|
||||
@@ -108,21 +100,6 @@ func diagnose(u uri.URI) *protocol.PublishDiagnosticsParams {
|
||||
return out
|
||||
}
|
||||
|
||||
// isEDAPlaybook reports whether the file is an EDA playbook document (it has
|
||||
// a top-level `trigger` key) rather than a routing binding file.
|
||||
func isEDAPlaybook(path string) bool {
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
var doc map[string]yaml.Node
|
||||
if yaml.NewDecoder(bytes.NewReader(data)).Decode(&doc) != nil {
|
||||
return false
|
||||
}
|
||||
_, ok := doc["trigger"]
|
||||
return ok
|
||||
}
|
||||
|
||||
// findingLine places a finding on path. A compiler-style `path:line:` prefix
|
||||
// gives the 0-based line; a finding naming path without a line goes on line
|
||||
// 0; a finding naming another file is not path's.
|
||||
|
||||
@@ -3,6 +3,7 @@ package archieui
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/app/servicekit"
|
||||
@@ -62,6 +63,10 @@ func compose(ctx context.Context, d deps) *webui.Server {
|
||||
Health: d.Health,
|
||||
ControlPlane: d.ControlPlane,
|
||||
Identities: d.Identities,
|
||||
Issuer: strings.TrimSpace(d.Options.OidcIssuer),
|
||||
}
|
||||
if subjects, ok := d.Identities.(identity.SubjectBinding); ok {
|
||||
srv.Subjects = subjects
|
||||
}
|
||||
if snapshots, ok := d.Store.(storecontract.ConfigSnapshotStore); ok {
|
||||
srv.ConfigSource = webui.RemoteConfigView(snapshots)
|
||||
@@ -116,10 +121,8 @@ func compose(ctx context.Context, d deps) *webui.Server {
|
||||
wireCatalog(d, srv)
|
||||
// The policy chain: wired together or not at all.
|
||||
srv.Refusals = refusalStore(d.Store)
|
||||
if d.Access != nil {
|
||||
srv.Access = d.Access
|
||||
srv.ReloadAccess = d.Access.Reload
|
||||
}
|
||||
srv.Access = d.Access
|
||||
srv.ReloadAccess = d.Access.Reload
|
||||
srv.Principals = d.Principals
|
||||
srv.Denials = d.Denials
|
||||
wireTaskLogs(d, srv)
|
||||
@@ -195,7 +198,7 @@ func wireCaptureSurfaces(d deps, srv *webui.Server) {
|
||||
MaxEvents: capture.MaxEvents,
|
||||
MaxBodyBytes: int64(capture.MaxBodyBytes),
|
||||
Publish: captureArrivalPublisher(d),
|
||||
Delivery: deliveryAuthorizer(d.Access),
|
||||
Delivery: d.Access,
|
||||
Refusals: refusalStore(d.Store),
|
||||
Log: d.Log,
|
||||
}
|
||||
@@ -221,15 +224,6 @@ func captureArrivalPublisher(d deps) func(context.Context, events.Event) {
|
||||
// a hung State Store must not hold the sender's webhook past this.
|
||||
const captureEventTimeout = 5 * time.Second
|
||||
|
||||
// deliveryAuthorizer keeps a missing chain a nil authorizer, which admits
|
||||
// every sender.
|
||||
func deliveryAuthorizer(chain *infraaccess.Live) access.DeliveryAuthorizer {
|
||||
if chain == nil {
|
||||
return nil
|
||||
}
|
||||
return chain
|
||||
}
|
||||
|
||||
func refusalStore(store storecontract.TaskStore) storecontract.CaptureRefusalStore {
|
||||
refusals, _ := store.(storecontract.CaptureRefusalStore)
|
||||
return refusals
|
||||
|
||||
@@ -176,7 +176,7 @@ func (o Options) validate() error {
|
||||
}
|
||||
if !webui.IsLoopback(o.Listen) && o.Token == "" {
|
||||
return fmt.Errorf(
|
||||
"ui listen address %q is non-loopback; a reachable dashboard requires a token (-token) or a token file to mint one (-token-file) (docs/prds/ui-service-boundary.md, listen and authentication)",
|
||||
"ui listen address %q is non-loopback; a reachable dashboard requires a token (-token) or a token file to mint one (-token-file) (deployments/README.md, running the dashboard)",
|
||||
o.Listen,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -18,7 +18,7 @@ import (
|
||||
// Store and, when one is configured, the sign-in provider.
|
||||
func newReadinessRegistry(o Options, tasks storecontract.TaskStore, chat messaging.ChatContract, chain *infraaccess.Live, provider *oidc.Provider) *health.Registry {
|
||||
probes := []health.Probe{
|
||||
readiness.NewProblemProbe("access_policies", problemSource(chain)),
|
||||
readiness.NewProblemProbe("access_policies", chain),
|
||||
captureRefusalProbe{store: refusalStore(tasks)},
|
||||
readiness.NewContractProbe("state_db", o.DependencyTimeout, func(ctx context.Context) error {
|
||||
_, err := tasks.StatusCounts(ctx)
|
||||
@@ -35,15 +35,6 @@ func newReadinessRegistry(o Options, tasks storecontract.TaskStore, chat messagi
|
||||
return health.NewRegistry(probes...)
|
||||
}
|
||||
|
||||
// problemSource keeps a missing chain a nil source, which the probe reports
|
||||
// as not wired.
|
||||
func problemSource(chain *infraaccess.Live) readiness.PolicyProblems {
|
||||
if chain == nil {
|
||||
return nil
|
||||
}
|
||||
return chain
|
||||
}
|
||||
|
||||
// captureRefusalProbe degrades when any source refused more events than it
|
||||
// accepted over the last hour: a sender being blocked, or someone probing.
|
||||
type captureRefusalProbe struct {
|
||||
|
||||
@@ -12,9 +12,6 @@ import (
|
||||
"slices"
|
||||
"time"
|
||||
|
||||
"google.golang.org/grpc/codes"
|
||||
"google.golang.org/grpc/status"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/app/servicekit"
|
||||
"github.com/samcharles93/archie-core/internal/domain/applystatus"
|
||||
"github.com/samcharles93/archie-core/internal/domain/presence"
|
||||
@@ -27,19 +24,14 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/webui"
|
||||
)
|
||||
|
||||
// buildAccessChain builds the live access engine over the stored policies.
|
||||
// An unreachable State Store returns nil and logs a warning.
|
||||
func buildAccessChain(ctx context.Context, store *staterpc.Client, log *slog.Logger) (*infraaccess.Live, error) {
|
||||
live, err := infraaccess.NewLive(ctx, store.ListPolicies, log)
|
||||
if status.Code(err) == codes.Unavailable {
|
||||
log.Warn("access chain unavailable; the credential check is the gate", "err", err)
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// buildAccessChain starts the live access engine over the stored policies.
|
||||
// It refuses every request until Run's first load succeeds and keeps
|
||||
// retrying, so a State Store or database that is down or slow at start
|
||||
// closes the dashboard rather than opening it or stopping it.
|
||||
func buildAccessChain(ctx context.Context, store *staterpc.Client, log *slog.Logger) *infraaccess.Live {
|
||||
live := infraaccess.NewPending(store.ListPolicies, log)
|
||||
go live.Run(ctx, applystatus.RestampInterval)
|
||||
return live, nil
|
||||
return live
|
||||
}
|
||||
|
||||
// Run serves the dashboard until ctx is cancelled. It resolves its own
|
||||
@@ -53,6 +45,7 @@ func Run(ctx context.Context, options Options) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
warnPlainHTTP(log, opts)
|
||||
var cleanups []func()
|
||||
defer func() {
|
||||
for _, cleanup := range slices.Backward(cleanups) {
|
||||
@@ -69,10 +62,7 @@ func Run(ctx context.Context, options Options) error {
|
||||
return err
|
||||
}
|
||||
cleanups = append(cleanups, closeGateway)
|
||||
chain, err := buildAccessChain(ctx, tasks, log)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
chain := buildAccessChain(ctx, tasks, log)
|
||||
provider, err := signInProvider(opts)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -131,6 +121,19 @@ func Run(ctx context.Context, options Options) error {
|
||||
return serve(ctx, listener, srv.Handler(), opts)
|
||||
}
|
||||
|
||||
// warnPlainHTTP names the exposure of a dashboard published off-host. The
|
||||
// process serves plain HTTP, so the session cookie and the provider token
|
||||
// cross the network in clear unless a TLS-terminating proxy sits in front of
|
||||
// it -- which [web].trust_forwarded_headers declares. A loopback bind is
|
||||
// reached over a tunnel or on the host, so it needs no warning.
|
||||
func warnPlainHTTP(log *slog.Logger, opts Options) {
|
||||
if webui.IsLoopback(opts.Listen) || opts.trustForwardedHeaders() {
|
||||
return
|
||||
}
|
||||
log.Warn("dashboard is reachable off-host over plain HTTP; session and provider tokens travel in clear unless TLS terminates in front of it (set [web].trust_forwarded_headers behind a TLS proxy)",
|
||||
"listen", opts.Listen)
|
||||
}
|
||||
|
||||
// wireHarness attaches the setup terminal to the daemon's session contract
|
||||
// when the operator configured one, and registers its cleanup.
|
||||
func wireHarness(opts Options, srv *webui.Server, cleanups *[]func()) error {
|
||||
|
||||
@@ -23,7 +23,7 @@ const ChannelSettingsKind = controlplanerpc.ChannelSettingsKind
|
||||
func seedChannels(cfg config.Config) any {
|
||||
chat := cfg.Chat
|
||||
return channelSettings{
|
||||
Operator: chat.Operator, Workspace: chat.Workspace, UnrestrictedFilesystem: chat.UnrestrictedFilesystem, ShowToolCalls: chat.ShowToolCalls, MaxSteps: chat.MaxSteps, Models: chat.Models,
|
||||
Operator: chat.Operator, Workspace: chat.Workspace, UnrestrictedFilesystem: chat.UnrestrictedFilesystem, ShowToolCalls: chat.ShowToolCalls, MaxSteps: chat.MaxSteps,
|
||||
Telegram: telegramSettings{AllowedUserIDs: chat.Telegram.AllowedUserIDs, Token: chat.Telegram.Token, CredentialConfigured: chat.Telegram.Token != (config.SecretRef{})},
|
||||
RateLimit: rateLimitSettings{Window: channelDuration(chat.RateLimit.Window), MaxRequests: chat.RateLimit.MaxRequests},
|
||||
}
|
||||
|
||||
@@ -3,7 +3,9 @@ package controlplane
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -50,6 +52,7 @@ type Server struct {
|
||||
store ResourceStore
|
||||
definitions map[string]Definition
|
||||
ordered []Definition
|
||||
steps workflow.StepRegistry
|
||||
}
|
||||
|
||||
// NewServer builds the control plane server the State Store serves. steps is
|
||||
@@ -66,7 +69,7 @@ func NewServer(resources ResourceStore, steps *workflow.Manager) (*Server, error
|
||||
for _, definition := range definitions {
|
||||
byKind[definition.Kind] = definition
|
||||
}
|
||||
return &Server{store: resources, definitions: byKind, ordered: definitions}, nil
|
||||
return &Server{store: resources, definitions: byKind, ordered: definitions, steps: registry}, nil
|
||||
}
|
||||
|
||||
func (s *Server) Catalog(context.Context, *pb.CatalogRequest) (*pb.CatalogResponse, error) {
|
||||
@@ -169,6 +172,9 @@ func (s *Server) Command(ctx context.Context, request *pb.CommandRequest) (*pb.C
|
||||
if err != nil {
|
||||
return nil, mapError(err)
|
||||
}
|
||||
if err := s.checkOrgWrite(ctx, requestOrg(ctx), request.Kind, value); err != nil {
|
||||
return nil, mapError(err)
|
||||
}
|
||||
resource, err := s.store.PutResource(ctx, storecontract.ResourceWrite{OrgID: requestOrg(ctx), Kind: request.Kind, Value: value, Actor: request.Actor, Source: request.Source, RequestID: request.RequestId, ExpectedVersion: request.ExpectedVersion, At: time.Now().UTC()})
|
||||
if err != nil {
|
||||
return nil, mapError(err)
|
||||
@@ -247,3 +253,20 @@ func mapError(err error) error {
|
||||
return status.Error(codes.Unavailable, err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// checkModelAliases refuses workflow definitions whose agent steps name a
|
||||
// model alias the writing org cannot resolve.
|
||||
func (s *Server) checkModelAliases(ctx context.Context, orgID string, value []byte) error {
|
||||
models, err := s.ModelsFor(ctx, orgID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var collection workflow.WorkflowDefinitionCollection
|
||||
if err := json.Unmarshal(value, &collection); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := workflow.CheckModelAliases(collection, s.steps, models.Aliases); err != nil {
|
||||
return fmt.Errorf("%w: %w", ErrValidation, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
// Schemas are not declared here: each Definition names its document type, and
|
||||
// Definition.Descriptor derives the JSON Schema from it (internal/app/controlplane/schema.go).
|
||||
func builtinDefinitions(steps workflow.StepRegistry, settings map[string]any) []Definition {
|
||||
definitions := []Definition{taskPolicyDefinition(), workflowDefinition(), workflowDefinitionsDefinition(steps, settings), workflowEnablementDefinition(), personaDefinition(), curatorDefinition(), soulDefinition(), scheduleDefinition(), agentProfileDefinition(), playbooksDefinition(), skillsDefinition(), credentialBindingsDefinition(), identityGrantsDefinition(), reviewSettingsDefinition()}
|
||||
definitions := []Definition{taskPolicyDefinition(), workflowDefinition(), workflowDefinitionsDefinition(steps, settings), workflowEnablementDefinition(), personaDefinition(), curatorDefinition(), soulDefinition(), scheduleDefinition(), agentProfileDefinition(), playbooksDefinition(), skillsDefinition(), credentialBindingsDefinition(), identityGrantsDefinition(), reviewSettingsDefinition(), orgModelPolicyDefinition()}
|
||||
definitions = append(definitions, modelDefinitions()...)
|
||||
definitions = append(definitions, operationalDefinitions()...)
|
||||
return definitions
|
||||
|
||||
@@ -2,7 +2,6 @@ package controlplane
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
@@ -75,21 +74,13 @@ func validServices(owner string, services []string) error {
|
||||
// each is its own resource, applies without a restart, and a stored value
|
||||
// replaces the file's outright.
|
||||
func layerCredentials(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, out *config.Config) error {
|
||||
if err := layerResource(ctx, reader, versions, CredentialBindingsKind, func(value []byte) error {
|
||||
var bindings []credentialBinding
|
||||
if err := json.Unmarshal(value, &bindings); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, CredentialBindingsKind, func(bindings []credentialBinding) error {
|
||||
out.Containers.Credentials = credentialBindingsSettings(bindings)
|
||||
return nil
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
return layerResource(ctx, reader, versions, IdentityGrantsKind, func(value []byte) error {
|
||||
var doc identityGrants
|
||||
if err := json.Unmarshal(value, &doc); err != nil {
|
||||
return err
|
||||
}
|
||||
return layerResource(ctx, reader, versions, IdentityGrantsKind, func(doc identityGrants) error {
|
||||
applyIdentityGrants(out, doc)
|
||||
return nil
|
||||
})
|
||||
|
||||
@@ -10,8 +10,8 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
ProviderSettingsKind = "provider-settings"
|
||||
ModelRoleAssignmentsKind = "model-role-assignments"
|
||||
ProviderSettingsKind = "provider-settings"
|
||||
ModelAliasesKind = "model-aliases"
|
||||
)
|
||||
|
||||
var bootDerivedProviderEnv = regexp.MustCompile(`^ARCHIE_PROVIDER_[0-9A-F]{16}_API_KEY$`)
|
||||
@@ -27,7 +27,7 @@ type providerDocument struct {
|
||||
func modelDefinitions() []Definition {
|
||||
return []Definition{
|
||||
{Kind: ProviderSettingsKind, Title: "Providers", ApplyMode: "live", Document: map[string]providerDocument{}, Seed: seedProviders, Validate: validateProviders},
|
||||
{Kind: ModelRoleAssignmentsKind, Title: "Model role assignments", ApplyMode: "live", Document: map[string]string{}, Seed: func(cfg config.Config) any { return cfg.Models }, Validate: validateModelRoles},
|
||||
{Kind: ModelAliasesKind, Title: "Model aliases", ApplyMode: "live", Document: map[string]string{}, Seed: func(cfg config.Config) any { return cfg.Models }, Validate: validateModelAliases},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -66,11 +66,16 @@ func rejectBootDerivedProviderEnv(name string, provider providerDocument) error
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateModelRoles(input []byte) error {
|
||||
var modelAliasName = regexp.MustCompile(`^[a-z0-9][a-z0-9._-]*$`)
|
||||
|
||||
func validateModelAliases(input []byte) error {
|
||||
return validateAs(input, func(models map[string]string) error {
|
||||
for role, model := range models {
|
||||
if strings.TrimSpace(role) == "" || !strings.Contains(model, "/") {
|
||||
return fmt.Errorf("model role %q must reference provider/model", role)
|
||||
for alias, model := range models {
|
||||
if !modelAliasName.MatchString(alias) {
|
||||
return fmt.Errorf("model alias %q must be lowercase letters, digits, '.', '_' or '-'", alias)
|
||||
}
|
||||
if provider, name, ok := strings.Cut(model, "/"); !ok || provider == "" || name == "" {
|
||||
return fmt.Errorf("model alias %q must reference provider/model", alias)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
|
||||
@@ -0,0 +1,234 @@
|
||||
package controlplane
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"maps"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
)
|
||||
|
||||
// OrgModelPolicyKind is the instance's per-org model policy. Only the
|
||||
// system org writes it.
|
||||
const OrgModelPolicyKind = "org-model-policy"
|
||||
|
||||
// orgModelPolicy is what one org may use. An org with no entry may use
|
||||
// nothing until the instance admin allows it; the system org is never
|
||||
// restricted.
|
||||
type orgModelPolicy struct {
|
||||
// Allowed lists the "provider/model" refs the org may use; "provider/*"
|
||||
// allows every model of an instance provider.
|
||||
Allowed []string `json:"allowed"`
|
||||
// OwnProviders lets the org add providers under its own credentials.
|
||||
OwnProviders bool `json:"own_providers"`
|
||||
}
|
||||
|
||||
func (p *orgModelPolicy) allows(ref string) bool {
|
||||
if p == nil {
|
||||
return false
|
||||
}
|
||||
provider, _, _ := strings.Cut(ref, "/")
|
||||
return slices.Contains(p.Allowed, ref) || slices.Contains(p.Allowed, provider+"/*")
|
||||
}
|
||||
|
||||
func orgModelPolicyDefinition() Definition {
|
||||
return Definition{
|
||||
Kind: OrgModelPolicyKind, Title: "Org model policy", ApplyMode: "live",
|
||||
Document: map[string]orgModelPolicy{},
|
||||
Seed: func(config.Config) any { return map[string]orgModelPolicy{} },
|
||||
Validate: func(input []byte) error {
|
||||
return validateAs(input, func(policies map[string]orgModelPolicy) error {
|
||||
for orgID, policy := range policies {
|
||||
if orgID == storecontract.DefaultOrgID {
|
||||
return fmt.Errorf("the system org uses every instance model; it takes no policy")
|
||||
}
|
||||
for _, ref := range policy.Allowed {
|
||||
if provider, model, ok := strings.Cut(ref, "/"); !ok || provider == "" || model == "" {
|
||||
return fmt.Errorf("org %q allows %q; use provider/model or provider/*", orgID, ref)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// ModelsFor resolves orgID's effective aliases and its own providers: the
|
||||
// instance aliases it may use, with its own aliases over them. An alias
|
||||
// whose model the org may not use is left out, so it fails as unknown.
|
||||
func (s *Server) ModelsFor(ctx context.Context, orgID string) (config.OrgModels, error) {
|
||||
instance, err := s.modelAliases(ctx, storecontract.DefaultOrgID)
|
||||
if err != nil || orgID == "" || orgID == storecontract.DefaultOrgID {
|
||||
return config.OrgModels{Org: orgID, Aliases: instance}, err
|
||||
}
|
||||
policy, err := s.orgPolicy(ctx, orgID)
|
||||
if err != nil {
|
||||
return config.OrgModels{}, err
|
||||
}
|
||||
providers, err := s.orgProviders(ctx, orgID, policy)
|
||||
if err != nil {
|
||||
return config.OrgModels{}, err
|
||||
}
|
||||
own, err := s.modelAliases(ctx, orgID)
|
||||
if err != nil {
|
||||
return config.OrgModels{}, err
|
||||
}
|
||||
aliases := make(map[string]string, len(instance)+len(own))
|
||||
for alias, ref := range instance {
|
||||
if policy.allows(ref) {
|
||||
aliases[alias] = ref
|
||||
}
|
||||
}
|
||||
for alias, ref := range own {
|
||||
if usable(ref, policy, providers) {
|
||||
aliases[alias] = ref
|
||||
} else {
|
||||
delete(aliases, alias)
|
||||
}
|
||||
}
|
||||
return config.OrgModels{Org: orgID, Aliases: aliases, Providers: providers}, nil
|
||||
}
|
||||
|
||||
func usable(ref string, policy *orgModelPolicy, own map[string]config.Provider) bool {
|
||||
provider, _, _ := strings.Cut(ref, "/")
|
||||
if _, ok := own[provider]; ok {
|
||||
return true
|
||||
}
|
||||
return policy.allows(ref)
|
||||
}
|
||||
|
||||
func (s *Server) orgPolicy(ctx context.Context, orgID string) (*orgModelPolicy, error) {
|
||||
var policies map[string]orgModelPolicy
|
||||
if err := s.readResource(ctx, storecontract.DefaultOrgID, OrgModelPolicyKind, &policies); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
policy, ok := policies[orgID]
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
return &policy, nil
|
||||
}
|
||||
|
||||
// orgProviders is orgID's own providers, empty unless its policy allows them.
|
||||
func (s *Server) orgProviders(ctx context.Context, orgID string, policy *orgModelPolicy) (map[string]config.Provider, error) {
|
||||
out := map[string]config.Provider{}
|
||||
if policy == nil || !policy.OwnProviders {
|
||||
return out, nil
|
||||
}
|
||||
var docs map[string]providerDocument
|
||||
if err := s.readResource(ctx, orgID, ProviderSettingsKind, &docs); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for name, doc := range docs {
|
||||
out[name] = config.Provider{Class: doc.Class, BaseURL: doc.BaseURL, APIKeyEnv: doc.APIKeyEnv}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (s *Server) modelAliases(ctx context.Context, orgID string) (map[string]string, error) {
|
||||
aliases := map[string]string{}
|
||||
return aliases, s.readResource(ctx, orgID, ModelAliasesKind, &aliases)
|
||||
}
|
||||
|
||||
// readResource decodes orgID's kind into into, leaving it untouched when the
|
||||
// org has none.
|
||||
func (s *Server) readResource(ctx context.Context, orgID, kind string, into any) error {
|
||||
resource, err := s.store.Resource(ctx, orgID, kind)
|
||||
if errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal(resource.Value, into); err != nil {
|
||||
return fmt.Errorf("read %s %s: %w", orgID, kind, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkOrgWrite applies the rules a resource's static validation cannot: the
|
||||
// policy is the system org's alone, an org's providers need its policy's
|
||||
// leave and its own credentials, and every alias must name a model the
|
||||
// writing org may use.
|
||||
func (s *Server) checkOrgWrite(ctx context.Context, orgID, kind string, value []byte) error {
|
||||
switch kind {
|
||||
case OrgModelPolicyKind:
|
||||
if orgID != storecontract.DefaultOrgID {
|
||||
return fmt.Errorf("%w: only the system org sets org model policy", ErrValidation)
|
||||
}
|
||||
case ProviderSettingsKind:
|
||||
if orgID != storecontract.DefaultOrgID {
|
||||
return s.checkOrgProviders(ctx, orgID, value)
|
||||
}
|
||||
case ModelAliasesKind:
|
||||
return s.checkAliasWrite(ctx, orgID, value)
|
||||
case WorkflowDefinitionsKind:
|
||||
return s.checkModelAliases(ctx, orgID, value)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) checkOrgProviders(ctx context.Context, orgID string, value []byte) error {
|
||||
var docs map[string]providerDocument
|
||||
if err := json.Unmarshal(value, &docs); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(docs) == 0 {
|
||||
return nil
|
||||
}
|
||||
policy, err := s.orgPolicy(ctx, orgID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if policy == nil || !policy.OwnProviders {
|
||||
return fmt.Errorf("%w: org %s may not add its own providers (instance Settings > Models > Org access)", ErrValidation, orgID)
|
||||
}
|
||||
var instance map[string]providerDocument
|
||||
if err := s.readResource(ctx, storecontract.DefaultOrgID, ProviderSettingsKind, &instance); err != nil {
|
||||
return err
|
||||
}
|
||||
for name, doc := range docs {
|
||||
if _, taken := instance[name]; taken {
|
||||
return fmt.Errorf("%w: provider %q is an instance provider; name yours differently", ErrValidation, name)
|
||||
}
|
||||
// The key comes from the org's own credential binding named after
|
||||
// the provider, never a secret this org could point at elsewhere.
|
||||
if doc.APIKey != (config.SecretRef{}) {
|
||||
return fmt.Errorf("%w: provider %q: an org provider takes its key from the org's credential binding named %q, not a secret reference", ErrValidation, name, name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) checkAliasWrite(ctx context.Context, orgID string, value []byte) error {
|
||||
var aliases map[string]string
|
||||
if err := json.Unmarshal(value, &aliases); err != nil {
|
||||
return err
|
||||
}
|
||||
if orgID == storecontract.DefaultOrgID {
|
||||
if _, ok := aliases[config.DefaultModelAlias]; !ok && len(aliases) > 0 {
|
||||
return fmt.Errorf("%w: model alias %q is required", ErrValidation, config.DefaultModelAlias)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
policy, err := s.orgPolicy(ctx, orgID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
providers, err := s.orgProviders(ctx, orgID, policy)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, alias := range slices.Sorted(maps.Keys(aliases)) {
|
||||
if !usable(aliases[alias], policy, providers) {
|
||||
return fmt.Errorf("%w: model alias %q: org %s may not use %q", ErrValidation, alias, orgID, aliases[alias])
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
package controlplane_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"maps"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/app/controlplane"
|
||||
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
|
||||
"github.com/samcharles93/archie-core/internal/domain/identity"
|
||||
"github.com/samcharles93/archie-core/internal/domain/org"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
|
||||
)
|
||||
|
||||
// An org reaches only the models the instance admin allows it and the
|
||||
// providers it was allowed to add; an org with no policy reaches nothing.
|
||||
func TestOrgModelPolicy(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
db := pgstore.Open(t)
|
||||
if err := db.BootstrapIdentities(ctx, nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.UpgradeDefaultOrg(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
steps, err := workflowsteps.NewManager()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
server, err := controlplane.NewServer(db, steps)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
owner, err := db.Create(ctx, identity.Identity{ID: "owner", Kind: identity.KindUser, DisplayName: "Owner", Lifecycle: identity.LifecycleActive}, identity.Audit{ActorID: identity.SystemID, Source: "test", RequestID: "owner", At: time.Now()})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, id := range []org.OrgID{"acme", "newco"} {
|
||||
if _, err := db.CreateOrg(ctx, org.Org{ID: id, Name: string(id)}, owner.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
sys := org.WithOrg(ctx, org.DefaultOrgID)
|
||||
acme := org.WithOrg(ctx, "acme")
|
||||
put := func(ctx context.Context, kind, value string) error {
|
||||
_, err := server.Command(ctx, &pb.CommandRequest{Kind: kind, Command: "replace", ValueJson: []byte(value), Actor: "test", Source: "test", RequestId: kind + value})
|
||||
return err
|
||||
}
|
||||
for _, w := range [][2]string{
|
||||
{controlplane.ProviderSettingsKind, `{"openai":{"class":"openai","api_key_ref":{}},"deepseek":{"class":"deepseek","api_key_ref":{}}}`},
|
||||
{controlplane.ModelAliasesKind, `{"default":"openai/gpt-6-luna","cheap":"deepseek/chat"}`},
|
||||
{controlplane.OrgModelPolicyKind, `{"acme":{"allowed":["openai/*"],"own_providers":true}}`},
|
||||
} {
|
||||
if err := put(sys, w[0], w[1]); err != nil {
|
||||
t.Fatalf("seed %s: %v", w[0], err)
|
||||
}
|
||||
}
|
||||
|
||||
for _, tt := range []struct {
|
||||
name, kind, value string
|
||||
ok bool
|
||||
}{
|
||||
{"an alias to a model the org may not use", controlplane.ModelAliasesKind, `{"fast":"deepseek/chat"}`, false},
|
||||
{"an org provider named like an instance one", controlplane.ProviderSettingsKind, `{"openai":{"class":"openai","api_key_ref":{}}}`, false},
|
||||
{"an org provider pointing at a secret", controlplane.ProviderSettingsKind, `{"mine":{"class":"openai","api_key_ref":{"engine":"env","key":"X"}}}`, false},
|
||||
{"an org writing the instance policy", controlplane.OrgModelPolicyKind, `{"acme":{"allowed":["deepseek/*"]}}`, false},
|
||||
{"an org provider of its own", controlplane.ProviderSettingsKind, `{"mine":{"class":"openai","api_key_ref":{}}}`, true},
|
||||
{"aliases to allowed and own models", controlplane.ModelAliasesKind, `{"fast":"openai/gpt-6-mini","own":"mine/m1"}`, true},
|
||||
} {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if err := put(acme, tt.kind, tt.value); (err == nil) != tt.ok {
|
||||
t.Fatalf("err = %v, want ok %v", err, tt.ok)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
for _, tt := range []struct {
|
||||
org string
|
||||
aliases, prov []string
|
||||
}{
|
||||
{"acme", []string{"default", "fast", "own"}, []string{"mine"}},
|
||||
{"newco", nil, nil},
|
||||
{string(org.DefaultOrgID), []string{"cheap", "default"}, nil},
|
||||
} {
|
||||
t.Run("resolve "+tt.org, func(t *testing.T) {
|
||||
models, err := server.ModelsFor(ctx, tt.org)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := slices.Sorted(maps.Keys(models.Aliases)); strings.Join(got, ",") != strings.Join(tt.aliases, ",") {
|
||||
t.Fatalf("aliases = %v, want %v", got, tt.aliases)
|
||||
}
|
||||
if got := slices.Sorted(maps.Keys(models.Providers)); strings.Join(got, ",") != strings.Join(tt.prov, ",") {
|
||||
t.Fatalf("providers = %v, want %v", got, tt.prov)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -68,11 +68,7 @@ func (r storeReader) Query(ctx context.Context, kind string, decode func([]byte)
|
||||
func runtimeConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReader, base config.Config) (config.Config, map[string]int64, error) {
|
||||
out := base.Clone()
|
||||
versions := map[string]int64{}
|
||||
if err := layerResource(ctx, reader, versions, ProviderSettingsKind, func(value []byte) error {
|
||||
var providers map[string]providerDocument
|
||||
if err := json.Unmarshal(value, &providers); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, ProviderSettingsKind, func(providers map[string]providerDocument) error {
|
||||
out.Providers = make(map[string]config.Provider, len(providers))
|
||||
for name, provider := range providers {
|
||||
if err := rejectBootDerivedProviderEnv(name, provider); err != nil {
|
||||
@@ -84,13 +80,9 @@ func runtimeConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReade
|
||||
}); err != nil {
|
||||
return config.Config{}, nil, err
|
||||
}
|
||||
// Stored role assignments replace cfg.Models outright.
|
||||
if err := layerResource(ctx, reader, versions, ModelRoleAssignmentsKind, func(value []byte) error {
|
||||
var roles map[string]string
|
||||
if err := json.Unmarshal(value, &roles); err != nil {
|
||||
return err
|
||||
}
|
||||
out.Models = roles
|
||||
// Stored aliases replace cfg.Models outright.
|
||||
if err := layerResource(ctx, reader, versions, ModelAliasesKind, func(aliases map[string]string) error {
|
||||
out.Models = aliases
|
||||
return nil
|
||||
}); err != nil {
|
||||
return config.Config{}, nil, err
|
||||
@@ -103,11 +95,7 @@ func runtimeConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReade
|
||||
return config.Config{}, nil, err
|
||||
}
|
||||
out.Chat, versions[ChannelSettingsKind] = chat, chatVersion
|
||||
if err := layerResource(ctx, reader, versions, SchedulingPolicyKind, func(value []byte) error {
|
||||
var policy schedulingPolicy
|
||||
if err := json.Unmarshal(value, &policy); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, SchedulingPolicyKind, func(policy schedulingPolicy) error {
|
||||
interval, err := time.ParseDuration(policy.PollInterval)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -125,11 +113,7 @@ func runtimeConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReade
|
||||
// The two fields are assigned individually, not `out.Review = ...`: a field
|
||||
// added to config.Review later keeps the file's value until the stored
|
||||
// document carries it.
|
||||
if err := layerResource(ctx, reader, versions, ReviewSettingsKind, func(value []byte) error {
|
||||
var settings reviewSettings
|
||||
if err := json.Unmarshal(value, &settings); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, ReviewSettingsKind, func(settings reviewSettings) error {
|
||||
out.Review.PrecisionGate = settings.PrecisionGate
|
||||
out.Review.ApproveBeforePost = settings.ApproveBeforePost
|
||||
return nil
|
||||
@@ -147,11 +131,7 @@ func runtimeToolConfigFrom(ctx context.Context, reader controlplanerpc.ResourceR
|
||||
out.DiffCapLines = new(policy.DiffCapLines)
|
||||
out.Notify.Webhook = policy.NotifyWebhook
|
||||
|
||||
if err := layerResource(ctx, reader, versions, ToolSettingsKind, func(value []byte) error {
|
||||
var settings toolSettings
|
||||
if err := json.Unmarshal(value, &settings); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, ToolSettingsKind, func(settings toolSettings) error {
|
||||
headers := make(map[string]map[string]string, len(out.Tools.MCPServers))
|
||||
fileParallelToolCalls := make(map[string]bool, len(out.Tools.MCPServers))
|
||||
for _, server := range out.Tools.MCPServers {
|
||||
@@ -184,21 +164,13 @@ func runtimeToolConfigFrom(ctx context.Context, reader controlplanerpc.ResourceR
|
||||
}); err != nil {
|
||||
return config.Config{}, nil, err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, PluginSettingsKind, func(value []byte) error {
|
||||
var settings pluginSettings
|
||||
if err := json.Unmarshal(value, &settings); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, PluginSettingsKind, func(settings pluginSettings) error {
|
||||
out.SkillsDir = settings.SkillsDir
|
||||
return nil
|
||||
}); err != nil {
|
||||
return config.Config{}, nil, err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, ContainerRuntimePoliciesKind, func(value []byte) error {
|
||||
var policies containerRuntimePolicies
|
||||
if err := json.Unmarshal(value, &policies); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, ContainerRuntimePoliciesKind, func(policies containerRuntimePolicies) error {
|
||||
// Keep the file-owned Profiles and RegistryAuth across the replacement.
|
||||
profiles := out.Containers.Profiles
|
||||
registryAuth := out.Containers.RegistryAuth
|
||||
@@ -210,11 +182,7 @@ func runtimeToolConfigFrom(ctx context.Context, reader controlplanerpc.ResourceR
|
||||
return config.Config{}, nil, err
|
||||
}
|
||||
// Stored profiles replace the file's outright.
|
||||
if err := layerResource(ctx, reader, versions, AgentProfileKind, func(value []byte) error {
|
||||
var profiles map[string]agentProfile
|
||||
if err := json.Unmarshal(value, &profiles); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := layerResource(ctx, reader, versions, AgentProfileKind, func(profiles map[string]agentProfile) error {
|
||||
out.Containers.Profiles = agentProfilesSettings(profiles)
|
||||
return nil
|
||||
}); err != nil {
|
||||
@@ -224,9 +192,25 @@ func runtimeToolConfigFrom(ctx context.Context, reader controlplanerpc.ResourceR
|
||||
return out, versions, err
|
||||
}
|
||||
|
||||
// layerResource decodes a stored resource and records its version. A kind
|
||||
// with no stored value is skipped.
|
||||
func layerResource(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, decode func([]byte) error) error {
|
||||
// layerResource decodes a stored JSON resource into a T, applies it and
|
||||
// records its version. A kind with no stored value is skipped.
|
||||
func layerResource[T any](ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, apply func(T) error) error {
|
||||
return layerRaw(ctx, reader, versions, kind, func(value []byte) error {
|
||||
var v T
|
||||
if err := json.Unmarshal(value, &v); err != nil {
|
||||
return err
|
||||
}
|
||||
return apply(v)
|
||||
})
|
||||
}
|
||||
|
||||
// layerResourceJSON decodes a stored resource over target, keeping fields
|
||||
// the stored document omits.
|
||||
func layerResourceJSON(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, target any) error {
|
||||
return layerRaw(ctx, reader, versions, kind, func(value []byte) error { return json.Unmarshal(value, target) })
|
||||
}
|
||||
|
||||
func layerRaw(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, decode func([]byte) error) error {
|
||||
version, found, err := reader.Query(ctx, kind, decode)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -237,10 +221,6 @@ func layerResource(ctx context.Context, reader controlplanerpc.ResourceReader, v
|
||||
return nil
|
||||
}
|
||||
|
||||
func layerResourceJSON(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, kind string, target any) error {
|
||||
return layerResource(ctx, reader, versions, kind, func(value []byte) error { return json.Unmarshal(value, target) })
|
||||
}
|
||||
|
||||
// query returns the version of the resource it decoded, so callers that layer
|
||||
// a resource in can report the version they applied. A kind the store holds no
|
||||
// value for is reported as not found rather than as an error: the caller leaves
|
||||
|
||||
+86
-27
@@ -1,40 +1,68 @@
|
||||
{
|
||||
"description": "Shared accept/reject fixtures for the control-plane validators the dashboard hand-mirrors in ui/src/settings/validation.ts. The control plane is the authority: every case's `valid` is what internal/app/controlplane accepts or refuses. The Go test in this package runs the real server validators over these cases; the node test ui/test/control-plane-validator-parity.test.ts runs the real dashboard validators over the same cases. A one-sided change to either rule makes one of the two tests fail, which is the tie that stops the two implementations drifting apart (archie-core-ui-dashboard-3). Add a case here, never only to one side.",
|
||||
"cases": {
|
||||
"model-role-assignments": [
|
||||
"model-aliases": [
|
||||
{
|
||||
"why": "a plain provider/model value",
|
||||
"value": { "builder": "openai/gpt-4o" },
|
||||
"value": {
|
||||
"default": "openai/gpt-4o"
|
||||
},
|
||||
"valid": true
|
||||
},
|
||||
{
|
||||
"why": "a trailing space: the server only asks for a slash, so the dashboard must not refuse it",
|
||||
"value": { "builder": "openai/gpt-4o " },
|
||||
"valid": true
|
||||
},
|
||||
{
|
||||
"why": "a leading space: the server only asks for a slash, so the dashboard must not refuse it",
|
||||
"value": { "builder": " openai/gpt-4o" },
|
||||
"why": "a trailing space: the server only splits on the slash, so the dashboard must not refuse it",
|
||||
"value": {
|
||||
"default": "openai/gpt-4o "
|
||||
},
|
||||
"valid": true
|
||||
},
|
||||
{
|
||||
"why": "a model id with a colon (a real ollama tag), which a stricter shape would wrongly refuse",
|
||||
"value": { "planner": "ollama/qwen2.5-coder:7b" },
|
||||
"value": {
|
||||
"default": "openai/gpt-4o",
|
||||
"fast": "ollama/qwen2.5-coder:7b"
|
||||
},
|
||||
"valid": true
|
||||
},
|
||||
{
|
||||
"why": "an empty table: nothing configured yet",
|
||||
"value": {},
|
||||
"valid": true
|
||||
},
|
||||
{
|
||||
"why": "no slash at all",
|
||||
"value": { "builder": "gpt-4o" },
|
||||
"value": {
|
||||
"default": "gpt-4o"
|
||||
},
|
||||
"valid": false
|
||||
},
|
||||
{
|
||||
"why": "an empty model half",
|
||||
"value": {
|
||||
"default": "openai/"
|
||||
},
|
||||
"valid": false
|
||||
},
|
||||
{
|
||||
"why": "an empty model: the server refuses it, so the dashboard must too (SystemModelsPage deletes the key rather than storing one)",
|
||||
"value": { "builder": "" },
|
||||
"value": {
|
||||
"default": ""
|
||||
},
|
||||
"valid": false
|
||||
},
|
||||
{
|
||||
"why": "a blank role key",
|
||||
"value": { "": "openai/gpt-4o" },
|
||||
"why": "aliases without the default alias: an org's own table inherits the instance default, so only the instance save requires it",
|
||||
"value": {
|
||||
"fast": "openai/gpt-4o"
|
||||
},
|
||||
"valid": true
|
||||
},
|
||||
{
|
||||
"why": "an alias name with capitals or spaces",
|
||||
"value": {
|
||||
"default": "openai/gpt-4o",
|
||||
"Fast model": "openai/gpt-4o"
|
||||
},
|
||||
"valid": false
|
||||
}
|
||||
],
|
||||
@@ -45,7 +73,9 @@
|
||||
"poll_interval": "1m0s",
|
||||
"max_retries": 0,
|
||||
"label": "archie",
|
||||
"dispatch": { "trigger": "assignee" }
|
||||
"dispatch": {
|
||||
"trigger": "assignee"
|
||||
}
|
||||
},
|
||||
"valid": true
|
||||
},
|
||||
@@ -54,7 +84,9 @@
|
||||
"value": {
|
||||
"poll_interval": "30s",
|
||||
"max_retries": 3,
|
||||
"dispatch": { "trigger": "assignee" }
|
||||
"dispatch": {
|
||||
"trigger": "assignee"
|
||||
}
|
||||
},
|
||||
"valid": true
|
||||
},
|
||||
@@ -63,7 +95,9 @@
|
||||
"value": {
|
||||
"poll_interval": "1h30m",
|
||||
"label": "archie",
|
||||
"dispatch": { "trigger": "either" }
|
||||
"dispatch": {
|
||||
"trigger": "either"
|
||||
}
|
||||
},
|
||||
"valid": true
|
||||
},
|
||||
@@ -71,7 +105,9 @@
|
||||
"why": "a fractional duration the server's parser accepts",
|
||||
"value": {
|
||||
"poll_interval": "1.5h",
|
||||
"dispatch": { "trigger": "assignee" }
|
||||
"dispatch": {
|
||||
"trigger": "assignee"
|
||||
}
|
||||
},
|
||||
"valid": true
|
||||
},
|
||||
@@ -80,7 +116,9 @@
|
||||
"value": {
|
||||
"poll_interval": "0s",
|
||||
"label": "archie",
|
||||
"dispatch": { "trigger": "assignee" }
|
||||
"dispatch": {
|
||||
"trigger": "assignee"
|
||||
}
|
||||
},
|
||||
"valid": false
|
||||
},
|
||||
@@ -89,7 +127,9 @@
|
||||
"value": {
|
||||
"poll_interval": "nonsense",
|
||||
"label": "archie",
|
||||
"dispatch": { "trigger": "assignee" }
|
||||
"dispatch": {
|
||||
"trigger": "assignee"
|
||||
}
|
||||
},
|
||||
"valid": false
|
||||
},
|
||||
@@ -98,7 +138,9 @@
|
||||
"value": {
|
||||
"poll_interval": "30s",
|
||||
"max_retries": 3,
|
||||
"dispatch": { "trigger": "label" }
|
||||
"dispatch": {
|
||||
"trigger": "label"
|
||||
}
|
||||
},
|
||||
"valid": false
|
||||
},
|
||||
@@ -108,7 +150,9 @@
|
||||
"poll_interval": "30s",
|
||||
"max_retries": 0,
|
||||
"label": "",
|
||||
"dispatch": { "trigger": "either" }
|
||||
"dispatch": {
|
||||
"trigger": "either"
|
||||
}
|
||||
},
|
||||
"valid": false
|
||||
}
|
||||
@@ -120,7 +164,10 @@
|
||||
{
|
||||
"id": "daily",
|
||||
"kind": "workflow",
|
||||
"schedule": { "kind": "interval", "interval": "24h0m0s" }
|
||||
"schedule": {
|
||||
"kind": "interval",
|
||||
"interval": "24h0m0s"
|
||||
}
|
||||
}
|
||||
],
|
||||
"valid": true
|
||||
@@ -132,7 +179,10 @@
|
||||
"id": "nightly",
|
||||
"kind": "workflow",
|
||||
"pool": "parallel",
|
||||
"schedule": { "kind": "cron", "cron": "0 3 * * *" }
|
||||
"schedule": {
|
||||
"kind": "cron",
|
||||
"cron": "0 3 * * *"
|
||||
}
|
||||
}
|
||||
],
|
||||
"valid": true
|
||||
@@ -143,7 +193,10 @@
|
||||
{
|
||||
"id": "",
|
||||
"kind": "workflow",
|
||||
"schedule": { "kind": "interval", "interval": "24h0m0s" }
|
||||
"schedule": {
|
||||
"kind": "interval",
|
||||
"interval": "24h0m0s"
|
||||
}
|
||||
}
|
||||
],
|
||||
"valid": false
|
||||
@@ -154,12 +207,18 @@
|
||||
{
|
||||
"id": "daily",
|
||||
"kind": "workflow",
|
||||
"schedule": { "kind": "interval", "interval": "24h0m0s" }
|
||||
"schedule": {
|
||||
"kind": "interval",
|
||||
"interval": "24h0m0s"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "daily",
|
||||
"kind": "workflow",
|
||||
"schedule": { "kind": "interval", "interval": "1h0m0s" }
|
||||
"schedule": {
|
||||
"kind": "interval",
|
||||
"interval": "1h0m0s"
|
||||
}
|
||||
}
|
||||
],
|
||||
"valid": false
|
||||
|
||||
@@ -169,10 +169,7 @@ func checkWorkflowDashboard(t *testing.T, db *pgstore.TaskDB, client *staterpc.C
|
||||
t.Helper()
|
||||
ctx := t.Context()
|
||||
policies := append(access.ShippedOrgPolicies("acme"), access.ShippedOrgPolicies(org.DefaultOrgID)...)
|
||||
chain, err := infraaccess.New(policies)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
chain := infraaccess.New(policies)
|
||||
srv := &webui.Server{Store: client, ControlPlane: client.ControlPlane(), Access: chain, Principals: client, Authenticate: func(ctx context.Context, token string) (identity.Identity, error) {
|
||||
return db.Get(ctx, identity.IdentityID(token))
|
||||
}}
|
||||
|
||||
@@ -56,7 +56,10 @@ func (p *WorkflowProjector) Apply(ctx context.Context, orgID, name, digest strin
|
||||
if err != nil {
|
||||
return fmt.Errorf("contributed workflow %q: %w", contentName, err)
|
||||
}
|
||||
entries = append(entries, workflow.WorkflowDefinitionEntry{ID: id, YAML: string(content)})
|
||||
if workflow.PackageOf(string(content)) != "" {
|
||||
return fmt.Errorf("contributed workflow %q: package is set by the installer, not the workflow", contentName)
|
||||
}
|
||||
entries = append(entries, workflow.WorkflowDefinitionEntry{ID: id, YAML: "package: " + strconv.Quote(name) + "\n" + string(content)})
|
||||
recorded = append(recorded, storepkg.ProjectionEntry{Family: storepkg.FamilyWorkflows, EntryID: id})
|
||||
}
|
||||
// One apply, one request: a retried PutResource reuses the same request ID
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
package prbench
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/samcharles93/ai-sdk/agent"
|
||||
"github.com/samcharles93/ai-sdk/core"
|
||||
"github.com/samcharles93/ai-sdk/runtime"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview/bench"
|
||||
)
|
||||
|
||||
// modelJudge asks one model, independent of the reviewer's, which posted
|
||||
// comment catches each golden.
|
||||
type modelJudge struct {
|
||||
runtime *runtime.Runtime
|
||||
modelRef string
|
||||
}
|
||||
|
||||
func (j *modelJudge) Judge(ctx context.Context, p bench.Problem, comments []prreview.ScoredFinding) ([]bench.Verdict, error) {
|
||||
provider, model, err := j.runtime.ChatProvider(ctx, j.modelRef)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("resolve judge model %q: %w", j.modelRef, err)
|
||||
}
|
||||
var verdicts []bench.Verdict
|
||||
sub := agent.Subagent{
|
||||
Provider: provider,
|
||||
Model: model,
|
||||
System: judgeSystemPrompt,
|
||||
Tools: core.ToolSet{"record_verdict": recordVerdictTool(len(p.Goldens), len(comments), &verdicts)},
|
||||
MaxSteps: 2*len(p.Goldens) + 3,
|
||||
}
|
||||
// The verdicts are the judgement, not the closing reply: a run that recorded
|
||||
// some and then ran out of steps is checked for completeness by the caller.
|
||||
if _, err := sub.Run(ctx, judgePrompt(p, comments)); err != nil && len(verdicts) == 0 {
|
||||
return nil, err
|
||||
}
|
||||
return verdicts, nil
|
||||
}
|
||||
|
||||
const judgeSystemPrompt = `You score a code reviewer against known defects. You are given the pull request's known defects (goldens) and the numbered comments the reviewer posted.
|
||||
|
||||
For each golden, decide whether any posted comment identifies the same underlying defect: the same location and the same root cause. A comment about the same file but a different problem is not a match, and neither is a vague comment that would only be right by coincidence. Wording does not need to match.
|
||||
|
||||
Call record_verdict exactly once per golden. Pass the index of the matching comment, or -1 when no comment matches, and a one-sentence reason. When several comments match, pass the one that states the defect most directly.`
|
||||
|
||||
func judgePrompt(p bench.Problem, comments []prreview.ScoredFinding) string {
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "Pull request: %s\n\n<goldens>\n", p.PRURL)
|
||||
for i, g := range p.Goldens {
|
||||
fmt.Fprintf(&b, "[%d] (%s) %s\n", i, g.Severity, g.Comment)
|
||||
}
|
||||
b.WriteString("</goldens>\n\n<comments>\n")
|
||||
for i, c := range comments {
|
||||
fmt.Fprintf(&b, "[%d] %s:%d-%d (%s) %s\n%s\n\n", i, c.File, c.LineStart, c.LineEnd, c.Severity, c.Title, c.Body)
|
||||
}
|
||||
b.WriteString("</comments>\n")
|
||||
return b.String()
|
||||
}
|
||||
|
||||
type verdictInput struct {
|
||||
Golden int `json:"golden" jsonschema:"description=Index of the golden this verdict is for."`
|
||||
Comment int `json:"comment" jsonschema:"description=Index of the comment that identifies the same defect, or -1 when none does."`
|
||||
Reason string `json:"reason" jsonschema:"description=One sentence on why the comment does or does not match."`
|
||||
}
|
||||
|
||||
func recordVerdictTool(goldens, comments int, verdicts *[]bench.Verdict) *core.Tool {
|
||||
return core.NewTypedTool(
|
||||
"record_verdict",
|
||||
"Record whether a posted comment catches one golden. Call once per golden.",
|
||||
func(_ context.Context, in verdictInput) (string, error) {
|
||||
if in.Golden < 0 || in.Golden >= goldens {
|
||||
return fmt.Sprintf("record_verdict rejected: golden must be 0..%d", goldens-1), nil
|
||||
}
|
||||
if in.Comment < bench.NoComment || in.Comment >= comments {
|
||||
return fmt.Sprintf("record_verdict rejected: comment must be -1..%d", comments-1), nil
|
||||
}
|
||||
for _, v := range *verdicts {
|
||||
if v.Golden == in.Golden {
|
||||
return fmt.Sprintf("record_verdict rejected: golden %d already has a verdict", in.Golden), nil
|
||||
}
|
||||
}
|
||||
*verdicts = append(*verdicts, bench.Verdict{Golden: in.Golden, Comment: in.Comment, Reason: in.Reason})
|
||||
return "verdict recorded", nil
|
||||
},
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
package prbench
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/agentrun"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview/bench"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/prsource"
|
||||
)
|
||||
|
||||
// liveReviewer runs the production pr-review decision phases on one problem.
|
||||
type liveReviewer struct {
|
||||
cfg config.Config
|
||||
agent agentrun.Runner
|
||||
source workflow.PRSource
|
||||
nextID atomic.Int64
|
||||
}
|
||||
|
||||
func (r *liveReviewer) Review(ctx context.Context, p bench.Problem) (bench.Review, error) {
|
||||
owner, repo, number, err := parsePRURL(p.PRURL)
|
||||
if err != nil {
|
||||
return bench.Review{}, err
|
||||
}
|
||||
slog.Info("benchmark review started", "problem", p.ID)
|
||||
tc := &workflow.TaskContext{
|
||||
Task: &workflow.Task{ID: r.nextID.Add(1), Attempt: 1, Owner: owner, Repo: repo, PRNumber: number, Workflow: "pr-review", Inputs: map[string]any{"depth": "deep"}},
|
||||
Cfg: r.cfg, Agent: r.agent, PRSource: r.source, Log: slog.Default(),
|
||||
}
|
||||
decision, err := workflow.ReviewPullRequest(ctx, tc)
|
||||
if err != nil {
|
||||
return bench.Review{}, err
|
||||
}
|
||||
// Non-nil empty comments means a completed pipeline found nothing.
|
||||
if decision.Comments == nil {
|
||||
decision.Comments = []prreview.ScoredFinding{}
|
||||
}
|
||||
slog.Info("benchmark review completed", "problem", p.ID, "comments", len(decision.Comments), "skipped_phases", decision.SkippedPhases)
|
||||
return bench.Review{Comments: decision.Comments, SkippedPhases: decision.SkippedPhases}, nil
|
||||
}
|
||||
|
||||
func parsePRURL(raw string) (string, string, int, error) {
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
return "", "", 0, fmt.Errorf("invalid pull request URL: %w", err)
|
||||
}
|
||||
parts := strings.Split(strings.Trim(u.Path, "/"), "/")
|
||||
if u.Scheme != "https" || u.Host != "github.com" || len(parts) != 4 || parts[2] != "pull" {
|
||||
return "", "", 0, fmt.Errorf("benchmark requires a GitHub pull request URL")
|
||||
}
|
||||
n, err := strconv.Atoi(parts[3])
|
||||
if err != nil || n <= 0 {
|
||||
return "", "", 0, fmt.Errorf("invalid pull request URL")
|
||||
}
|
||||
return parts[0], parts[1], n, nil
|
||||
}
|
||||
|
||||
// publicGitHub reads the benchmark's public pull requests anonymously. The
|
||||
// diff and archive come from hosts outside the API rate limit, so a full run
|
||||
// makes one API call per pull request. GH_TOKEN or GITHUB_TOKEN, when set,
|
||||
// only lifts that API limit.
|
||||
type publicGitHub struct {
|
||||
token string
|
||||
mu sync.Mutex
|
||||
prs map[string]prsource.PullRequest
|
||||
}
|
||||
|
||||
func newPublicGitHub() *publicGitHub {
|
||||
token := os.Getenv("GH_TOKEN")
|
||||
if token == "" {
|
||||
token = os.Getenv("GITHUB_TOKEN")
|
||||
}
|
||||
return &publicGitHub{token: token, prs: map[string]prsource.PullRequest{}}
|
||||
}
|
||||
|
||||
func (g *publicGitHub) get(ctx context.Context, rawURL string, api bool) (io.ReadCloser, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if api && g.token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+g.token)
|
||||
}
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
_ = resp.Body.Close()
|
||||
return nil, fmt.Errorf("GET %s: %s", rawURL, resp.Status)
|
||||
}
|
||||
return resp.Body, nil
|
||||
}
|
||||
|
||||
func (g *publicGitHub) GetPullRequest(ctx context.Context, owner, repo string, number int) (prsource.PullRequest, error) {
|
||||
key := fmt.Sprintf("%s/%s#%d", owner, repo, number)
|
||||
g.mu.Lock()
|
||||
pr, ok := g.prs[key]
|
||||
g.mu.Unlock()
|
||||
if ok {
|
||||
return pr, nil
|
||||
}
|
||||
body, err := g.get(ctx, fmt.Sprintf("https://api.github.com/repos/%s/%s/pulls/%d", owner, repo, number), true)
|
||||
if err != nil {
|
||||
return prsource.PullRequest{}, err
|
||||
}
|
||||
defer body.Close()
|
||||
var raw struct {
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
Head struct {
|
||||
SHA string `json:"sha"`
|
||||
} `json:"head"`
|
||||
}
|
||||
if err := json.NewDecoder(body).Decode(&raw); err != nil {
|
||||
return prsource.PullRequest{}, err
|
||||
}
|
||||
pr = prsource.PullRequest{Title: raw.Title, Body: raw.Body, HeadSHA: raw.Head.SHA}
|
||||
g.mu.Lock()
|
||||
g.prs[key] = pr
|
||||
g.mu.Unlock()
|
||||
return pr, nil
|
||||
}
|
||||
|
||||
func (g *publicGitHub) GetPullRequestDiff(ctx context.Context, owner, repo string, number int) (string, error) {
|
||||
body, err := g.get(ctx, fmt.Sprintf("https://github.com/%s/%s/pull/%d.diff", owner, repo, number), false)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer body.Close()
|
||||
data, err := io.ReadAll(body)
|
||||
return string(data), err
|
||||
}
|
||||
|
||||
func (g *publicGitHub) GetRepoArchive(ctx context.Context, owner, repo, ref string) (io.ReadCloser, error) {
|
||||
return g.get(ctx, fmt.Sprintf("https://codeload.github.com/%s/%s/tar.gz/%s", owner, repo, ref), false)
|
||||
}
|
||||
@@ -0,0 +1,638 @@
|
||||
[
|
||||
{
|
||||
"id": "cal_dot_com#10600",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/10600",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The exported function TwoFactor handles backup codes and is in BackupCode.tsx. Inconsistent naming.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "Error message mentions 'backup code login' but this is a disable endpoint, not login",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "Backup code validation is case-sensitive due to the use of indexOf(). This causes validation to fail if a user enters uppercase hex characters, as backup codes should be case-insensitive for a better user experience.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "Because backupCodes are decrypted and mutated in memory before being written back, two concurrent login requests using the same backupCode could both pass this check and update, so a single backup code may effectively be accepted more than once if used concurrently, weakening the intended one-time-use semantics.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cal_dot_com#10967",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/10967",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Potential null reference if mainHostDestinationCalendar is undefined if evt.destinationCalendar is null or an empty array ",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "The optional chaining on mainHostDestinationCalendar?.integration is redundant since you already check mainHostDestinationCalendar in the ternary condition.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "Logic error: when externalCalendarId is provided, you're searching for a calendar where externalId === externalCalendarId, but this will always fail since you're looking for a calendar that matches itself. Should likely find by credentialId or use different logic.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "Logic inversion in organization creation: The slug property is now conditionally set when IS_TEAM_BILLING_ENABLED is true, instead of when it's false as originally intended. This change, combined with requestedSlug still being set when IS_TEAM_BILLING_ENABLED is true, results in both properties being set when billing is enabled, and neither when disabled",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "The Calendar interface now requires createEvent(event, credentialId), but some implementations (e.g., Lark/Office365) still declare createEvent(event) only—this breaks the interface contract (also applies to other locations in the PR).",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cal_dot_com#11059",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/11059",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The parseRefreshTokenResponse function incorrectly sets refresh_token to the hardcoded string 'refresh_token' when it's missing from the OAuth refresh token response. This invalidates the token, breaking subsequent token refreshes and causing authentication failures.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "Invalid Zod schema syntax. Computed property keys like [z.string().toString()] are not valid in Zod object schemas and will cause runtime errors. ",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "parseRefreshTokenResponse returns a Zod safeParse result ({ success, data, error }), not the credential key object. Persisting that as key stores the wrapper instead of the token payload; we should store the parsed data or use schema parse.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "When APP_CREDENTIAL_SHARING_ENABLED and CALCOM_CREDENTIAL_SYNC_ENDPOINT are set, the refreshFunction helper returns the fetch Response, but several callers (for example GoogleCalendarService.refreshAccessToken expecting res.data, and HubspotCalendarService.refreshAccessToken expecting a HubspotToken) assume it returns the integration-specific token object. That mismatch will cause runtime errors in the sync-enabled path unless the return type or those call sites are adjusted.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "When the sync endpoint path is used, res is a fetch Response and has no .data; res?.data will be undefined and token.access_token will throw at runtime. This relies on a consistent return shape from refreshOAuthTokens, which isn’t guaranteed currently.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cal_dot_com#14740",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/14740",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Case sensitivity bypass in email blacklist",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "The logic for checking team admin/owner permissions is incorrect. This condition uses AND (&&) which requires both isTeamAdmin AND isTeamOwner to be true, but it should use OR (||) since a user needs to be either an admin OR an owner to have permission.",
|
||||
"severity": "Critical"
|
||||
},
|
||||
{
|
||||
"comment": "This calls the email sender with the original guests, so existing attendees included in the input will be treated as new when sending notifications, leading to incorrect emails.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "uniqueGuests filters out existing attendees and blacklisted emails but does not deduplicate duplicates within the input; createMany can insert duplicate attendee rows if the client submits repeated emails.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "Starting with an array containing an empty string may cause validation issues. Consider starting with an empty array [] and handling the empty state in the MultiEmail component instead.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cal_dot_com#14943",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/14943",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Using retryCount: reminder.retryCount + 1 reads a possibly stale value and can lose increments under concurrency; consider an atomic increment via Prisma (increment: 1) to avoid race conditions (also applies to the similar update in the catch block).",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "The deletion logic in scheduleSMSReminders.ts incorrectly deletes non-SMS workflow reminders (e.g., Email, WhatsApp) that have retryCount > 1. This occurs because the retryCount condition within the OR clause for deletion lacks a method: WorkflowMethods.SMS filter, causing it to apply to all reminder types instead of only SMS reminders, which is the intended scope of this function.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cal_dot_com#22345",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/22345",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "In getBaseConditions(), the else if (filterConditions) and final else branches are unreachable. This is because getAuthorizationConditions() always returns a non-null Prisma.Sql object, making authConditions always truthy, which means only the first two if/else if conditions are ever evaluated.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "Fetching userIdsFromOrg only when teamsFromOrg.length > 0 can exclude org-level members for orgs without child teams; consider deriving from teamIds (which includes orgId) or removing the guard so org-only orgs still include member user bookings.",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cal_dot_com#22532",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/22532",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The updateManyByCredentialId call uses an empty data object, which prevents Prisma's @updatedAt decorator from updating the updatedAt timestamp. This results in inaccurate cache status tracking, as the timestamp isn't updated when the cache is refreshed. To fix this, explicitly set the updatedAt field.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "logic: macOS-specific sed syntax with empty string after -i flag will fail on Linux systems",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cal_dot_com#7232",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/7232",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Asynchronous functions deleteScheduledEmailReminder and deleteScheduledSMSReminder are called without await inside forEach loops. This occurs during booking rescheduling/cancellation, and workflow/workflow step deletion/updates. Consequently, scheduled workflow reminders may not be reliably cancelled, potentially leaving them active.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "When immediateDelete is true, the deleteScheduledEmailReminder function cancels the SendGrid email but fails to delete the corresponding WorkflowReminder record from the database. This creates orphaned database entries and is inconsistent with the immediateDelete: false path, which marks the record as cancelled. The SendGrid DELETE API call is also omitted in this path.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cal_dot_com#8087",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/8087",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Consider adding try-catch around the await to handle import failures gracefully",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "The code uses forEach with async callbacks, which causes asynchronous operations (e.g., calendar/video event deletions, payment refunds) to run concurrently without being awaited. This 'fire-and-forget' behavior leads to unhandled promise rejections, race conditions, and incomplete cleanup, as surrounding try-catch blocks cannot properly handle errors from these unawaited promises. Replace forEach with for...of loops or Promise.all() with map() to ensure proper sequential execution and error handling.",
|
||||
"severity": "Critical"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cal_dot_com#8330",
|
||||
"pr_url": "https://github.com/calcom/cal.com/pull/8330",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Incorrect end time calculation using slotStartTime instead of slotEndTime",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "Using === for dayjs object comparison will always return false as it compares object references, not values. Use .isSame() method instead: dayjs(date.start).add(utcOffset, 'minutes').isSame(dayjs(date.end).add(utcOffset, minutes))",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#103633",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/103633",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The Check operation exhibits asymmetric cache trust logic: cached permission grants are trusted and returned immediately, but cached denials from the same permission cache are ignored, leading to a fresh database lookup. This allows stale cached grants to provide access to revoked resources, posing a security risk. ",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "The test comment says the cached permissions 'allow access', but the map stores false for dashboards:uid:dash1, so checkPermission will still treat this scope as not allowed.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#106778",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/106778",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The rendered GrafanaRuleListItem is missing the required key prop for React list items. This can cause rendering issues when the list order changes.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "RuleActionsButtons is invoked with only promRule, but SilenceGrafanaRuleDrawer inside RuleActionsButtons still depends on a Grafana Ruler rule being present, so for Grafana rules coming from list views the 'Silence notifications' menu entry (now driven by Grafana Prom abilities) will toggle showSilenceDrawer without ever rendering the drawer. This means clicking 'Silence notifications' for these rules has no visible effect, even when abilities indicate silencing is allowed.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#107534",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/107534",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The applyTemplateVariables method is called with request.filters as the third parameter, but this parameter is not used in the corresponding test setup.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#76186",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/76186",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The ContextualLoggerMiddleware methods (QueryData, CallResource, CheckHealth, CollectMetrics) panic when a nil request is received. This occurs because they directly access req.PluginContext (via the instrumentContext function) without first checking if req is nil. This is a regression, as previous middleware layers gracefully handled nil requests.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "The traceID is no longer logged for plugin requests. During a refactoring, the tracing import and the logic to extract and add traceID from the context to log parameters were removed from the LoggerMiddleware. The newly introduced ContextualLoggerMiddleware does not add this information, resulting in missing traceID in plugin request logs and impacting debugging and request tracing capabilities.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#79265",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/79265",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Race condition: Multiple concurrent requests could pass the device count check simultaneously and create devices beyond the limit. Consider using a database transaction or lock.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "Anonymous authentication now fails entirely if anonDeviceService.TagDevice returns ErrDeviceLimitReached. Previously, device tagging was asynchronous and non-blocking. This change prevents anonymous users from authenticating when the device limit is reached.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "This call won’t compile: dbSession.Exec(args...) is given a []interface{} where the first element is the query, but Exec’s signature requires a first parameter of type string (not an interface{} splat).",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "Returning ErrDeviceLimitReached when no rows were updated is misleading; the device might not exist.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "Time window calculation inconsistency: Using device.UpdatedAt.UTC().Add(-anonymousDeviceExpiration) as the lower bound but device.UpdatedAt as the current time may not match the intended logic. Consider using time.Now().UTC() consistently.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#80329",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/80329",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The code uses Error log level for what appears to be debugging information. This will pollute error logs in production. Consider using Debug or Info level instead.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#90045",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/90045",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The context is being created with d.Log instead of the log variable that was initialized with additional context values (name, kind, method). This means those values won't be propagated to the logging context.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "Bug: calling recordLegacyDuration when storage operation fails should be recordStorageDuration.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "Inconsistency: using name instead of options.Kind for metrics recording differs from other methods.",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#90939",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/90939",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The GetWebAssets function implements an incomplete double-checked locking pattern for caching web assets. The function first checks if the cache is populated using a read lock (RLock), and if the cache is empty, it acquires a write lock to populate it. However, it fails to re-check whether the cache was populated by another goroutine while waiting to acquire the write lock.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "In addition to the missing double-check, the function has a critical flaw in its error handling: it unconditionally assigns the fetch result to the cache (line 69: entryPointAssetsCache = result) regardless of whether the fetch succeeded or failed. When an error occurs during asset fetching, result is nil, and this nil value overwrites any previously valid cache entry.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#94942",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/94942",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The enableSqlExpressions function has flawed logic that always returns false, effectively disabling SQL expressions unconditionally:",
|
||||
"severity": "Critical"
|
||||
},
|
||||
{
|
||||
"comment": "Several methods such as NewInMemoryDB().RunCommands and db.QueryFramesInto return 'not implemented'.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "grafana#97529",
|
||||
"pr_url": "https://github.com/grafana/grafana/pull/97529",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "A race condition in BuildIndex allows multiple goroutines to concurrently build the same expensive index for the same key. This is caused by moving the b.cacheMu lock from protecting the entire function to only protecting the final cache assignment. ",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "Calling s.search.TotalDocs() here may race with concurrent index creation: TotalDocs iterates b.cache without synchronization, and the event watcher goroutine started just above could trigger BuildIndex writes concurrently, potentially causing a concurrent map read/write panic.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#32918",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/32918",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Recursive caching call using session instead of delegate",
|
||||
"severity": "Critical"
|
||||
},
|
||||
{
|
||||
"comment": "Cleanup reference uses incorrect alias - should be 'idp-alias-' + i instead of 'alias'.",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#33832",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/33832",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Returns wrong provider (default keystore instead of BouncyCastle)",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "Dead code exists where ASN1Encoder instances are created and written to, but their results are immediately discarded. The actual encoding is performed by new ASN1Encoder instances created in the subsequent return statement, rendering the earlier operations useless.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#36880",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/36880",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Inconsistent feature flag bug causing orphaned permissions. The AdminPermissions event listener, responsible for cleaning up permissions upon role, client, or group removal, is incorrectly guarded by the ADMIN_FINE_GRAINED_AUTHZ (V1) feature flag. This is inconsistent with other methods in the class that use ADMIN_FINE_GRAINED_AUTHZ_V2. Consequently, if ADMIN_FINE_GRAINED_AUTHZ_V2 is enabled but V1 is not, the permission cleanup logic will not execute, leading to orphaned permission data. Cleanup should occur regardless of which fine-grained authorization version is enabled.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "In hasPermission(ClientModel client, String scope), the resource lookup uses findByName(server, client.getId(), server.getId()), but AdminPermissionsSchema.getOrCreateResource creates per-client resources with the owner set to resourceServer.getClientId(), so this lookup will never find those resources and will always fall back to the 'all-clients' resource, effectively ignoring client-specific permissions.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "In getClientsWithPermission(String scope), iterating resourceStore.findByType(server, AdminPermissionsSchema.CLIENTS_RESOURCE_TYPE) and returning resource.getName() will only ever consider the type-level 'Clients' resource (per-client resources have no type) and return its name, while AvailableRoleMappingResource#getRoleIdsWithPermissions expects actual client IDs to pass to realm.getClientById, which can lead to incorrect behavior or a null client and subsequent failures.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#36882",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/36882",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Incorrect method call for exit codes. The picocli.exit() method calls System.exit() directly, which is problematic:",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#37038",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/37038",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Incorrect permission check in canManage() method",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "In getGroupIdsWithViewPermission, hasPermission is called with groupResource.getId() and the same groupResource.getId() is added to granted, but hasPermission resolves resources by name (treating the argument as a group id) and the GroupPermissionEvaluator contract says this method returns group IDs that are later used as UserModel.GROUPS and in getUsersCount group filters. This mismatch means per-group VIEW_MEMBERS/MANAGE_MEMBERS permissions may not yield the expected group IDs for filtering and counts, and evaluation may effectively only look at the type-level 'all-groups' resource; consider revisiting whether this should operate on the underlying group ids (resource names) instead so it aligns with the JPA queries and the interface contract.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#37429",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/37429",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The translation is in Italian instead of Lithuanian. This should be translated to Lithuanian to match the file's locale (messages_lt.properties).",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "The totpStep1 value uses Traditional Chinese terms in the Simplified Chinese file (zh_CN), which is likely incorrect for this locale. Please verify the locale‑appropriate translation.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "The anchor sanitization logic has a potential issue where it consumes English matcher groups without proper validation. If the translated text has more anchor tags than the English text, this could lead to incorrect validation results.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "The method name 'santizeAnchors' should be 'sanitizeAnchors' (missing 'i').",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#37634",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/37634",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Wrong parameter in null check (grantType vs. rawTokenId)",
|
||||
"severity": "Critical"
|
||||
},
|
||||
{
|
||||
"comment": "In isAccessTokenId, the substring for the grant shortcut and the equality check look inverted: the grant shortcut occupies indices 4–5 (substring(4,6)), and a match should return true (combined with UUID check), not false.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "Javadoc mentions \"usually like 3-letters shortcut\" but some implementations use 2-letter shortcuts (\"ac\", \"cc\", \"rt\", \"te\", \"pc\", \"ci\", \"ro\"). Consider updating documentation to reflect actual usage pattern.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": " Catching generic RuntimeException is too broad. The implementation throws IllegalArgumentException specifically - catch that instead for more precise testing.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#38446",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/38446",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Unsafe raw List deserialization without type safety. Calling Optional.get() directly on the Optional returned by RecoveryAuthnCodesUtils.getCredential(user) without checking isPresent() can lead to a NoSuchElementException if the Optional is empty.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "After creating the RecoveryAuthnCodesCredentialModel, consider setting its id from the stored credential (e.g., myUser.recoveryCodes.getId()); otherwise getId() will be null and downstream removal by id (e.g., removeStoredCredentialById in the authenticator flow) may not work.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#40940",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/40940",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Returning null from getSubGroupsCount() violates the GroupModel contract (Javadoc says it never returns null) and may lead to NPEs in callers that expect a non-null count.",
|
||||
"severity": "Critical"
|
||||
},
|
||||
{
|
||||
"comment": "The reader thread isn’t waited for; flipping deletedAll to true and asserting immediately can race and miss exceptions added just after the flag change, making this test flaky.",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "keycloak#41249",
|
||||
"pr_url": "https://github.com/keycloak/keycloak/pull/41249",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "ConditionalPasskeysEnabled() called without UserModel parameter",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "With isConditionalPasskeysEnabled(UserModel user) requiring user != null, authenticate(...) will not call webauthnAuth.fillContextForm(context) on the initial login page where context.getUser() is still null, so conditional passkey UI will not be set up for first-time passkey login. Consider whether this should also be enabled when no user has been selected yet so ID-less passkey authentication on the initial login form continues to work.",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sentry#67876",
|
||||
"pr_url": "https://github.com/getsentry/sentry/pull/67876",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Null reference if github_authenticated_user state is missing",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "OAuth state uses pipeline.signature (static) instead of a per-request random value",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "The code attempts to access integration.metadata[sender][login] without checking for the existence of the sender key. This causes a KeyError for integrations where the sender metadata was not set during creation",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sentry#77754",
|
||||
"pr_url": "https://github.com/getsentry/sentry/pull/77754",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Shared mutable default in dataclass timestamp",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "The method name has a typo: test_from_dict_inalid_data should be test_from_dict_invalid_data.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "Method name says 'empty_array' but tests empty dict - consider renaming to 'test_from_dict_empty_dict' for clarity.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "to_dict() returns a datetime for queued; if this dict is passed in task kwargs (e.g., via apply_async), JSON serialization may fail depending on the serializer, which can cause enqueue errors.",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sentry#80168",
|
||||
"pr_url": "https://github.com/getsentry/sentry/pull/80168",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "MetricAlertDetectorHandler inherits from StatefulDetectorHandler but only contains pass, failing to implement its required abstract methods: counter_names (property), get_dedupe_value(), get_group_key_values(), and build_occurrence_and_event_data(). This will cause a TypeError at runtime when the class is instantiated.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "Docstring says this returns a list of DetectorEvaluationResult, but the method now returns a dict keyed by DetectorGroupKey. Consider updating the docstring to match the new return type.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sentry#80528",
|
||||
"pr_url": "https://github.com/getsentry/sentry/pull/80528",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The function modifies the config variable to include display values but then returns the original monitor.config instead of the modified version.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "The code fetches MonitorCheckIn objects by ID when the required data already exists in previous_checkins. This creates an unnecessary database query.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sentry#92393",
|
||||
"pr_url": "https://github.com/getsentry/sentry/pull/92393",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "OptimizedCursorPaginator negative-offset branch slices QuerySet with a negative start index",
|
||||
"severity": "Critical"
|
||||
},
|
||||
{
|
||||
"comment": "BasePaginator negative-offset branch slices QuerySet with a negative start index",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "OptimizedCursorPaginator.get_item_key uses floor/ceil on a datetime key (order_by='-datetime'), causing TypeError.",
|
||||
"severity": "High"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sentry#93824",
|
||||
"pr_url": "https://github.com/getsentry/sentry/pull/93824",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "Inconsistent metric tagging with 'shard' and 'shards'",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "Fixed sleep in tests can be flaky; wait on condition instead",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "Because flusher processes are created via multiprocessing.get_context('spawn').Process, they are instances of multiprocessing.context.SpawnProcess, which on POSIX is not a subclass of multiprocessing.Process, so this isinstance check will always be false and hung processes won't be killed here.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "Sleep in test_consumer.py won’t actually wait because time.sleep was monkeypatched above; consider restoring sleep or using a different sync to ensure the flusher has time to process.",
|
||||
"severity": "Medium"
|
||||
},
|
||||
{
|
||||
"comment": "Breaking out of the loop when the deadline has elapsed can skip terminating remaining flusher processes, potentially leaving them running after shutdown; consider ensuring termination is attempted even if the deadline is exceeded.",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sentry#94376",
|
||||
"pr_url": "https://github.com/getsentry/sentry/pull/94376",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "sample_rate = 0.0 is falsy and skipped",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "Using Python’s built-in hash() to build cache keys is non-deterministic across processes (hash randomization), so keys won’t match across workers and invalidate_upsampling_cache may fail to delete them. Use a deterministic serialization of project_ids for the cache key.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "The upsampling eligibility check passes the outer dataset instead of the actual dataset used by scoped_dataset. In paths where the query ultimately runs against discover (e.g., dashboard split) while the original dataset is metrics, upsampling may be skipped even when all projects are allowlisted.",
|
||||
"severity": "Medium"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "sentry#95633",
|
||||
"pr_url": "https://github.com/getsentry/sentry/pull/95633",
|
||||
"goldens": [
|
||||
{
|
||||
"comment": "The queue.shutdown() method with 'immediate=False' parameter may not exist in the standard Python queue module. This could cause AttributeError at runtime. Verify the correct API or implement a custom shutdown mechanism.",
|
||||
"severity": "High"
|
||||
},
|
||||
{
|
||||
"comment": "The magic number 50 for max_wait is used repeatedly throughout the tests. Consider extracting this as a named constant to improve maintainability.",
|
||||
"severity": "Low"
|
||||
},
|
||||
{
|
||||
"comment": "The test test_thread_queue_parallel_error_handling has a docstring that doesn't match the test implementation.",
|
||||
"severity": "Low"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,199 @@
|
||||
// Package prbench runs the PR reviewer benchmark: the Martian Code-Review-Bench
|
||||
// problems, the production review pipeline, and a judge model, all on the
|
||||
// providers and model aliases the instance already configures.
|
||||
package prbench
|
||||
|
||||
import (
|
||||
"context"
|
||||
_ "embed"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/agentexec"
|
||||
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
|
||||
"github.com/samcharles93/archie-core/internal/app/servicekit"
|
||||
"github.com/samcharles93/archie-core/internal/buildinfo"
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview/bench"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/prsource"
|
||||
)
|
||||
|
||||
// acceptanceRecall is the PRD's golden micro-recall bar for a full run.
|
||||
const acceptanceRecall = 0.70
|
||||
|
||||
// martian holds the 38 runnable problems of the Martian Code-Review-Bench
|
||||
// offline set: those whose pull request still exists upstream.
|
||||
//
|
||||
//go:embed martian.json
|
||||
var martian []byte
|
||||
|
||||
// Problems returns the benchmark problems, optionally narrowed to ids.
|
||||
func Problems(ids []string) ([]bench.Problem, error) {
|
||||
var all []bench.Problem
|
||||
if err := json.Unmarshal(martian, &all); err != nil {
|
||||
return nil, fmt.Errorf("decode benchmark problems: %w", err)
|
||||
}
|
||||
if len(ids) == 0 {
|
||||
return all, nil
|
||||
}
|
||||
var out []bench.Problem
|
||||
for _, id := range ids {
|
||||
i := slices.IndexFunc(all, func(p bench.Problem) bool { return p.ID == id })
|
||||
if i < 0 {
|
||||
return nil, fmt.Errorf("no benchmark problem %q", id)
|
||||
}
|
||||
out = append(out, all[i])
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Options configures one benchmark run. Model and Judge each name a model
|
||||
// alias of the config or a provider/model ref; empty means the "default" alias.
|
||||
type Options struct {
|
||||
Config string
|
||||
Model string
|
||||
Judge string
|
||||
OutDir string
|
||||
Problems []string
|
||||
Concurrency int
|
||||
}
|
||||
|
||||
// Run reviews and judges the selected problems, writes the report into
|
||||
// Options.OutDir, and prints it to w.
|
||||
func Run(ctx context.Context, opts Options, w io.Writer) (bench.Summary, error) {
|
||||
problems, err := Problems(opts.Problems)
|
||||
if err != nil {
|
||||
return bench.Summary{}, err
|
||||
}
|
||||
doc, err := configuration.New(nil).File(opts.Config)
|
||||
if err != nil {
|
||||
return bench.Summary{}, err
|
||||
}
|
||||
cfg := doc.Config
|
||||
if err := servicekit.ResolveProviders(&cfg, slog.Default()); err != nil {
|
||||
return bench.Summary{}, err
|
||||
}
|
||||
reviewRef, err := resolveRef(cfg.Models, opts.Model)
|
||||
if err != nil {
|
||||
return bench.Summary{}, err
|
||||
}
|
||||
judgeRef, err := resolveRef(cfg.Models, opts.Judge)
|
||||
if err != nil {
|
||||
return bench.Summary{}, err
|
||||
}
|
||||
rt := modelloop.NewRuntime(agentexec.ProvidersFromConfig(cfg.Providers), cfg.ModelLimits)
|
||||
if rt == nil {
|
||||
return bench.Summary{}, fmt.Errorf("%s configures no providers", opts.Config)
|
||||
}
|
||||
// The pipeline resolves its model through the default alias.
|
||||
cfg.Models = map[string]string{config.DefaultModelAlias: reviewRef}
|
||||
cfg.Review.ApproveBeforePost = false
|
||||
cfg.Review.PrecisionGate = false
|
||||
reviewer := &liveReviewer{cfg: cfg, agent: modelloop.NewLoopRunner(rt, slog.Default()), source: prsource.New(newPublicGitHub())}
|
||||
judge := &modelJudge{runtime: rt, modelRef: judgeRef}
|
||||
|
||||
results := bench.Run(ctx, problems, reviewer, judge, opts.Concurrency)
|
||||
summary := bench.Summarize(results)
|
||||
report := render(results, summary, reviewRef, judgeRef, len(opts.Problems) == 0)
|
||||
path, err := writeReport(opts.OutDir, report)
|
||||
if err != nil {
|
||||
return summary, err
|
||||
}
|
||||
fmt.Fprint(w, report)
|
||||
fmt.Fprintf(w, "\nreport written to %s\n", path)
|
||||
if summary.Failed != 0 {
|
||||
return summary, fmt.Errorf("%d benchmark problem(s) failed; results are incomplete", summary.Failed)
|
||||
}
|
||||
return summary, nil
|
||||
}
|
||||
|
||||
// Main is the `archied prbench` subcommand.
|
||||
func Main(args []string, stdout, stderr io.Writer) int {
|
||||
var opts Options
|
||||
var problems string
|
||||
flags := flag.NewFlagSet("archied prbench", flag.ContinueOnError)
|
||||
flags.SetOutput(stderr)
|
||||
flags.StringVar(&opts.Config, "config", configuration.DefaultConfigPath(), "config file whose providers and model aliases serve the run")
|
||||
flags.StringVar(&opts.Model, "model", "", "model alias or provider/model for the reviewer (default: the default alias)")
|
||||
flags.StringVar(&opts.Judge, "judge-model", "", "model alias or provider/model for the judge (default: the default alias; use a different model from the reviewer's for an independent judge)")
|
||||
flags.StringVar(&opts.OutDir, "out", "docs/benchmarks", "directory the report is written to")
|
||||
flags.StringVar(&problems, "problems", "", "comma-separated problem ids for a partial run (default: all 38)")
|
||||
flags.IntVar(&opts.Concurrency, "concurrency", 4, "problems reviewed at once")
|
||||
if err := flags.Parse(args); err != nil {
|
||||
if errors.Is(err, flag.ErrHelp) {
|
||||
return 0
|
||||
}
|
||||
return 2
|
||||
}
|
||||
if problems != "" {
|
||||
opts.Problems = strings.Split(problems, ",")
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if _, err := Run(ctx, opts, stdout); err != nil {
|
||||
fmt.Fprintln(stderr, "archied prbench:", err)
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// resolveRef accepts a provider/model ref as is and otherwise looks the alias up.
|
||||
func resolveRef(aliases map[string]string, v string) (string, error) {
|
||||
v = strings.TrimSpace(v)
|
||||
if strings.Contains(v, "/") {
|
||||
return v, nil
|
||||
}
|
||||
return config.ResolveModel(aliases, v, config.PurposeAgent)
|
||||
}
|
||||
|
||||
func render(results []bench.Result, s bench.Summary, reviewRef, judgeRef string, full bool) string {
|
||||
var b strings.Builder
|
||||
scope, verdict := "partial run (not the acceptance run)", ""
|
||||
if full {
|
||||
scope = "full run"
|
||||
verdict = fmt.Sprintf("\nAcceptance (recall >= %.2f): %s\n", acceptanceRecall, passFail(s.Failed == 0 && s.Recall >= acceptanceRecall))
|
||||
}
|
||||
fmt.Fprintf(&b, "# PR review benchmark\n\nDate: %s\nArchie version: %s\nReview model: %s\nJudge model: %s\nScope: %s\n\n",
|
||||
time.Now().UTC().Format("2006-01-02"), buildinfo.Version, reviewRef, judgeRef, scope)
|
||||
fmt.Fprintf(&b, "Problems %d (failed %d) recall %.3f (%d/%d) golden-only precision %.3f (%d/%d)\n%s\n",
|
||||
s.Problems, s.Failed, s.Recall, s.Hits, s.Goldens, s.Precision, s.Matched, s.Comments, verdict)
|
||||
b.WriteString("| Problem | Recall | Comments | Notes |\n| --- | --- | --- | --- |\n")
|
||||
for _, r := range results {
|
||||
recall, notes := fmt.Sprintf("%d/%d", r.Hits(), len(r.Problem.Goldens)), ""
|
||||
switch {
|
||||
case r.Err != "":
|
||||
recall, notes = "-", "failed: "+strings.ReplaceAll(r.Err, "|", "/")
|
||||
case len(r.SkippedPhases) != 0:
|
||||
notes = "skipped phases: " + strings.Join(r.SkippedPhases, ", ")
|
||||
}
|
||||
fmt.Fprintf(&b, "| %s | %s | %d | %s |\n", r.Problem.ID, recall, len(r.Comments), notes)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func passFail(ok bool) string {
|
||||
if ok {
|
||||
return "PASS"
|
||||
}
|
||||
return "FAIL"
|
||||
}
|
||||
|
||||
func writeReport(dir, report string) (string, error) {
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
return "", err
|
||||
}
|
||||
path := filepath.Join(dir, "pr-review-"+time.Now().UTC().Format("20060102-150405")+".md")
|
||||
return path, os.WriteFile(path, []byte(report), 0o644)
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package statestore
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
)
|
||||
|
||||
// packageUpdateInterval is how often auto packages are checked against the
|
||||
// catalogue. Catalogue releases are infrequent, so hours is enough.
|
||||
const packageUpdateInterval = 6 * time.Hour
|
||||
|
||||
// autoUpdatePackages updates auto packages now and then every interval until
|
||||
// ctx ends. A failed tick is logged and retried on the next one.
|
||||
func (b *server) autoUpdatePackages(ctx context.Context, packages storepkg.Manager) {
|
||||
if packages == nil {
|
||||
return
|
||||
}
|
||||
ticker := time.NewTicker(packageUpdateInterval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
if err := packages.UpdateAuto(ctx); err != nil && ctx.Err() == nil {
|
||||
b.log.Error("auto-update packages", "err", err)
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -26,11 +26,7 @@ type guardedPolicies struct {
|
||||
}
|
||||
|
||||
func (g guardedPolicies) PutPolicy(ctx context.Context, p access.Policy) (int64, error) {
|
||||
validator, err := accessengine.New(nil)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := validator.Validate(p); err != nil {
|
||||
if err := accessengine.New(nil).Validate(p); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := g.checkLockout(ctx, func(current []access.Policy) []access.Policy {
|
||||
@@ -55,10 +51,7 @@ func (g guardedPolicies) checkLockout(ctx context.Context, change func([]access.
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
engine, err := accessengine.New(change(current))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
engine := accessengine.New(change(current))
|
||||
orgs, err := g.owners.ListOrgs(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
@@ -26,6 +26,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/domain/presence"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
"github.com/samcharles93/archie-core/internal/domain/usage"
|
||||
infraaccess "github.com/samcharles93/archie-core/internal/infrastructure/access"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/readiness"
|
||||
@@ -142,7 +143,8 @@ func Run(ctx context.Context, options Options) error { //nolint:cyclop // the co
|
||||
b.announceReady()
|
||||
|
||||
deps := b.stateStoreDeps(grants)
|
||||
deps.ControlPlane = control
|
||||
controlPlaneDeps(&deps, control)
|
||||
go b.autoUpdatePackages(ctx, deps.Packages)
|
||||
return serveStateStore(ctx, listener, deps, opts)
|
||||
}
|
||||
|
||||
@@ -180,10 +182,7 @@ func (b *server) seedAndValidatePolicies(ctx context.Context) error {
|
||||
if err != nil {
|
||||
return fmt.Errorf("load stored policies: %w", err)
|
||||
}
|
||||
engine, err := infraaccess.New(stored)
|
||||
if err != nil {
|
||||
return fmt.Errorf("validate stored policies: %w", err)
|
||||
}
|
||||
engine := infraaccess.New(stored)
|
||||
for _, problem := range engine.Problems() {
|
||||
b.log.Error("stored access policy is invalid and denies its level",
|
||||
"policy", problem.Policy.ID, "level", problem.Policy.Level, "err", problem.Err)
|
||||
@@ -375,6 +374,9 @@ func (b *server) executionDeps(deps *staterpc.Deps) {
|
||||
if ec, ok := b.st.(storecontract.ExecutionCanceller); ok {
|
||||
deps.Canceller = ec
|
||||
}
|
||||
if ur, ok := b.st.(usage.Recorder); ok {
|
||||
deps.Usage = ur
|
||||
}
|
||||
if sr, ok := b.st.(storecontract.StepReader); ok {
|
||||
deps.StepReader = sr
|
||||
}
|
||||
@@ -581,3 +583,13 @@ func newPackageProjections(pool *pgxpool.Pool) (map[string]storepkg.FamilyProjec
|
||||
storepkg.FamilyProfiles: profiles,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// controlPlaneDeps serves the control plane and the per-org model answers
|
||||
// built on it.
|
||||
func controlPlaneDeps(deps *staterpc.Deps, control *controlplane.Server) {
|
||||
deps.ControlPlane = control
|
||||
deps.OrgModels = func(ctx context.Context, orgID string) (map[string]string, map[string]config.Provider, error) {
|
||||
models, err := control.ModelsFor(ctx, orgID)
|
||||
return models.Aliases, models.Providers, err
|
||||
}
|
||||
}
|
||||
@@ -27,6 +27,7 @@ var gatewayCommandSpecs = []commandSpec{
|
||||
{Command: "personality", Description: "Choose a communication style", Usage: "/personality [name]"},
|
||||
{Command: "spawn", Description: "Create a tracked task", Usage: "/spawn [identity=name] [repo=owner/name] [workflow=name] <title>"},
|
||||
{Command: "settings", Description: "Inspect or change runtime settings", Usage: "/settings [list|get <kind>|set <kind> <field>=<value> ...]"},
|
||||
{Command: "approve", Description: "Approve a waiting task", Usage: "/approve [identity=name] <task-id>"},
|
||||
{Command: "cancel", Description: "Cancel a queued or waiting task", Usage: "/cancel [identity=name] <task-id>"},
|
||||
{Command: "start", Description: "Confirm that Archie is running", Usage: "/start"},
|
||||
{Command: "new", Description: "Start a fresh session, clearing conversation history", Usage: "/new [title]"},
|
||||
|
||||
@@ -5,7 +5,8 @@ import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"sort"
|
||||
"maps"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -236,11 +237,7 @@ func formatInstallFailure(err error) string {
|
||||
}
|
||||
|
||||
func formatVersionChanges(result releaseupdate.Result) string {
|
||||
ids := make([]string, 0, len(result.Installed))
|
||||
for id := range result.Installed {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
sort.Strings(ids)
|
||||
ids := slices.Sorted(maps.Keys(result.Installed))
|
||||
parts := make([]string, 0, len(ids))
|
||||
for _, id := range ids {
|
||||
installed := result.Installed[id]
|
||||
@@ -266,11 +263,7 @@ func formatPendingReport(report releaseupdate.Report, running map[string]string)
|
||||
return formatHealthyReport(report, report.Verify(running))
|
||||
}
|
||||
|
||||
ids := make([]string, 0, len(report.Previous))
|
||||
for id := range report.Previous {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
sort.Strings(ids)
|
||||
ids := slices.Sorted(maps.Keys(report.Previous))
|
||||
versions := make([]string, 0, len(ids))
|
||||
for _, id := range ids {
|
||||
versions = append(versions, fmt.Sprintf("%s %s", id, report.Previous[id]))
|
||||
|
||||
@@ -348,8 +348,8 @@ type Config struct {
|
||||
Dispatch Dispatch `toml:"dispatch" yaml:"dispatch"`
|
||||
Review Review `toml:"review" yaml:"review"`
|
||||
|
||||
// Models maps a role ("planner", "builder", "triage") to a runtime
|
||||
// model ref ("provider/model").
|
||||
// Models maps a model alias ("default", "fast", "embedding") to a
|
||||
// runtime model ref ("provider/model"). Resolve it with ResolveModel.
|
||||
Models map[string]string `toml:"models" yaml:"models"`
|
||||
|
||||
Providers map[string]Provider `toml:"providers" yaml:"providers"`
|
||||
@@ -513,7 +513,6 @@ func (c Config) Clone() Config {
|
||||
c.Providers = maps.Clone(c.Providers)
|
||||
c.Repos = cloneRepos(c.Repos)
|
||||
c.Identities = cloneIdentities(c.Identities)
|
||||
c.Chat.Models = append([]string(nil), c.Chat.Models...)
|
||||
c.Chat.Telegram.AllowedUserIDs = append([]int64(nil), c.Chat.Telegram.AllowedUserIDs...)
|
||||
c.Bindings.PreviousEncryptionKeys = append([]SecretRef(nil), c.Bindings.PreviousEncryptionKeys...)
|
||||
c.Tools.MCPServers = cloneMCPServers(c.Tools.MCPServers)
|
||||
@@ -864,11 +863,7 @@ type ChatConfig struct {
|
||||
// channel. Off by default.
|
||||
ShowToolCalls bool `toml:"show_tool_calls" yaml:"show_tool_calls"`
|
||||
// MaxSteps caps model/tool round-trips per chat turn. Zero uses the default.
|
||||
MaxSteps int `toml:"max_steps" yaml:"max_steps"`
|
||||
// Models is the optional interactive-chat model catalog. When empty,
|
||||
// chat falls back to the distinct model references assigned to workflow
|
||||
// roles in the top-level [models] table.
|
||||
Models []string `toml:"models" yaml:"models"`
|
||||
MaxSteps int `toml:"max_steps" yaml:"max_steps"`
|
||||
Telegram TelegramConfig `toml:"telegram" yaml:"telegram"`
|
||||
// RateLimit budgets inbound messages per channel and sender. Off unless set.
|
||||
// Seeds the channel-settings resource.
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"maps"
|
||||
"slices"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ModelPurpose is what a model is used for. Each purpose has a default alias
|
||||
// that an empty alias resolves to.
|
||||
type ModelPurpose string
|
||||
|
||||
const (
|
||||
PurposeAgent ModelPurpose = "agent"
|
||||
PurposeChat ModelPurpose = "chat"
|
||||
PurposeEmbedding ModelPurpose = "embedding"
|
||||
PurposeTranscription ModelPurpose = "transcription"
|
||||
)
|
||||
|
||||
// DefaultModelAlias is the alias agent steps and chat use when none is named.
|
||||
const DefaultModelAlias = "default"
|
||||
|
||||
// DefaultAlias is the alias an empty alias means for purpose.
|
||||
func (p ModelPurpose) DefaultAlias() string {
|
||||
switch p {
|
||||
case PurposeEmbedding, PurposeTranscription:
|
||||
return string(p)
|
||||
default:
|
||||
return DefaultModelAlias
|
||||
}
|
||||
}
|
||||
|
||||
// ResolveModel returns the "provider/model" ref aliases maps alias to, or
|
||||
// purpose's default alias when alias is empty. An unknown alias is an error,
|
||||
// never a substitute.
|
||||
func ResolveModel(aliases map[string]string, alias string, purpose ModelPurpose) (string, error) {
|
||||
alias = strings.TrimSpace(alias)
|
||||
if alias == "" {
|
||||
alias = purpose.DefaultAlias()
|
||||
}
|
||||
ref := strings.TrimSpace(aliases[alias])
|
||||
if ref == "" {
|
||||
return "", fmt.Errorf("no model alias %q is configured (Settings > Models)", alias)
|
||||
}
|
||||
return ref, nil
|
||||
}
|
||||
|
||||
// ChatAliases are the aliases chat may switch between: every alias but the
|
||||
// embedding and transcription ones, sorted. Never nil.
|
||||
func ChatAliases(aliases map[string]string) []string {
|
||||
out := make([]string, 0, len(aliases))
|
||||
for _, alias := range slices.Sorted(maps.Keys(aliases)) {
|
||||
if alias != string(PurposeEmbedding) && alias != string(PurposeTranscription) {
|
||||
out = append(out, alias)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// OrgModels is what one org resolves models against: the aliases it may use
|
||||
// and its own providers, which are never the instance's.
|
||||
type OrgModels struct {
|
||||
Org string
|
||||
Aliases map[string]string
|
||||
Providers map[string]Provider
|
||||
}
|
||||
|
||||
// WithOrgModels is c for a run in models.Org: its aliases, and the instance
|
||||
// providers with the org's own beside them.
|
||||
func (c Config) WithOrgModels(models OrgModels) Config {
|
||||
c.Models = models.Aliases
|
||||
if len(models.Providers) > 0 {
|
||||
providers := maps.Clone(c.Providers)
|
||||
if providers == nil {
|
||||
providers = map[string]Provider{}
|
||||
}
|
||||
maps.Copy(providers, models.Providers)
|
||||
c.Providers = providers
|
||||
}
|
||||
return c
|
||||
}
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/docker/sandbox-kit-spec/v3/spec"
|
||||
"github.com/moby/moby/api/pkg/stdcopy"
|
||||
"github.com/moby/moby/api/types/container"
|
||||
"github.com/moby/moby/api/types/mount"
|
||||
@@ -81,10 +82,8 @@ func StartKit(ctx context.Context, cli *client.Client, s KitSpec) (string, error
|
||||
if _, err := cli.ContainerStart(ctx, created.ID, client.ContainerStartOptions{}); err != nil {
|
||||
return fail(fmt.Errorf("start kit container: %w", err))
|
||||
}
|
||||
for i, h := range s.Launch.Install {
|
||||
if err := runHook(ctx, cli, created.ID, h); err != nil {
|
||||
return fail(fmt.Errorf("install hook %d: %w", i+1, err))
|
||||
}
|
||||
if err := installKit(ctx, cli, created.ID, s.Launch); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
if s.InstallDone != nil {
|
||||
s.InstallDone()
|
||||
@@ -95,10 +94,15 @@ func StartKit(ctx context.Context, cli *client.Client, s KitSpec) (string, error
|
||||
}
|
||||
}
|
||||
for _, f := range s.Launch.Context {
|
||||
if err := writeKitFile(ctx, cli, created.ID, "0", f.Path, f.Content, f.Mode, nil); err != nil {
|
||||
if err := writeKitContext(ctx, cli, created.ID, f); err != nil {
|
||||
return fail(fmt.Errorf("write agent context %s: %w", f.Path, err))
|
||||
}
|
||||
}
|
||||
for _, skill := range s.Launch.Skills {
|
||||
if err := stageKitSkill(ctx, cli, created.ID, s.Launch.Harness.User, skill); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
}
|
||||
for i, h := range s.Launch.Startup {
|
||||
if err := runHook(ctx, cli, created.ID, h); err != nil {
|
||||
return fail(fmt.Errorf("startup hook %d: %w", i+1, err))
|
||||
@@ -107,6 +111,82 @@ func StartKit(ctx context.Context, cli *client.Client, s KitSpec) (string, error
|
||||
return created.ID, nil
|
||||
}
|
||||
|
||||
func installKit(ctx context.Context, cli *client.Client, id string, launch kit.Launch) error {
|
||||
for _, f := range launch.InstallFiles {
|
||||
if err := writeKitFile(ctx, cli, id, launch.Harness.User, f.Path, f.Content, f.Mode, f.Overwrite); err != nil {
|
||||
return fmt.Errorf("write install credential file %s: %w", f.Path, err)
|
||||
}
|
||||
}
|
||||
for i, h := range launch.Install {
|
||||
if err := runHook(ctx, cli, id, h); err != nil {
|
||||
return fmt.Errorf("install hook %d: %w", i+1, err)
|
||||
}
|
||||
}
|
||||
for _, f := range launch.InstallFiles {
|
||||
code, out, err := ExecIn(ctx, cli, id, launch.Harness.User, []string{"KIT_PATH=" + f.Path}, []string{"sh", "-c", `rm -f -- "$KIT_PATH"`})
|
||||
if err != nil {
|
||||
return fmt.Errorf("remove install credential file %s: %w", f.Path, err)
|
||||
}
|
||||
if code != 0 {
|
||||
return fmt.Errorf("remove install credential file %s: exit %d: %s", f.Path, code, out)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func stageKitSkill(ctx context.Context, cli *client.Client, id, user string, skill kit.Skill) error {
|
||||
code, out, err := ExecIn(ctx, cli, id, "0", []string{"SKILL_SOURCE=" + skill.Source, "SKILL_TARGET=" + skill.Target, "SKILL_USER=" + user}, []string{"sh", "-ec", `
|
||||
[ -d "$SKILL_SOURCE" ] && [ -f "$SKILL_SOURCE/SKILL.md" ] || exit 1
|
||||
[ "$SKILL_SOURCE" != "$SKILL_TARGET" ] || exit 0
|
||||
at="$SKILL_TARGET"
|
||||
while [ "$at" != / ]; do
|
||||
[ ! -L "$at" ] || exit 1
|
||||
at=$(dirname "$at")
|
||||
done
|
||||
[ ! -e "$SKILL_TARGET" ] || exit 0
|
||||
mkdir -p "$(dirname "$SKILL_TARGET")"
|
||||
cp -R "$SKILL_SOURCE" "$SKILL_TARGET"
|
||||
chown -R "$SKILL_USER" "$SKILL_TARGET"
|
||||
`})
|
||||
if err != nil {
|
||||
return fmt.Errorf("stage bundled skill %s: %w", skill.Source, err)
|
||||
}
|
||||
if code != 0 {
|
||||
return fmt.Errorf("stage bundled skill %s: exit %d: %s", skill.Source, code, out)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Replace only Archie's managed guidance, retaining the image's instructions.
|
||||
func writeKitContext(ctx context.Context, cli *client.Client, id string, f spec.File) error {
|
||||
code, out, err := ExecIn(ctx, cli, id, "0", []string{"KIT_PATH=" + f.Path, "KIT_CONTENT=" + base64.StdEncoding.EncodeToString([]byte(f.Content))}, []string{"sh", "-ec", `
|
||||
at="$KIT_PATH"
|
||||
while [ "$at" != / ]; do
|
||||
[ ! -L "$at" ] || exit 1
|
||||
at=$(dirname "$at")
|
||||
done
|
||||
[ ! -e "$KIT_PATH" ] || [ -f "$KIT_PATH" ] || exit 1
|
||||
mkdir -p "$(dirname "$KIT_PATH")"
|
||||
tmp=$(mktemp "$KIT_PATH.XXXXXX")
|
||||
trap 'rm -f "$tmp"' 0
|
||||
if [ -f "$KIT_PATH" ]; then
|
||||
awk '/^<!-- BEGIN ARCHIE -->$/ { managed=1; next } /^<!-- END ARCHIE -->$/ { managed=0; next } !managed { print }' "$KIT_PATH" > "$tmp"
|
||||
fi
|
||||
printf '%s\n' '<!-- BEGIN ARCHIE -->' >> "$tmp"
|
||||
printf %s "$KIT_CONTENT" | base64 -d >> "$tmp"
|
||||
printf '\n%s\n' '<!-- END ARCHIE -->' >> "$tmp"
|
||||
chmod 0644 "$tmp"
|
||||
mv "$tmp" "$KIT_PATH"
|
||||
`})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if code != 0 {
|
||||
return fmt.Errorf("exited %d: %s", code, out)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RemoveKitVolumes removes an execution's Kit volumes once the execution
|
||||
// has ended; a retry of the same execution reuses them until then.
|
||||
func RemoveKitVolumes(ctx context.Context, cli *client.Client, volumes []kit.Volume) error {
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
package container
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/docker/sandbox-kit-spec/v3/spec"
|
||||
"github.com/moby/moby/client"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/agentrun"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/kit"
|
||||
)
|
||||
|
||||
func TestKitRuntimeDiscovery(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("requires Docker")
|
||||
}
|
||||
cli, err := client.New(client.FromEnv)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer cli.Close()
|
||||
shared := t.TempDir()
|
||||
if err := os.Chmod(shared, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(shared, "SKILL.md"), []byte("shared body"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
symlink bool
|
||||
}{
|
||||
{name: "preserves and replaces managed guidance"},
|
||||
{name: "refuses symlink guidance", symlink: true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
script := `test -f /tmp/install.json; mkdir -p /tmp/discovery /tmp/bundle; printf 'SKILL body' > /tmp/bundle/SKILL.md; printf 'image instructions\n' > /tmp/discovery/AGENTS.md`
|
||||
if tc.symlink {
|
||||
script += `; mv /tmp/discovery/AGENTS.md /tmp/other; ln -s /tmp/other /tmp/discovery/AGENTS.md`
|
||||
}
|
||||
launch := kit.Launch{
|
||||
Harness: agentrun.HarnessSpec{User: "0"}, Install: []kit.Hook{{User: "0", Argv: []string{"sh", "-ec", script}}}, InstallFiles: []spec.File{{Path: "/tmp/install.json", Content: "install sentinel"}},
|
||||
Files: []spec.File{{Path: "/tmp/runtime.json", Content: "runtime sentinel"}},
|
||||
Startup: []kit.Hook{{User: "0", Argv: []string{"sh", "-ec", "test ! -e /tmp/install.json; test -f /tmp/runtime.json"}}},
|
||||
Context: []spec.File{{Path: "/tmp/discovery/AGENTS.md", Content: "first guidance"}}, Skills: []kit.Skill{{Source: "/tmp/bundle", Target: "/tmp/skills/bundle"}},
|
||||
}
|
||||
id, err := StartKit(t.Context(), cli, KitSpec{Image: "alpine:3", Worker: []string{"tail", "-f", "/dev/null"}, Binds: []string{shared + ":/tmp/skills/shared:ro,z"}, Launch: launch})
|
||||
if tc.symlink {
|
||||
if err == nil {
|
||||
_, _ = cli.ContainerRemove(context.Background(), id, client.ContainerRemoveOptions{Force: true})
|
||||
t.Fatal("symlink was followed")
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
_, _ = cli.ContainerRemove(context.Background(), id, client.ContainerRemoveOptions{Force: true})
|
||||
})
|
||||
if err := writeKitContext(t.Context(), cli, id, spec.File{Path: "/tmp/discovery/AGENTS.md", Content: "replacement guidance"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
code, out, err := ExecIn(t.Context(), cli, id, "0", nil, []string{"sh", "-ec", "cat /tmp/discovery/AGENTS.md; cat /tmp/skills/bundle/SKILL.md; cat /tmp/skills/shared/SKILL.md; if echo overwrite > /tmp/skills/shared/SKILL.md 2>/dev/null; then exit 1; fi"})
|
||||
if err != nil || code != 0 {
|
||||
t.Fatalf("read discovery: %d %q %v", code, out, err)
|
||||
}
|
||||
for _, want := range []string{"image instructions", "replacement guidance", "SKILL body", "shared body"} {
|
||||
if !strings.Contains(out, want) {
|
||||
t.Errorf("missing %q in %q", want, out)
|
||||
}
|
||||
}
|
||||
if strings.Contains(out, "first guidance") || strings.Count(out, "<!-- BEGIN ARCHIE -->") != 1 {
|
||||
t.Fatalf("managed guidance was not replaced: %q", out)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
package container
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strconv"
|
||||
|
||||
"github.com/moby/moby/api/types/container"
|
||||
"github.com/moby/moby/api/types/mount"
|
||||
"github.com/moby/moby/client"
|
||||
)
|
||||
|
||||
// Reown hands dir back to uid:gid with a throwaway root container. A task
|
||||
// container that was SIGKILLed never restores ownership of the files it
|
||||
// created as root, and the non-root daemon has no privilege to do it itself.
|
||||
func (p *Pool) Reown(ctx context.Context, dir string, uid, gid int) error {
|
||||
cfg := p.current()
|
||||
if err := p.ensureImage(ctx, cfg, cfg.Image); err != nil {
|
||||
return err
|
||||
}
|
||||
resp, err := p.cli.ContainerCreate(ctx, client.ContainerCreateOptions{
|
||||
Config: &container.Config{
|
||||
Image: cfg.Image,
|
||||
Entrypoint: []string{"chown", "-R", strconv.Itoa(uid) + ":" + strconv.Itoa(gid), "/w"},
|
||||
Labels: map[string]string{"archie-daemon": "true"},
|
||||
},
|
||||
HostConfig: &container.HostConfig{
|
||||
NetworkMode: "none",
|
||||
Mounts: []mount.Mount{{Type: mount.TypeBind, Source: dir, Target: "/w"}},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("create reown container: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
_, _ = p.cli.ContainerRemove(context.WithoutCancel(ctx), resp.ID, client.ContainerRemoveOptions{Force: true})
|
||||
}()
|
||||
wait := p.cli.ContainerWait(ctx, resp.ID, client.ContainerWaitOptions{Condition: container.WaitConditionNextExit})
|
||||
if _, err := p.cli.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {
|
||||
return fmt.Errorf("start reown container: %w", err)
|
||||
}
|
||||
select {
|
||||
case res := <-wait.Result:
|
||||
if res.StatusCode != 0 {
|
||||
return fmt.Errorf("reown %s: chown exited %d", dir, res.StatusCode)
|
||||
}
|
||||
return nil
|
||||
case err := <-wait.Error:
|
||||
return fmt.Errorf("wait for reown container: %w", err)
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
}
|
||||
}
|
||||
+2769
-1426
File diff suppressed because it is too large.
Load diff
@@ -27,6 +27,10 @@ const (
|
||||
StateStoreService_ListInstalledPackages_FullMethodName = "/state.v1.StateStoreService/ListInstalledPackages"
|
||||
StateStoreService_AcceptPackageAuthority_FullMethodName = "/state.v1.StateStoreService/AcceptPackageAuthority"
|
||||
StateStoreService_RemoveInstalledPackage_FullMethodName = "/state.v1.StateStoreService/RemoveInstalledPackage"
|
||||
StateStoreService_UpdatePackage_FullMethodName = "/state.v1.StateStoreService/UpdatePackage"
|
||||
StateStoreService_ApprovePackageUpdate_FullMethodName = "/state.v1.StateStoreService/ApprovePackageUpdate"
|
||||
StateStoreService_RollbackPackage_FullMethodName = "/state.v1.StateStoreService/RollbackPackage"
|
||||
StateStoreService_SetPackageUpdatePolicy_FullMethodName = "/state.v1.StateStoreService/SetPackageUpdatePolicy"
|
||||
StateStoreService_RegisterTaskGrant_FullMethodName = "/state.v1.StateStoreService/RegisterTaskGrant"
|
||||
StateStoreService_RevokeTaskGrant_FullMethodName = "/state.v1.StateStoreService/RevokeTaskGrant"
|
||||
StateStoreService_ResolveTaskGrant_FullMethodName = "/state.v1.StateStoreService/ResolveTaskGrant"
|
||||
@@ -39,17 +43,23 @@ const (
|
||||
StateStoreService_RetireIdentity_FullMethodName = "/state.v1.StateStoreService/RetireIdentity"
|
||||
StateStoreService_ResolveIdentitySubject_FullMethodName = "/state.v1.StateStoreService/ResolveIdentitySubject"
|
||||
StateStoreService_BindIdentitySubject_FullMethodName = "/state.v1.StateStoreService/BindIdentitySubject"
|
||||
StateStoreService_ClaimIdentitySubject_FullMethodName = "/state.v1.StateStoreService/ClaimIdentitySubject"
|
||||
StateStoreService_AddPersonalToken_FullMethodName = "/state.v1.StateStoreService/AddPersonalToken"
|
||||
StateStoreService_ListPersonalTokens_FullMethodName = "/state.v1.StateStoreService/ListPersonalTokens"
|
||||
StateStoreService_RevokePersonalToken_FullMethodName = "/state.v1.StateStoreService/RevokePersonalToken"
|
||||
StateStoreService_EnqueueIssue_FullMethodName = "/state.v1.StateStoreService/EnqueueIssue"
|
||||
StateStoreService_EnqueueChatTask_FullMethodName = "/state.v1.StateStoreService/EnqueueChatTask"
|
||||
StateStoreService_ActiveTasksByOrigin_FullMethodName = "/state.v1.StateStoreService/ActiveTasksByOrigin"
|
||||
StateStoreService_UnpostedOutcomes_FullMethodName = "/state.v1.StateStoreService/UnpostedOutcomes"
|
||||
StateStoreService_MarkOutcomePosted_FullMethodName = "/state.v1.StateStoreService/MarkOutcomePosted"
|
||||
StateStoreService_ClaimNext_FullMethodName = "/state.v1.StateStoreService/ClaimNext"
|
||||
StateStoreService_ClaimByIssue_FullMethodName = "/state.v1.StateStoreService/ClaimByIssue"
|
||||
StateStoreService_Transition_FullMethodName = "/state.v1.StateStoreService/Transition"
|
||||
StateStoreService_Update_FullMethodName = "/state.v1.StateStoreService/Update"
|
||||
StateStoreService_StartStep_FullMethodName = "/state.v1.StateStoreService/StartStep"
|
||||
StateStoreService_RecordUsage_FullMethodName = "/state.v1.StateStoreService/RecordUsage"
|
||||
StateStoreService_TaskModels_FullMethodName = "/state.v1.StateStoreService/TaskModels"
|
||||
StateStoreService_CallerModels_FullMethodName = "/state.v1.StateStoreService/CallerModels"
|
||||
StateStoreService_FinishStep_FullMethodName = "/state.v1.StateStoreService/FinishStep"
|
||||
StateStoreService_CancelExecution_FullMethodName = "/state.v1.StateStoreService/CancelExecution"
|
||||
StateStoreService_ListSteps_FullMethodName = "/state.v1.StateStoreService/ListSteps"
|
||||
@@ -171,6 +181,16 @@ type StateStoreServiceClient interface {
|
||||
// Administrative until org-scoped principals land.
|
||||
AcceptPackageAuthority(ctx context.Context, in *AcceptPackageAuthorityRequest, opts ...grpc.CallOption) (*AcceptPackageAuthorityResponse, error)
|
||||
RemoveInstalledPackage(ctx context.Context, in *RemoveInstalledPackageRequest, opts ...grpc.CallOption) (*RemoveInstalledPackageResponse, error)
|
||||
// UpdatePackage moves a package to the digest the verified catalogue lists:
|
||||
// a non-widening update applies, a widening one waits as pending.
|
||||
// ApprovePackageUpdate applies the pending update and accepts its authority;
|
||||
// RollbackPackage restores the previous digest and its accepted authority.
|
||||
UpdatePackage(ctx context.Context, in *UpdatePackageRequest, opts ...grpc.CallOption) (*UpdatePackageResponse, error)
|
||||
ApprovePackageUpdate(ctx context.Context, in *ApprovePackageUpdateRequest, opts ...grpc.CallOption) (*ApprovePackageUpdateResponse, error)
|
||||
RollbackPackage(ctx context.Context, in *RollbackPackageRequest, opts ...grpc.CallOption) (*RollbackPackageResponse, error)
|
||||
// SetPackageUpdatePolicy sets "manual" or "auto"; the State Store updates
|
||||
// auto packages on a timer.
|
||||
SetPackageUpdatePolicy(ctx context.Context, in *SetPackageUpdatePolicyRequest, opts ...grpc.CallOption) (*SetPackageUpdatePolicyResponse, error)
|
||||
// Administrative task credential lifecycle. Worker credentials cannot call these.
|
||||
RegisterTaskGrant(ctx context.Context, in *RegisterTaskGrantRequest, opts ...grpc.CallOption) (*RegisterTaskGrantResponse, error)
|
||||
RevokeTaskGrant(ctx context.Context, in *RevokeTaskGrantRequest, opts ...grpc.CallOption) (*RevokeTaskGrantResponse, error)
|
||||
@@ -188,6 +208,7 @@ type StateStoreServiceClient interface {
|
||||
// identity store.
|
||||
ResolveIdentitySubject(ctx context.Context, in *ResolveIdentitySubjectRequest, opts ...grpc.CallOption) (*ResolveIdentitySubjectResponse, error)
|
||||
BindIdentitySubject(ctx context.Context, in *BindIdentitySubjectRequest, opts ...grpc.CallOption) (*BindIdentitySubjectResponse, error)
|
||||
ClaimIdentitySubject(ctx context.Context, in *ClaimIdentitySubjectRequest, opts ...grpc.CallOption) (*ClaimIdentitySubjectResponse, error)
|
||||
// Personal API tokens act on the caller's own principal: the State Store
|
||||
// refuses a call that carries none.
|
||||
AddPersonalToken(ctx context.Context, in *AddPersonalTokenRequest, opts ...grpc.CallOption) (*AddPersonalTokenResponse, error)
|
||||
@@ -197,6 +218,10 @@ type StateStoreServiceClient interface {
|
||||
EnqueueIssue(ctx context.Context, in *EnqueueIssueRequest, opts ...grpc.CallOption) (*EnqueueIssueResponse, error)
|
||||
EnqueueChatTask(ctx context.Context, in *EnqueueChatTaskRequest, opts ...grpc.CallOption) (*EnqueueChatTaskResponse, error)
|
||||
ActiveTasksByOrigin(ctx context.Context, in *ActiveTasksByOriginRequest, opts ...grpc.CallOption) (*ActiveTasksByOriginResponse, error)
|
||||
// UnpostedOutcomes lists finished chat tasks whose outcome has not yet been
|
||||
// posted to their conversation; MarkOutcomePosted records that it was.
|
||||
UnpostedOutcomes(ctx context.Context, in *UnpostedOutcomesRequest, opts ...grpc.CallOption) (*UnpostedOutcomesResponse, error)
|
||||
MarkOutcomePosted(ctx context.Context, in *MarkOutcomePostedRequest, opts ...grpc.CallOption) (*MarkOutcomePostedResponse, error)
|
||||
ClaimNext(ctx context.Context, in *ClaimNextRequest, opts ...grpc.CallOption) (*ClaimNextResponse, error)
|
||||
ClaimByIssue(ctx context.Context, in *ClaimByIssueRequest, opts ...grpc.CallOption) (*ClaimByIssueResponse, error)
|
||||
Transition(ctx context.Context, in *TransitionRequest, opts ...grpc.CallOption) (*TransitionResponse, error)
|
||||
@@ -204,6 +229,16 @@ type StateStoreServiceClient interface {
|
||||
// Step executions (docs/prds/execution-tree-state-machine.md). The run
|
||||
// credential authorises both on its own execution only.
|
||||
StartStep(ctx context.Context, in *StartStepRequest, opts ...grpc.CallOption) (*StartStepResponse, error)
|
||||
// RecordUsage appends one model call's usage record. A run credential may
|
||||
// record usage only for its own task.
|
||||
RecordUsage(ctx context.Context, in *RecordUsageRequest, opts ...grpc.CallOption) (*RecordUsageResponse, error)
|
||||
// TaskModels answers what a task's models resolve against: the aliases its
|
||||
// org may use and the org's own providers. The org is the task's, never
|
||||
// the caller's.
|
||||
TaskModels(ctx context.Context, in *TaskModelsRequest, opts ...grpc.CallOption) (*TaskModelsResponse, error)
|
||||
// CallerModels answers the model aliases the caller's org may use. The org
|
||||
// is the calling principal's; a call with none is the system org's.
|
||||
CallerModels(ctx context.Context, in *CallerModelsRequest, opts ...grpc.CallOption) (*CallerModelsResponse, error)
|
||||
FinishStep(ctx context.Context, in *FinishStepRequest, opts ...grpc.CallOption) (*FinishStepResponse, error)
|
||||
// Cancellation. Deliberately administrative (deny-by-default for a
|
||||
// task-scoped grant): a cancel is a dashboard, API and dispatch action
|
||||
@@ -468,6 +503,46 @@ func (c *stateStoreServiceClient) RemoveInstalledPackage(ctx context.Context, in
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) UpdatePackage(ctx context.Context, in *UpdatePackageRequest, opts ...grpc.CallOption) (*UpdatePackageResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(UpdatePackageResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_UpdatePackage_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) ApprovePackageUpdate(ctx context.Context, in *ApprovePackageUpdateRequest, opts ...grpc.CallOption) (*ApprovePackageUpdateResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(ApprovePackageUpdateResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_ApprovePackageUpdate_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) RollbackPackage(ctx context.Context, in *RollbackPackageRequest, opts ...grpc.CallOption) (*RollbackPackageResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(RollbackPackageResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_RollbackPackage_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) SetPackageUpdatePolicy(ctx context.Context, in *SetPackageUpdatePolicyRequest, opts ...grpc.CallOption) (*SetPackageUpdatePolicyResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(SetPackageUpdatePolicyResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_SetPackageUpdatePolicy_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) RegisterTaskGrant(ctx context.Context, in *RegisterTaskGrantRequest, opts ...grpc.CallOption) (*RegisterTaskGrantResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(RegisterTaskGrantResponse)
|
||||
@@ -588,6 +663,16 @@ func (c *stateStoreServiceClient) BindIdentitySubject(ctx context.Context, in *B
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) ClaimIdentitySubject(ctx context.Context, in *ClaimIdentitySubjectRequest, opts ...grpc.CallOption) (*ClaimIdentitySubjectResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(ClaimIdentitySubjectResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_ClaimIdentitySubject_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) AddPersonalToken(ctx context.Context, in *AddPersonalTokenRequest, opts ...grpc.CallOption) (*AddPersonalTokenResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(AddPersonalTokenResponse)
|
||||
@@ -648,6 +733,26 @@ func (c *stateStoreServiceClient) ActiveTasksByOrigin(ctx context.Context, in *A
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) UnpostedOutcomes(ctx context.Context, in *UnpostedOutcomesRequest, opts ...grpc.CallOption) (*UnpostedOutcomesResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(UnpostedOutcomesResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_UnpostedOutcomes_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) MarkOutcomePosted(ctx context.Context, in *MarkOutcomePostedRequest, opts ...grpc.CallOption) (*MarkOutcomePostedResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(MarkOutcomePostedResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_MarkOutcomePosted_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) ClaimNext(ctx context.Context, in *ClaimNextRequest, opts ...grpc.CallOption) (*ClaimNextResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(ClaimNextResponse)
|
||||
@@ -698,6 +803,36 @@ func (c *stateStoreServiceClient) StartStep(ctx context.Context, in *StartStepRe
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) RecordUsage(ctx context.Context, in *RecordUsageRequest, opts ...grpc.CallOption) (*RecordUsageResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(RecordUsageResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_RecordUsage_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) TaskModels(ctx context.Context, in *TaskModelsRequest, opts ...grpc.CallOption) (*TaskModelsResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(TaskModelsResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_TaskModels_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) CallerModels(ctx context.Context, in *CallerModelsRequest, opts ...grpc.CallOption) (*CallerModelsResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(CallerModelsResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_CallerModels_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) FinishStep(ctx context.Context, in *FinishStepRequest, opts ...grpc.CallOption) (*FinishStepResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(FinishStepResponse)
|
||||
@@ -1681,6 +1816,16 @@ type StateStoreServiceServer interface {
|
||||
// Administrative until org-scoped principals land.
|
||||
AcceptPackageAuthority(context.Context, *AcceptPackageAuthorityRequest) (*AcceptPackageAuthorityResponse, error)
|
||||
RemoveInstalledPackage(context.Context, *RemoveInstalledPackageRequest) (*RemoveInstalledPackageResponse, error)
|
||||
// UpdatePackage moves a package to the digest the verified catalogue lists:
|
||||
// a non-widening update applies, a widening one waits as pending.
|
||||
// ApprovePackageUpdate applies the pending update and accepts its authority;
|
||||
// RollbackPackage restores the previous digest and its accepted authority.
|
||||
UpdatePackage(context.Context, *UpdatePackageRequest) (*UpdatePackageResponse, error)
|
||||
ApprovePackageUpdate(context.Context, *ApprovePackageUpdateRequest) (*ApprovePackageUpdateResponse, error)
|
||||
RollbackPackage(context.Context, *RollbackPackageRequest) (*RollbackPackageResponse, error)
|
||||
// SetPackageUpdatePolicy sets "manual" or "auto"; the State Store updates
|
||||
// auto packages on a timer.
|
||||
SetPackageUpdatePolicy(context.Context, *SetPackageUpdatePolicyRequest) (*SetPackageUpdatePolicyResponse, error)
|
||||
// Administrative task credential lifecycle. Worker credentials cannot call these.
|
||||
RegisterTaskGrant(context.Context, *RegisterTaskGrantRequest) (*RegisterTaskGrantResponse, error)
|
||||
RevokeTaskGrant(context.Context, *RevokeTaskGrantRequest) (*RevokeTaskGrantResponse, error)
|
||||
@@ -1698,6 +1843,7 @@ type StateStoreServiceServer interface {
|
||||
// identity store.
|
||||
ResolveIdentitySubject(context.Context, *ResolveIdentitySubjectRequest) (*ResolveIdentitySubjectResponse, error)
|
||||
BindIdentitySubject(context.Context, *BindIdentitySubjectRequest) (*BindIdentitySubjectResponse, error)
|
||||
ClaimIdentitySubject(context.Context, *ClaimIdentitySubjectRequest) (*ClaimIdentitySubjectResponse, error)
|
||||
// Personal API tokens act on the caller's own principal: the State Store
|
||||
// refuses a call that carries none.
|
||||
AddPersonalToken(context.Context, *AddPersonalTokenRequest) (*AddPersonalTokenResponse, error)
|
||||
@@ -1707,6 +1853,10 @@ type StateStoreServiceServer interface {
|
||||
EnqueueIssue(context.Context, *EnqueueIssueRequest) (*EnqueueIssueResponse, error)
|
||||
EnqueueChatTask(context.Context, *EnqueueChatTaskRequest) (*EnqueueChatTaskResponse, error)
|
||||
ActiveTasksByOrigin(context.Context, *ActiveTasksByOriginRequest) (*ActiveTasksByOriginResponse, error)
|
||||
// UnpostedOutcomes lists finished chat tasks whose outcome has not yet been
|
||||
// posted to their conversation; MarkOutcomePosted records that it was.
|
||||
UnpostedOutcomes(context.Context, *UnpostedOutcomesRequest) (*UnpostedOutcomesResponse, error)
|
||||
MarkOutcomePosted(context.Context, *MarkOutcomePostedRequest) (*MarkOutcomePostedResponse, error)
|
||||
ClaimNext(context.Context, *ClaimNextRequest) (*ClaimNextResponse, error)
|
||||
ClaimByIssue(context.Context, *ClaimByIssueRequest) (*ClaimByIssueResponse, error)
|
||||
Transition(context.Context, *TransitionRequest) (*TransitionResponse, error)
|
||||
@@ -1714,6 +1864,16 @@ type StateStoreServiceServer interface {
|
||||
// Step executions (docs/prds/execution-tree-state-machine.md). The run
|
||||
// credential authorises both on its own execution only.
|
||||
StartStep(context.Context, *StartStepRequest) (*StartStepResponse, error)
|
||||
// RecordUsage appends one model call's usage record. A run credential may
|
||||
// record usage only for its own task.
|
||||
RecordUsage(context.Context, *RecordUsageRequest) (*RecordUsageResponse, error)
|
||||
// TaskModels answers what a task's models resolve against: the aliases its
|
||||
// org may use and the org's own providers. The org is the task's, never
|
||||
// the caller's.
|
||||
TaskModels(context.Context, *TaskModelsRequest) (*TaskModelsResponse, error)
|
||||
// CallerModels answers the model aliases the caller's org may use. The org
|
||||
// is the calling principal's; a call with none is the system org's.
|
||||
CallerModels(context.Context, *CallerModelsRequest) (*CallerModelsResponse, error)
|
||||
FinishStep(context.Context, *FinishStepRequest) (*FinishStepResponse, error)
|
||||
// Cancellation. Deliberately administrative (deny-by-default for a
|
||||
// task-scoped grant): a cancel is a dashboard, API and dispatch action
|
||||
@@ -1922,6 +2082,18 @@ func (UnimplementedStateStoreServiceServer) AcceptPackageAuthority(context.Conte
|
||||
func (UnimplementedStateStoreServiceServer) RemoveInstalledPackage(context.Context, *RemoveInstalledPackageRequest) (*RemoveInstalledPackageResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method RemoveInstalledPackage not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) UpdatePackage(context.Context, *UpdatePackageRequest) (*UpdatePackageResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method UpdatePackage not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) ApprovePackageUpdate(context.Context, *ApprovePackageUpdateRequest) (*ApprovePackageUpdateResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method ApprovePackageUpdate not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) RollbackPackage(context.Context, *RollbackPackageRequest) (*RollbackPackageResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method RollbackPackage not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) SetPackageUpdatePolicy(context.Context, *SetPackageUpdatePolicyRequest) (*SetPackageUpdatePolicyResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method SetPackageUpdatePolicy not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) RegisterTaskGrant(context.Context, *RegisterTaskGrantRequest) (*RegisterTaskGrantResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method RegisterTaskGrant not implemented")
|
||||
}
|
||||
@@ -1958,6 +2130,9 @@ func (UnimplementedStateStoreServiceServer) ResolveIdentitySubject(context.Conte
|
||||
func (UnimplementedStateStoreServiceServer) BindIdentitySubject(context.Context, *BindIdentitySubjectRequest) (*BindIdentitySubjectResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method BindIdentitySubject not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) ClaimIdentitySubject(context.Context, *ClaimIdentitySubjectRequest) (*ClaimIdentitySubjectResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method ClaimIdentitySubject not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) AddPersonalToken(context.Context, *AddPersonalTokenRequest) (*AddPersonalTokenResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method AddPersonalToken not implemented")
|
||||
}
|
||||
@@ -1976,6 +2151,12 @@ func (UnimplementedStateStoreServiceServer) EnqueueChatTask(context.Context, *En
|
||||
func (UnimplementedStateStoreServiceServer) ActiveTasksByOrigin(context.Context, *ActiveTasksByOriginRequest) (*ActiveTasksByOriginResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method ActiveTasksByOrigin not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) UnpostedOutcomes(context.Context, *UnpostedOutcomesRequest) (*UnpostedOutcomesResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method UnpostedOutcomes not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) MarkOutcomePosted(context.Context, *MarkOutcomePostedRequest) (*MarkOutcomePostedResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method MarkOutcomePosted not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) ClaimNext(context.Context, *ClaimNextRequest) (*ClaimNextResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method ClaimNext not implemented")
|
||||
}
|
||||
@@ -1991,6 +2172,15 @@ func (UnimplementedStateStoreServiceServer) Update(context.Context, *UpdateReque
|
||||
func (UnimplementedStateStoreServiceServer) StartStep(context.Context, *StartStepRequest) (*StartStepResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method StartStep not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) RecordUsage(context.Context, *RecordUsageRequest) (*RecordUsageResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method RecordUsage not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) TaskModels(context.Context, *TaskModelsRequest) (*TaskModelsResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method TaskModels not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) CallerModels(context.Context, *CallerModelsRequest) (*CallerModelsResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method CallerModels not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) FinishStep(context.Context, *FinishStepRequest) (*FinishStepResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method FinishStep not implemented")
|
||||
}
|
||||
@@ -2435,6 +2625,78 @@ func _StateStoreService_RemoveInstalledPackage_Handler(srv interface{}, ctx cont
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_UpdatePackage_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(UpdatePackageRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).UpdatePackage(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_UpdatePackage_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).UpdatePackage(ctx, req.(*UpdatePackageRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_ApprovePackageUpdate_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(ApprovePackageUpdateRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).ApprovePackageUpdate(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_ApprovePackageUpdate_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).ApprovePackageUpdate(ctx, req.(*ApprovePackageUpdateRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_RollbackPackage_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(RollbackPackageRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).RollbackPackage(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_RollbackPackage_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).RollbackPackage(ctx, req.(*RollbackPackageRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_SetPackageUpdatePolicy_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(SetPackageUpdatePolicyRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).SetPackageUpdatePolicy(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_SetPackageUpdatePolicy_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).SetPackageUpdatePolicy(ctx, req.(*SetPackageUpdatePolicyRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_RegisterTaskGrant_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(RegisterTaskGrantRequest)
|
||||
if err := dec(in); err != nil {
|
||||
@@ -2651,6 +2913,24 @@ func _StateStoreService_BindIdentitySubject_Handler(srv interface{}, ctx context
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_ClaimIdentitySubject_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(ClaimIdentitySubjectRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).ClaimIdentitySubject(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_ClaimIdentitySubject_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).ClaimIdentitySubject(ctx, req.(*ClaimIdentitySubjectRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_AddPersonalToken_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(AddPersonalTokenRequest)
|
||||
if err := dec(in); err != nil {
|
||||
@@ -2759,6 +3039,42 @@ func _StateStoreService_ActiveTasksByOrigin_Handler(srv interface{}, ctx context
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_UnpostedOutcomes_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(UnpostedOutcomesRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).UnpostedOutcomes(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_UnpostedOutcomes_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).UnpostedOutcomes(ctx, req.(*UnpostedOutcomesRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_MarkOutcomePosted_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(MarkOutcomePostedRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).MarkOutcomePosted(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_MarkOutcomePosted_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).MarkOutcomePosted(ctx, req.(*MarkOutcomePostedRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_ClaimNext_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(ClaimNextRequest)
|
||||
if err := dec(in); err != nil {
|
||||
@@ -2849,6 +3165,60 @@ func _StateStoreService_StartStep_Handler(srv interface{}, ctx context.Context,
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_RecordUsage_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(RecordUsageRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).RecordUsage(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_RecordUsage_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).RecordUsage(ctx, req.(*RecordUsageRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_TaskModels_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(TaskModelsRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).TaskModels(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_TaskModels_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).TaskModels(ctx, req.(*TaskModelsRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_CallerModels_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(CallerModelsRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).CallerModels(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_CallerModels_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).CallerModels(ctx, req.(*CallerModelsRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_FinishStep_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(FinishStepRequest)
|
||||
if err := dec(in); err != nil {
|
||||
@@ -4541,6 +4911,22 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
|
||||
MethodName: "RemoveInstalledPackage",
|
||||
Handler: _StateStoreService_RemoveInstalledPackage_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "UpdatePackage",
|
||||
Handler: _StateStoreService_UpdatePackage_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "ApprovePackageUpdate",
|
||||
Handler: _StateStoreService_ApprovePackageUpdate_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "RollbackPackage",
|
||||
Handler: _StateStoreService_RollbackPackage_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "SetPackageUpdatePolicy",
|
||||
Handler: _StateStoreService_SetPackageUpdatePolicy_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "RegisterTaskGrant",
|
||||
Handler: _StateStoreService_RegisterTaskGrant_Handler,
|
||||
@@ -4589,6 +4975,10 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
|
||||
MethodName: "BindIdentitySubject",
|
||||
Handler: _StateStoreService_BindIdentitySubject_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "ClaimIdentitySubject",
|
||||
Handler: _StateStoreService_ClaimIdentitySubject_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "AddPersonalToken",
|
||||
Handler: _StateStoreService_AddPersonalToken_Handler,
|
||||
@@ -4613,6 +5003,14 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
|
||||
MethodName: "ActiveTasksByOrigin",
|
||||
Handler: _StateStoreService_ActiveTasksByOrigin_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "UnpostedOutcomes",
|
||||
Handler: _StateStoreService_UnpostedOutcomes_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "MarkOutcomePosted",
|
||||
Handler: _StateStoreService_MarkOutcomePosted_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "ClaimNext",
|
||||
Handler: _StateStoreService_ClaimNext_Handler,
|
||||
@@ -4633,6 +5031,18 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
|
||||
MethodName: "StartStep",
|
||||
Handler: _StateStoreService_StartStep_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "RecordUsage",
|
||||
Handler: _StateStoreService_RecordUsage_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "TaskModels",
|
||||
Handler: _StateStoreService_TaskModels_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "CallerModels",
|
||||
Handler: _StateStoreService_CallerModels_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "FinishStep",
|
||||
Handler: _StateStoreService_FinishStep_Handler,
|
||||
|
||||
@@ -2,8 +2,12 @@ package daemon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||
)
|
||||
|
||||
// callWaitBuffer is the watcher's subscription buffer. The bus carries the
|
||||
@@ -16,13 +20,16 @@ const callWaitBuffer = 1024
|
||||
// daemon's own bus, republished from the container running the workflow, and
|
||||
// carry the caller's task id.
|
||||
func (d *Daemon) watchCallWaits(ctx context.Context) {
|
||||
if d.Bus == nil {
|
||||
reader, ok := d.Store.(storecontract.StepReader)
|
||||
if d.Bus == nil || !ok {
|
||||
return
|
||||
}
|
||||
sub := d.Bus.Subscribe(callWaitBuffer)
|
||||
dispatcher := d.taskDispatcher()
|
||||
go func() {
|
||||
defer sub.Close()
|
||||
ticker := time.NewTicker(time.Second)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
@@ -31,23 +38,38 @@ func (d *Daemon) watchCallWaits(ctx context.Context) {
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
applyCallWait(dispatcher, e)
|
||||
if e.Kind == events.KindWorkflowCallStarted || e.Kind == events.KindWorkflowCallFinished || e.Kind == events.KindStageStart || e.Kind == events.KindStageFinish {
|
||||
d.reconcileCallWaits(ctx, dispatcher, reader)
|
||||
}
|
||||
case <-ticker.C:
|
||||
d.reconcileCallWaits(ctx, dispatcher, reader)
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// applyCallWait suspends the run a call start names and resumes it when the
|
||||
// call finishes. It stays non-blocking: a full subscriber buffer drops events,
|
||||
// and a dropped start leaves a waiting run holding the capacity its callee
|
||||
// needs, so this must never wait on anything.
|
||||
func applyCallWait(dispatcher *taskDispatcher, e events.Event) {
|
||||
switch e.Kind {
|
||||
case events.KindWorkflowCallStarted:
|
||||
if waiting, _ := e.Data["wait"].(bool); waiting {
|
||||
dispatcher.Suspend(e.TaskID)
|
||||
// Events only prompt a read; periodic reads recover dropped starts and finishes.
|
||||
func (d *Daemon) reconcileCallWaits(ctx context.Context, dispatcher *taskDispatcher, reader storecontract.StepReader) {
|
||||
dispatcher.mu.Lock()
|
||||
holds := make([]*taskHold, 0, len(dispatcher.holds))
|
||||
for _, hold := range dispatcher.holds {
|
||||
holds = append(holds, hold)
|
||||
}
|
||||
dispatcher.mu.Unlock()
|
||||
for _, hold := range holds {
|
||||
readCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
steps, err := reader.ListSteps(readCtx, hold.id, hold.attempt)
|
||||
cancel()
|
||||
if err != nil {
|
||||
d.Log.Warn("read call waits", "task", hold.id, "err", err)
|
||||
continue
|
||||
}
|
||||
case events.KindWorkflowCallFinished:
|
||||
dispatcher.Resume(e.TaskID)
|
||||
waits := 0
|
||||
for _, step := range steps {
|
||||
if step.Kind == task.StepKindCall && step.CalledExecutionID != 0 && (step.Status == taskstate.StepPending || step.Status == taskstate.StepRunning) {
|
||||
waits++
|
||||
}
|
||||
}
|
||||
dispatcher.setCallWaits(hold, waits)
|
||||
}
|
||||
}
|
||||
+120
-103
@@ -14,7 +14,6 @@ import (
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/agentexec"
|
||||
@@ -152,6 +151,12 @@ type Daemon struct {
|
||||
RunCredentials RunCredentialIssuer
|
||||
// KitLauncher starts tasks whose agent profile is a Kit. Nil parks them.
|
||||
KitLauncher KitLauncher
|
||||
// ModelEgress gives native task containers model access without provider
|
||||
// keys. Nil parks a task whose providers need a key.
|
||||
ModelEgress ModelEgress
|
||||
// TaskModels answers the model aliases a task's org resolves against.
|
||||
// Nil runs every task on the instance aliases.
|
||||
TaskModels TaskModels
|
||||
// TaskRunReadyTimeout bounds how long runViaAgent retries a taskrun request
|
||||
// while the new container has not subscribed yet. Zero uses
|
||||
// defaultTaskRunReadyTimeout.
|
||||
@@ -185,10 +190,9 @@ type Daemon struct {
|
||||
IdentityRepository identity.Repository
|
||||
RootIdentityID identity.IdentityID
|
||||
|
||||
// KindWorkflows and LabelWorkflows are the routing bindings loaded at
|
||||
// startup, sent to the agent. Nil means defaults.
|
||||
KindWorkflows workflow.KindWorkflows
|
||||
LabelWorkflows workflow.LabelWorkflows
|
||||
// PackageAuthorities bounds the tools a package workflow's run is offered.
|
||||
PackageAuthorities PackageAuthorities
|
||||
|
||||
// Playbooks are the loaded EDA playbooks, consulted before kind/label
|
||||
// bindings when pinning a workflow. Nil means none.
|
||||
Playbooks interface {
|
||||
@@ -224,11 +228,10 @@ type Daemon struct {
|
||||
// to use.
|
||||
running runningTasks
|
||||
|
||||
// lastPollAt is when the most recent poll pass began, in Unix
|
||||
// nanoseconds (zero = no pass has started yet), read by
|
||||
// LastPollAt. Atomic because runIdentities runs one poll goroutine per
|
||||
// identity and they all stamp this one "is the poller alive" reading.
|
||||
lastPollAt atomic.Int64
|
||||
// lastPollAt maps each poll loop (an identity ID, or "" for the root
|
||||
// loop) to when its latest pass began, as Unix nanoseconds. LastPollAt
|
||||
// reports the oldest, so one wedged loop is not hidden by a healthy one.
|
||||
lastPollAt sync.Map
|
||||
}
|
||||
|
||||
// IdentityRunner bundles identity-specific state for a single agent
|
||||
@@ -434,10 +437,10 @@ func (d *Daemon) runIdentities(ctx context.Context) error {
|
||||
// drains or reconciles -- those are store-wide and run in the shared
|
||||
// maintainAndDrain loop.
|
||||
func (d *Daemon) pollForIdentity(ctx context.Context, id *IdentityRunner) {
|
||||
d.markPoll(string(id.ID))
|
||||
if !d.identityActive(ctx, id.ID) {
|
||||
return
|
||||
}
|
||||
d.markPoll()
|
||||
cfg := configForIdentity(d.Cfg.Get(), id.Cfg)
|
||||
for _, repo := range identityRepositories(cfg.Repos, id.Repos) {
|
||||
issues, complete := d.pollIssuesWithConfig(ctx, id.Forge, cfg, repo)
|
||||
@@ -1026,25 +1029,27 @@ type taskHold struct {
|
||||
baton chan struct{}
|
||||
// slot is true while the run counts against the global limit.
|
||||
slot bool
|
||||
// waits counts the callees the run is waiting on: the first Suspend gives
|
||||
// capacity back and the last Resume takes it again.
|
||||
// waits counts the callees the run is waiting on: an open call gives capacity
|
||||
// back and the last closed call takes it again.
|
||||
waits int
|
||||
// finished is true once the run's process returned.
|
||||
finished bool
|
||||
// id is the task this hold belongs to, named by the run once it knows the
|
||||
// task's id.
|
||||
id int64
|
||||
id int64
|
||||
attempt int
|
||||
}
|
||||
|
||||
// name records which task's capacity this hold is, so a wait on a callee can
|
||||
// give it back by task id. Zero names nothing: a task arriving on the NATS
|
||||
// stream has no id until the store claims its row inside the run.
|
||||
func (h *taskHold) name(id int64) {
|
||||
func (h *taskHold) name(id int64, attempt int) {
|
||||
if id == 0 {
|
||||
return
|
||||
}
|
||||
h.dispatcher.mu.Lock()
|
||||
h.id = id
|
||||
h.attempt = attempt
|
||||
h.dispatcher.holds[id] = h
|
||||
h.dispatcher.mu.Unlock()
|
||||
}
|
||||
@@ -1125,47 +1130,25 @@ func (d *taskDispatcher) finishRun(hold *taskHold) {
|
||||
d.slotsFree.Broadcast()
|
||||
}
|
||||
|
||||
// Suspend gives a run's capacity back while it waits on a callee, so the
|
||||
// callee can start. Calls are refcounted per task: a run waiting on callees
|
||||
// from parallel branches releases once. An unknown task, or one whose run has
|
||||
// already returned, is a no-op.
|
||||
func (d *taskDispatcher) Suspend(id int64) {
|
||||
// setCallWaits applies a durable snapshot without accumulating duplicate events.
|
||||
func (d *taskDispatcher) setCallWaits(hold *taskHold, waits int) {
|
||||
d.mu.Lock()
|
||||
hold := d.holds[id]
|
||||
if hold == nil || hold.finished {
|
||||
if d.holds[hold.id] != hold || hold.finished {
|
||||
d.mu.Unlock()
|
||||
return
|
||||
}
|
||||
hold.waits++
|
||||
if hold.waits == 1 {
|
||||
previous := hold.waits
|
||||
hold.waits = waits
|
||||
if waits > 0 {
|
||||
d.giveBackLocked(hold)
|
||||
}
|
||||
d.mu.Unlock()
|
||||
d.slotsFree.Broadcast()
|
||||
}
|
||||
|
||||
// Resume takes a run's capacity back when one of its waits ends. Only the last
|
||||
// wait does: a run waiting on callees from parallel branches keeps nothing
|
||||
// until all of them are done. An unknown task, one whose run has returned, and
|
||||
// one that never suspended are all no-ops.
|
||||
func (d *taskDispatcher) Resume(id int64) {
|
||||
d.mu.Lock()
|
||||
hold := d.holds[id]
|
||||
if hold == nil || hold.finished || hold.waits == 0 {
|
||||
d.mu.Unlock()
|
||||
return
|
||||
if waits > 0 {
|
||||
d.slotsFree.Broadcast()
|
||||
} else if previous > 0 {
|
||||
// Reacquiring may block behind another task; the caller keeps running.
|
||||
go d.reacquire(hold)
|
||||
}
|
||||
hold.waits--
|
||||
if hold.waits > 0 {
|
||||
d.mu.Unlock()
|
||||
return
|
||||
}
|
||||
d.mu.Unlock()
|
||||
// The two waits below can block, and neither may hold up the bus
|
||||
// subscriber calling this or the resumed run, which is already executing
|
||||
// inside its container: the execution cap can be exceeded by at most the
|
||||
// number of simultaneous resumes, and re-converges as tasks finish.
|
||||
go d.reacquire(hold)
|
||||
}
|
||||
|
||||
// reacquire returns a resumed run's capacity: its place in its repo's chain
|
||||
@@ -1176,7 +1159,7 @@ func (d *taskDispatcher) Resume(id int64) {
|
||||
func (d *taskDispatcher) reacquire(hold *taskHold) {
|
||||
d.mu.Lock()
|
||||
if hold.finished || hold.waits > 0 {
|
||||
// The run returned, or started waiting again, after the Resume that
|
||||
// The run returned, or started waiting again, after the reconciliation that
|
||||
// queued this: it needs no capacity.
|
||||
d.mu.Unlock()
|
||||
return
|
||||
@@ -1265,10 +1248,10 @@ func (d *taskDispatcher) Wait() {
|
||||
}
|
||||
|
||||
func (d *Daemon) poll(ctx context.Context) {
|
||||
d.markPoll("")
|
||||
if !d.identityActive(ctx, d.RootIdentityID) {
|
||||
return
|
||||
}
|
||||
d.markPoll()
|
||||
for _, repo := range d.Cfg.Get().Repos {
|
||||
issues, complete := d.pollIssues(ctx, repo)
|
||||
if complete {
|
||||
@@ -1593,7 +1576,7 @@ func (d *Daemon) reconcilePRs(ctx context.Context) {
|
||||
func (d *Daemon) process(ctx context.Context, task *workflow.Task, hold *taskHold) {
|
||||
// A task off the NATS stream only has an id once the store has claimed its
|
||||
// row, inside this call, so its capacity is named here.
|
||||
hold.name(task.ID)
|
||||
hold.name(task.ID, task.Attempt)
|
||||
// The workflow may name the identity the run acts as, which everything
|
||||
// below is chosen by, so it is settled before anything else.
|
||||
if !d.adoptWorkflowIdentity(ctx, task) {
|
||||
@@ -1666,14 +1649,17 @@ func (d *Daemon) process(ctx context.Context, task *workflow.Task, hold *taskHol
|
||||
// teardownStorage runs after the workflow completes. The Docker backend is
|
||||
// a no-op; future backends (temp volumes, NFS leases) use this hook.
|
||||
func (d *Daemon) teardownStorage(ctx context.Context, task *workflow.Task, repo config.Repo, workDir string) {
|
||||
if d.Storage != nil {
|
||||
_ = d.Storage.Teardown(ctx, storage.TaskRef{
|
||||
WorktreeDir: workDir,
|
||||
Ecosystem: repo.Ecosystem,
|
||||
PersistentStorage: repo.PersistentStorage,
|
||||
Owner: task.Owner,
|
||||
Repo: task.Repo,
|
||||
})
|
||||
if d.Storage == nil {
|
||||
return
|
||||
}
|
||||
if err := d.Storage.Teardown(ctx, storage.TaskRef{
|
||||
WorktreeDir: workDir,
|
||||
Ecosystem: repo.Ecosystem,
|
||||
PersistentStorage: repo.PersistentStorage,
|
||||
Owner: task.Owner,
|
||||
Repo: task.Repo,
|
||||
}); err != nil {
|
||||
d.Log.Warn("storage teardown failed", "task", task.ID, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1705,7 +1691,13 @@ func (d *Daemon) prepareWorkspace(ctx context.Context, task *workflow.Task, tree
|
||||
target = worktree.Target(task.Branch)
|
||||
}
|
||||
// Every task gets an independent full clone.
|
||||
dir, branch, err := trees.Prepare(ctx, task.Owner, task.Repo, repo.BaseBranch(), task.IssueNumber, task.Title, task.Body, task.Labels, target)
|
||||
prepare := func() (string, string, error) {
|
||||
return trees.Prepare(ctx, task.Owner, task.Repo, repo.BaseBranch(), task.IssueNumber, task.Title, task.Body, task.Labels, target)
|
||||
}
|
||||
dir, branch, err := prepare()
|
||||
if d.reownDenied(ctx, trees.Dir(task.Owner, task.Repo, task.IssueNumber), err) {
|
||||
dir, branch, err = prepare()
|
||||
}
|
||||
if err != nil {
|
||||
reason := "worktree prepare failed: " + err.Error()
|
||||
if target != worktree.Fresh {
|
||||
@@ -1748,7 +1740,7 @@ func (d *Daemon) cleanupTerminalTaskWorktree(ctx context.Context, task *workflow
|
||||
if d.worktreeHoldsUncapturedWork(cleanupCtx, trees, task) {
|
||||
return
|
||||
}
|
||||
if err := trees.Cleanup(task.Owner, task.Repo, task.IssueNumber); err != nil {
|
||||
if err := d.cleanupWorktree(cleanupCtx, trees, task.Owner, task.Repo, task.IssueNumber); err != nil {
|
||||
d.Log.Warn("terminal worktree cleanup failed", "task", task.ID, "err", err)
|
||||
}
|
||||
}
|
||||
@@ -1848,35 +1840,29 @@ func (d *Daemon) acquireTaskContainer(
|
||||
|
||||
credential, revokeCredential, err := d.runCredential(task)
|
||||
if err != nil {
|
||||
if terr := d.Storage.Teardown(ctx, storage.TaskRef{
|
||||
WorktreeDir: workDir,
|
||||
Ecosystem: repo.Ecosystem,
|
||||
PersistentStorage: repo.PersistentStorage,
|
||||
Owner: task.Owner,
|
||||
Repo: task.Repo,
|
||||
}); terr != nil {
|
||||
d.Log.Warn("storage teardown after run credential failure failed", "err", terr)
|
||||
}
|
||||
d.teardownStorage(ctx, task, repo, workDir)
|
||||
park("run credential failed", err)
|
||||
return nil, "", nil, false
|
||||
}
|
||||
|
||||
ctr, err := d.ContainerPool.Acquire(ctx, image, mounts, d.containerEnv(task, credential))
|
||||
model, closeModel, err := d.openModelEgress(ctx, task, credential)
|
||||
if err != nil {
|
||||
revokeCredential()
|
||||
d.teardownStorage(ctx, task, repo, workDir)
|
||||
park("model egress failed", err)
|
||||
return nil, "", nil, false
|
||||
}
|
||||
revokeRun := revokeCredential
|
||||
revokeCredential = func() { closeModel(); revokeRun() }
|
||||
|
||||
ctr, err := d.ContainerPool.Acquire(ctx, image, append(mounts, model.Mounts...), append(d.containerEnv(task, credential), model.Env...))
|
||||
if err != nil {
|
||||
revokeCredential()
|
||||
// Roll back the storage we just set up: the mounts were created
|
||||
// for this task but no container will use them. Leaking them until
|
||||
// a later TTL sweep is not acceptable on a backend that allocates
|
||||
// real resources (volumes, NFS leases).
|
||||
if terr := d.Storage.Teardown(ctx, storage.TaskRef{
|
||||
WorktreeDir: workDir,
|
||||
Ecosystem: repo.Ecosystem,
|
||||
PersistentStorage: repo.PersistentStorage,
|
||||
Owner: task.Owner,
|
||||
Repo: task.Repo,
|
||||
}); terr != nil {
|
||||
d.Log.Warn("storage teardown after acquire failure failed", "err", terr)
|
||||
}
|
||||
d.teardownStorage(ctx, task, repo, workDir)
|
||||
park("container acquire failed", err)
|
||||
return nil, "", nil, false
|
||||
}
|
||||
@@ -1960,7 +1946,11 @@ func (d *Daemon) recordPark(ctx context.Context, taskID int64, reason string) {
|
||||
// credential is the task's run credential; the agent presents it to push the
|
||||
// task's branch.
|
||||
func (d *Daemon) runViaAgent(ctx context.Context, task *workflow.Task, repo config.Repo, profile config.AgentProfile, harness *agentrun.HarnessSpec, credential string) {
|
||||
cfg := d.configFor(task)
|
||||
cfg, err := d.taskConfig(ctx, task)
|
||||
if err != nil {
|
||||
d.parkRunningTask(ctx, task.ID, err.Error(), taskstate.ParkTransient)
|
||||
return
|
||||
}
|
||||
taskCfg := cfg.ForTask()
|
||||
d.captureAttemptConfig(ctx, task, taskCfg)
|
||||
req := taskrun.Request{
|
||||
@@ -1970,10 +1960,8 @@ func (d *Daemon) runViaAgent(ctx context.Context, task *workflow.Task, repo conf
|
||||
Providers: agentexec.ProvidersFromConfig(cfg.Providers),
|
||||
MCPServers: cfg.Tools.MCPServers,
|
||||
RunCredential: credential,
|
||||
KindWorkflows: d.KindWorkflows,
|
||||
LabelWorkflows: d.LabelWorkflows,
|
||||
WorkflowDefinition: task.WorkflowDefinitionYAML,
|
||||
Tools: profile.Tools,
|
||||
Tools: d.toolAllowlist(ctx, task, profile.Tools),
|
||||
Harness: harness,
|
||||
}
|
||||
data, err := json.Marshal(req)
|
||||
@@ -2076,6 +2064,9 @@ func (d *Daemon) pinWorkflowDefinition(ctx context.Context, task *workflow.Task)
|
||||
if !enablement.Enabled(task.Org, task.Workflow) {
|
||||
return pinFailure{taskstate.ParkNeedsHuman, fmt.Errorf("workflow %s is disabled", task.Workflow)}
|
||||
}
|
||||
if err := d.checkPackageTrigger(ctx, task); err != nil {
|
||||
return pinFailure{taskstate.ParkNeedsHuman, err}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -2149,14 +2140,14 @@ func (d *Daemon) runActionPlaybooks(ctx context.Context, task *workflow.Task) {
|
||||
}
|
||||
|
||||
// resolveWorkflowID picks the definition to pin: a matching playbook, then
|
||||
// the kind/label bindings. A task already naming a workflow keeps it.
|
||||
// the kind defaults. A task already naming a workflow keeps it.
|
||||
func (d *Daemon) resolveWorkflowID(task *workflow.Task, available map[string]struct{}) (string, error) {
|
||||
if d.Playbooks == nil || task.Workflow != "" {
|
||||
return workflow.ResolveWorkflowID(task, available, d.KindWorkflows, d.LabelWorkflows)
|
||||
return workflow.ResolveWorkflowID(task, available)
|
||||
}
|
||||
decision, matched := d.Playbooks.Dispatch(playbookInput(task))
|
||||
if !matched {
|
||||
return workflow.ResolveWorkflowID(task, available, d.KindWorkflows, d.LabelWorkflows)
|
||||
return workflow.ResolveWorkflowID(task, available)
|
||||
}
|
||||
if _, defined := available[decision.Workflow]; !defined {
|
||||
// Reported, not silently downgraded to the binding's choice: the
|
||||
@@ -2290,13 +2281,6 @@ func (d *Daemon) containerEnv(task *workflow.Task, stateStoreToken string) []str
|
||||
// back before the daemon reads it to push.
|
||||
env = append(env, fmt.Sprintf("WORKTREE_UID=%d", os.Getuid()))
|
||||
env = append(env, fmt.Sprintf("WORKTREE_GID=%d", os.Getgid()))
|
||||
for _, p := range d.configFor(task).Providers {
|
||||
if p.APIKeyEnv != "" {
|
||||
if v := os.Getenv(p.APIKeyEnv); v != "" {
|
||||
env = append(env, p.APIKeyEnv+"="+v)
|
||||
}
|
||||
}
|
||||
}
|
||||
return env
|
||||
}
|
||||
|
||||
@@ -2468,19 +2452,26 @@ func (d *Daemon) allowConcurrentForTask(task *workflow.Task) bool {
|
||||
return ok && repo.AllowConcurrent
|
||||
}
|
||||
|
||||
// LastPollAt reports when the daemon last began a poll pass, or the zero
|
||||
// time when no pass has started yet.
|
||||
// LastPollAt reports when the least recently polled loop last began a pass,
|
||||
// or the zero time when no pass has started yet.
|
||||
func (d *Daemon) LastPollAt() time.Time {
|
||||
ns := d.lastPollAt.Load()
|
||||
if ns == 0 {
|
||||
var oldest int64
|
||||
d.lastPollAt.Range(func(_, v any) bool {
|
||||
if ns, ok := v.(int64); ok && (oldest == 0 || ns < oldest) {
|
||||
oldest = ns
|
||||
}
|
||||
return true
|
||||
})
|
||||
if oldest == 0 {
|
||||
return time.Time{}
|
||||
}
|
||||
return time.Unix(0, ns).UTC()
|
||||
return time.Unix(0, oldest).UTC()
|
||||
}
|
||||
|
||||
// markPoll stamps the start of a poll pass, so a hung pass shows as stale.
|
||||
func (d *Daemon) markPoll() {
|
||||
d.lastPollAt.Store(time.Now().UnixNano())
|
||||
// markPoll stamps the start of a poll pass for one loop, so a hung pass shows
|
||||
// as stale.
|
||||
func (d *Daemon) markPoll(loop string) {
|
||||
d.lastPollAt.Store(loop, time.Now().UnixNano())
|
||||
}
|
||||
|
||||
// errContainerExited is the cancellation cause when a task's agent container
|
||||
@@ -2523,7 +2514,33 @@ func (d *Daemon) RemoveWorktree(owner, repo, identity string, issue int) error {
|
||||
if trees == nil {
|
||||
return nil
|
||||
}
|
||||
return trees.Cleanup(owner, repo, issue)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
|
||||
defer cancel()
|
||||
return d.cleanupWorktree(ctx, trees, owner, repo, issue)
|
||||
}
|
||||
|
||||
// cleanupWorktree removes a task's clone, first handing back any root-owned
|
||||
// files a killed container left in it.
|
||||
func (d *Daemon) cleanupWorktree(ctx context.Context, trees *worktree.Manager, owner, repo string, issue int) error {
|
||||
err := trees.Cleanup(owner, repo, issue)
|
||||
if d.reownDenied(ctx, trees.Dir(owner, repo, issue), err) {
|
||||
return trees.Cleanup(owner, repo, issue)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// reownDenied repairs dir when err says the daemon was refused access to it,
|
||||
// and reports whether the caller should retry. A SIGKILLed task container
|
||||
// leaves root-owned files that only a container can chown back.
|
||||
func (d *Daemon) reownDenied(ctx context.Context, dir string, err error) bool {
|
||||
if err == nil || !errors.Is(err, fs.ErrPermission) || d.ContainerPool == nil {
|
||||
return false
|
||||
}
|
||||
if rerr := d.ContainerPool.Reown(ctx, dir, os.Getuid(), os.Getgid()); rerr != nil {
|
||||
d.Log.Warn("worktree ownership repair failed", "dir", dir, "err", rerr)
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// InFlight counts the same submitted work that graceful shutdown waits for,
|
||||
|
||||
@@ -2,7 +2,6 @@ package daemon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log/slog"
|
||||
"runtime"
|
||||
"sync"
|
||||
@@ -12,8 +11,10 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/eventbus"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||
)
|
||||
|
||||
// runDeadline bounds a wait for something that must happen. Only a broken
|
||||
@@ -139,12 +140,25 @@ func slotHolders(d *taskDispatcher) int {
|
||||
return held
|
||||
}
|
||||
|
||||
func changeCallWaits(d *taskDispatcher, delta int, id int64) {
|
||||
d.mu.Lock()
|
||||
hold := d.holds[id]
|
||||
waits := 0
|
||||
if hold != nil {
|
||||
waits = max(0, hold.waits+delta)
|
||||
}
|
||||
d.mu.Unlock()
|
||||
if hold != nil {
|
||||
d.setCallWaits(hold, waits)
|
||||
}
|
||||
}
|
||||
|
||||
// submitRun queues a task whose process reports its start on started and then
|
||||
// blocks until released. Naming the run's capacity is what the daemon's own
|
||||
// process does, where the task's id first exists.
|
||||
func submitRun(d *taskDispatcher, id int64, repo string, started chan<- int64, released <-chan struct{}) {
|
||||
d.Submit(context.Background(), &workflow.Task{ID: id, Owner: "acme", Repo: repo}, func(_ context.Context, task *workflow.Task, hold *taskHold) {
|
||||
hold.name(task.ID)
|
||||
hold.name(task.ID, task.Attempt)
|
||||
started <- task.ID
|
||||
<-released
|
||||
})
|
||||
@@ -164,11 +178,11 @@ func TestSuspendReleasesCapacityForTheTaskBehindIt(t *testing.T) {
|
||||
submitRun(d, 2, "app", started, b)
|
||||
awaitNoStart(t, started, "while the waiting run held the only slot and the repo's place")
|
||||
|
||||
d.Suspend(1)
|
||||
changeCallWaits(d, 1, 1)
|
||||
awaitStart(t, started, 2)
|
||||
close(b)
|
||||
|
||||
d.Resume(1)
|
||||
changeCallWaits(d, -1, 1)
|
||||
awaitState(t, "task 1 did not take its slot and its place back", func() bool { return holdsCapacity(d, 1) })
|
||||
|
||||
submitRun(d, 3, "app", started, c)
|
||||
@@ -193,11 +207,11 @@ func TestResumedRunKeepsItsPlaceInTheReposChain(t *testing.T) {
|
||||
submitRun(d, 2, "app", started, b)
|
||||
awaitNoStart(t, started, "while the repo's place was another run's")
|
||||
|
||||
d.Suspend(1)
|
||||
changeCallWaits(d, 1, 1)
|
||||
awaitStart(t, started, 2)
|
||||
close(b)
|
||||
|
||||
d.Resume(1)
|
||||
changeCallWaits(d, -1, 1)
|
||||
awaitState(t, "task 1 did not take its slot and its place back", func() bool { return holdsCapacity(d, 1) })
|
||||
submitRun(d, 3, "app", started, c)
|
||||
awaitNoStart(t, started, "while the resumed run held the repo's place, with slots free")
|
||||
@@ -217,8 +231,8 @@ func TestResumeTakesCapacityBackOnlyAfterTheLastWait(t *testing.T) {
|
||||
|
||||
submitRun(d, 1, "app", started, a)
|
||||
awaitStart(t, started, 1)
|
||||
d.Suspend(1)
|
||||
d.Suspend(1)
|
||||
changeCallWaits(d, 1, 1)
|
||||
changeCallWaits(d, 1, 1)
|
||||
|
||||
// The callee queued behind it runs, and returns.
|
||||
submitRun(d, 2, "app", started, b)
|
||||
@@ -228,11 +242,11 @@ func TestResumeTakesCapacityBackOnlyAfterTheLastWait(t *testing.T) {
|
||||
|
||||
// One wait outstanding: the run keeps no capacity while its other callee
|
||||
// runs.
|
||||
d.Resume(1)
|
||||
changeCallWaits(d, -1, 1)
|
||||
awaitNoState(t, "task 1 took capacity back with a callee still running", func() bool { return holdsCapacity(d, 1) })
|
||||
|
||||
// With the last wait done it takes capacity back.
|
||||
d.Resume(1)
|
||||
changeCallWaits(d, -1, 1)
|
||||
awaitState(t, "task 1 did not take its capacity back", func() bool { return holdsCapacity(d, 1) })
|
||||
|
||||
close(a)
|
||||
@@ -253,13 +267,13 @@ func TestResumeTwiceDoesNotCountASecondSlot(t *testing.T) {
|
||||
|
||||
submitRun(d, 1, "app", started, a)
|
||||
awaitStart(t, started, 1)
|
||||
d.Suspend(1)
|
||||
changeCallWaits(d, 1, 1)
|
||||
|
||||
// X takes the repo's place while A waits on its callee, so A's first
|
||||
// re-acquire queues its next place behind X.
|
||||
submitRun(d, 2, "app", started, x)
|
||||
awaitStart(t, started, 2)
|
||||
d.Resume(1)
|
||||
changeCallWaits(d, -1, 1)
|
||||
awaitState(t, "task 1 did not queue its place back", func() bool {
|
||||
_, place, _ := capacity(d, 1)
|
||||
return place
|
||||
@@ -268,8 +282,8 @@ func TestResumeTwiceDoesNotCountASecondSlot(t *testing.T) {
|
||||
// That callee ends and the run starts its next call at once: the place the
|
||||
// first re-acquire claimed is given back with it, and the second re-acquire
|
||||
// takes a slot immediately.
|
||||
d.Suspend(1)
|
||||
d.Resume(1)
|
||||
changeCallWaits(d, 1, 1)
|
||||
changeCallWaits(d, -1, 1)
|
||||
awaitState(t, "task 1 did not take its slot back", func() bool { return holdsCapacity(d, 1) })
|
||||
|
||||
// X leaves, so the first re-acquire can take a slot as well.
|
||||
@@ -308,13 +322,13 @@ func TestSecondReacquireDoesNotQueueASecondPlace(t *testing.T) {
|
||||
|
||||
submitRun(d, 1, "app", started, a)
|
||||
awaitStart(t, started, 1)
|
||||
d.Suspend(1)
|
||||
changeCallWaits(d, 1, 1)
|
||||
|
||||
// X holds the repo's place, so the run's re-acquire queues behind it and
|
||||
// stays there.
|
||||
submitRun(d, 2, "app", started, x)
|
||||
awaitStart(t, started, 2)
|
||||
d.Resume(1)
|
||||
changeCallWaits(d, -1, 1)
|
||||
awaitState(t, "task 1 did not queue its place back", func() bool {
|
||||
_, place, _ := capacity(d, 1)
|
||||
return place
|
||||
@@ -363,14 +377,14 @@ func TestResumeAfterTheRunReturnedLeavesNoCapacity(t *testing.T) {
|
||||
|
||||
submitRun(d, 1, "app", started, a)
|
||||
awaitStart(t, started, 1)
|
||||
d.Suspend(1)
|
||||
changeCallWaits(d, 1, 1)
|
||||
|
||||
submitRun(d, 2, "app", started, b)
|
||||
awaitStart(t, started, 2)
|
||||
|
||||
// The resume is queued for a slot and a place in the chain, and overtaken by
|
||||
// the run returning before either comes free.
|
||||
d.Resume(1)
|
||||
changeCallWaits(d, -1, 1)
|
||||
awaitState(t, "task 1 did not take its place in the chain back", func() bool {
|
||||
_, place, _ := capacity(d, 1)
|
||||
return place
|
||||
@@ -394,49 +408,7 @@ func TestResumeAfterTheRunReturnedLeavesNoCapacity(t *testing.T) {
|
||||
close(c)
|
||||
|
||||
// A Resume arriving after the run returned is a no-op.
|
||||
d.Resume(1)
|
||||
d.Wait()
|
||||
}
|
||||
|
||||
// callEvent builds one call step's event as the daemon receives it: the event
|
||||
// the container's workflow engine published, JSON-encoded and decoded again by
|
||||
// the agent-event bridge, so the wait flag arrives as a decoded bool.
|
||||
func callEvent(t *testing.T, kind string, taskID int64, wait bool) events.Event {
|
||||
t.Helper()
|
||||
raw, err := json.Marshal(events.Event{Kind: kind, TaskID: taskID, Data: map[string]any{"wait": wait}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var e events.Event
|
||||
if err := json.Unmarshal(raw, &e); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return e
|
||||
}
|
||||
|
||||
// Only a call the caller waits on releases its capacity, and the call
|
||||
// finishing takes it back.
|
||||
func TestCallWaitEventsReleaseCapacity(t *testing.T) {
|
||||
d := newTaskDispatcher(1, nil)
|
||||
started := make(chan int64, 4)
|
||||
a, b := make(chan struct{}), make(chan struct{})
|
||||
|
||||
submitRun(d, 1, "app", started, a)
|
||||
awaitStart(t, started, 1)
|
||||
submitRun(d, 2, "app", started, b)
|
||||
|
||||
// A call that does not wait is not waited on: nothing is released.
|
||||
applyCallWait(d, callEvent(t, events.KindWorkflowCallStarted, 1, false))
|
||||
awaitNoStart(t, started, "while the call that started does not wait")
|
||||
|
||||
applyCallWait(d, callEvent(t, events.KindWorkflowCallStarted, 1, true))
|
||||
awaitStart(t, started, 2)
|
||||
close(b)
|
||||
|
||||
applyCallWait(d, callEvent(t, events.KindWorkflowCallFinished, 1, false))
|
||||
awaitState(t, "task 1 did not take its capacity back", func() bool { return holdsCapacity(d, 1) })
|
||||
|
||||
close(a)
|
||||
changeCallWaits(d, -1, 1)
|
||||
d.Wait()
|
||||
}
|
||||
|
||||
@@ -447,6 +419,65 @@ func TestCallWaitsWithoutABus(t *testing.T) {
|
||||
d.WaitForTasks()
|
||||
}
|
||||
|
||||
type callWaitStore struct {
|
||||
storecontract.TaskStore
|
||||
mu sync.Mutex
|
||||
steps []task.StepExecution
|
||||
}
|
||||
|
||||
func (s *callWaitStore) ListSteps(_ context.Context, id int64, _ int) ([]task.StepExecution, error) {
|
||||
if id != 1 {
|
||||
return nil, nil
|
||||
}
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return append([]task.StepExecution(nil), s.steps...), nil
|
||||
}
|
||||
|
||||
func TestCallWaitsRecoverDroppedStart(t *testing.T) {
|
||||
store := &callWaitStore{steps: []task.StepExecution{
|
||||
{Kind: task.StepKindCall, Status: taskstate.StepRunning, CalledExecutionID: 2},
|
||||
{Kind: task.StepKindCall, Status: taskstate.StepRunning, CalledExecutionID: 3},
|
||||
}}
|
||||
cfg := config.Config{}
|
||||
cfg.Containers.MaxConcurrency = 1
|
||||
d := &Daemon{Cfg: config.NewHolder(cfg), Store: store, Bus: events.NewBus(), Log: slog.New(slog.DiscardHandler)}
|
||||
dispatcher := d.taskDispatcher()
|
||||
started := make(chan int64, 2)
|
||||
a, b := make(chan struct{}), make(chan struct{})
|
||||
defer func() { close(a); dispatcher.Wait() }()
|
||||
defer func() {
|
||||
if b != nil {
|
||||
close(b)
|
||||
}
|
||||
}()
|
||||
submitRun(dispatcher, 1, "app", started, a)
|
||||
awaitStart(t, started, 1)
|
||||
submitRun(dispatcher, 2, "app", started, b)
|
||||
// The live start event was dropped; only the recorded call remains.
|
||||
d.watchCallWaits(t.Context())
|
||||
awaitStart(t, started, 2)
|
||||
close(b)
|
||||
b = nil
|
||||
// Repeated reads must not accumulate waits; a partial finish stays suspended.
|
||||
store.mu.Lock()
|
||||
store.steps[0].Status = taskstate.StepSucceeded
|
||||
store.mu.Unlock()
|
||||
awaitState(t, "remaining call was not reconciled", func() bool {
|
||||
dispatcher.mu.Lock()
|
||||
defer dispatcher.mu.Unlock()
|
||||
return dispatcher.holds[1].waits == 1
|
||||
})
|
||||
if holdsCapacity(dispatcher, 1) {
|
||||
t.Fatal("parent reacquired capacity with a call still open")
|
||||
}
|
||||
// The finish event is also missing; the durable close restores capacity.
|
||||
store.mu.Lock()
|
||||
store.steps[1].Status = taskstate.StepSucceeded
|
||||
store.mu.Unlock()
|
||||
awaitState(t, "parent did not recover its capacity", func() bool { return holdsCapacity(dispatcher, 1) })
|
||||
}
|
||||
|
||||
// stubTasks is a task bus with an empty stream, so a drain pass falls through
|
||||
// to the store's ClaimNext.
|
||||
type stubTasks struct{}
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
workflowtask "github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/kit"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/kitrun"
|
||||
"github.com/samcharles93/archie-core/internal/storage"
|
||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||
)
|
||||
|
||||
@@ -112,3 +113,61 @@ func (d *Daemon) removeEndedKitVolumes(ctx context.Context, taskID int64, volume
|
||||
d.Log.Warn("kit volume removal failed", "task", taskID, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
// ModelEgress opens a native task's model proxy session. The container holds
|
||||
// a sentinel under each provider's key variable and reaches the providers
|
||||
// through the egress proxy, which swaps in the org's bound key.
|
||||
type ModelEgress interface {
|
||||
Open(token, org string, providers map[string]config.Provider) (ModelSession, error)
|
||||
Close(token string)
|
||||
}
|
||||
|
||||
// TaskModels answers what a task's org resolves models against.
|
||||
type TaskModels interface {
|
||||
TaskModels(ctx context.Context, taskID int64) (config.OrgModels, error)
|
||||
}
|
||||
|
||||
// taskConfig is task's configuration with its org's model aliases and own
|
||||
// providers.
|
||||
func (d *Daemon) taskConfig(ctx context.Context, task *workflow.Task) (config.Config, error) {
|
||||
cfg := d.configFor(task)
|
||||
if d.TaskModels == nil {
|
||||
return cfg, nil
|
||||
}
|
||||
models, err := d.TaskModels.TaskModels(ctx, task.ID)
|
||||
if err != nil {
|
||||
return config.Config{}, fmt.Errorf("resolve the task's models: %w", err)
|
||||
}
|
||||
return cfg.WithOrgModels(models), nil
|
||||
}
|
||||
|
||||
// ModelSession is what a native container needs for its model proxy session.
|
||||
type ModelSession struct {
|
||||
Env []string
|
||||
Mounts []storage.Mount
|
||||
}
|
||||
|
||||
// openModelEgress opens task's model session, or refuses: provider keys
|
||||
// never reach a container, so without the proxy no model call can run.
|
||||
func (d *Daemon) openModelEgress(ctx context.Context, task *workflow.Task, credential string) (ModelSession, func(), error) {
|
||||
cfg, err := d.taskConfig(ctx, task)
|
||||
if err != nil {
|
||||
return ModelSession{}, nil, err
|
||||
}
|
||||
if d.ModelEgress == nil {
|
||||
for _, p := range cfg.Providers {
|
||||
if p.APIKeyEnv != "" {
|
||||
return ModelSession{}, nil, fmt.Errorf("model egress proxy is unavailable on this daemon, and provider keys never enter a container")
|
||||
}
|
||||
}
|
||||
return ModelSession{}, func() {}, nil
|
||||
}
|
||||
if credential == "" {
|
||||
return ModelSession{}, nil, fmt.Errorf("model egress needs a run credential, and no State Store issues one")
|
||||
}
|
||||
session, err := d.ModelEgress.Open(credential, task.OrgOrDefault(), cfg.Providers)
|
||||
if err != nil {
|
||||
return ModelSession{}, nil, err
|
||||
}
|
||||
return session, func() { d.ModelEgress.Close(credential) }, nil
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"slices"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
)
|
||||
|
||||
// PackageAuthorities answers what the operator accepted for an installed
|
||||
// package. A package that is not installed, or not accepted, has none.
|
||||
type PackageAuthorities interface {
|
||||
AcceptedAuthority(ctx context.Context, name string) (storepkg.Authority, error)
|
||||
}
|
||||
|
||||
// toolAllowlist is the tool allowlist a task runs with. nil offers every tool;
|
||||
// a non-nil list offers exactly its members. A run of a package's workflow is
|
||||
// bounded by the tools accepted for that package, so a missing, unaccepted or
|
||||
// unreadable package offers none.
|
||||
func (d *Daemon) toolAllowlist(ctx context.Context, task *workflow.Task, profileTools []string) []string {
|
||||
name := workflow.PackageOf(task.WorkflowDefinitionYAML)
|
||||
if name == "" {
|
||||
if len(profileTools) == 0 {
|
||||
return nil
|
||||
}
|
||||
return profileTools
|
||||
}
|
||||
var accepted []string
|
||||
if d.PackageAuthorities != nil {
|
||||
authority, err := d.PackageAuthorities.AcceptedAuthority(ctx, name)
|
||||
if err != nil {
|
||||
d.Log.Warn("package authority unavailable; offering no tools", "task", task.ID, "package", name, "err", err)
|
||||
} else {
|
||||
accepted = authority.Tools
|
||||
}
|
||||
}
|
||||
allowed := []string{}
|
||||
for _, tool := range accepted {
|
||||
if len(profileTools) == 0 || slices.Contains(profileTools, tool) {
|
||||
allowed = append(allowed, tool)
|
||||
}
|
||||
}
|
||||
return allowed
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
)
|
||||
|
||||
type acceptedTools map[string]storepkg.Authority
|
||||
|
||||
func (a acceptedTools) AcceptedAuthority(_ context.Context, name string) (storepkg.Authority, error) {
|
||||
if authority, ok := a[name]; ok {
|
||||
return authority, nil
|
||||
}
|
||||
return storepkg.Authority{}, errors.New("not installed")
|
||||
}
|
||||
|
||||
// TestToolAllowlistFollowsPackageAuthority pins that a package workflow is
|
||||
// offered only the tools accepted for its package, and none when the package
|
||||
// is missing or unaccepted, while an operator workflow keeps its profile's list.
|
||||
func TestToolAllowlistFollowsPackageAuthority(t *testing.T) {
|
||||
d := &Daemon{Log: slog.Default(), PackageAuthorities: acceptedTools{
|
||||
"triage": {Tools: []string{"search", "fetch"}},
|
||||
"unsigned": {},
|
||||
}}
|
||||
tests := []struct {
|
||||
name string
|
||||
yaml string
|
||||
profile []string
|
||||
want []string
|
||||
}{
|
||||
{"operator workflow without a profile list", "id: a\n", nil, nil},
|
||||
{"operator workflow keeps its profile list", "id: a\n", []string{"x"}, []string{"x"}},
|
||||
{"package workflow gets its accepted tools", "package: triage\nid: a\n", nil, []string{"search", "fetch"}},
|
||||
{"profile narrows the accepted tools", "package: triage\nid: a\n", []string{"fetch", "shell"}, []string{"fetch"}},
|
||||
{"profile cannot widen the accepted tools", "package: triage\nid: a\n", []string{"shell"}, []string{}},
|
||||
{"package with nothing accepted", "package: unsigned\nid: a\n", nil, []string{}},
|
||||
{"package that is gone", "package: gone\nid: a\n", nil, []string{}},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := d.toolAllowlist(context.Background(), &workflow.Task{WorkflowDefinitionYAML: tt.yaml}, tt.profile)
|
||||
if !reflect.DeepEqual(got, tt.want) {
|
||||
t.Fatalf("got %#v, want %#v", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
)
|
||||
|
||||
// startTriggers names how a task was started, in the vocabulary a package's
|
||||
// accepted authority lists: "source:<source>" for where the task came from,
|
||||
// "call" for a workflow.call, "binding:<id>" for a binding dispatch and
|
||||
// "label:<name>" for each issue label.
|
||||
func startTriggers(task *workflow.Task) []string {
|
||||
source := task.Source
|
||||
if source == "" {
|
||||
source = workflow.SourceForge
|
||||
}
|
||||
triggers := []string{"source:" + source}
|
||||
if task.CallParentTaskID != 0 {
|
||||
triggers = append(triggers, "call")
|
||||
}
|
||||
if task.BindingID != "" {
|
||||
triggers = append(triggers, "binding:"+task.BindingID)
|
||||
}
|
||||
for label := range strings.SplitSeq(task.Labels, ",") {
|
||||
if label = strings.TrimSpace(label); label != "" {
|
||||
triggers = append(triggers, "label:"+label)
|
||||
}
|
||||
}
|
||||
return triggers
|
||||
}
|
||||
|
||||
// checkPackageTrigger refuses to run a package's workflow unless one of the
|
||||
// ways the task started is a trigger accepted for that package. A missing,
|
||||
// unaccepted or unreadable package accepts none. Operator workflows are not
|
||||
// bounded here.
|
||||
func (d *Daemon) checkPackageTrigger(ctx context.Context, task *workflow.Task) error {
|
||||
name := workflow.PackageOf(task.WorkflowDefinitionYAML)
|
||||
if name == "" {
|
||||
return nil
|
||||
}
|
||||
if d.PackageAuthorities == nil {
|
||||
return fmt.Errorf("package %s: no accepted authority is available", name)
|
||||
}
|
||||
authority, err := d.PackageAuthorities.AcceptedAuthority(ctx, name)
|
||||
if err != nil {
|
||||
return fmt.Errorf("package %s: accepted authority unavailable: %w", name, err)
|
||||
}
|
||||
started := startTriggers(task)
|
||||
for _, accepted := range authority.Triggers {
|
||||
if slices.Contains(started, accepted) {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("package %s: none of the triggers that started this task (%s) is accepted", name, strings.Join(started, ", "))
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
)
|
||||
|
||||
// TestCheckPackageTrigger pins that a package workflow starts only by a
|
||||
// trigger accepted for its package, failing closed when none is accepted or
|
||||
// the package is unknown, while operator workflows are unbounded.
|
||||
func TestCheckPackageTrigger(t *testing.T) {
|
||||
d := &Daemon{PackageAuthorities: acceptedTools{
|
||||
"triage": storepkg.Authority{Triggers: []string{"label:triage", "binding:b1", "call"}},
|
||||
"chatty": storepkg.Authority{Triggers: []string{"source:chat"}},
|
||||
"unsigned": storepkg.Authority{},
|
||||
}}
|
||||
tests := []struct {
|
||||
name string
|
||||
task workflow.Task
|
||||
wantErr bool
|
||||
}{
|
||||
{"operator workflow", workflow.Task{WorkflowDefinitionYAML: "id: a\n"}, false},
|
||||
{"accepted label", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", Labels: "bug, triage"}, false},
|
||||
{"accepted binding", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", BindingID: "b1"}, false},
|
||||
{"accepted call", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", CallParentTaskID: 7}, false},
|
||||
{"accepted source", workflow.Task{WorkflowDefinitionYAML: "package: chatty\nid: a\n", Source: "chat"}, false},
|
||||
{"unaccepted label", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", Labels: "bug"}, true},
|
||||
{"unaccepted binding", workflow.Task{WorkflowDefinitionYAML: "package: triage\nid: a\n", BindingID: "b2"}, true},
|
||||
{"forge source not accepted", workflow.Task{WorkflowDefinitionYAML: "package: chatty\nid: a\n"}, true},
|
||||
{"nothing accepted", workflow.Task{WorkflowDefinitionYAML: "package: unsigned\nid: a\n", Labels: "triage"}, true},
|
||||
{"package that is gone", workflow.Task{WorkflowDefinitionYAML: "package: gone\nid: a\n", Labels: "triage"}, true},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := d.checkPackageTrigger(context.Background(), &tt.task)
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Fatalf("err = %v, wantErr %v", err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -181,4 +181,8 @@ var (
|
||||
// ErrPolicyLockout is returned when a change would leave an org with no
|
||||
// owner who may manage its policies.
|
||||
ErrPolicyLockout = errors.New("access: change leaves no owner able to manage policies")
|
||||
// ErrChainUnavailable reports that no policy engine has loaded yet: the
|
||||
// request was refused without evaluating any policy, so it is an
|
||||
// availability failure, not a denial.
|
||||
ErrChainUnavailable = errors.New("access: policy chain not loaded")
|
||||
)
|
||||
@@ -131,6 +131,12 @@ type Decision struct {
|
||||
// Allowed is the shared permit decision: no level denied.
|
||||
func Allowed() Decision { return Decision{Allowed: true} }
|
||||
|
||||
// Unavailable is the refusal of a chain that has not loaded: nothing was
|
||||
// evaluated, so it is an availability failure rather than a denial.
|
||||
func Unavailable() Decision {
|
||||
return Decision{Level: LevelInstance, Err: ErrChainUnavailable}
|
||||
}
|
||||
|
||||
// DeniedAt records a denial decided at a level by the given policies.
|
||||
func DeniedAt(level Level, policies []string) Decision {
|
||||
return Decision{Level: level, Policies: policies}
|
||||
|
||||
@@ -30,10 +30,10 @@ func Processes() []string { return []string{Daemon, Gateway, Messaging} }
|
||||
// Applicability names the consumers of each resource, including boot-only settings.
|
||||
func Applicability() map[string][]string {
|
||||
kinds := map[string][]string{}
|
||||
for _, kind := range []string{"task-policy", "provider-settings", "model-role-assignments", "repository-policies", "scheduling-policy", "tool-settings", "plugin-settings", "review-settings", "channel-settings", "container-runtime-policies", "credential-bindings", "identity-grants", "agent-profiles", "workflow-execution-settings", "extension-settings", "workflow-definitions", "workflow-enablement", "schedules", "eda-playbooks"} {
|
||||
for _, kind := range []string{"task-policy", "provider-settings", "model-aliases", "repository-policies", "scheduling-policy", "tool-settings", "plugin-settings", "review-settings", "channel-settings", "container-runtime-policies", "credential-bindings", "identity-grants", "agent-profiles", "workflow-execution-settings", "extension-settings", "workflow-definitions", "workflow-enablement", "schedules", "eda-playbooks"} {
|
||||
kinds[kind] = []string{Daemon}
|
||||
}
|
||||
for _, kind := range []string{"provider-settings", "model-role-assignments", "tool-settings", "channel-settings", "plugin-settings", "extension-settings", "personas", "soul", "skills", "curators"} {
|
||||
for _, kind := range []string{"provider-settings", "model-aliases", "tool-settings", "channel-settings", "plugin-settings", "extension-settings", "personas", "soul", "skills", "curators"} {
|
||||
kinds[kind] = append(kinds[kind], Gateway)
|
||||
}
|
||||
for _, kind := range []string{"channel-settings", "extension-settings"} {
|
||||
|
||||
@@ -2,7 +2,8 @@ package binding
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"maps"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/mapping"
|
||||
@@ -113,10 +114,5 @@ func (b Binding) RepositoryFrom(values map[string]any) (owner, repo string, ok b
|
||||
}
|
||||
|
||||
func sortedKeys[V any](m map[string]V) []string {
|
||||
keys := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
return keys
|
||||
return slices.Sorted(maps.Keys(m))
|
||||
}
|
||||
@@ -7,22 +7,21 @@ import (
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
|
||||
)
|
||||
|
||||
// HealthStatus is a curator's point-in-time health.
|
||||
type HealthStatus string
|
||||
// HealthStatus is a point-in-time health status.
|
||||
type HealthStatus = lifecycle.HealthStatus
|
||||
|
||||
const (
|
||||
HealthHealthy HealthStatus = "healthy"
|
||||
HealthDegraded HealthStatus = "degraded"
|
||||
HealthUnhealthy HealthStatus = "unhealthy"
|
||||
HealthHealthy = lifecycle.HealthHealthy
|
||||
HealthDegraded = lifecycle.HealthDegraded
|
||||
HealthUnhealthy = lifecycle.HealthUnhealthy
|
||||
)
|
||||
|
||||
// Health is a point-in-time curator health report.
|
||||
type Health struct {
|
||||
Status HealthStatus
|
||||
Message string
|
||||
}
|
||||
// Health is a point-in-time health report.
|
||||
type Health = lifecycle.Health
|
||||
|
||||
// Manifest is a curator's declared shape: what input wakes it, how often it
|
||||
// runs, which tools it may reach, and which memory engine it uses. The
|
||||
|
||||
@@ -1,50 +1,26 @@
|
||||
package curator
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"slices"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
|
||||
)
|
||||
|
||||
// Registry errors.
|
||||
var (
|
||||
// ErrDuplicate is returned when a curator name is already registered.
|
||||
ErrDuplicate = errors.New("duplicate curator")
|
||||
// ErrStarted is returned when registration or lifecycle is attempted
|
||||
// after the registry has started.
|
||||
ErrStarted = errors.New("curator registry already started")
|
||||
ErrDuplicate = lifecycle.ErrDuplicate
|
||||
ErrStarted = lifecycle.ErrStarted
|
||||
)
|
||||
|
||||
type registryState uint8
|
||||
|
||||
const (
|
||||
stateIdle registryState = iota
|
||||
stateRunning
|
||||
stateStopped
|
||||
)
|
||||
|
||||
type curatorStatus struct {
|
||||
started bool
|
||||
startErr error
|
||||
}
|
||||
|
||||
// Registry owns curator registration and lifecycle. A curator's failure
|
||||
// affects only that curator.
|
||||
type Registry struct {
|
||||
mu sync.Mutex
|
||||
host Registrar
|
||||
curators map[string]CuratorEngine
|
||||
status map[string]curatorStatus
|
||||
order []string
|
||||
state registryState
|
||||
*lifecycle.Registry[CuratorEngine, Registrar]
|
||||
host Registrar
|
||||
|
||||
// activity tracks recent per-curator run history for observability.
|
||||
// It has its own mutex and is safe to
|
||||
// use independently of r.mu.
|
||||
// It has its own mutex.
|
||||
activity *activityTracker
|
||||
}
|
||||
|
||||
@@ -56,9 +32,8 @@ func NewRegistry(host Registrar) *Registry {
|
||||
host.Clock = systemClock{}
|
||||
}
|
||||
return &Registry{
|
||||
Registry: lifecycle.New[CuratorEngine, Registrar]("curator", host.Log),
|
||||
host: host,
|
||||
curators: make(map[string]CuratorEngine),
|
||||
status: make(map[string]curatorStatus),
|
||||
activity: newActivityTracker(),
|
||||
}
|
||||
}
|
||||
@@ -77,7 +52,7 @@ func (r *Registry) Activity(name string) (Activity, bool) {
|
||||
// Register validates the curator's manifest and binds it to a filtered host
|
||||
// view. Failures affect only that curator.
|
||||
func (r *Registry) Register(c CuratorEngine) error {
|
||||
if isNilEngine(c) {
|
||||
if lifecycle.IsNil(c) {
|
||||
return errors.New("curator: nil engine")
|
||||
}
|
||||
m := c.Manifest()
|
||||
@@ -87,21 +62,13 @@ func (r *Registry) Register(c CuratorEngine) error {
|
||||
if err := r.validateDeclared(m); err != nil {
|
||||
return err
|
||||
}
|
||||
return r.Add(c, r.filter(m))
|
||||
}
|
||||
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.state != stateIdle {
|
||||
return ErrStarted
|
||||
}
|
||||
if _, dup := r.curators[c.Name()]; dup {
|
||||
return fmt.Errorf("%w: %s", ErrDuplicate, c.Name())
|
||||
}
|
||||
if err := bindSafely(c, r.filter(m)); err != nil {
|
||||
return err
|
||||
}
|
||||
r.curators[c.Name()] = c
|
||||
r.order = append(r.order, c.Name())
|
||||
return nil
|
||||
// Host returns the raw registrar bundle the registry was built with. The
|
||||
// daemon itself is never part of it.
|
||||
func (r *Registry) Host() Registrar {
|
||||
return r.host
|
||||
}
|
||||
|
||||
// validateDeclared enforces "only the host services a curator's declared
|
||||
@@ -152,180 +119,9 @@ func agentic(m Manifest) bool {
|
||||
return len(m.Tools) > 0 || m.Skills || m.MemoryEngine != "" || m.Conversations
|
||||
}
|
||||
|
||||
// bindSafely isolates a panicking Bind: a curator that panics while
|
||||
// receiving its host access is refused without taking the registry down.
|
||||
func bindSafely(c CuratorEngine, host Registrar) (err error) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
err = fmt.Errorf("curator %s: bind panic: %v", c.Name(), p)
|
||||
}
|
||||
}()
|
||||
c.Bind(host)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Host returns the raw registrar bundle the registry was built with — the
|
||||
// shared services (events, clock, model) the runtime and future surfaces
|
||||
// use. It is narrow typed; the daemon itself is never part of it.
|
||||
func (r *Registry) Host() Registrar {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
return r.host
|
||||
}
|
||||
|
||||
// Get returns the registered curator by name.
|
||||
func (r *Registry) Get(name string) (CuratorEngine, bool) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
c, ok := r.curators[name]
|
||||
return c, ok
|
||||
}
|
||||
|
||||
// Names returns the registered curator names in sorted order.
|
||||
func (r *Registry) Names() []string {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
names := make([]string, 0, len(r.curators))
|
||||
for name := range r.curators {
|
||||
names = append(names, name)
|
||||
}
|
||||
slices.Sort(names)
|
||||
return names
|
||||
}
|
||||
|
||||
// Start starts every registered curator in registration order. A curator
|
||||
// whose Start fails (error or panic) is marked unhealthy and skipped; the
|
||||
// rest start regardless, and all failures are joined into the returned
|
||||
// error. This is the family's failure-isolation boundary for lifecycle.
|
||||
func (r *Registry) Start(ctx context.Context) error {
|
||||
r.mu.Lock()
|
||||
if r.state == stateRunning {
|
||||
r.mu.Unlock()
|
||||
return ErrStarted
|
||||
}
|
||||
r.state = stateRunning
|
||||
order := slices.Clone(r.order)
|
||||
r.mu.Unlock()
|
||||
|
||||
var errs []error
|
||||
for _, name := range order {
|
||||
c := r.curators[name]
|
||||
err := startSafely(ctx, c)
|
||||
r.mu.Lock()
|
||||
if err != nil {
|
||||
r.status[name] = curatorStatus{startErr: err}
|
||||
if r.host.Log != nil {
|
||||
r.host.Log.Error("curator lifecycle failed", "curator", name, "phase", "start", "err", err)
|
||||
}
|
||||
errs = append(errs, fmt.Errorf("curator %s: %w", name, err))
|
||||
} else {
|
||||
r.status[name] = curatorStatus{started: true}
|
||||
}
|
||||
r.mu.Unlock()
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
func startSafely(ctx context.Context, c CuratorEngine) (err error) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
err = fmt.Errorf("start panic: %v", p)
|
||||
}
|
||||
}()
|
||||
return c.Start(ctx)
|
||||
}
|
||||
|
||||
// Health reports every registered curator's health. A curator whose Start
|
||||
// failed is reported unhealthy with the failure; otherwise the curator's own
|
||||
// Health is used. A panicking Health is reported unhealthy, not propagated.
|
||||
func (r *Registry) Health(ctx context.Context) map[string]Health {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
out := make(map[string]Health, len(r.curators))
|
||||
for name, c := range r.curators {
|
||||
if st, ok := r.status[name]; ok && st.startErr != nil {
|
||||
out[name] = Health{Status: HealthUnhealthy, Message: st.startErr.Error()}
|
||||
continue
|
||||
}
|
||||
out[name] = healthSafely(ctx, c)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func healthSafely(ctx context.Context, c CuratorEngine) (h Health) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
h = Health{Status: HealthUnhealthy, Message: fmt.Sprintf("health panic: %v", p)}
|
||||
}
|
||||
}()
|
||||
return c.Health(ctx)
|
||||
}
|
||||
|
||||
// Stop stops started curators in reverse order, bounded by ctx, and joins
|
||||
// their errors.
|
||||
func (r *Registry) Stop(ctx context.Context) error {
|
||||
r.mu.Lock()
|
||||
if r.state == stateIdle || r.state == stateStopped {
|
||||
r.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
r.state = stateStopped
|
||||
order := slices.Clone(r.order)
|
||||
r.mu.Unlock()
|
||||
|
||||
var errs []error
|
||||
for _, name := range slices.Backward(order) {
|
||||
|
||||
r.mu.Lock()
|
||||
st, ok := r.status[name]
|
||||
r.mu.Unlock()
|
||||
if !ok || !st.started {
|
||||
continue
|
||||
}
|
||||
c := r.curators[name]
|
||||
if err := stopSafely(ctx, c); err != nil {
|
||||
if r.host.Log != nil {
|
||||
r.host.Log.Error("curator lifecycle failed", "curator", name, "phase", "stop", "err", err)
|
||||
}
|
||||
errs = append(errs, fmt.Errorf("curator %s: %w", name, err))
|
||||
}
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
func stopSafely(ctx context.Context, c CuratorEngine) (err error) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
err = fmt.Errorf("stop panic: %v", p)
|
||||
}
|
||||
}()
|
||||
return c.Stop(ctx)
|
||||
}
|
||||
|
||||
func isNilEngine(c CuratorEngine) bool {
|
||||
v := reflect.ValueOf(c)
|
||||
switch v.Kind() {
|
||||
case reflect.Invalid:
|
||||
return true
|
||||
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
|
||||
return v.IsNil()
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (r *Registry) replaceDefinitions(candidate *Registry) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.order = slices.DeleteFunc(r.order, func(name string) bool {
|
||||
if _, custom := r.curators[name].(*DefinitionEngine); !custom {
|
||||
return false
|
||||
}
|
||||
delete(r.curators, name)
|
||||
delete(r.status, name)
|
||||
return true
|
||||
})
|
||||
for _, name := range candidate.order {
|
||||
r.curators[name] = candidate.curators[name]
|
||||
r.order = append(r.order, name)
|
||||
}
|
||||
r.Replace(func(c CuratorEngine) bool {
|
||||
_, custom := c.(*DefinitionEngine)
|
||||
return custom
|
||||
}, candidate.Registry)
|
||||
}
|
||||
@@ -3,7 +3,9 @@
|
||||
package expr
|
||||
|
||||
import (
|
||||
"maps"
|
||||
"reflect"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
@@ -252,11 +254,7 @@ func actionReferences(ast *cel.Ast) ([]string, bool) {
|
||||
}
|
||||
})
|
||||
celast.PreOrderVisit(checked.Expr(), visitor)
|
||||
ids := make([]string, 0, len(seen))
|
||||
for id := range seen {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
sort.Strings(ids)
|
||||
ids := slices.Sorted(maps.Keys(seen))
|
||||
return ids, identCount == staticCount
|
||||
}
|
||||
|
||||
|
||||
@@ -8,8 +8,9 @@ import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"maps"
|
||||
"reflect"
|
||||
"sort"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/eda/module/log"
|
||||
@@ -164,11 +165,7 @@ func decodeResultStruct(kind string, t reflect.Type, raw map[string]any) (any, e
|
||||
byName[name] = out.Field(i)
|
||||
}
|
||||
|
||||
keys := make([]string, 0, len(raw))
|
||||
for key := range raw {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
keys := slices.Sorted(maps.Keys(raw))
|
||||
for _, key := range keys {
|
||||
if _, ok := byName[key]; !ok {
|
||||
return nil, fmt.Errorf("module %s: unknown result field %q", kind, key)
|
||||
|
||||
@@ -6,10 +6,11 @@ import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"fmt"
|
||||
"maps"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"sort"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
@@ -235,7 +236,7 @@ func Load(dir string, modules Modules) (*Store, error) {
|
||||
// Compile validates and compiles playbook documents in id order. Any invalid
|
||||
// playbook fails the whole set: a partially valid set never runs.
|
||||
func Compile(docs []Document, modules Modules) (*Store, error) {
|
||||
sort.Slice(docs, func(i, j int) bool { return docs[i].ID < docs[j].ID })
|
||||
slices.SortFunc(docs, func(a, b Document) int { return strings.Compare(a.ID, b.ID) })
|
||||
store := &Store{modules: modules}
|
||||
for _, doc := range docs {
|
||||
pb, err := compileOne(doc, modules)
|
||||
@@ -481,11 +482,7 @@ func compileArgs(path string, lines map[string]int, label, kind string, raw map[
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
keys := make([]string, 0, len(raw))
|
||||
for key := range raw {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
keys := slices.Sorted(maps.Keys(raw))
|
||||
args := make(map[string]*expr.Program, len(raw))
|
||||
for _, key := range keys {
|
||||
prg, err := compileExpr(at(path, lines[key]), argsLabel(label, key), raw[key], env)
|
||||
@@ -512,11 +509,7 @@ func validateArgsKeys(path string, lines map[string]int, label, kind string, arg
|
||||
for field := range t.Fields() {
|
||||
fields[strings.ToLower(field.Name)] = struct{}{}
|
||||
}
|
||||
keys := make([]string, 0, len(raw))
|
||||
for key := range raw {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
keys := slices.Sorted(maps.Keys(raw))
|
||||
loc := kind + " kind"
|
||||
if label != "" {
|
||||
loc = label + " " + loc
|
||||
@@ -671,11 +664,7 @@ func (a Action) whenHolds(ctx expr.Context) bool {
|
||||
// the first evaluation error.
|
||||
func (a Action) evalArgs(ctx expr.Context) (map[string]any, error) {
|
||||
out := make(map[string]any, len(a.Args))
|
||||
keys := make([]string, 0, len(a.Args))
|
||||
for key := range a.Args {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
keys := slices.Sorted(maps.Keys(a.Args))
|
||||
for _, key := range keys {
|
||||
if a.Args[key] == nil {
|
||||
continue
|
||||
|
||||
@@ -17,6 +17,12 @@ type Subject struct {
|
||||
Subject string `json:"subject"`
|
||||
}
|
||||
|
||||
// EmailSubject is the placeholder binding of a person added by email: the
|
||||
// issuer's verified address stands in until their first sign-in claims it.
|
||||
func EmailSubject(issuer, email string) Subject {
|
||||
return Subject{Issuer: issuer, Subject: "email:" + strings.ToLower(strings.TrimSpace(email))}
|
||||
}
|
||||
|
||||
func (s Subject) Validate() error {
|
||||
if strings.TrimSpace(s.Issuer) == "" || strings.TrimSpace(s.Subject) == "" {
|
||||
return fmt.Errorf("%w: issuer and subject are required", ErrInvalid)
|
||||
@@ -28,6 +34,9 @@ func (s Subject) Validate() error {
|
||||
// verification, never an assertion from the caller.
|
||||
type Credential struct {
|
||||
Subject Subject
|
||||
// Email is the address the provider vouched for (email_verified), empty
|
||||
// otherwise. It resolves a person added by email before any subject is known.
|
||||
Email string
|
||||
Scopes []string
|
||||
Expires time.Time
|
||||
}
|
||||
@@ -69,6 +78,9 @@ type SubjectResolver interface {
|
||||
// configured with resolves to an identity, and a subject a provider asserts
|
||||
// resolves to the same identity.
|
||||
ResolveSubject(context.Context, Subject) (Identity, error)
|
||||
// ClaimSubject moves a placeholder binding onto subject and returns its
|
||||
// identity; a placeholder that is gone (already claimed) is ErrNotFound.
|
||||
ClaimSubject(ctx context.Context, placeholder, subject Subject) (Identity, error)
|
||||
}
|
||||
|
||||
// ProviderSession is what a completed authorization flow yields: the credential
|
||||
@@ -126,6 +138,11 @@ func Resolve(ctx context.Context, subjects SubjectResolver, credential Credentia
|
||||
return Identity{}, fmt.Errorf("%w: %w", ErrCredentialRejected, err)
|
||||
}
|
||||
value, err := subjects.ResolveSubject(ctx, credential.Subject)
|
||||
if errors.Is(err, ErrNotFound) && credential.Email != "" {
|
||||
// The first sign-in binds the provider's subject in place of the
|
||||
// address, so later sign-ins rest on the subject alone.
|
||||
value, err = subjects.ClaimSubject(ctx, EmailSubject(credential.Subject.Issuer, credential.Email), credential.Subject)
|
||||
}
|
||||
if err != nil {
|
||||
if errors.Is(err, ErrNotFound) {
|
||||
return Identity{}, fmt.Errorf("%w: %s", ErrSubjectUnbound, credential.Subject.Subject)
|
||||
|
||||
@@ -3,9 +3,11 @@ package image
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"maps"
|
||||
"slices"
|
||||
"sync"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
|
||||
)
|
||||
|
||||
// ErrDuplicate is returned when a provider name is already registered.
|
||||
@@ -28,7 +30,7 @@ func NewRegistry() *Registry {
|
||||
// Registration fails on a nil provider, an invalid Capability, or a
|
||||
// duplicate name.
|
||||
func (r *Registry) Register(p Provider) error {
|
||||
if isNilProvider(p) {
|
||||
if lifecycle.IsNil(p) {
|
||||
return errors.New("image: nil provider")
|
||||
}
|
||||
if err := p.Capability().Validate(); err != nil {
|
||||
@@ -56,12 +58,7 @@ func (r *Registry) Get(name string) (Provider, bool) {
|
||||
func (r *Registry) Names() []string {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
names := make([]string, 0, len(r.providers))
|
||||
for name := range r.providers {
|
||||
names = append(names, name)
|
||||
}
|
||||
slices.Sort(names)
|
||||
return names
|
||||
return slices.Sorted(maps.Keys(r.providers))
|
||||
}
|
||||
|
||||
// ByClass returns the sorted names of registered providers matching class.
|
||||
@@ -77,14 +74,3 @@ func (r *Registry) ByClass(class ProviderClass) []string {
|
||||
slices.Sort(names)
|
||||
return names
|
||||
}
|
||||
|
||||
func isNilProvider(p Provider) bool {
|
||||
v := reflect.ValueOf(p)
|
||||
switch v.Kind() {
|
||||
case reflect.Invalid:
|
||||
return true
|
||||
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
|
||||
return v.IsNil()
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,247 @@
|
||||
// Package lifecycle is the registration and start/health/stop machinery
|
||||
// shared by engine families such as curators and memory engines.
|
||||
package lifecycle
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"maps"
|
||||
"reflect"
|
||||
"slices"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// HealthStatus is an engine's point-in-time health.
|
||||
type HealthStatus string
|
||||
|
||||
const (
|
||||
HealthHealthy HealthStatus = "healthy"
|
||||
HealthDegraded HealthStatus = "degraded"
|
||||
HealthUnhealthy HealthStatus = "unhealthy"
|
||||
)
|
||||
|
||||
// Health is a point-in-time engine health report.
|
||||
type Health struct {
|
||||
Status HealthStatus
|
||||
Message string
|
||||
}
|
||||
|
||||
// Registry errors.
|
||||
var (
|
||||
// ErrDuplicate is returned when an engine name is already registered.
|
||||
ErrDuplicate = errors.New("duplicate engine")
|
||||
// ErrStarted is returned when registration or lifecycle is attempted
|
||||
// after the registry has started.
|
||||
ErrStarted = errors.New("registry already started")
|
||||
)
|
||||
|
||||
// Engine is a named, host-bound component with a lifecycle.
|
||||
type Engine[H any] interface {
|
||||
Name() string
|
||||
Bind(host H)
|
||||
Start(ctx context.Context) error
|
||||
Health(ctx context.Context) Health
|
||||
Stop(ctx context.Context) error
|
||||
}
|
||||
|
||||
type state uint8
|
||||
|
||||
const (
|
||||
stateIdle state = iota
|
||||
stateRunning
|
||||
stateStopped
|
||||
)
|
||||
|
||||
type status struct {
|
||||
started bool
|
||||
startErr error
|
||||
}
|
||||
|
||||
// Registry owns registration and lifecycle for one engine family. An
|
||||
// engine's failure, including a panic, affects only that engine.
|
||||
type Registry[E Engine[H], H any] struct {
|
||||
kind string
|
||||
log *slog.Logger
|
||||
mu sync.Mutex
|
||||
engines map[string]E
|
||||
status map[string]status
|
||||
order []string
|
||||
state state
|
||||
}
|
||||
|
||||
// New builds an empty registry. kind prefixes errors and log lines; a nil
|
||||
// log disables lifecycle logging.
|
||||
func New[E Engine[H], H any](kind string, log *slog.Logger) *Registry[E, H] {
|
||||
return &Registry[E, H]{
|
||||
kind: kind,
|
||||
log: log,
|
||||
engines: make(map[string]E),
|
||||
status: make(map[string]status),
|
||||
}
|
||||
}
|
||||
|
||||
// IsNil reports whether v is nil or an interface holding a nil value.
|
||||
func IsNil(v any) bool {
|
||||
rv := reflect.ValueOf(v)
|
||||
switch rv.Kind() {
|
||||
case reflect.Invalid:
|
||||
return true
|
||||
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
|
||||
return rv.IsNil()
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Add binds e to host and registers it. A panicking Bind refuses the engine.
|
||||
func (r *Registry[E, H]) Add(e E, host H) error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.state != stateIdle {
|
||||
return fmt.Errorf("%s: %w", r.kind, ErrStarted)
|
||||
}
|
||||
name := e.Name()
|
||||
if _, dup := r.engines[name]; dup {
|
||||
return fmt.Errorf("%w: %s %s", ErrDuplicate, r.kind, name)
|
||||
}
|
||||
if err := guard(func() error { e.Bind(host); return nil }); err != nil {
|
||||
return fmt.Errorf("%s %s: bind %w", r.kind, name, err)
|
||||
}
|
||||
r.engines[name] = e
|
||||
r.order = append(r.order, name)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Get returns the registered engine by name.
|
||||
func (r *Registry[E, H]) Get(name string) (E, bool) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
e, ok := r.engines[name]
|
||||
return e, ok
|
||||
}
|
||||
|
||||
// Names returns the registered engine names in sorted order.
|
||||
func (r *Registry[E, H]) Names() []string {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
return slices.Sorted(maps.Keys(r.engines))
|
||||
}
|
||||
|
||||
// Start starts every engine in registration order. An engine whose Start
|
||||
// fails is marked unhealthy and skipped; the rest start regardless, and all
|
||||
// failures are joined into the returned error. A stopped registry may be
|
||||
// started again.
|
||||
func (r *Registry[E, H]) Start(ctx context.Context) error {
|
||||
r.mu.Lock()
|
||||
if r.state == stateRunning {
|
||||
r.mu.Unlock()
|
||||
return fmt.Errorf("%s: %w", r.kind, ErrStarted)
|
||||
}
|
||||
r.state = stateRunning
|
||||
order := slices.Clone(r.order)
|
||||
r.mu.Unlock()
|
||||
|
||||
var errs []error
|
||||
for _, name := range order {
|
||||
e, _ := r.Get(name)
|
||||
err := guard(func() error { return e.Start(ctx) })
|
||||
r.mu.Lock()
|
||||
if err != nil {
|
||||
r.status[name] = status{startErr: err}
|
||||
errs = append(errs, r.fail(name, "start", err))
|
||||
} else {
|
||||
r.status[name] = status{started: true}
|
||||
}
|
||||
r.mu.Unlock()
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
// Health reports every engine's health. An engine whose Start failed, or
|
||||
// whose Health panics, is reported unhealthy.
|
||||
func (r *Registry[E, H]) Health(ctx context.Context) map[string]Health {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
out := make(map[string]Health, len(r.engines))
|
||||
for name, e := range r.engines {
|
||||
if st := r.status[name]; st.startErr != nil {
|
||||
out[name] = Health{Status: HealthUnhealthy, Message: st.startErr.Error()}
|
||||
continue
|
||||
}
|
||||
out[name] = healthSafely(ctx, e)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Stop stops started engines in reverse order, bounded by ctx, and joins
|
||||
// their errors.
|
||||
func (r *Registry[E, H]) Stop(ctx context.Context) error {
|
||||
r.mu.Lock()
|
||||
if r.state != stateRunning {
|
||||
r.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
r.state = stateStopped
|
||||
order := slices.Clone(r.order)
|
||||
r.mu.Unlock()
|
||||
|
||||
var errs []error
|
||||
for _, name := range slices.Backward(order) {
|
||||
r.mu.Lock()
|
||||
e, st := r.engines[name], r.status[name]
|
||||
r.mu.Unlock()
|
||||
if !st.started {
|
||||
continue
|
||||
}
|
||||
if err := guard(func() error { return e.Stop(ctx) }); err != nil {
|
||||
errs = append(errs, r.fail(name, "stop", err))
|
||||
}
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
// Replace removes the engines drop selects and appends every engine from
|
||||
// src, keeping src's order. Call it only while stopped.
|
||||
func (r *Registry[E, H]) Replace(drop func(E) bool, src *Registry[E, H]) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.order = slices.DeleteFunc(r.order, func(name string) bool {
|
||||
if !drop(r.engines[name]) {
|
||||
return false
|
||||
}
|
||||
delete(r.engines, name)
|
||||
delete(r.status, name)
|
||||
return true
|
||||
})
|
||||
for _, name := range src.order {
|
||||
r.engines[name] = src.engines[name]
|
||||
r.order = append(r.order, name)
|
||||
}
|
||||
}
|
||||
|
||||
func (r *Registry[E, H]) fail(name, phase string, err error) error {
|
||||
if r.log != nil {
|
||||
r.log.Error(r.kind+" lifecycle failed", "name", name, "phase", phase, "err", err)
|
||||
}
|
||||
return fmt.Errorf("%s %s: %w", r.kind, name, err)
|
||||
}
|
||||
|
||||
// guard runs fn, converting a panic into an error.
|
||||
func guard(fn func() error) (err error) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
err = fmt.Errorf("panic: %v", p)
|
||||
}
|
||||
}()
|
||||
return fn()
|
||||
}
|
||||
|
||||
func healthSafely[E Engine[H], H any](ctx context.Context, e E) (h Health) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
h = Health{Status: HealthUnhealthy, Message: fmt.Sprintf("health panic: %v", p)}
|
||||
}
|
||||
}()
|
||||
return e.Health(ctx)
|
||||
}
|
||||
@@ -8,22 +8,22 @@ import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
|
||||
"github.com/samcharles93/archie-core/internal/domain/org"
|
||||
)
|
||||
|
||||
// HealthStatus is an engine's point-in-time health.
|
||||
type HealthStatus string
|
||||
// HealthStatus is a point-in-time health status.
|
||||
type HealthStatus = lifecycle.HealthStatus
|
||||
|
||||
const (
|
||||
HealthHealthy HealthStatus = "healthy"
|
||||
HealthDegraded HealthStatus = "degraded"
|
||||
HealthUnhealthy HealthStatus = "unhealthy"
|
||||
HealthHealthy = lifecycle.HealthHealthy
|
||||
HealthDegraded = lifecycle.HealthDegraded
|
||||
HealthUnhealthy = lifecycle.HealthUnhealthy
|
||||
)
|
||||
|
||||
// Health is a point-in-time engine health report.
|
||||
type Health struct {
|
||||
Status HealthStatus
|
||||
Message string
|
||||
}
|
||||
// Health is a point-in-time health report.
|
||||
type Health = lifecycle.Health
|
||||
|
||||
// Manifest is an engine's declared shape. RequiresNetwork marks an engine
|
||||
// that calls an external service, so a caller can size timeouts and
|
||||
@@ -78,6 +78,9 @@ type Scope struct {
|
||||
Kind ScopeKind
|
||||
Agent AgentID
|
||||
User IdentityID
|
||||
// Org owns the region. Every kind, global included, is per org: one
|
||||
// org's facts never reach another org's turns. Empty is the system org.
|
||||
Org org.OrgID
|
||||
}
|
||||
|
||||
// Validate rejects an unknown kind, a missing component, and — just as
|
||||
@@ -116,6 +119,14 @@ func (s Scope) Validate() error {
|
||||
// Key returns the scope's length-prefixed storage key, or "" for an invalid
|
||||
// scope.
|
||||
func (s Scope) Key() string {
|
||||
key := s.kindKey()
|
||||
if key == "" || s.Org == "" || s.Org == org.DefaultOrgID {
|
||||
return key
|
||||
}
|
||||
return "org:" + lengthPrefixed(string(s.Org)) + ":" + key
|
||||
}
|
||||
|
||||
func (s Scope) kindKey() string {
|
||||
switch s.Kind {
|
||||
case ScopeGlobal:
|
||||
return "global"
|
||||
@@ -151,12 +162,15 @@ type Subject struct {
|
||||
// carries no per-person identity (the dashboard, a webhook) — which
|
||||
// means "show none of it", never "fall back to a wider scope".
|
||||
UserID IdentityID
|
||||
// Org is the org the turn runs in; every scope it reads or writes
|
||||
// belongs to that org.
|
||||
Org org.OrgID
|
||||
}
|
||||
|
||||
// Scopes returns the subject's readable scopes, narrowest first: agent-user,
|
||||
// agent, user, global.
|
||||
func (s Subject) Scopes() []Scope {
|
||||
return append(s.WritableScopes(), Scope{Kind: ScopeGlobal})
|
||||
return append(s.WritableScopes(), Scope{Kind: ScopeGlobal, Org: s.Org})
|
||||
}
|
||||
|
||||
// WritableScopes returns the scopes a write for this subject may name —
|
||||
@@ -165,13 +179,13 @@ func (s Subject) Scopes() []Scope {
|
||||
func (s Subject) WritableScopes() []Scope {
|
||||
var scopes []Scope
|
||||
if s.AgentID != "" && s.UserID != "" {
|
||||
scopes = append(scopes, Scope{Kind: ScopeAgentUser, Agent: s.AgentID, User: s.UserID})
|
||||
scopes = append(scopes, Scope{Kind: ScopeAgentUser, Agent: s.AgentID, User: s.UserID, Org: s.Org})
|
||||
}
|
||||
if s.AgentID != "" {
|
||||
scopes = append(scopes, Scope{Kind: ScopeAgent, Agent: s.AgentID})
|
||||
scopes = append(scopes, Scope{Kind: ScopeAgent, Agent: s.AgentID, Org: s.Org})
|
||||
}
|
||||
if s.UserID != "" {
|
||||
scopes = append(scopes, Scope{Kind: ScopeUser, User: s.UserID})
|
||||
scopes = append(scopes, Scope{Kind: ScopeUser, User: s.UserID, Org: s.Org})
|
||||
}
|
||||
return scopes
|
||||
}
|
||||
|
||||
@@ -1,45 +1,22 @@
|
||||
package memory
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"slices"
|
||||
"sync"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/lifecycle"
|
||||
)
|
||||
|
||||
// Registry errors.
|
||||
var (
|
||||
// ErrDuplicate is returned when an engine name is already registered.
|
||||
ErrDuplicate = errors.New("duplicate memory engine")
|
||||
// ErrStarted is returned when registration or lifecycle is attempted
|
||||
// after the registry has started.
|
||||
ErrStarted = errors.New("memory registry already started")
|
||||
ErrDuplicate = lifecycle.ErrDuplicate
|
||||
ErrStarted = lifecycle.ErrStarted
|
||||
)
|
||||
|
||||
type registryState uint8
|
||||
|
||||
const (
|
||||
stateIdle registryState = iota
|
||||
stateRunning
|
||||
stateStopped
|
||||
)
|
||||
|
||||
type engineStatus struct {
|
||||
started bool
|
||||
startErr error
|
||||
}
|
||||
|
||||
// Registry owns memory engine registration and lifecycle. An engine's
|
||||
// failure affects only that engine.
|
||||
type Registry struct {
|
||||
mu sync.Mutex
|
||||
host Registrar
|
||||
engines map[string]MemoryEngine
|
||||
status map[string]engineStatus
|
||||
order []string
|
||||
state registryState
|
||||
*lifecycle.Registry[MemoryEngine, Registrar]
|
||||
host Registrar
|
||||
}
|
||||
|
||||
// NewRegistry builds the family registry. A nil Clock is replaced with the
|
||||
@@ -48,182 +25,17 @@ func NewRegistry(host Registrar) *Registry {
|
||||
if host.Clock == nil {
|
||||
host.Clock = systemClock{}
|
||||
}
|
||||
return &Registry{
|
||||
host: host,
|
||||
engines: make(map[string]MemoryEngine),
|
||||
status: make(map[string]engineStatus),
|
||||
}
|
||||
return &Registry{Registry: lifecycle.New[MemoryEngine, Registrar]("memory engine", nil), host: host}
|
||||
}
|
||||
|
||||
// Register validates the engine's declared shape and adds it, binding the
|
||||
// engine to the registrar. Registration fails on nil engines, invalid
|
||||
// manifests, duplicate names, or a panicking Bind — and a failure affects
|
||||
// only that engine.
|
||||
// Register validates the engine's declared shape and binds it to the
|
||||
// registrar. A failure affects only that engine.
|
||||
func (r *Registry) Register(e MemoryEngine) error {
|
||||
if isNilEngine(e) {
|
||||
if lifecycle.IsNil(e) {
|
||||
return errors.New("memory: nil engine")
|
||||
}
|
||||
if err := e.Manifest().Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.state != stateIdle {
|
||||
return ErrStarted
|
||||
}
|
||||
if _, dup := r.engines[e.Name()]; dup {
|
||||
return fmt.Errorf("%w: %s", ErrDuplicate, e.Name())
|
||||
}
|
||||
if err := bindSafely(e, r.host); err != nil {
|
||||
return err
|
||||
}
|
||||
r.engines[e.Name()] = e
|
||||
r.order = append(r.order, e.Name())
|
||||
return nil
|
||||
}
|
||||
|
||||
// bindSafely isolates a panicking Bind: an engine that panics while
|
||||
// receiving its host access is refused without taking the registry down.
|
||||
func bindSafely(e MemoryEngine, host Registrar) (err error) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
err = fmt.Errorf("memory engine %s: bind panic: %v", e.Name(), p)
|
||||
}
|
||||
}()
|
||||
e.Bind(host)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Get returns the registered engine by name.
|
||||
func (r *Registry) Get(name string) (MemoryEngine, bool) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
e, ok := r.engines[name]
|
||||
return e, ok
|
||||
}
|
||||
|
||||
// Names returns the registered engine names in sorted order.
|
||||
func (r *Registry) Names() []string {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
names := make([]string, 0, len(r.engines))
|
||||
for name := range r.engines {
|
||||
names = append(names, name)
|
||||
}
|
||||
slices.Sort(names)
|
||||
return names
|
||||
}
|
||||
|
||||
// Start starts every registered engine in registration order. An engine
|
||||
// whose Start fails (error or panic) is marked unhealthy and skipped; the
|
||||
// rest start regardless, and all failures are joined into the returned
|
||||
// error. This is the family's failure-isolation boundary for lifecycle.
|
||||
func (r *Registry) Start(ctx context.Context) error {
|
||||
r.mu.Lock()
|
||||
if r.state != stateIdle {
|
||||
r.mu.Unlock()
|
||||
return ErrStarted
|
||||
}
|
||||
r.state = stateRunning
|
||||
order := slices.Clone(r.order)
|
||||
r.mu.Unlock()
|
||||
|
||||
var errs []error
|
||||
for _, name := range order {
|
||||
e := r.engines[name]
|
||||
err := startSafely(ctx, e)
|
||||
r.mu.Lock()
|
||||
if err != nil {
|
||||
r.status[name] = engineStatus{startErr: err}
|
||||
errs = append(errs, fmt.Errorf("memory engine %s: %w", name, err))
|
||||
} else {
|
||||
r.status[name] = engineStatus{started: true}
|
||||
}
|
||||
r.mu.Unlock()
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
func startSafely(ctx context.Context, e MemoryEngine) (err error) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
err = fmt.Errorf("start panic: %v", p)
|
||||
}
|
||||
}()
|
||||
return e.Start(ctx)
|
||||
}
|
||||
|
||||
// Health reports every registered engine's health. An engine whose Start
|
||||
// failed is reported unhealthy with the failure; otherwise the engine's own
|
||||
// Health is used. A panicking Health is reported unhealthy, not propagated.
|
||||
func (r *Registry) Health(ctx context.Context) map[string]Health {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
out := make(map[string]Health, len(r.engines))
|
||||
for name, e := range r.engines {
|
||||
if st, ok := r.status[name]; ok && st.startErr != nil {
|
||||
out[name] = Health{Status: HealthUnhealthy, Message: st.startErr.Error()}
|
||||
continue
|
||||
}
|
||||
out[name] = healthSafely(ctx, e)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func healthSafely(ctx context.Context, e MemoryEngine) (h Health) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
h = Health{Status: HealthUnhealthy, Message: fmt.Sprintf("health panic: %v", p)}
|
||||
}
|
||||
}()
|
||||
return e.Health(ctx)
|
||||
}
|
||||
|
||||
// Stop stops started engines in reverse order, bounded by ctx, and joins
|
||||
// their errors.
|
||||
func (r *Registry) Stop(ctx context.Context) error {
|
||||
r.mu.Lock()
|
||||
if r.state == stateIdle || r.state == stateStopped {
|
||||
r.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
r.state = stateStopped
|
||||
order := slices.Clone(r.order)
|
||||
r.mu.Unlock()
|
||||
|
||||
var errs []error
|
||||
for _, name := range slices.Backward(order) {
|
||||
r.mu.Lock()
|
||||
st, ok := r.status[name]
|
||||
r.mu.Unlock()
|
||||
if !ok || !st.started {
|
||||
continue
|
||||
}
|
||||
e := r.engines[name]
|
||||
if err := stopSafely(ctx, e); err != nil {
|
||||
errs = append(errs, fmt.Errorf("memory engine %s: %w", name, err))
|
||||
}
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
func stopSafely(ctx context.Context, e MemoryEngine) (err error) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
err = fmt.Errorf("stop panic: %v", p)
|
||||
}
|
||||
}()
|
||||
return e.Stop(ctx)
|
||||
}
|
||||
|
||||
func isNilEngine(e MemoryEngine) bool {
|
||||
v := reflect.ValueOf(e)
|
||||
switch v.Kind() {
|
||||
case reflect.Invalid:
|
||||
return true
|
||||
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
|
||||
return v.IsNil()
|
||||
}
|
||||
return false
|
||||
return r.Add(e, r.host)
|
||||
}
|
||||
@@ -15,7 +15,6 @@ type ChatSnapshot struct {
|
||||
// frontends display it rather than stamping their own build, so a
|
||||
// partially upgraded deployment cannot report itself as matched.
|
||||
Version string
|
||||
RestartAvailable bool
|
||||
CancellationAvailable bool
|
||||
// VoiceAvailable reports that a voice clip is transcribed into the turn.
|
||||
VoiceAvailable bool
|
||||
|
||||
@@ -10,7 +10,7 @@ var localCommands = []string{
|
||||
"/new", "/reset", "/topic", "/retry", "/undo",
|
||||
"/title", "/branch", "/fork", "/compress", "/compact",
|
||||
"/whoami", "/profile", "/sessions", "/resume", "/delete", "/agents",
|
||||
"/personality", "/help", "/version", "/update", "/restart",
|
||||
"/personality", "/help", "/version", "/update",
|
||||
}
|
||||
|
||||
// CommandSpec describes a local command for adapter-provided discovery.
|
||||
@@ -49,7 +49,6 @@ var localCommandSpecs = []CommandSpec{
|
||||
{Command: "/help", Description: "See what Archie can do", Usage: "/help"},
|
||||
{Command: "/version", Description: "Show installed Archie versions", Usage: "/version"},
|
||||
{Command: "/update", Description: "Check for Archie updates", Usage: "/update"},
|
||||
{Command: "/restart", Description: "Reload the chat adapter", Usage: "/restart"},
|
||||
}
|
||||
|
||||
// LocalCommands returns the command names Route answers from local state.
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"maps"
|
||||
"slices"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -323,11 +324,7 @@ func renderSetting(resource SettingResource, descriptor SettingDescriptor) strin
|
||||
}
|
||||
return fmt.Sprintf("%s (version %d):\n%s", resource.Kind, resource.Version, value)
|
||||
}
|
||||
names := make([]string, 0, len(resource.Value))
|
||||
for name := range resource.Value {
|
||||
names = append(names, name)
|
||||
}
|
||||
sort.Strings(names)
|
||||
names := slices.Sorted(maps.Keys(resource.Value))
|
||||
lines := []string{fmt.Sprintf("%s (version %d):", resource.Kind, resource.Version)}
|
||||
for _, name := range names {
|
||||
field := descriptor.Fields[name]
|
||||
|
||||
@@ -22,3 +22,18 @@ func OrgFromContext(ctx context.Context) OrgID {
|
||||
}
|
||||
return DefaultOrgID
|
||||
}
|
||||
|
||||
type scopeContextKey struct{}
|
||||
|
||||
// WithScope confines a request's reads to one org. A request without a scope
|
||||
// is an internal service acting across orgs, such as the daemon running every
|
||||
// org's work or webhook intake resolving a source by its path.
|
||||
func WithScope(ctx context.Context, id OrgID) context.Context {
|
||||
return context.WithValue(ctx, scopeContextKey{}, id)
|
||||
}
|
||||
|
||||
// Scope returns the org WithScope confined ctx to.
|
||||
func Scope(ctx context.Context) (OrgID, bool) {
|
||||
id, ok := ctx.Value(scopeContextKey{}).(OrgID)
|
||||
return id, ok && id != ""
|
||||
}
|
||||
@@ -81,6 +81,7 @@ type HarnessSecretStore interface {
|
||||
// status guard. It is separate from the already broad lifecycle contract so
|
||||
// consumers that only run tasks do not acquire an operator-only capability.
|
||||
type TaskArchiver interface {
|
||||
OutcomePoster
|
||||
ArchiveTask(ctx context.Context, taskID int64, fromStatus string, audit events.Event) (eventID int64, err error)
|
||||
ClearTerminalTasks(ctx context.Context) (int64, error)
|
||||
}
|
||||
@@ -120,6 +121,16 @@ type RemediationStarter interface {
|
||||
SetReviewCursors(ctx context.Context, taskID, reviewCursor, commentCursor int64) error
|
||||
}
|
||||
|
||||
// OutcomePoster tracks which finished chat tasks have reported back to their
|
||||
// conversation.
|
||||
type OutcomePoster interface {
|
||||
// UnpostedOutcomes returns finished chat tasks whose outcome has not been
|
||||
// posted to the conversation that created them.
|
||||
UnpostedOutcomes(ctx context.Context) ([]*task.Task, error)
|
||||
// MarkOutcomePosted records that a task's outcome was posted.
|
||||
MarkOutcomePosted(ctx context.Context, taskID int64) error
|
||||
}
|
||||
|
||||
// TaskQueries groups read-only task accessors.
|
||||
type TaskQueries interface {
|
||||
TaskByIssue(ctx context.Context, owner, repo string, number int) (*task.Task, error)
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
package storepkg
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"slices"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type fakeAuto struct {
|
||||
refs []OrgPackage
|
||||
listErr error
|
||||
failing string
|
||||
tried []string
|
||||
}
|
||||
|
||||
func (f *fakeAuto) ListAutoUpdate(context.Context) ([]OrgPackage, error) { return f.refs, f.listErr }
|
||||
|
||||
func (f *fakeAuto) UpdatePackage(_ context.Context, _, name string) (Installed, error) {
|
||||
f.tried = append(f.tried, name)
|
||||
if name == f.failing {
|
||||
return Installed{}, errors.New("catalogue unreachable")
|
||||
}
|
||||
return Installed{}, nil
|
||||
}
|
||||
|
||||
func (f *fakeAuto) ApprovePackageUpdate(context.Context, string, string) (Installed, error) {
|
||||
return Installed{}, nil
|
||||
}
|
||||
|
||||
func (f *fakeAuto) RollbackPackage(context.Context, string, string) (Installed, error) {
|
||||
return Installed{}, nil
|
||||
}
|
||||
|
||||
func (f *fakeAuto) SetPackageUpdatePolicy(context.Context, string, string, string) (Installed, error) {
|
||||
return Installed{}, nil
|
||||
}
|
||||
func (f *fakeAuto) UpdateAuto(context.Context) error { return nil }
|
||||
|
||||
// A tick updates every package the store lists as auto, and one failing
|
||||
// package neither stops the rest nor hides its error.
|
||||
func TestUpdateAuto(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
fake fakeAuto
|
||||
tried []string
|
||||
wantErr bool
|
||||
}{
|
||||
{name: "no auto packages", tried: nil},
|
||||
{name: "every auto package is tried", fake: fakeAuto{refs: []OrgPackage{{"o", "a"}, {"o", "b"}}}, tried: []string{"a", "b"}},
|
||||
{name: "a failure does not stop the rest", fake: fakeAuto{refs: []OrgPackage{{"o", "a"}, {"o", "b"}}, failing: "a"}, tried: []string{"a", "b"}, wantErr: true},
|
||||
{name: "a failed listing is an error", fake: fakeAuto{listErr: errors.New("db down")}, wantErr: true},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := updateAuto(t.Context(), &tt.fake, &tt.fake)
|
||||
if (err != nil) != tt.wantErr {
|
||||
t.Fatalf("err = %v; wantErr %v", err, tt.wantErr)
|
||||
}
|
||||
if !slices.Equal(tt.fake.tried, tt.tried) {
|
||||
t.Fatalf("tried %v; want %v", tt.fake.tried, tt.tried)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -35,6 +35,22 @@ type Installed struct {
|
||||
// pinned digest. A nil Authority means the operator has not accepted;
|
||||
// until then the package's authority at use time is nothing.
|
||||
AcceptedAuthority *Authority
|
||||
// Pending is an update waiting for an admin to approve widened authority.
|
||||
// The live pin and AcceptedAuthority stay as they were until then.
|
||||
Pending *Pin
|
||||
// PendingAuthority is what the pending update declares.
|
||||
PendingAuthority *Authority
|
||||
// Previous is the pin an update or rollback replaced, with the authority
|
||||
// accepted against it, so a rollback restores both.
|
||||
Previous *Pin
|
||||
// PreviousAccepted is the authority accepted against Previous.
|
||||
PreviousAccepted *Authority
|
||||
}
|
||||
|
||||
// Pin is an immutable package reference at a digest.
|
||||
type Pin struct {
|
||||
Reference string
|
||||
Digest string
|
||||
}
|
||||
|
||||
// Repository owns installed package records and dependency-safe deletion.
|
||||
@@ -46,6 +62,22 @@ type Repository interface {
|
||||
// Accept records the operator's accepted authority against the package's
|
||||
// currently pinned digest.
|
||||
Accept(context.Context, string, string, Authority) error
|
||||
UpdateRecords
|
||||
}
|
||||
|
||||
// UpdateRecords is the part of Repository the update lifecycle writes.
|
||||
type UpdateRecords interface {
|
||||
// SetPending records an update waiting for approval, leaving the live pin.
|
||||
SetPending(ctx context.Context, orgID, name string, pending Pin, declared Authority) error
|
||||
// Replace swaps the live pin, descriptor, layer and accepted authority for
|
||||
// next's, keeping the replaced pin and its accepted authority as Previous
|
||||
// and clearing Pending.
|
||||
Replace(ctx context.Context, next Installed) error
|
||||
// SetUpdatePolicy sets whether a package follows the catalogue on its own.
|
||||
SetUpdatePolicy(ctx context.Context, orgID, name, policy string) error
|
||||
// ListAutoUpdate names every installed package, in any org, whose policy
|
||||
// is auto.
|
||||
ListAutoUpdate(ctx context.Context) ([]OrgPackage, error)
|
||||
}
|
||||
|
||||
// Registry reads a package by immutable OCI manifest digest.
|
||||
@@ -62,6 +94,17 @@ type Manager interface {
|
||||
AcceptPackageAuthority(ctx context.Context, orgID, name string, accepted Authority) (Installed, error)
|
||||
ListCatalogue(ctx context.Context) (Catalogue, error)
|
||||
InstallFromCatalogue(ctx context.Context, orgID, name string) (Installed, error)
|
||||
OrgUpdates
|
||||
}
|
||||
|
||||
// OrgUpdates is the update lifecycle of an installed package in one org.
|
||||
type OrgUpdates interface {
|
||||
UpdatePackage(ctx context.Context, orgID, name string) (Installed, error)
|
||||
ApprovePackageUpdate(ctx context.Context, orgID, name string) (Installed, error)
|
||||
RollbackPackage(ctx context.Context, orgID, name string) (Installed, error)
|
||||
SetPackageUpdatePolicy(ctx context.Context, orgID, name, policy string) (Installed, error)
|
||||
// UpdateAuto runs UpdatePackage for every auto package.
|
||||
UpdateAuto(ctx context.Context) error
|
||||
}
|
||||
|
||||
// Installations is Manager as a remote caller sees it: the State Store acts
|
||||
@@ -74,6 +117,15 @@ type Installations interface {
|
||||
AcceptPackageAuthority(ctx context.Context, name string, accepted Authority) (Installed, error)
|
||||
ListCatalogue(ctx context.Context) (Catalogue, error)
|
||||
InstallFromCatalogue(ctx context.Context, name string) (Installed, error)
|
||||
Updates
|
||||
}
|
||||
|
||||
// Updates is OrgUpdates as a remote caller sees it.
|
||||
type Updates interface {
|
||||
UpdatePackage(ctx context.Context, name string) (Installed, error)
|
||||
ApprovePackageUpdate(ctx context.Context, name string) (Installed, error)
|
||||
RollbackPackage(ctx context.Context, name string) (Installed, error)
|
||||
SetPackageUpdatePolicy(ctx context.Context, name, policy string) (Installed, error)
|
||||
}
|
||||
|
||||
// Service validates the registry content before persisting an installation.
|
||||
@@ -157,7 +209,7 @@ func (s Service) Install(ctx context.Context, orgID, name, reference, digest str
|
||||
}
|
||||
installed := Installed{
|
||||
OrgID: orgID, Name: name, Reference: reference, Digest: digest,
|
||||
Descriptor: descriptor, Layer: layer, UpdatePolicy: "manual",
|
||||
Descriptor: descriptor, Layer: layer, UpdatePolicy: PolicyManual,
|
||||
}
|
||||
contents, err := s.resolveFamilyContents(descriptor, layer)
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
package storepkg
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// Update policies. Manual is the default; an auto package follows the
|
||||
// catalogue, with a widening update still waiting for approval.
|
||||
const (
|
||||
PolicyManual = "manual"
|
||||
PolicyAuto = "auto"
|
||||
)
|
||||
|
||||
// ErrInvalidPolicy: the update policy is neither manual nor auto.
|
||||
var ErrInvalidPolicy = errors.New("store package update policy must be manual or auto")
|
||||
|
||||
// OrgPackage names one installed package.
|
||||
type OrgPackage struct{ OrgID, Name string }
|
||||
|
||||
// SetPackageUpdatePolicy sets whether name follows the catalogue on its own.
|
||||
func (s Service) SetPackageUpdatePolicy(ctx context.Context, orgID, name, policy string) (Installed, error) {
|
||||
if policy != PolicyManual && policy != PolicyAuto {
|
||||
return Installed{}, ErrInvalidPolicy
|
||||
}
|
||||
if err := s.Store.SetUpdatePolicy(ctx, orgID, name, policy); err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
return s.Store.Get(ctx, orgID, name)
|
||||
}
|
||||
|
||||
// UpdateAuto runs UpdatePackage for every auto package.
|
||||
func (s Service) UpdateAuto(ctx context.Context) error {
|
||||
return updateAuto(ctx, s.Store, s)
|
||||
}
|
||||
|
||||
// updateAuto tries every auto package: one that fails does not stop the rest.
|
||||
func updateAuto(ctx context.Context, store interface {
|
||||
ListAutoUpdate(context.Context) ([]OrgPackage, error)
|
||||
}, updates OrgUpdates,
|
||||
) error {
|
||||
refs, err := store.ListAutoUpdate(ctx)
|
||||
if err != nil {
|
||||
return fmt.Errorf("list auto-update packages: %w", err)
|
||||
}
|
||||
var errs []error
|
||||
for _, ref := range refs {
|
||||
if _, err := updates.UpdatePackage(ctx, ref.OrgID, ref.Name); err != nil {
|
||||
errs = append(errs, fmt.Errorf("update %s/%s: %w", ref.OrgID, ref.Name, err))
|
||||
}
|
||||
}
|
||||
return errors.Join(errs...)
|
||||
}
|
||||
|
||||
var (
|
||||
// ErrNoPendingUpdate: there is no update waiting for approval.
|
||||
ErrNoPendingUpdate = errors.New("store package has no pending update")
|
||||
// ErrNoPrevious: the package has no earlier version to roll back to.
|
||||
ErrNoPrevious = errors.New("store package has no previous version")
|
||||
)
|
||||
|
||||
// Widens reports whether an update's declared authority asks for more than
|
||||
// the operator accepted. A package never accepted holds no authority, so
|
||||
// any grant it declares is wider.
|
||||
func (i Installed) Widens(next Authority) bool {
|
||||
var held Authority
|
||||
if i.AcceptedAuthority != nil {
|
||||
held = *i.AcceptedAuthority
|
||||
}
|
||||
return !held.Covers(next)
|
||||
}
|
||||
|
||||
// UpdatePackage moves name to the digest the verified catalogue lists. An
|
||||
// update that does not widen authority applies at once; a widening one waits
|
||||
// as pending until ApprovePackageUpdate, with the live pin untouched.
|
||||
func (s Service) UpdatePackage(ctx context.Context, orgID, name string) (Installed, error) {
|
||||
installed, err := s.Store.Get(ctx, orgID, name)
|
||||
if err != nil {
|
||||
return Installed{}, fmt.Errorf("installed package %q: %w", name, err)
|
||||
}
|
||||
catalogue, err := s.ListCatalogue(ctx)
|
||||
if err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
for _, entry := range catalogue.Packages {
|
||||
if entry.Name != name {
|
||||
continue
|
||||
}
|
||||
pin := Pin{Reference: entry.Reference, Digest: entry.Digest}
|
||||
if pin.Digest == installed.Digest || (installed.Pending != nil && installed.Pending.Digest == pin.Digest) {
|
||||
return installed, nil
|
||||
}
|
||||
next, contents, err := s.prepare(ctx, orgID, name, pin)
|
||||
if err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
if installed.Widens(next.Descriptor.Authority) {
|
||||
if err := s.Store.SetPending(ctx, orgID, name, pin, next.Descriptor.Authority); err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
return s.Store.Get(ctx, orgID, name)
|
||||
}
|
||||
next.AcceptedAuthority = installed.AcceptedAuthority
|
||||
return s.swap(ctx, installed, next, contents)
|
||||
}
|
||||
return Installed{}, fmt.Errorf("catalogue package %q: %w", name, ErrNotFound)
|
||||
}
|
||||
|
||||
// ApprovePackageUpdate applies the pending update and accepts the authority
|
||||
// it declares.
|
||||
func (s Service) ApprovePackageUpdate(ctx context.Context, orgID, name string) (Installed, error) {
|
||||
installed, err := s.Store.Get(ctx, orgID, name)
|
||||
if err != nil {
|
||||
return Installed{}, fmt.Errorf("installed package %q: %w", name, err)
|
||||
}
|
||||
if installed.Pending == nil {
|
||||
return Installed{}, fmt.Errorf("package %q: %w", name, ErrNoPendingUpdate)
|
||||
}
|
||||
next, contents, err := s.prepare(ctx, orgID, name, *installed.Pending)
|
||||
if err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
accepted := next.Descriptor.Authority
|
||||
next.AcceptedAuthority = &accepted
|
||||
return s.swap(ctx, installed, next, contents)
|
||||
}
|
||||
|
||||
// RollbackPackage reinstalls the previous digest with the authority accepted
|
||||
// against it.
|
||||
func (s Service) RollbackPackage(ctx context.Context, orgID, name string) (Installed, error) {
|
||||
installed, err := s.Store.Get(ctx, orgID, name)
|
||||
if err != nil {
|
||||
return Installed{}, fmt.Errorf("installed package %q: %w", name, err)
|
||||
}
|
||||
if installed.Previous == nil {
|
||||
return Installed{}, fmt.Errorf("package %q: %w", name, ErrNoPrevious)
|
||||
}
|
||||
next, contents, err := s.prepare(ctx, orgID, name, *installed.Previous)
|
||||
if err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
next.AcceptedAuthority = installed.PreviousAccepted
|
||||
return s.swap(ctx, installed, next, contents)
|
||||
}
|
||||
|
||||
// prepare fetches and validates the package at pin and resolves its
|
||||
// contributions before anything live changes.
|
||||
func (s Service) prepare(ctx context.Context, orgID, name string, pin Pin) (Installed, map[string]map[string][]byte, error) {
|
||||
descriptor, layer, err := s.Registry.Fetch(ctx, pin.Reference, pin.Digest)
|
||||
if err != nil {
|
||||
return Installed{}, nil, fmt.Errorf("fetch package: %w", err)
|
||||
}
|
||||
if err := descriptor.Validate(); err != nil {
|
||||
return Installed{}, nil, fmt.Errorf("validate package: %w", err)
|
||||
}
|
||||
contents, err := s.resolveFamilyContents(descriptor, layer)
|
||||
if err != nil {
|
||||
return Installed{}, nil, err
|
||||
}
|
||||
return Installed{
|
||||
OrgID: orgID, Name: name, Reference: pin.Reference, Digest: pin.Digest,
|
||||
Descriptor: descriptor, Layer: layer,
|
||||
}, contents, nil
|
||||
}
|
||||
|
||||
// swap replaces installed with next: old contributions come out, new ones go
|
||||
// in, then the record flips. A failure puts the old contributions back so
|
||||
// the record, which still names the old pin, stays true.
|
||||
func (s Service) swap(ctx context.Context, installed, next Installed, contents map[string]map[string][]byte) (Installed, error) {
|
||||
if err := s.withdraw(ctx, installed); err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
err := s.project(ctx, next, contents)
|
||||
if err == nil {
|
||||
err = s.Store.Replace(ctx, next)
|
||||
}
|
||||
if err != nil {
|
||||
return Installed{}, errors.Join(err, s.withdraw(ctx, next), s.restore(ctx, installed))
|
||||
}
|
||||
return s.Store.Get(ctx, installed.OrgID, installed.Name)
|
||||
}
|
||||
|
||||
func (s Service) restore(ctx context.Context, installed Installed) error {
|
||||
contents, err := s.resolveFamilyContents(installed.Descriptor, installed.Layer)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.project(ctx, installed, contents)
|
||||
}
|
||||
Loaded 100 of 301 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user