Compare commits

...
91 Commits
Author SHA1 Message Date
sam 673a5bc2ce chore(release): v1.52.0
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-09 13:09:59 +11:00
sam df5ee243b1 feat(harness): create the setup PTY at the browser's size
The setup terminal created its PTY at a fixed 24x80 while the browser
measured its own box and threw the measurement away, so a full-screen CLI
login drew into a corner of a much larger xterm.

OpenRequest now carries a ConsoleSize with the target frame; the route
reads cols/rows from the WebSocket URL, falling back to 24x80 when they
are absent or out of range, and the daemon creates the PTY at that
geometry. The component sends the size it measured before connecting.

Live resize is the follow-on: it needs a control channel on the WebSocket,
which today is a raw byte copy.
2026-10-09 13:07:29 +11:00
sam 71fa670512 fix(harness): keep the setup container alive with archie's own binary
A setup session started its container with "sleep infinity" and opened the
PTY with "sh -l". Neither binary is guaranteed: the spec capability whose
floor is a shell and bash is com.docker.sandbox/sbx@1, and archie's
supported map does not list it, so a Kit image need not provide either. A
missing sleep killed the container at start and a missing sh failed the
exec, both as an opaque error.

The container now runs archie-agent hold as its entrypoint. The agent
binary is archie's own and already mounted into every Kit, so the
keepalive needs nothing the image may lack and the container env stays
empty. Before opening the PTY, harnesssession probes for bash then sh and
returns ErrNoShell naming the Kit reference when neither is present.

Test: TestPickShell covers bash-preferred, sh-only, neither, and a probe
that cannot run.
2026-10-09 13:05:05 +11:00
sam d91ab1ab0f fix(channels): normalize stored settings and deduplicate save errors 2026-10-09 13:02:23 +11:00
sam 1eb05f0d24 test(ui): prune brittle checks and document agent rules 2026-10-09 12:51:57 +11:00
sam a49e939609 fix(ui): clean dependencies and add task table sorting 2026-10-09 12:46:05 +11:00
sam e76e2fd317 chore(lint): pin the linter to 2.14.0
The setup task is the only installer, so the pin follows the version in
use rather than holding it back, and CI follows the pin.
2026-10-09 12:32:05 +11:00
sam 2418a81a9e fix(egress): use context-aware dial and request in the token-endpoint test
net.Dial and httptest.NewRequest fail the noctx linter, so the gate was red
on the commit that added the test.
2026-10-09 12:27:45 +11:00
sam c4a441d1c3 chore(task): install the pinned tooling with task setup
task setup installs the golangci-lint CI pins, the protobuf generators and
the dashboard dependencies, and warns by name about the tools it cannot
install (Go, Task, Node, Docker).

The linter version now lives in one file read by both the setup task and
both workflows, so a local checkout and the gate cannot run different
linters.
2026-10-09 12:27:45 +11:00
sam c5c145f325 test(archieui): restore the archie-ui import-boundary check
Only the daemon owns containers, and the dashboard consumes a harness
session over its gRPC contract. Go imports are atomic, so a single
reference relinks a runtime, and the boundary holds only while something
re-measures it. cmd/archie-ui/architecture_test.go enforced this until it
was purged with the test suite; nothing enforced it since.

The restored check runs go list -deps . and fails if the dashboard links
internal/container, internal/app/archied or internal/agentexec. It is
transitive, which is what "links" means, and runs in the existing test
step.
2026-10-09 11:42:39 +11:00
sam 3683ce9082 fix(egress): refuse a token endpoint the run is not granted
forward intercepted a declared OAuth credential's token endpoint only when
the run was granted the service. For an optional credential it fell
through to the reverse proxy, so the provider's real access and refresh
tokens reached the container un-sentinelized, contradicting the harness
contract's "the real credential never enters the container".

A matched token endpoint is now refused when the service is not granted
(403 for an optional credential; a required one keeps its existing 502).
The table test drives forward against a recording upstream and asserts
both the status and that the upstream was never contacted.
2026-10-09 11:42:39 +11:00
sam 80dc6ec8d0 fix(deps): pin sandbox-kit-spec to the API the code is written against
The dependency bump moved sandbox-kit-spec from v3.0.0-m.6 to m.8 without
adapting the code, so the tree no longer builds: CredentialCapability.Phase
is now spec.Phases, not a string (internal/infrastructure/egress), and
Client.Assemble became a package-level fetch.Assemble that takes a Loader
and a LayerValidator instead of spec.MergeOptions (internal/infrastructure/
kitrun).

Migrating to m.8 is not a version-string change: m.8 pushes kit integrity,
safe layer extraction and resource limits onto the caller, so it is
security-relevant work, not an unblock. Pin back to m.6, which the code is
written against, and leave the migration to a deliberate change.

Verified: go build ./..., go vet ./..., go test -short ./... all pass with
this pin. The gate's ui:check and lint steps fail for unrelated
toolchain reasons recorded separately.
2026-10-09 11:42:33 +11:00
sam 12d6585586 ```
chore: update ui dependencies
```
2026-10-09 11:27:48 +11:00
sam 53916b0f96 chore: bump Go dependencies and toolchain 2026-10-09 11:23:04 +11:00
sam badbbf84bd fix(config): normalize default org for harness credentials 2026-10-09 10:01:35 +11:00
sam d98352d003 feat(settings): classify startup and live runtime settings 2026-10-09 09:57:57 +11:00
sam 542bd88040 feat(logging): aggregate recent service logs in dashboard 2026-10-09 09:57:57 +11:00
sam 81384939a2 fix(gateway): attribute chat settings to signed-in users 2026-10-09 09:57:57 +11:00
sam c8ed5619c6 fix(webui): explain personal token availability 2026-10-09 09:57:57 +11:00
sam 3b79155602 refactor(webui): remove unused configuration schema 2026-10-09 09:57:57 +11:00
sam 552ee93a47 fix(settings): show configuration reload failures 2026-10-09 09:57:57 +11:00
sam 7bf429b599 fix(gateway): read task logs through state store 2026-10-09 09:57:57 +11:00
sam 0792f0ba84 fix(ui): render unknown routes and correct status description 2026-10-09 09:57:56 +11:00
sam 02ab747ca9 fix(configuration): reject unknown conf.d files 2026-10-09 09:57:56 +11:00
sam c53a98ee87 feat(curators): edit and apply definitions live 2026-10-09 09:57:56 +11:00
sam 1e4aa8badc feat(settings): edit profiles bindings and playbooks 2026-10-09 09:57:56 +11:00
sam 1e76c99b48 feat(settings): show resource consumers and apply status 2026-10-09 09:57:56 +11:00
sam ff9b6c7196 feat(presence): report workers and broker on status 2026-10-09 09:57:56 +11:00
sam 7680dc5079 feat(presence): expose service dependency checks 2026-10-09 09:57:56 +11:00
sam d0370ed9f7 fix(daemon): inherit live settings for identity tasks 2026-10-09 09:57:56 +11:00
sam 7e3b3dbd78 test: verify enablement reports channel running 2026-10-09 02:24:47 +11:00
sam 6b71dc8108 fix(workflow): enforce optional result booleans only when required 2026-10-09 02:22:19 +11:00
sam 307f2fc9e2 fix(messaging): refuse failed channels as enablement success 2026-10-09 02:22:19 +11:00
sam 455de3d799 fix(workflow): retry a transient read while awaiting a callee 2026-10-09 02:21:49 +11:00
sam 4e74ed37b4 fix(workflow): preserve a reference's type between steps 2026-10-09 02:10:43 +11:00
sam 7ef58293be feat(harness): serve a daemon-side setup session to the dashboard
The dashboard's setup terminal was a 503: it had a WebSocket route, an
authorizer and an xterm.js surface, but nothing owned a container. The
daemon owns containers, so the session contract lands there and the
dashboard consumes it over a new gRPC contract.

One session concept, two targets. A setup session is addressed by Kit
profile and is ephemeral; a run session (archie-core-qi91) is addressed by
task and attempt and is bounded by the task. This slice serves setup and
answers Unimplemented for run.

The contract, the authorisation, the lifetime and the egress phase are
settled in docs/prds/external-agent-harness.md, "Session contract". The
short version: the daemon serves harness.v1.HarnessService; the dashboard
authorizes the operator as an update on the secret and then dials the
daemon with the harness service token, required even on loopback because a
session is a shell holding an org's credential; the setup container runs
no worker, mounts no worktree, holds a keepalive, and carries an empty
container environment. Install hooks run at the install phase and the
session enters runtime before the shell, so a runtime-phase credential's
token endpoint is the one the proxy intercepts.

Capture is gated by the egress proxy on a granted service, and a setup
session has no task, so the daemon registers a session grant: the
session's proxy token resolves to the org's bindings for the Kit's
declared services. The real token never enters the container: the token
endpoint response is sentinelized and stored as the org's harness secret.

Every teardown path releases the pool slot, revokes the egress session and
the grant, removes the sandbox network, and removes the session's volumes.
The listener requires HARNESS_TOKEN and stops with a 10s grace before
Stop, so an open session cannot hold shutdown to the watchdog leash.
2026-10-09 01:38:41 +11:00
sam 27c9647a13 fix(container): reap orphaned processes in Kit containers 2026-10-09 01:37:57 +11:00
sam 89541eb3b0 chore(lint): drop nolint directives that suppress nothing
29 of 59 directives did nothing. Removing each one individually and
re-running the gate revealed no issue from any of them: 20 named gosec,
which is not enabled in .golangci.yml and never has been, plus 4 nilerr
in functions that return no error, 2 dupl where dupl reports nothing
repo-wide, and one each of contextcheck, nestif and unused. The funlen
name on settings.go:147 is gone for the same reason; its gocyclo and
cyclop names are both live.

The unused directive in webui/authorize.go sat on `var _ = slog.Default`,
a statement that only existed to keep the log/slog import alive. Both go.

The 30 remaining directives all suppress a real issue.
2026-10-08 23:29:19 +11:00
sam 3a3a714cc4 fix(modelcatalog): never serve a disabled provider from the ambient key
disableProvider already cleared a failed provider's credential, but that
made it indistinguishable from a provider nobody configured, so
usableProvider fell back to the catalog's ambient env var (e.g.
OPENAI_API_KEY) and sent turns with a key the operator did not choose.
Mark the provider Disabled when its key fails to resolve, refuse it in
usableProvider, and carry the flag through mergeProviders so a catalog
refresh cannot resurrect it.
2026-10-08 22:59:23 +11:00
sam b107e4452a fix(webui): keep the dashboard token out of the startup log
DashboardURL no longer takes a token, so the startup line cannot embed
?t=<token> in the printed URL. The log now names the token file instead.
2026-10-08 22:57:28 +11:00
sam 2b514cb526 feat(scheduling): refuse a job payload that names a dangerous command
A scheduled job runs unattended, so a payload that restarts the daemon,
formats a disk or powers the machine down is refused when the job is
created rather than when it fires. scheduling.RefuseDangerousJob is the
one rule every creation path reaches: the dashboard, any control-plane
client and the config seed all replace the schedules document through the
same validator, so a new token is added in one place.
2026-10-08 22:45:28 +11:00
sam 630c30de79 fix(archiemessaging): apply channel enablement live 2026-10-08 22:44:15 +11:00
sam a69b3455d3 refactor(tools): back generate_video with ai-sdk's minimax video provider
Delete the hand-rolled MiniMax HTTP/poll client; ai-sdk v0.1.43 already
ships provider/minimax implementing video.Provider with the same endpoints,
defaults and poll semantics. The tool now takes a video.Provider and the
composition root builds it, matching the transcription and embedding
capabilities.
2026-10-08 22:42:49 +11:00
sam ddf8383784 fix(archiemessaging): recover a channel that fails on its own 2026-10-08 22:36:09 +11:00
sam c1bbeba18f fix(archied): wait for in-flight tasks on a single-cycle run
A drain pass no longer waits for the tasks it submitted, so -once (the
systemd timer and test invocation) returned while their containers were
still running and the process then tore them down. Wait for the pass's
tasks here, restoring what the flag means.
2026-10-08 22:31:07 +11:00
sam 1f99fb592e fix(daemon): let a run waiting on a callee release its capacity
A wait:true workflow.call run held its global slot and its repo's place for
the callee's whole life, and the drain pass waited for its own tasks, so a
callee queued by that pass was never claimed. With containers.max_concurrency
saturated, or a repo that does not allow concurrency, the wait could not end.

A drain pass now returns once the stream and the claim fallback are empty, and
shutdown waits for in-flight work once. Suspend/Resume give a run's slot and
repo place back while it waits on a callee, driven by the workflow call events
on the daemon's bus; a hold records one slot and one place, so a second
re-acquire gives back what it took.
2026-10-08 22:31:07 +11:00
sam aae01431b4 refactor(workflow): keep the call key change inside the lint gate
golangci-lint failed on the change and had not before: gofumpt wants
StartCall's same-typed parameters grouped, and runWorkflowCall reached a
cyclomatic complexity of 16. The unwaited call's early close moves into
closeUnwaitedCall on *TaskContext -- it needs only the call step, the
callee and the settings -- which leaves runWorkflowCall below the limit
with its behaviour unchanged.

The walk's id-less call case is deleted: ParseDefinition now refuses that
definition before validateWorkflowCalls runs, so it passed for the wrong
reason. The refusal is pinned where it happens instead, by the compile
case in TestWorkflowCallKeyIsTheDeclaredPath, now asserting the whole
message so it still names the step it refused.
2026-10-08 22:31:07 +11:00
sam 3c99f69fee fix(workflow): key a call on its callee, and free a terminal child's key
A call's key was the step's declared path alone, so one task running a
second definition at a colliding path -- workflow.handoff, human.approve
and a definition re-pin all change it -- was handed the first callee's
child, waited on the wrong workflow and published its outputs. The key
now carries the callee too, while the step row and the call's events stay
named and staged for the call site, so a rail pairing an event with its
step still attaches them.

A child that reached dead or closed_wont_do can never do the work and
holds no result the caller can use, so it leaves the call key's index and
frees the key for the caller's retry to start a fresh child. parked and
rejected keep theirs: a parked child is recoverable by retrying it, and a
rejected child ran.

The id a call step must declare is now refused where the definition is
compiled, which is also the path a pinned definition takes, so an id that
spells a step type is accepted again and no stage needs a heuristic.
2026-10-08 22:31:07 +11:00
sam b7fd13923a fix(workflow): key a workflow.call on its declared step path
A call site had no durable identity, so any retry of it -- a lost
EnqueueCallTask reply, a failed call-step record, a requeued attempt that
re-runs the stage -- started a second child and its work happened twice.

The call step's declared path within the run is that identity: wrapStep
sets it (branch-qualified, restored on return), a workflow.call step must
declare an id so two call sites cannot share one, and the key crosses
EnqueueCallTaskRequest.call_key into tasks.call_key, which EnqueueCallTask
upserts on so the retry returns the child already started.
2026-10-08 22:31:07 +11:00
sam 0a47a95cc3 fix(agentexec): enforce capture tool result schemas 2026-10-08 22:30:12 +11:00
sam ef8d9af469 fix(workflow): apply step retry and on_failure to command.run 2026-10-08 22:22:13 +11:00
sam 549f36a1f2 fix(ui): stop the command palette scrolling sideways 2026-10-08 21:58:22 +11:00
sam 187e6e81ee fix(ui): anchor the command palette and stop scroll stealing its highlight 2026-10-08 21:57:20 +11:00
sam e60ad551f0 fix(webhook): authorize delivery against the source's org 2026-10-08 21:43:28 +11:00
sam a0617ad437 test(postgres): derive the org fixture source from its insert 2026-10-08 21:40:24 +11:00
sam 411736fdaa fix(captures): keep count-retention inside the inserted capture's org 2026-10-08 21:27:54 +11:00
sam bea6567933 fix(captures): scope the capture list to the acting org 2026-10-08 21:26:07 +11:00
sam d6b693a9b2 fix(sources): store the acting org on source creation 2026-10-08 21:25:49 +11:00
sam b8bedf4d7b fix(capture): inherit source org on ingestion 2026-10-08 21:02:21 +11:00
sam 5c53dfb91b fix(workflow): resolve definitions in the owning org 2026-10-08 20:41:08 +11:00
sam e0cbbc0895 chore(release): v1.51.1
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-08 01:42:29 +11:00
sam 40dd546564 fix(release): a dependency bump is releasable for every binary 2026-10-08 01:42:27 +11:00
sam b89d8cbab2 fix(deps): ai-sdk v0.1.43 sends gpt-6 tool calls through the Responses API 2026-10-08 01:41:44 +11:00
sam 4b62912343 chore(release): v1.51.0
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-08 01:37:28 +11:00
sam 8a8c8cf7e9 feat(chat): the dashboard chat records voice, downloads files a turn sends, and offers /model and /personality choices 2026-10-08 01:37:21 +11:00
sam 201529414b fix(gateway): providers resolve again when a secret engine starts after boot 2026-10-08 01:34:55 +11:00
sam 3da1f918a4 chore(release): v1.50.1
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-08 00:40:03 +11:00
sam f8d8e5b1e2 fix(update): the watchdog logs the health report it rolled back on 2026-10-08 00:39:57 +11:00
sam 826d4849df chore(release): v1.50.0
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
deploy / gate (push) Canceled after 33s
2026-10-08 00:25:35 +11:00
sam 0a18bfe21b feat(chat): the dashboard chat answers tool approvals and questions, and /update offers Install 2026-10-08 00:25:23 +11:00
sam 5976b94fbc fix(gatewayrpc): carry tool approvals and ask_user questions between a channel and the gateway turn 2026-10-08 00:22:37 +11:00
sam 698548d6ef chore(release): v1.49.4
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-07 23:58:52 +11:00
sam d0584279e5 fix(ui): tabbed Status and Extensions, one Add MCP server, repos page renders without repos, save bar rests at the viewport edge 2026-10-07 23:46:34 +11:00
sam 62f248fd2c fix(ui): the Versions table's status column ends at the card's edge 2026-10-07 23:35:12 +11:00
sam 8f2750a266 chore(release): v1.49.3
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
2026-10-07 23:34:21 +11:00
sam 2ea7f7cd38 fix(ui): the release channel control sits at the Updates card's edge, not mid-row 2026-10-07 23:34:13 +11:00
sam 75b3810856 fix(archieui): the Versions card compares against the running daemon, so an available update can be installed 2026-10-07 23:29:03 +11:00
sam 30419532ba chore(release): v1.49.2
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
deploy / gate (push) Canceled after 1m51s
2026-10-07 23:18:11 +11:00
sam fff67838e0 fix(releaseupdate): read GitHub releases directly; drop go-selfupdate and its openpgp dependency 2026-10-07 23:18:03 +11:00
sam 15ebe24927 chore(release): v1.49.1
deploy / gate (push) Failing after 3m43s
deploy / archied-image (push) Skipped
deploy / agent-image (push) Skipped
deploy / dist-zip (push) Skipped
2026-10-07 23:10:38 +11:00
sam b2c1629925 fix(ui): an expired session reloads to the sign-in page instead of showing a banner 2026-10-07 23:05:52 +11:00
sam 0e8fcf70a0 chore(release): v1.49.0
deploy / gate (push) Failing after 5m18s
deploy / archied-image (push) Skipped
deploy / agent-image (push) Skipped
deploy / dist-zip (push) Skipped
2026-10-07 22:17:00 +11:00
sam 4673b435b7 fix(extension): confine egress only for packages that declare egress hosts 2026-10-07 22:15:06 +11:00
sam 1e062fd7f0 fix(releaseupdate): reuse a found release for ten minutes to stay under GitHub's unauthenticated rate limit 2026-10-07 01:17:21 +11:00
sam 74a3ec925c feat(releaseupdate): archied update check finds releases through go-selfupdate; archie-update-check is deleted 2026-10-07 01:12:11 +11:00
sam 4944ec8473 chore(release): v1.48.1
deploy / gate (push) Failing after 3m28s
deploy / archied-image (push) Skipped
deploy / agent-image (push) Skipped
deploy / dist-zip (push) Skipped
2026-10-07 01:00:30 +11:00
sam 204b65a0e3 fix(controlplane): update-settings selects a release channel; services run only the installed update scripts 2026-10-07 01:00:28 +11:00
sam cbedb8a504 chore(release): v1.48.0
deploy / gate (push) Failing after 5m4s
deploy / archied-image (push) Skipped
deploy / agent-image (push) Skipped
deploy / dist-zip (push) Skipped
2026-10-07 00:57:07 +11:00
sam a8dbab0cff feat(controlplane): update commands are an update-settings kind edited on Status, defaulting to the installed scripts 2026-10-07 00:57:01 +11:00
sam 789ff73fe9 fix(scripts): archie-update-install loads the archie env file so a shell run can back up the bundled database 2026-10-07 00:53:23 +11:00
sam 5176938e95 ci(deploy): build the archied and agent images in parallel so the release zip waits only on the agent image 2026-10-07 00:35:13 +11:00
340 changed files with 14623 additions and 10817 deletions

No files matched your search

+43 -12
View File
@@ -66,7 +66,7 @@ jobs:
- name: Install quality tools
run: |
go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.13.2
go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$(cat .golangci-version)
go install mvdan.cc/gofumpt@v0.11.0
go install github.com/go-task/task/v3/cmd/task@v3.48.0
@@ -170,9 +170,7 @@ jobs:
echo "This commit is not a release, so the images are published as :edge and sha-<sha>; :latest keeps naming the newest release."
fi
build-and-push:
outputs:
agent_digest: ${{ steps.agent.outputs.digest }}
archied-image:
needs: gate
runs-on: ubuntu-latest
steps:
@@ -264,6 +262,43 @@ jobs:
cache-from: type=gha,scope=archied
cache-to: type=gha,mode=max,scope=archied
agent-image:
outputs:
agent_digest: ${{ steps.agent.outputs.digest }}
needs: gate
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
# Annotated tags: dereference to the commit, as archied-image does.
- name: Resolve the commit this ref points at
id: commit
run: |
sha="$(git rev-parse "${GITHUB_REF}^{commit}")"
echo "sha=$sha" >> "$GITHUB_OUTPUT"
echo "short=${sha:0:12}" >> "$GITHUB_OUTPUT"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to GHCR
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
- name: Resolve runtime version
id: versions
run: |
RUNTIME_TAG="$(git tag --points-at HEAD --list 'archie/v*' | sort -V | tail -1)"
RELEASE_TAG="$(git tag --points-at HEAD --list 'v*' | sort -V | tail -1)"
RUNTIME_VERSION="${RUNTIME_TAG#archie/}"
[ -n "$RUNTIME_VERSION" ] || RUNTIME_VERSION="${RELEASE_TAG#v}"
echo "runtime_tag=$RUNTIME_TAG" >> "$GITHUB_OUTPUT"
echo "release_tag=$RELEASE_TAG" >> "$GITHUB_OUTPUT"
echo "runtime=$RUNTIME_VERSION" >> "$GITHUB_OUTPUT"
- name: Build and push archie-agent
id: agent
if: steps.versions.outputs.runtime_tag != '' || steps.versions.outputs.release_tag != ''
@@ -283,14 +318,10 @@ jobs:
# Packages every host-run binary (archied, archie-gateway,
# archie-state-store, archie-ui, archie-messaging) as a
# linux/amd64 zip for a
# non-Docker install. Independent of build-and-push: it never touches the
# archied/archie-agent image jobs, and a gateway-only release (no
# archie/v* tag) still produces this zip -- it's keyed to the archied tag
# alone. archie-agent is deliberately excluded: it only ever runs inside
# the Docker image build-and-push already publishes, never as a host
# binary (archie-core-8cda.4.7.1).
# non-Docker install. It waits only for the agent image, whose digest
# release.json pins; archie-agent itself never ships as a host binary.
dist-zip:
needs: [gate, build-and-push]
needs: [gate, agent-image]
# Not gated on github.ref. A release is pushed as main plus both tags at
# one SHA, the concurrency group below collapses those into a single run,
# and the survivor is whichever ref happened to start last -- for
@@ -352,7 +383,7 @@ jobs:
done
cp deployments/INSTRUCTIONS.md CHANGELOG.md "$name/"
cp scripts/archie-update-* "$name/"
digest='${{ needs.build-and-push.outputs.agent_digest }}'
digest='${{ needs.agent-image.outputs.agent_digest }}'
test -n "$digest"
jq --arg image "ghcr.io/samcharles93/archie-agent@$digest" '. + {agent_image: $image}' release.json > "$name/release.json"
# README.md doesn't exist yet in this repo; include it once it does
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
- name: Install quality tools
run: |
go install github.com/go-task/task/v3/cmd/task@v3.48.0
go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.13.2
go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$(cat .golangci-version)
go install mvdan.cc/gofumpt@v0.11.0
bash tools/proto.sh install
+1
View File
@@ -0,0 +1 @@
v2.14.0
+106
View File
@@ -5,6 +5,112 @@ release's per-component sections are labelled beneath its heading.
## [Unreleased]
## [1.52.0] - 2026-10-09
### archied
- The harness page's setup terminal opens a real session: it starts an ephemeral container from a Kit profile, runs the CLI's own login and captures the OAuth tokens at the provider's token endpoint. Point the dashboard at the daemon with `[services.harness].target` and a `HARNESS_TOKEN` and it stops answering 503; the terminal is created at your browser's size. Takeover of a live task is not in this release.
- A credential's token endpoint is refused when the run is not granted that service, instead of proxy-ing the provider's real tokens into the container.
- Settings → Profiles, Credential bindings and Playbooks are editable from the dashboard, and curator definitions can be changed and applied without a restart.
- Settings say whether each one applies live or needs a restart, show what is consuming resources, and report whether an apply landed.
- Status shows each service's dependency checks, the agent workers and the broker.
- The Logs page reads recent logs from every service, not just the daemon.
- Captures and webhook sources are scoped to the org that owns them: the list, retention and delivery authorization all act in the right org, and a workflow definition resolves in its own org.
- Chat settings are attributed to the signed-in user, and Settings → Tokens explains when a personal token cannot be created.
- A channel enabled after Archie started works immediately; a channel that fails on its own recovers; a failed channel is no longer reported as enabled, and stored channel settings no longer save duplicate or unnormalized values.
- The task table sorts, the command palette no longer scrolls sideways or loses its highlight, unknown routes render a not-found page, and configuration reload failures are shown instead of passing silently.
- A scheduled job whose payload names a dangerous command is refused.
- A disabled model provider is never served from the ambient API key.
- `generate_video` is backed by ai-sdk's MiniMax video provider.
- A dashboard token is no longer written to the startup log.
### archie-agent
- `workflow.call`: a call is keyed on its declared step path, its key is freed once the callee reaches a terminal state, and a caller waiting on a callee releases its capacity instead of holding it.
- `command.run` honours step retry and `on_failure`.
- A value passed between steps keeps its declared type, and optional result booleans are enforced only when the workflow declares them.
- Capture tools have their results checked against the declared schema.
- A single-cycle run waits for its in-flight tasks before exiting, and identity tasks inherit live settings.
- Kit containers run Docker's init as PID 1, so orphaned processes are reaped instead of accumulating inside a long task.
## [1.51.1] - 2026-10-08
### archied
- Chat with OpenAI's gpt-6 models (gpt-6-luna and the rest) works with tools; a turn failed with "Function tools with reasoning_effort are not supported".
## [1.51.0] - 2026-10-08
### archied
- A secret engine enabled after Archie started (Bitwarden, for example) now supplies its keys straight away; providers that depended on it no longer stay disabled until a restart.
- The dashboard chat has a microphone button: record a voice message and it is transcribed like a Telegram voice note. It shows only when a `transcription` model role is set under Settings → Models.
- A file Archie sends in the dashboard chat is a download link for an hour, instead of a note that it can't be delivered.
- Typing `/model` or `/personality` on its own in the dashboard chat offers the choices as buttons.
## [1.50.1] - 2026-10-08
### archied
- When an update is rolled back, the log names the health check that failed instead of only "unhealthy".
## [1.50.0] - 2026-10-08
### archied
- Tools that need your approval ask for it again, and Archie can ask you a question or offer choices mid-reply. Since the services were split these never reached Telegram or the dashboard: approval-gated tools failed and questions were never asked.
- The dashboard chat shows approvals as Approve / Always this session / Deny, choices as buttons, and questions with a reply field.
- `/update` in the dashboard chat lists what's available with Install and Not now.
- Settings → Status → Versions installs the update it shows; its Install button sent an empty request.
## [1.49.4] - 2026-10-07
### archied
- Settings → Status and Settings → Extensions are split into tabs instead of one long page; the Versions table's status column ends at the card's edge.
- Tools & MCP has one "Add MCP server" button; the transport is chosen in the form, and an unnamed new server is not an unsaved change.
- Settings → Repositories shows the add field when no repositories are configured, instead of a blank page.
- The unsaved-changes bar always rests at the bottom of the window.
## [1.49.3] - 2026-10-07
### archied
- Settings → Status → Versions shows the Gateway as "Update available" with an Install button when a newer release exists; it read "Unknown" with no way to install. The Runtime row stays "Unknown" until an agent task has run.
- The release channel buttons sit at the Updates card's right edge instead of mid-row.
## [1.49.2] - 2026-10-07
### archied
- Carries the 1.49.0 and 1.49.1 changes, whose builds were never published. The update check reads GitHub releases directly and still offers only releases with a verifiable linux-amd64 zip.
## [1.49.1] - 2026-10-07
### archied
- When the dashboard session expires, the page returns to the sign-in screen instead of showing errors on every page until a manual refresh.
## [1.49.0] - 2026-10-07
### archied
- Extensions that declare no egress hosts (vault, email, gitea) reach the network again; only packages that declare their hosts are confined to them. The Accept dialog shows "network: unrestricted" for the rest.
- The update check finds releases itself, and reuses a found release for ten minutes so repeated checks stay under GitHub's rate limit.
## [1.48.1] - 2026-10-07
### archied
- **Settings → Status → Updates** now chooses only the release channel (stable, prereleases, or a pinned version). Update checks and installs always run the `archie-update-*` scripts installed beside the binaries; the dashboard can no longer set the command that runs.
## [1.48.0] - 2026-10-07
### archied
- **Settings → Status → Updates** edits the update check and install commands. They default to the `archie-update-*` scripts installed beside the binaries, so a zip install shows **Install** on the Versions card with nothing configured. The `update_check_command` and `update_install_command` keys under `[chat.telegram]` are no longer read; remove them from `config.toml`.
- `archie-update-install` run from a shell now reads `~/.config/archie/env`, so the pre-update backup of the bundled database no longer fails with `no password supplied`. Run it with `--auto` to install the newest release.
## [1.47.3] - 2026-10-07
### archied
+29
View File
@@ -9,6 +9,35 @@ tasks:
cmds:
- task --list
setup:
desc: Install the pinned tooling and dependencies a checkout needs
cmds:
# Report what this task cannot install for you, so a missing Go, Node or
# Docker fails here by name rather than somewhere inside the gate.
- |
set -eu
missing=
for tool in go task node npm docker; do
command -v "$tool" >/dev/null 2>&1 || missing="$missing $tool"
done
if [ -n "$missing" ]; then
echo "warning: not installed (AGENTS.md, Build and gate):$missing" >&2
fi
# The pin is one file, read here and by .github/workflows, so the linter
# running locally and the one running the gate cannot drift apart.
- |
set -eu
want=$(cat {{.ROOT_DIR}}/.golangci-version)
have=$(golangci-lint --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1 || true)
if [ "$have" = "${want#v}" ]; then
echo "golangci-lint $want already installed"
else
echo "installing golangci-lint $want (have: ${have:-none})"
GOBIN="$(go env GOPATH)/bin" go install "github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$want"
fi
- task: proto:install
- task: ui:deps
fmt:
desc: Fix and format Go source with the gate's configured formatters
cmds:
+4
View File
@@ -252,6 +252,10 @@ _runs without a repository._
| `wait` | bool | Wait for the child run to finish before continuing. |
| `workflow` | text | The workflow to start as a child run. |
The step must declare an `id`: the step's id is the key of the call it starts,
so a retry of that call site returns the child already started instead of
starting a second one.
### `workflow.finish`
_runs without a repository._
+14
View File
@@ -19,6 +19,17 @@ import (
type workerRunner func(context.Context, agentworker.Settings, *slog.Logger) error
// runHold blocks until signalled and exits 0. It is the keepalive for a Kit
// container that runs no worker: the agent binary is archie's own and is
// mounted into every Kit, so the container needs no binary the image may not
// ship, and holds no environment of its own.
func runHold() int {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
<-ctx.Done()
return 0
}
func main() {
os.Exit(run())
}
@@ -28,6 +39,9 @@ func run() int {
}
func runCommand(args []string, getenv func(string) string, stderr io.Writer, runWorker workerRunner) int {
if len(args) > 0 && args[0] == "hold" {
return runHold()
}
if len(args) > 0 && args[0] == "relay" {
return runRelay(args[1:], stderr, agentworker.RunRelay)
}
+41
View File
@@ -0,0 +1,41 @@
// This file enforces the dashboard's process boundary
// (docs/prds/external-agent-harness.md, "Session contract", Transport): the
// dashboard must not link the container or daemon internals, because only the
// daemon owns containers. Go imports are atomic, so a single reference
// relinks a runtime, and the boundary holds only while something re-measures
// it. archie-core-h3s0 names this check; it was purged with the test suite and
// is restored here.
package main
import (
"context"
"os/exec"
"strings"
"testing"
"time"
)
func TestDashboardDoesNotLinkContainerRuntime(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
out, err := exec.CommandContext(ctx, "go", "list", "-deps", ".").Output()
if err != nil {
t.Fatalf("go list -deps .: %v", err)
}
linked := map[string]bool{}
for line := range strings.SplitSeq(strings.TrimSpace(string(out)), "\n") {
linked[line] = true
}
for _, pkg := range []string{
"github.com/samcharles93/archie-core/internal/container",
"github.com/samcharles93/archie-core/internal/app/archied",
"github.com/samcharles93/archie-core/internal/agentexec",
} {
t.Run(pkg, func(t *testing.T) {
if linked[pkg] {
t.Errorf("cmd/archie-ui links %s: only the daemon owns containers", pkg)
}
})
}
}
+2
View File
@@ -38,6 +38,8 @@ func run() int {
flag.StringVar(&options.Gateway.Token, "gateway-token", "", "bearer token presented to archie-gateway")
flag.StringVar(&options.State.Target, "state-target", "", "archie-state-store gRPC address (defaults to [services.state].target)")
flag.StringVar(&options.State.Token, "state-token", "", "bearer token presented to archie-state-store")
flag.StringVar(&options.Harness.Target, "harness-target", "", "archied harness session gRPC address (defaults to [services.harness].target); empty leaves the setup terminal unwired")
flag.StringVar(&options.Harness.Token, "harness-token", "", "bearer token presented to the daemon's harness contract")
flag.DurationVar(&options.DependencyTimeout, "dependency-timeout", 0, "per-dependency readiness probe timeout (default 5s)")
flag.DurationVar(&options.EventPollInterval, "event-poll-interval", 0, "how often the activity feed polls the State Store for new events (default 1s)")
// BoolFunc rather than BoolVar so an unset flag stays distinguishable
+2 -2
View File
@@ -16,8 +16,8 @@ func main() {
if archied.IsSetupArgs(args) {
os.Exit(archied.RunSetup(args[1:], os.Stdin, os.Stdout, os.Stderr))
}
if len(args) > 0 && args[0] == "select-release" {
os.Exit(archied.RunReleaseChannel(args[1:], os.Stdin, os.Stdout, os.Stderr))
if len(args) > 0 && args[0] == "update" {
os.Exit(archied.RunUpdate(args[1:], os.Stdout, os.Stderr))
}
if len(args) > 0 && args[0] == "update-image" {
os.Exit(archied.RunUpdateImage(args[1:], os.Stderr))
+15 -11
View File
@@ -17,9 +17,6 @@
# database_url = "postgres://archie:archie@127.0.0.1:5432/archie?sslmode=disable"
# skills_dir = "~/.local/share/archie/skills" # SKILL.md search path; defaults under work_dir when unset
# workflow_routing_file = "~/.local/share/archie/workflow-routing.yaml" # kind->workflow rebinding (bug/feature)
# workflow_labels_file = "~/.local/share/archie/workflow-labels.yaml" # arbitrary forge label -> workflow (EDA playbook slice 2)
# playbook_dirs = ["~/.local/share/archie/workflows"] # list of dirs of *.yaml binding files, merged with the two above (slice 3); a key bound twice across sources is a load failure
poll_interval = "60s"
label = "archie"
@@ -63,7 +60,8 @@ token = { engine = "env", key = "ARCHIE_GITHUB_TOKEN" }
# intake per identity is a pending migration, not a setting.
# [[identities]] runs more than one bot identity from a single daemon, each
# with its own forge/dispatch/models/providers/budgets/notify. Intake
# with its own forge account and repository membership. Runtime settings
# are shared and edited through the control plane. Intake
# settings are rejected per identity, and [forge].intake = "webhook"/"both"
# is rejected alongside [[identities]]. Minimal shape:
# [[identities]]
@@ -146,16 +144,9 @@ class = "ollama"
# "openrouter/openai/gpt-5.6-sol",
# ]
# Optional self-hosted update adapter. Both values are argv arrays (never
# shell snippets). The check command must write a releaseupdate.Snapshot JSON
# document to stdout; configure the installer as well to show Approve/Defer.
# [chat.telegram]
# The bot token from @BotFather, resolved through a configured secret engine.
# token = { engine = "bws", key = "TELEGRAM_BOT_TOKEN" }
# Run by archie-messaging, which serves /update, so these commands execute on
# whichever host runs that process.
# update_check_command = ["/usr/local/bin/archie-update-check"]
# update_install_command = ["/usr/local/bin/archie-update-install"]
# Email and the generic inbound webhook are archipelago channel extensions
# (channels/email, channels/webhook): install, accept and enable them from the
@@ -299,6 +290,19 @@ target = "127.0.0.1:9090" # REQUIRED. The State Store gRPC address arch
# # same remote store. See deployments/README.md for how to
# # start archie-state-store.
# The daemon's harness session contract: the dashboard's setup terminal opens
# a PTY into an ephemeral Kit container over it. Both ends are optional and
# the dashboard's setup terminal degrades to 503 without them: the daemon
# serves the contract only when target_token (or HARNESS_TOKEN) is set, and
# archie-ui dials it only when a target is set.
# [services.harness]
# target = "127.0.0.1:8587" # The address archie-ui dials. Empty leaves the
# # setup terminal unwired.
# target_token = "..." # REQUIRED, even on loopback: a session is a
# # shell in a container that holds an org credential.
# # The daemon reads the same value to authenticate its
# # clients. Leave unset to disable the listener.
# NATS is the sole autonomous workflow handoff. mode changes only where the
# broker runs; every workflow still runs in a managed task container:
# "embedded" – in-process nats-server (the default; no external server)
+5 -4
View File
@@ -92,14 +92,15 @@ an install (or an upgrade across v1.30.0) needs its unit as well -- see
`archie-update-install` downloads the approved release's Linux/amd64 archive
and verifies its `SHA256SUMS` entry before executing or installing it. Releases
must include `release.json`, all host binaries, and the three updater scripts.
must include `release.json`, all host binaries, and the two updater scripts.
Older releases without this contract cannot be installed through the artifact
path. `--from-source` (or `ARCHIE_UPDATE_SOURCE_BUILD=1`) explicitly clones and
builds the selected release instead.
The update check command accepts `--channel stable` (default), `--channel next`
(includes prereleases), or `--channel exact-pin --pin 1.2.3`. Set these arguments
in the control plane's channel settings alongside the existing check command.
`archied update check` finds the newest GitHub release with a linux-amd64 zip on
the channel chosen under Settings → Status → Updates: stable, next (includes
prereleases) or a pinned version. From a shell it reads
`ARCHIE_UPDATE_RELEASE_CHANNEL` and `ARCHIE_UPDATE_PIN`, or `-channel` and `-pin`.
The notification destination remains separate from the release channel.
For scheduled, unattended runs, `archie-update-install --auto` discovers the
+24
View File
@@ -65,7 +65,19 @@
"/news/archie/1.47.1/": "/news/1.47.1/",
"/news/archie/1.47.2/": "/news/1.47.2/",
"/news/archie/1.47.3/": "/news/1.47.3/",
"/news/archie/1.48.0/": "/news/1.48.0/",
"/news/archie/1.48.1/": "/news/1.48.1/",
"/news/archie/1.49.0/": "/news/1.49.0/",
"/news/archie/1.49.1/": "/news/1.49.1/",
"/news/archie/1.49.2/": "/news/1.49.2/",
"/news/archie/1.49.3/": "/news/1.49.3/",
"/news/archie/1.49.4/": "/news/1.49.4/",
"/news/archie/1.5.0/": "/news/1.5.0/",
"/news/archie/1.50.0/": "/news/1.50.0/",
"/news/archie/1.50.1/": "/news/1.50.1/",
"/news/archie/1.51.0/": "/news/1.51.0/",
"/news/archie/1.51.1/": "/news/1.51.1/",
"/news/archie/1.52.0/": "/news/1.52.0/",
"/news/archie/1.6.0/": "/news/1.6.0/",
"/news/archie/1.7.0/": "/news/1.7.0/",
"/news/archie/1.8.0/": "/news/1.8.0/",
@@ -147,7 +159,19 @@
"/news/archied/1.47.1/": "/news/1.47.1/",
"/news/archied/1.47.2/": "/news/1.47.2/",
"/news/archied/1.47.3/": "/news/1.47.3/",
"/news/archied/1.48.0/": "/news/1.48.0/",
"/news/archied/1.48.1/": "/news/1.48.1/",
"/news/archied/1.49.0/": "/news/1.49.0/",
"/news/archied/1.49.1/": "/news/1.49.1/",
"/news/archied/1.49.2/": "/news/1.49.2/",
"/news/archied/1.49.3/": "/news/1.49.3/",
"/news/archied/1.49.4/": "/news/1.49.4/",
"/news/archied/1.5.0/": "/news/1.5.0/",
"/news/archied/1.50.0/": "/news/1.50.0/",
"/news/archied/1.50.1/": "/news/1.50.1/",
"/news/archied/1.51.0/": "/news/1.51.0/",
"/news/archied/1.51.1/": "/news/1.51.1/",
"/news/archied/1.52.0/": "/news/1.52.0/",
"/news/archied/1.6.0/": "/news/1.6.0/",
"/news/archied/1.7.0/": "/news/1.7.0/",
"/news/archied/1.8.0/": "/news/1.8.0/",
+60
View File
@@ -1,4 +1,64 @@
[
{
"version": "1.52.0",
"date": "2026-10-09",
"body": "### archied\n\n- The harness page's setup terminal opens a real session: it starts an ephemeral container from a Kit profile, runs the CLI's own login and captures the OAuth tokens at the provider's token endpoint. Point the dashboard at the daemon with `[services.harness].target` and a `HARNESS_TOKEN` and it stops answering 503; the terminal is created at your browser's size. Takeover of a live task is not in this release.\n- A credential's token endpoint is refused when the run is not granted that service, instead of proxy-ing the provider's real tokens into the container.\n- Settings → Profiles, Credential bindings and Playbooks are editable from the dashboard, and curator definitions can be changed and applied without a restart.\n- Settings say whether each one applies live or needs a restart, show what is consuming resources, and report whether an apply landed.\n- Status shows each service's dependency checks, the agent workers and the broker.\n- The Logs page reads recent logs from every service, not just the daemon.\n- Captures and webhook sources are scoped to the org that owns them: the list, retention and delivery authorization all act in the right org, and a workflow definition resolves in its own org.\n- Chat settings are attributed to the signed-in user, and Settings → Tokens explains when a personal token cannot be created.\n- A channel enabled after Archie started works immediately; a channel that fails on its own recovers; a failed channel is no longer reported as enabled, and stored channel settings no longer save duplicate or unnormalized values.\n- The task table sorts, the command palette no longer scrolls sideways or loses its highlight, unknown routes render a not-found page, and configuration reload failures are shown instead of passing silently.\n- A scheduled job whose payload names a dangerous command is refused.\n- A disabled model provider is never served from the ambient API key.\n- `generate_video` is backed by ai-sdk's MiniMax video provider.\n- A dashboard token is no longer written to the startup log.\n\n### archie-agent\n\n- `workflow.call`: a call is keyed on its declared step path, its key is freed once the callee reaches a terminal state, and a caller waiting on a callee releases its capacity instead of holding it.\n- `command.run` honours step retry and `on_failure`.\n- A value passed between steps keeps its declared type, and optional result booleans are enforced only when the workflow declares them.\n- Capture tools have their results checked against the declared schema.\n- A single-cycle run waits for its in-flight tasks before exiting, and identity tasks inherit live settings.\n- Kit containers run Docker's init as PID 1, so orphaned processes are reaped instead of accumulating inside a long task."
},
{
"version": "1.51.1",
"date": "2026-10-08",
"body": "### archied\n\n- Chat with OpenAI's gpt-6 models (gpt-6-luna and the rest) works with tools; a turn failed with \"Function tools with reasoning_effort are not supported\"."
},
{
"version": "1.51.0",
"date": "2026-10-08",
"body": "### archied\n\n- A secret engine enabled after Archie started (Bitwarden, for example) now supplies its keys straight away; providers that depended on it no longer stay disabled until a restart.\n- The dashboard chat has a microphone button: record a voice message and it is transcribed like a Telegram voice note. It shows only when a `transcription` model role is set under Settings → Models.\n- A file Archie sends in the dashboard chat is a download link for an hour, instead of a note that it can't be delivered.\n- Typing `/model` or `/personality` on its own in the dashboard chat offers the choices as buttons."
},
{
"version": "1.50.1",
"date": "2026-10-08",
"body": "### archied\n\n- When an update is rolled back, the log names the health check that failed instead of only \"unhealthy\"."
},
{
"version": "1.50.0",
"date": "2026-10-08",
"body": "### archied\n\n- Tools that need your approval ask for it again, and Archie can ask you a question or offer choices mid-reply. Since the services were split these never reached Telegram or the dashboard: approval-gated tools failed and questions were never asked.\n- The dashboard chat shows approvals as Approve / Always this session / Deny, choices as buttons, and questions with a reply field.\n- `/update` in the dashboard chat lists what's available with Install and Not now.\n- Settings → Status → Versions installs the update it shows; its Install button sent an empty request."
},
{
"version": "1.49.4",
"date": "2026-10-07",
"body": "### archied\n\n- Settings → Status and Settings → Extensions are split into tabs instead of one long page; the Versions table's status column ends at the card's edge.\n- Tools & MCP has one \"Add MCP server\" button; the transport is chosen in the form, and an unnamed new server is not an unsaved change.\n- Settings → Repositories shows the add field when no repositories are configured, instead of a blank page.\n- The unsaved-changes bar always rests at the bottom of the window."
},
{
"version": "1.49.3",
"date": "2026-10-07",
"body": "### archied\n\n- Settings → Status → Versions shows the Gateway as \"Update available\" with an Install button when a newer release exists; it read \"Unknown\" with no way to install. The Runtime row stays \"Unknown\" until an agent task has run.\n- The release channel buttons sit at the Updates card's right edge instead of mid-row."
},
{
"version": "1.49.2",
"date": "2026-10-07",
"body": "### archied\n\n- Carries the 1.49.0 and 1.49.1 changes, whose builds were never published. The update check reads GitHub releases directly and still offers only releases with a verifiable linux-amd64 zip."
},
{
"version": "1.49.1",
"date": "2026-10-07",
"body": "### archied\n\n- When the dashboard session expires, the page returns to the sign-in screen instead of showing errors on every page until a manual refresh."
},
{
"version": "1.49.0",
"date": "2026-10-07",
"body": "### archied\n\n- Extensions that declare no egress hosts (vault, email, gitea) reach the network again; only packages that declare their hosts are confined to them. The Accept dialog shows \"network: unrestricted\" for the rest.\n- The update check finds releases itself, and reuses a found release for ten minutes so repeated checks stay under GitHub's rate limit."
},
{
"version": "1.48.1",
"date": "2026-10-07",
"body": "### archied\n\n- **Settings → Status → Updates** now chooses only the release channel (stable, prereleases, or a pinned version). Update checks and installs always run the `archie-update-*` scripts installed beside the binaries; the dashboard can no longer set the command that runs."
},
{
"version": "1.48.0",
"date": "2026-10-07",
"body": "### archied\n\n- **Settings → Status → Updates** edits the update check and install commands. They default to the `archie-update-*` scripts installed beside the binaries, so a zip install shows **Install** on the Versions card with nothing configured. The `update_check_command` and `update_install_command` keys under `[chat.telegram]` are no longer read; remove them from `config.toml`.\n- `archie-update-install` run from a shell now reads `~/.config/archie/env`, so the pre-update backup of the bundled database no longer fails with `no password supplied`. Run it with `--auto` to install the newest release."
},
{
"version": "1.47.3",
"date": "2026-10-07",
+354
View File
@@ -0,0 +1,354 @@
# External agent harnesses
**Status:** Approved
**Scope:** the harness model and, in "Session contract", the duplex session
that the setup terminal and the run-scoped live view both use. Code cites
this file for both (`ui/src/harness/SetupTerminal.vue`,
`internal/infrastructure/postgres/queries/harness_secrets.sql`).
## Decision
A workflow stage can run on an external coding-agent CLI instead of the
built-in agent loop: Claude Code, Codex, Pi, OMP, GitHub Copilot CLI, or any
other CLI packaged as a Docker Sandbox Kit v3. Each authenticates with an API
key or the operator's subscription login, and can carry the operator's own
skills, plugins, subagents and instructions.
Two boundaries hold regardless of what the harness does:
- **The real credential never enters the container.** The container sees
sentinel values; archie's egress proxy substitutes the real credential on
outbound requests and performs OAuth refresh on the host.
- **Archie's workflow guarantees are enforced outside the harness.** Git,
protected paths, read-only stages, gates and budgets are checked by archie
around the process and against the worktree afterwards.
## Kits
A harness is packaged as a Kit v3 image: an OCI image whose
`vnd.docker.sandbox.kit.descriptor` manifest annotation declares what the
harness needs. Archie reads descriptors with the published
`github.com/docker/sandbox-kit-spec/v3/spec` package and does not maintain
its own descriptor format. Published Kits run unmodified.
Archie is a partial Kit runtime. It implements these capability types:
| Type | Archie behaviour |
| ------------------- | ----------------------------------------------------------------------------------------- |
| `credential@1` | Proxy-managed only, described below. |
| `network-policy@1` | Enforced by the egress proxy. |
| `volume@1` | A Docker volume keyed by WorkflowExecution, deleted with it. |
| `lifecycle@1` | Install and startup hooks run on container create. |
| `agent-sessions@1` | The headless `prompt` and `resume` invocations for the stage. |
| `agent-skills@1` | The operator's skills store (`skills_dir`), mounted read-only whatever the Kit asks. |
| `agent-context@1` | The workload's profile file beside the workspace, indexing each Kit's guidance. |
| `resources@1` | Container CPU and memory limits. |
Every other type fails closed: a required entry refuses the launch, an
optional one is skipped and recorded on the step. `credential@1` with
`passthrough: true` and any credential for a forge service are refused.
The operator's own setup (plugins, subagents, instructions, settings) is a
mixin Kit, published with the harness Kit as a Kit set: the set merges their
layers into one image. It is versioned image content, rewritten on every
create, never state a run can modify.
## Contract
A harness run is a second implementation of `agentexec.Runner`. It takes the
same `Request` and returns the same `Result`, so workflows, the daemon and
the State Store do not know which runner ran a stage.
`agent-sessions@1` supplies the invocation. Archie supplies, per CLI, an
output adapter that translates the CLI's JSON output stream into
`ToolCallReporter` calls, `Usage` and a session ID. A CLI with no adapter can
run stages with no capture tools, reporting no usage; its token budget falls
back to wall clock.
Workflow definition validation rejects, at load time, a stage whose needs the
selected harness cannot meet: capture tools without an adapter, or a gate
retry without a `resume` verb.
## Selection
A harness is an agent profile. `docs/prds/event-automation.md` defines a
profile as the execution environment a workflow selects; a harness profile
names one Kit pinned by digest, a workload Kit or a published Kit set, in
place of a container image. Archie runs that Kit's image and composes no Kits
itself; a bare mixin is refused. Everything else about profiles
holds: a workflow names its profile, a stage may name a different one, and a
profile holds no secrets.
`archie-agent` builds the stage's runner from the stage's profile: the
ai-sdk agent loop for an image profile, the harness runner for a Kit
profile. Image profiles, and every workflow already written against them,
are unchanged. A Kit profile named by no workflow is a validation error.
A stage naming a different profile than its workflow's is a separate run,
not a mid-container runner switch: a Kit task runs in one container
provisioned for one profile (Container layout), so a stage that needs a
different one is its own callee, started the way `workflow.call` already
starts one (`docs/prds/event-automation.md` "Workflows call workflows"),
with its own agent profile resolved off its own task row. This needs no
mechanism beyond `workflow.call` plus per-task profile resolution
(`daemon.pinTaskProfile`), both of which already treat every task's
profile independently of any caller's -- see
`TestWorkflowCallerConformance` (staterpc) for the callee's independent
`Workflow` field and `TestWorkflowCallGivesACalleeItsOwnProfile` (daemon)
for the Kit-vs-image runner selection that independence produces.
Profiles are org resources in the State Store, so a Kit profile belongs to
one org, is granted to its workspaces like any profile, and applies without
a restart.
## Organisations
`docs/prds/orgs-and-access.md` is the boundary; a harness adds nothing that
crosses it.
- A Kit profile, its credential bindings and its OAuth tokens belong to one
org. No record is shared between orgs.
- A harness run is a run like any other: it acts as the workflow's identity,
under the run credential issued at dispatch.
- The egress proxy injects a credential only when the run credential carries
the secret the binding names. A Kit that declares a service the identity is
not granted gets no credential: a required one refuses the launch, an
optional one is skipped.
- A Kit's network policy is what the Kit asks for; the identity's network
grants are what the org allows. The proxy enforces the intersection.
- Concurrency slots are counted per binding within its org.
## Credentials
A credential binding maps a Kit's `credential@1` `service` name to an org
secret, which the org grants to identities like any secret. The secret holds
one of:
- **API key:** a value in the org's secret store.
- **OAuth:** tokens captured by the setup terminal, stored in the State Store
under the at-rest envelope of `binding-secret-encryption.md` with its own
domain separator.
The egress proxy resolves the binding for each request against the run
credential. The container gets the Kit's sentinel in the named variable or
credential file. OAuth refresh happens at the proxy, which intercepts the
Kit's `tokenEndpoint`, so a refreshed token is written to the State Store and
never returned to the container.
Runs sharing an OAuth binding share its subscription's rate limit.
`max_concurrent` on the binding (default 1) caps its simultaneous runs. A
stage waiting for a slot is `pending`, not `running`.
## Session contract
A **session** is a duplex PTY into a Kit container the daemon owns. One
concept covers both targets; only the address, the lifetime and the
established egress phase differ.
| | Setup session | Run session (deferred) |
| --- | --- | --- |
| Address | `{org, profile}` | `{org, task, attempt}` |
| Container | ephemeral, from the profile's Kit | the task's running container |
| Purpose | run the CLI's own login so the proxy captures the tokens | watch and optionally take over |
| Lifetime | bounded by the session TTL; the container is removed on close | bounded by the task |
- **Transport.** The daemon serves `harness.v1.HarnessService`, a bidi gRPC
stream, and the dashboard implements its `HarnessTerminal` interface
(`internal/webui`) over that client; the browser's WebSocket route is
unchanged. The contract lives on the daemon because the daemon owns
containers: the dashboard process must not link `internal/container`,
`internal/app/archied` or `internal/agentexec`.
- **Authorisation.** Opening a session is the `update` action on the secret,
decided by the dashboard's authorizer before the route runs
(`internal/webui/authorize.go`). The dashboard then dials the daemon with
the harness service token. The daemon does not re-authorise a human; it
trusts the asserted `{org, profile}` exactly as the task-action RPC trusts
an asserted actor. Because that authority is a shell in a container holding
an org's credential, the harness listener requires `HARNESS_TOKEN` even on
loopback, unlike the State Store's loopback convention.
- **Egress phase.** Token interception is filtered by phase
(`oauth.go`'s `rule.runtime == atRun`), and `EnterRuntime` is one-way. A
setup session therefore follows a run's lifecycle exactly: install hooks at
the install phase, `EnterRuntime` before the shell. A credential whose
token endpoint must be captured declares `phase: runtime`; a credential
capturing at install phase is captured by the install hooks that run before
the switch.
- **Credentials.** Capture is gated: the proxy intercepts a token endpoint
only for a service the session is granted (`Proxy.granted`). A setup session
has no task, so the daemon registers a session grant: its proxy token
resolves to `{org, the Kit's declared credential services the org binds}`.
That is the intersection the `update`-on-secret authorisation covers. The
real token never enters the container: the response is sentinelized and
stored as the org's harness secret.
- **Container discipline.** The setup container runs a keepalive entrypoint
with an empty `Config.Env`; the PTY is an exec as the Kit's harness user
with the harness env and nothing inherited from any worker. No worktree is
mounted. It is acquired through the container pool, so it carries the
`archie-daemon` label, the concurrency cap and the max-uptime reaper; the
session TTL is the smaller bound. Closing revokes the egress session,
removes the sandbox network and the Kit volumes, then releases the
container.
- **Kit unavailable.** The daemon serves the contract only when it has a
container pool and a Kit launcher; otherwise there is no listener, and a
client's refused connection is gRPC's `Unavailable`. The dashboard's
existing failure surface is the honest degrade; `/api/harness`'s `terminal`
field means "a client is configured", not "the daemon can run a Kit".
- **Run target.** Deferred (`archie-core-qi91`). `Open` answers `Unimplemented`
for it in the setup slice. A finished or absent attempt will be `NotFound`,
not `Unavailable`.
- **Audit.** Opening and closing a session and a successful capture are
recorded with the asserted org and profile.
Each session is the only interactive path into a harness container.
## Egress
Only Kit profiles run on the egress path. An image profile keeps the network
access its identity is granted, because workflows such as a network
investigator need raw DNS, ICMP and arbitrary TCP that no HTTP proxy
carries.
- **Sandbox network.** Each Kit run gets its own internal Docker bridge on
which the host holds no address. Host services, including those listening
on every interface, are unreachable from it by any address.
- **Relay.** One relay container, `archie-agent relay`, sits on the sandbox
networks under a fixed alias and on the host-reachable network. It
forwards three fixed ports, to the proxy, NATS and the State Store, and can
reach nothing else. It holds no secrets.
- **Proxy.** The proxy runs inside `archied`, listening on the host gateway
of the host-reachable network. It terminates TLS with a per-daemon CA
installed into the container at create, identifies the container by a
per-run proxy token, and applies that run's network policy and credential
inject rules. Upstream dials refuse loopback, and refuse internal addresses
unless the policy names the exact IP, so an allowed name cannot resolve
back into the host.
## Container layout
A Kit task runs in one container, from the Kit composition's image, on the
task's sandbox network. The worker and every harness invocation share it;
they are separated by user, not by container, so a run's cost scales with
the run and not with how many agents it fans out to.
- **Worker.** Archie mounts its own `archie-agent` binary read-only and runs
it as the container's entrypoint, as root. It holds the task's NATS and
State Store credentials in its own environment and in root-only files,
runs the workflow, and serves `archie-agent mcp`. The Kit image's
entrypoint is not PID 1.
- **Harness.** Every invocation runs as the Kit's user (the image's `User`,
default uid 1000) with an explicit environment: the Kit's variables, the
proxy environment and the credential sentinels, and nothing inherited from
the worker. That user cannot read the worker's environment or files, or
trace it. Sibling invocations in one run share the user and the worktree.
- **Hooks.** `lifecycle@1` install and startup hooks run as the user each
declares, before the worker accepts work.
- **Mounts.** The task worktree at the Kit's workspace directory, owned by the
Kit's user, and the read-only skills store. Nothing else from the host.
- **Network.** The relay forwards three fixed ports on the sandbox network:
the egress proxy, NATS and the State Store. The worker reaches NATS and the
State Store with its credentials; a harness can open those ports but holds
no credential either accepts.
- **Volumes.** `volume@1` paths persist for the life of the
WorkflowExecution, which covers session resume across gate iterations and
retries. They are not shared between executions.
## Archie tools inside the harness
Capture tools, which are how a stage returns structured results, are served
by `archie-agent mcp`, a stdio MCP server registered through the Kit's MCP
configuration or the adapter's flags. It serves the stage's `CaptureTools`
and the central tools the agent profile's `AllowTools` permits, never the
`workspace` toolset. The runner's `ToolLimits` bound its results; the
harness's own built-in tools are outside archie's limits.
## Enforcement around the harness
| Property | Enforcement |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Model never runs git | No forge credential can be bound. Archie records `HEAD` and every ref before the run; any change fails the step (`failed`, policy class). The workflow commits the stage's changes. |
| Protected paths | A diff touching a `Protection` path fails the step. |
| Read-only stage | Any worktree diff fails the step. |
| Gate | Archie runs the stage's gate after the harness exits. On failure it resumes the session with the gate output, up to the stage's iteration budget. |
| Budget | Archie kills the process on wall-clock timeout, or when streamed usage passes the token budget. |
| Cancellation | Cancelling the step kills the process group, then the container. |
| Egress | Only the proxy is reachable; the Kit's network policy decides the rest. |
Harness settings that deny git or protected paths are added where the CLI
supports them, to save wasted turns. They are never the enforcement.
## Execution tree
A harness run is one `agent` StepExecution. Subagents the harness spawns
internally are not recorded as children. Their usage is included in the
step's `tokens_used` when the adapter reports it.
## Agents never hold a harness
An Agent reaches a harness only by invoking a Workflow whose stage names it.
No Agent tool, chat tool or capability exposes a harness, its setup terminal
or its credential. The stage boundary is what carries the container, the git
and path checks, the gate and the budget; a harness outside a stage has none
of them.
## Out of scope
- Chat through a harness. The gateway chat loop stays on the built-in
runtime.
- `network-policy@2` HTTP method and path rules; a Kit requiring it is
refused.
- Recording the harness's internal subagents as StepExecutions.
- Cost in currency.
- Choosing a harness automatically per task.
## Plan
1. Descriptor resolution with the Kit spec package; capability support
table and fail-closed refusal.
2. Egress proxy: internal network, per-daemon CA, per-container token,
`network-policy@1`.
3. `credential@1`: API-key bindings, sentinel rendering, inject rules.
4. Harness `Runner`: container create with worktree, volumes, lifecycle
hooks and the mounted `archie-agent`; `agent-sessions@1` invocation;
worktree checks for git, protected paths and read-only; budget and
cancellation.
5. Claude Code output adapter; `archie-agent mcp`; gate-and-resume loop.
6. Kit profiles, runner selection from the stage's profile, and definition
validation.
7. OAuth bindings as org secrets: token-endpoint interception, encrypted
storage, per-binding concurrency; dashboard setup terminal.
8. Output adapters for Codex, Pi, OMP and GitHub Copilot CLI.
## Verification
- The same workflow definition completes on `builtin` and on the Claude Code
Kit with the same `Result` shape and captures.
- After `/login` in the setup terminal, the container's credential file holds
only sentinels, the State Store holds the encrypted token, and a task run
authenticates.
- A real API key or OAuth token never appears in the container environment,
filesystem, worktree, task log or event stream.
- A request to a host outside the Kit's runtime allow list fails, including a
direct connection that bypasses the proxy settings.
- A harness instructed to `git commit` fails its step, and the task branch is
unchanged.
- A harness edit to a protected test file fails the step in a TDD fix stage.
- A read-only feasibility stage that writes a file fails.
- A file a run writes into a Kit-managed path such as the CLI's settings file
is gone on the next execution.
- A Kit requiring an unsupported capability is refused at launch with the
capability named.
- A stage with a failing gate resumes the same session with the gate output,
then succeeds or exhausts its iterations.
- Two tasks sharing an OAuth binding with `max_concurrent = 1` run one at a
time; the second waits as `pending`.
- Stopping a run kills the harness process and removes the container.
- A harness invocation cannot read the worker's NATS or State Store
credential from its environment, its files or `/proc`.
- A run fanning out to many agent invocations uses one container.
- A Kit profile in org A cannot be selected, and its binding cannot be
resolved, by a workflow in org B.
- A harness run whose identity is not granted a Kit's required credential is
refused at launch; the proxy never injects a secret outside the run
credential.
- The firewall investigation example runs unchanged on an image profile with
direct network access.
+28 -28
View File
@@ -1,13 +1,13 @@
module github.com/samcharles93/archie-core
go 1.27.0
go 1.27.2
require (
cel.dev/expr v0.25.3 // indirect
dario.cat/mergo v1.0.2 // indirect
github.com/Azure/go-ansiterm v0.0.0-20260917205352-e937bb47801a // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/ProtonMail/go-crypto v1.5.1 // indirect
github.com/Azure/go-ansiterm v0.0.0-20261008225312-e8a9660d7aea // indirect
github.com/Microsoft/go-winio v0.6.3 // indirect
github.com/ProtonMail/go-crypto v1.5.2 // indirect
github.com/antithesishq/antithesis-sdk-go v0.8.0 // indirect
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
@@ -19,7 +19,7 @@ require (
github.com/containerd/platforms v1.0.0-rc.5 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/go-connections v0.8.1 // indirect
github.com/docker/go-connections v0.8.2 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dustin/go-humanize v1.1.0 // indirect
github.com/ebitengine/purego v0.11.1 // indirect
@@ -34,17 +34,15 @@ require (
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/google/jsonschema-go v0.4.3 // indirect
github.com/hashicorp/yamux v0.1.2 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/jackc/puddle/v2 v2.2.3 // indirect
github.com/kevinburke/ssh_config v1.6.0 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/cpuid/v2 v2.4.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/lufia/plan9stats v0.0.0-20260802145828-341c2f0c90b5 // indirect
github.com/magiconair/properties v1.18.12 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-colorable v0.1.16 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/mfridman/interpolate v0.0.2 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
@@ -58,15 +56,15 @@ require (
github.com/nats-io/jwt/v2 v2.8.2 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
github.com/oklog/run v1.1.0 // indirect
github.com/pjbgf/sha1cd v0.6.0 // indirect
github.com/oklog/run v1.2.0 // indirect
github.com/pjbgf/sha1cd v0.7.0 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/power-devops/perfstat v0.0.0-20260916203055-22a1a467d9f0 // indirect
github.com/segmentio/asm v1.2.1 // indirect
github.com/segmentio/encoding v0.5.4 // indirect
github.com/sergi/go-diff v1.4.0 // indirect
github.com/sethvargo/go-retry v0.4.0 // indirect
github.com/shirou/gopsutil/v4 v4.26.8 // indirect
github.com/sethvargo/go-retry v0.5.0 // indirect
github.com/shirou/gopsutil/v4 v4.26.9 // indirect
github.com/sirupsen/logrus v1.10.2 // indirect
github.com/stretchr/testify v1.12.1 // indirect
github.com/tklauser/go-sysconf v0.4.0 // indirect
@@ -74,19 +72,20 @@ require (
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0 // indirect
go.opentelemetry.io/otel v1.46.0 // indirect
go.opentelemetry.io/otel/metric v1.46.0 // indirect
go.opentelemetry.io/otel/trace v1.46.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.72.0 // indirect
go.opentelemetry.io/otel v1.47.0 // indirect
go.opentelemetry.io/otel/log v1.47.0 // indirect
go.opentelemetry.io/otel/metric v1.47.0 // indirect
go.opentelemetry.io/otel/trace v1.47.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba // indirect
golang.org/x/exp v0.0.0-20261007192929-f45ad48fbe92 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/text v0.42.0 // indirect
golang.org/x/time v0.16.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260921155816-b14227669459 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260921155816-b14227669459 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20261005182115-fad411399dd8 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20261005182115-fad411399dd8 // indirect
modernc.org/libc v1.77.1 // indirect
modernc.org/sqlite v1.59.0 // indirect
)
@@ -97,33 +96,34 @@ require (
github.com/cedar-policy/cedar-go v1.8.0
github.com/coreos/go-oidc/v3 v3.21.0
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.6
github.com/go-git/go-billy/v6 v6.0.0-alpha.2
github.com/go-git/go-git/v6 v6.0.0-alpha.5
github.com/go-git/go-billy/v6 v6.0.0-beta.1
github.com/go-git/go-git/v6 v6.0.0-beta.1
github.com/go-telegram/bot v1.27.0
github.com/google/jsonschema-go v0.4.3
github.com/google/uuid v1.6.0
github.com/hashicorp/go-hclog v1.6.3
github.com/hashicorp/go-plugin v1.8.0
github.com/jackc/pgx/v5 v5.11.0
github.com/moby/moby/api v1.56.0
github.com/moby/moby/client v0.6.0
github.com/moby/moby/api v1.56.1
github.com/moby/moby/client v0.6.2
github.com/modelcontextprotocol/go-sdk v1.8.0
github.com/nats-io/nats-server/v2 v2.15.0
github.com/nats-io/nats.go v1.54.0
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.1
github.com/pressly/goose/v3 v3.28.0
github.com/samcharles93/ai-sdk v0.1.42
github.com/samcharles93/ai-sdk v0.1.43
github.com/testcontainers/testcontainers-go v0.44.0
github.com/testcontainers/testcontainers-go/modules/postgres v0.44.0
go.lsp.dev/jsonrpc2 v1.0.1
go.lsp.dev/protocol v1.0.1
go.lsp.dev/uri v1.0.1
golang.org/x/mod v0.41.0
golang.org/x/net v0.59.0
golang.org/x/net v0.60.0
golang.org/x/oauth2 v0.37.0
golang.org/x/sys v0.48.0
golang.org/x/term v0.46.0
google.golang.org/grpc v1.85.0-dev.0.20260825072537-93e31b48545e
google.golang.org/grpc v1.86.0-dev
google.golang.org/protobuf v1.36.12
gopkg.in/yaml.v3 v3.0.1
oras.land/oras-go/v2 v2.6.2
+58 -59
View File
@@ -6,14 +6,14 @@ dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
github.com/Azure/go-ansiterm v0.0.0-20260917205352-e937bb47801a h1:4hxax9ktNjSDoFn1tZSeL6gXMTRMpO0FzjdKfT01jgY=
github.com/Azure/go-ansiterm v0.0.0-20260917205352-e937bb47801a/go.mod h1:3EWSSOZ50kb+arhww0qIaEXHToib/3FjBj9Jj5lcP5g=
github.com/Azure/go-ansiterm v0.0.0-20261008225312-e8a9660d7aea h1:NdVLskj4PaItpD133shAdyHicndDbcDGpManoqR032g=
github.com/Azure/go-ansiterm v0.0.0-20261008225312-e8a9660d7aea/go.mod h1:Ue4ymb+lBljb9NjkybMTsOYOSjH/cJTim9nrhvRJAUU=
github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk=
github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/ProtonMail/go-crypto v1.5.1 h1:pTrLDQHyOT8y3DFYIpijgPBTw/7E2GLMimutvOlceuE=
github.com/ProtonMail/go-crypto v1.5.1/go.mod h1:/RaSu30DaKO4RY+XdV/ACcCcZkGr7AhUIduq5sjzzCo=
github.com/Microsoft/go-winio v0.6.3 h1:M/52tsoyTfwaxajLnXZm7PqkbMGRH8iSWP+8fcg9MQ4=
github.com/Microsoft/go-winio v0.6.3/go.mod h1:ma0QpxizD4fyyzJ1C7MIblymTD2Nxz5Mho/3CepwIZg=
github.com/ProtonMail/go-crypto v1.5.2 h1:cucYnvqcY7UOXVD//mSyjeaPY0SSN3v5cDkYPxumINk=
github.com/ProtonMail/go-crypto v1.5.2/go.mod h1:/RaSu30DaKO4RY+XdV/ACcCcZkGr7AhUIduq5sjzzCo=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
github.com/antithesishq/antithesis-sdk-go v0.8.0 h1:bVV5ZAuwaREn4klRSJusYaA+A7NXq8Qi9C/knCaQ2N4=
@@ -50,8 +50,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/docker/go-connections v0.8.1 h1:JibmG5hULs5qXSr/cp/w3Pw5fZuStt4MOHMUExb29/M=
github.com/docker/go-connections v0.8.1/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-connections v0.8.2 h1:9d9DEokx5W7qlr0shkFXIxEA7+TTD+llwzcykSlxMNc=
github.com/docker/go-connections v0.8.2/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/sandbox-kit-spec/v3 v3.0.0-m.6 h1:co7vn6GgImYEGO/KPlhPRJT9i5EPWgOKcB9q7bNJlFQ=
@@ -71,12 +71,12 @@ github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c=
github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU=
github.com/go-git/gcfg/v2 v2.0.2 h1:MY5SIIfTGGEMhdA7d7JePuVVxtKL7Hp+ApGDJAJ7dpo=
github.com/go-git/gcfg/v2 v2.0.2/go.mod h1:/lv2NsxvhepuMrldsFilrgct6pxzpGdSRC13ydTLSLs=
github.com/go-git/go-billy/v6 v6.0.0-alpha.2 h1:1Sv5WemXL8CxKrAx1gioJ+uHNb2bZJhiQLfwSZ4Et8c=
github.com/go-git/go-billy/v6 v6.0.0-alpha.2/go.mod h1:r/bsv9i/iDyyEU8/Z6mjC+YraOVwie1ddfUqBCElKXQ=
github.com/go-git/go-billy/v6 v6.0.0-beta.1 h1:OJ4r207zcEbeq/DJNajQUOMd+pph3r9A7hRq1Sjl1bA=
github.com/go-git/go-billy/v6 v6.0.0-beta.1/go.mod h1:QGOWN4CzmtRfpFeCw5paG3RJyKdTwsfGASkxBVkIaAU=
github.com/go-git/go-git-fixtures/v6 v6.0.0-alpha.1 h1:gmqi2jvsreu0s8JMLylYDFq4sbjHwwlhktMw0DUg3mA=
github.com/go-git/go-git-fixtures/v6 v6.0.0-alpha.1/go.mod h1:ECf1MqJlBdYpKggBrOXjo/0EnvRZx6D++I86UYjPgAQ=
github.com/go-git/go-git/v6 v6.0.0-alpha.5 h1:sE+OlkHgYWNMVmN1s9sR7uyFgsWLtxcNWse/vBYKxRE=
github.com/go-git/go-git/v6 v6.0.0-alpha.5/go.mod h1:3IjhiZnM+uBmUrOGSeqrJpsmi4Vd0H2NZO/uK2a7d0s=
github.com/go-git/go-git/v6 v6.0.0-beta.1 h1:1t+vJYGDBN6f9GRWskgWx5UjXPFUa7wfffZZZ2C5GtA=
github.com/go-git/go-git/v6 v6.0.0-beta.1/go.mod h1:ugDlr38Wwc+fOw85J+Oluh5TNXQtJgYMcriOljg3l4s=
github.com/go-jose/go-jose/v4 v4.1.5 h1:RjgjO2LOtWOJKUC5wpwY9LR3B3vwVAz6JS2YHfYU6eA=
github.com/go-jose/go-jose/v4 v4.1.5/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-json-experiment/json v0.0.0-20260820222146-c27c302e5fc3 h1:UADEEmDKgfXbtnGJZ97beY5XLo9ZechG1nlU4KnRrkE=
@@ -115,16 +115,14 @@ github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7Ulw
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.11.0 h1:IzBBtyK9AHqf98cctWFifYSci2hgQR/cd56wB4p+ogg=
github.com/jackc/pgx/v5 v5.11.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/jackc/puddle/v2 v2.2.3 h1:f8HoREISb3l5eO9coPgK7U3hKLhWGokOFVcThl5jrao=
github.com/jackc/puddle/v2 v2.2.3/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/jhump/protoreflect v1.17.0 h1:qOEr613fac2lOuTgWN4tPAtLL7fUSbuJL5X5XumQh94=
github.com/jhump/protoreflect v1.17.0/go.mod h1:h9+vUUL38jiBzck8ck+6G/aeMX8Z4QUY/NiJPwPNi+8=
github.com/kevinburke/ssh_config v1.6.0 h1:J1FBfmuVosPHf5GRdltRLhPJtJpTlMdKTBjRgTaQBFY=
github.com/kevinburke/ssh_config v1.6.0/go.mod h1:q2RIzfka+BXARoNexmF9gkxEX7DmvbW9P4hIVx2Kg4M=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw=
github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
@@ -140,8 +138,8 @@ github.com/magiconair/properties v1.18.12 h1:sT9zQpvTB3B4gzrX0tmZNTEaGyg8Zw55MFY
github.com/magiconair/properties v1.18.12/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-colorable v0.1.16 h1:+2YjeAe7xP1OQbZW1jERPWWCDLlNqFNCvjiqGiAt84A=
github.com/mattn/go-colorable v0.1.16/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
@@ -156,10 +154,10 @@ github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3N
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/go-archive v0.3.3 h1:OxxR9paxsluYi+zDUEXTTaIxtkK3viymW+Ka7vRhhME=
github.com/moby/go-archive v0.3.3/go.mod h1:Npdv43fFqlhZW7Xo8fbm3ZMYFvAGNviUPqX21VERbcE=
github.com/moby/moby/api v1.56.0 h1:GQzua3NA599ASSIICx0iFgiJeO9YkdDARvQsm23ZZuQ=
github.com/moby/moby/api v1.56.0/go.mod h1:sZ+THbVWkjOmBPPfbnzdD/G1LuIexWhqlSHHPTDQ1Uk=
github.com/moby/moby/client v0.6.0 h1:AJjEB21QPbXSXjDsZorFBoDZPhMrfbpaPLgSMAW9Bgs=
github.com/moby/moby/client v0.6.0/go.mod h1:OCo00wNRyA3m4lmJ228W3JbyCN4ZNNYjpOXiJydBdcQ=
github.com/moby/moby/api v1.56.1 h1:PpWkvVPB7Fr/No8w+TfyJ/I6rWZ2YPZhT3JorIg+06c=
github.com/moby/moby/api v1.56.1/go.mod h1:sZ+THbVWkjOmBPPfbnzdD/G1LuIexWhqlSHHPTDQ1Uk=
github.com/moby/moby/client v0.6.2 h1:ekSPlMccrZGcw6Scb8T9D39qE0mB+XupU/yS0zNw9N4=
github.com/moby/moby/client v0.6.2/go.mod h1:DGcnO7qmPJrqobx/ggQgQ/AUk+2wgVdQv7uxB0/mEsc=
github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x084U=
github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/sys/mount v0.3.5 h1:eS3fsZTjHaBihwjp4/+5Z3jxqLXYsbwxqpVSfFv3M00=
@@ -188,14 +186,14 @@ github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/oklog/run v1.1.0 h1:GEenZ1cK0+q0+wsJew9qUg/DyD8k3JzYsZAi5gYi2mA=
github.com/oklog/run v1.1.0/go.mod h1:sVPdnTZT1zYwAJeCMu2Th4T21pA3FPOQRfWjQlk7DVU=
github.com/oklog/run v1.2.0 h1:O8x3yXwah4A73hJdlrwo/2X6J62gE5qTMusH0dvz60E=
github.com/oklog/run v1.2.0/go.mod h1:mgDbKRSwPhJfesJ4PntqFUbKQRZ50NgmZTSPlFA0YFk=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M=
github.com/pjbgf/sha1cd v0.6.0 h1:3WJ8Wz8gvDz29quX1OcEmkAlUg9diU4GxJHqs0/XiwU=
github.com/pjbgf/sha1cd v0.6.0/go.mod h1:lhpGlyHLpQZoxMv8HcgXvZEhcGs0PG/vsZnEJ7H0iCM=
github.com/pjbgf/sha1cd v0.7.0 h1:ZRNPKHj+gfkLBf0KJv/p1Hmz+7bqCT5o311YX0nq+DA=
github.com/pjbgf/sha1cd v0.7.0/go.mod h1:pKR5Li+qTCo+ebqITZfwPlJGoCFCvSO00qxjbNtTUFQ=
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ=
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
@@ -207,18 +205,18 @@ github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/samcharles93/ai-sdk v0.1.42 h1:YV4O6oV0OLIqDAnEWAotPyw19K1VH2YoAT0He7aW+Lg=
github.com/samcharles93/ai-sdk v0.1.42/go.mod h1:u4zrQ9/Q58L5mNEut4bNQ1ks+RCKq71LN5EDz2N/0Zk=
github.com/samcharles93/ai-sdk v0.1.43 h1:u/OER6FuRv/KkS/5xnRttABPD2UNHR2BmcJeRe07hyw=
github.com/samcharles93/ai-sdk v0.1.43/go.mod h1:u4zrQ9/Q58L5mNEut4bNQ1ks+RCKq71LN5EDz2N/0Zk=
github.com/segmentio/asm v1.2.1 h1:DTNbBqs57ioxAD4PrArqftgypG4/qNpXoJx8TVXxPR0=
github.com/segmentio/asm v1.2.1/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs=
github.com/segmentio/encoding v0.5.4 h1:OW1VRern8Nw6ITAtwSZ7Idrl3MXCFwXHPgqESYfvNt0=
github.com/segmentio/encoding v0.5.4/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0=
github.com/sergi/go-diff v1.4.0 h1:n/SP9D5ad1fORl+llWyN+D6qoUETXNZARKjyY2/KVCw=
github.com/sergi/go-diff v1.4.0/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
github.com/sethvargo/go-retry v0.4.0 h1:9qy1OoIAxBL+gBYnkTnTnWle5wlfsXQlwRzIbbpdqPw=
github.com/sethvargo/go-retry v0.4.0/go.mod h1:tvsjdKG6xfiCx4LSiUZ06kcv38xvdVQwv8R6/VnnVWg=
github.com/shirou/gopsutil/v4 v4.26.8 h1:YQMTF/1J50B5+Y0vlo1eDRf5DoR7Gk69hY+8wjYkQeo=
github.com/shirou/gopsutil/v4 v4.26.8/go.mod h1:5O9FjBiXoTDFatIWjZZosqj4pV0DRtLx598xGbBehzM=
github.com/sethvargo/go-retry v0.5.0 h1:JYfFKmVJ8ER06GReN1efzc5cm52BrVsFVzmcHiKJwfY=
github.com/sethvargo/go-retry v0.5.0/go.mod h1:tvsjdKG6xfiCx4LSiUZ06kcv38xvdVQwv8R6/VnnVWg=
github.com/shirou/gopsutil/v4 v4.26.9 h1:CaBo/hBFqvlJoLvAQEKdHdf5GD97/MN2ACfyv63+2Ig=
github.com/shirou/gopsutil/v4 v4.26.9/go.mod h1:nKH+8wX2zxr/mDbsR+AAy6Qd2z4ZOQ7wfQaEp+yrYmg=
github.com/sirupsen/logrus v1.10.2 h1:G2SED73/qrAu6YwbdxOD6peLkCBI3z7L+ykJFTXJBBo=
github.com/sirupsen/logrus v1.10.2/go.mod h1:SLEg8TqYulVKKfIGHldVp2K2aYz2DKSVBq4g/H5bR7Q=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -250,30 +248,32 @@ go.lsp.dev/uri v1.0.1 h1:ywxJN1UXoQ+v0I9lkoQNMmsm7lub5G9ZG7KXT8YTx30=
go.lsp.dev/uri v1.0.1/go.mod h1:06a0ghafs4PuqkLsJGUmlgPHw6+kH79edRXjOT/rA8s=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0 h1:3g7B90UzBltIDKq1/5mrTGxTnOFDV0ICOhLoxiZ8jlg=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0/go.mod h1:Ef8SuTh59BT7+ofpDxN9z+yOlc4t2GjLmKDgYNJL/NU=
go.opentelemetry.io/otel v1.46.0 h1:FHt5/CDyVxi/8IM1CH7VE/rRgq3kLHa2mSTVMO8AWyc=
go.opentelemetry.io/otel v1.46.0/go.mod h1:Gj3SEScelsNC45tp4nSxRYlS+f5iez7W8XPMCt905kE=
go.opentelemetry.io/otel/metric v1.46.0 h1:yBnkXvgV7AXFILZc5K6IZe/CBFF3OS7BJ8ov6/lj0K8=
go.opentelemetry.io/otel/metric v1.46.0/go.mod h1:iPmdWqifKUdzziPkvvzIJXITl56fQx2mGM/DHLB3/2o=
go.opentelemetry.io/otel/sdk v1.46.0 h1:h5CNQQjEbuQXY/JfZtgt3i7HVFV3aHPO2OAwO2eTYPI=
go.opentelemetry.io/otel/sdk v1.46.0/go.mod h1:GAERFXFt5SYCEB+YiKUbMBeza6UaDH7GmGOZEfh2gSM=
go.opentelemetry.io/otel/sdk/metric v1.46.0 h1:0piZ26EG4RBfebb2jhDH6ERCYHoVWduc3kLgPCwSnSE=
go.opentelemetry.io/otel/sdk/metric v1.46.0/go.mod h1:I1PbKrdVc8Qu8HYVDNtqVIwLwjNrhsV/uFuxfwg8mO4=
go.opentelemetry.io/otel/trace v1.46.0 h1:OULy7ccdJnZtJ0UDYFOIGaCmiWzJ8Vi2G/Rsu60qs1c=
go.opentelemetry.io/otel/trace v1.46.0/go.mod h1:J7GAXweO77XSFkB/rmAqk9D6ihszhFjLU+d9WuUxDLI=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.72.0 h1:LxwW/9ctSCv+QkE/cLR7M91ZIkXNMqJtEMi1vCw9U8s=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.72.0/go.mod h1:tOsftB4SslBwwErVEPaenU2RpThXWPIU8DoJHEC4dyw=
go.opentelemetry.io/otel v1.47.0 h1:j7ALJ/zgkS7Z6aeJW09p8VC9804bC+PpeTfCD4XPnOM=
go.opentelemetry.io/otel v1.47.0/go.mod h1:8wS9O2qfXrYrzp6hIF/HOYJJf/wIhFPhR2xLuP+iXQU=
go.opentelemetry.io/otel/log v1.47.0 h1:cOTS1CcLbSQeZKanGJ+0JpF/+t4PELi3O3bbl2lqCcI=
go.opentelemetry.io/otel/log v1.47.0/go.mod h1:9byitSQ5pLC6PpqwGXjqdMKya6ZTswHRZh2vvXT33nw=
go.opentelemetry.io/otel/metric v1.47.0 h1:4PptaldXx3Eat1XjMZ68pPJEs5wrhlemctZE9a3UdWY=
go.opentelemetry.io/otel/metric v1.47.0/go.mod h1:ADGSXxRrXM6bjbvLo535EstVFlPpPYZm4LBKixjDHwU=
go.opentelemetry.io/otel/sdk v1.47.0 h1:zWXEr4j2lFefG87TU6Yg8a7ngfohIKFZHKp0Hf5hC6I=
go.opentelemetry.io/otel/sdk v1.47.0/go.mod h1:VUc24kiOeoGsxG8G9ULx3fWKvB7jMhnGE8Oi607lgR0=
go.opentelemetry.io/otel/sdk/metric v1.47.0 h1:lfISg2j93VT6yqdk9OfUaZmw/GfcZqCCV3jdXtsPnKw=
go.opentelemetry.io/otel/sdk/metric v1.47.0/go.mod h1:ypLp+mW1Nt2x+Szt3b5/i1syodyts49lMOwxpDI3VGw=
go.opentelemetry.io/otel/trace v1.47.0 h1:JOjX/Oci8K94QHddo+bbfya/Ai/nf6/dt9ZfrFNWSrM=
go.opentelemetry.io/otel/trace v1.47.0/go.mod h1:jNaSLa2PZEYFG6fRjJABAu+bw4FS08uDmPg28lTghu0=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba h1:Ck8QetSgk912qxWLMCKxd0in+aiyBQyDSMae6e/xmpU=
golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba/go.mod h1:50RgIsmK7OwqzTTeqcSXQW8SswW0o8fRcDxmqGluJ8E=
golang.org/x/exp v0.0.0-20261007192929-f45ad48fbe92 h1:kc+Gi2MFkd0KWtdDlVxVwjw4KB98/77usJDFj+6YFQ8=
golang.org/x/exp v0.0.0-20261007192929-f45ad48fbe92/go.mod h1:SRjTzy5WcYqRRqquLUvHeeovSwLO0CZ3xcltZNO4Fck=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues=
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
golang.org/x/net v0.60.0 h1:79p50tfZlm0J9YfoDsSi639qSXNGVwEzOPLCxM2FsYU=
golang.org/x/net v0.60.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
golang.org/x/oauth2 v0.37.0 h1:JUlcxA8oAtauLfiH8FX2/FkAWHAdi0QtGCGc+hofE98=
golang.org/x/oauth2 v0.37.0/go.mod h1:IxwZNxUULJmpBFf9K/9NTMSIfZZuvuTy1gGxhigP/58=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
@@ -284,7 +284,6 @@ golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
@@ -296,16 +295,16 @@ golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s=
golang.org/x/tools v0.50.0 h1:c2ifzfcuY7L90lZ2aKd8S4K2NpASF08SZx9ZuJkHmSU=
golang.org/x/tools v0.50.0/go.mod h1:7ulVMw3831Mwi5EZD6RomGyffr4VFjuNYXf2BbCEAV0=
golang.org/x/tools v0.51.0 h1:k4Xc/1Om9jwkBJBo4NVLMSARBoWtK10mx+W5BnXCeAI=
golang.org/x/tools v0.51.0/go.mod h1:9eEncMayCV6zRMGhR5eZEC2iBx98qWcF1HZ9Z7wJOoA=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
google.golang.org/genproto/googleapis/api v0.0.0-20260921155816-b14227669459 h1:GS9OIt/j7c8bvBjYNgnKQysVfmV7e4jM0H8ZK95G4t8=
google.golang.org/genproto/googleapis/api v0.0.0-20260921155816-b14227669459/go.mod h1:PX5/4vemwVoXtwEcRDWwcR1/r0qrosfx3qoVADMwnVE=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260921155816-b14227669459 h1:b0xCahf3FK2m2Cv0p4vTozGPWncCvLfwV86UNg8xWU8=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260921155816-b14227669459/go.mod h1:OaIUM3+LpYcK2GXM4FTmhWoIq371Owdr+Cc7/BsYHHc=
google.golang.org/grpc v1.85.0-dev.0.20260825072537-93e31b48545e h1:ar+feAPij1Znzug45m6AoShUCc5AaR8FLA3I6LZd94k=
google.golang.org/grpc v1.85.0-dev.0.20260825072537-93e31b48545e/go.mod h1:7aY/KlAHanMAVIWHnk8wloQsdlfwzDz/PP9vCbJG6CU=
google.golang.org/genproto/googleapis/api v0.0.0-20261005182115-fad411399dd8 h1:cFbkNigDInw+wVTXc+CyhUsoHGJCw0BR6mlNoRVfre0=
google.golang.org/genproto/googleapis/api v0.0.0-20261005182115-fad411399dd8/go.mod h1:QK6p7IgD7IZEZJBhhSUCib5X2vMOzlrGCGUw2XyVDh0=
google.golang.org/genproto/googleapis/rpc v0.0.0-20261005182115-fad411399dd8 h1:0SXBqli5dnuCKMjI/X3FM226zEaEa0GUcSCJtN1vL2o=
google.golang.org/genproto/googleapis/rpc v0.0.0-20261005182115-fad411399dd8/go.mod h1:OaIUM3+LpYcK2GXM4FTmhWoIq371Owdr+Cc7/BsYHHc=
google.golang.org/grpc v1.86.0-dev h1:FLq7hcApuJeJ70J9+k0FvmXsNCwS8B2/MeAfYdy3GHY=
google.golang.org/grpc v1.86.0-dev/go.mod h1:Ovl0ECo4xx5r4kn/6d4BPSNB7OIFuu6EAjOzjtVAKaM=
google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc=
google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+46 -1
View File
@@ -8,6 +8,8 @@ import (
"slices"
"strings"
"github.com/google/jsonschema-go/jsonschema"
"github.com/samcharles93/archie-core/internal/domain/agentrun"
)
@@ -62,7 +64,7 @@ func ProtectionMatcher(p agentrun.Protection, readOnly bool) func(string) bool {
func ValidateCaptureArgs(spec agentrun.CaptureTool, value json.RawMessage) (string, bool) {
var object map[string]json.RawMessage
if err := json.Unmarshal(value, &object); err != nil {
return spec.Name + " rejected: arguments must be a JSON object", false //nolint:nilerr // the agent loop must see malformed tool arguments as feedback it can correct, not as a failed tool call
return spec.Name + " rejected: arguments must be a JSON object", false // the agent loop must see malformed tool arguments as feedback it can correct, not as a failed tool call
}
for _, check := range []func(agentrun.CaptureTool, map[string]json.RawMessage) (string, bool){
checkRequiredFields,
@@ -74,9 +76,52 @@ func ValidateCaptureArgs(spec agentrun.CaptureTool, value json.RawMessage) (stri
return rejection, false
}
}
// The derived checks above cover the common cases with friendly wording;
// the schema is what enforces the rest of what the tool advertised --
// enums, numeric types, nested objects, array shapes -- so a call the
// agent could only make valid is rejected with the reason, not recorded.
return checkSchema(spec, value)
}
// checkSchema validates the arguments against the tool's advertised JSON
// Schema. A tool with no schema, or one that will not resolve, is accepted
// here: a schema that never resolves is a definition bug, refused where the
// tool is built (workflow's agentResultTool), so the agent is never blamed
// for it.
func checkSchema(spec agentrun.CaptureTool, value json.RawMessage) (string, bool) {
if len(spec.Parameters) == 0 {
return "", true
}
resolved, err := resolveCaptureSchema(spec.Parameters)
if err != nil {
return "", true
}
var instance any
if err := json.Unmarshal(value, &instance); err != nil {
return spec.Name + " rejected: arguments must be JSON", false
}
if err := resolved.Validate(instance); err != nil {
return fmt.Sprintf("%s rejected: %v", spec.Name, err), false
}
return "", true
}
// resolveCaptureSchema parses and resolves a capture tool's JSON Schema. The
// workflow parser resolves the same schema when it builds the tool, so an
// unusable schema is refused at definition time rather than silently skipped
// here.
func resolveCaptureSchema(params json.RawMessage) (*jsonschema.Resolved, error) {
var schema jsonschema.Schema
if err := json.Unmarshal(params, &schema); err != nil {
return nil, fmt.Errorf("capture tool schema: %w", err)
}
resolved, err := schema.Resolve(nil)
if err != nil {
return nil, fmt.Errorf("capture tool schema: %w", err)
}
return resolved, nil
}
func checkRequiredFields(spec agentrun.CaptureTool, object map[string]json.RawMessage) (string, bool) {
for _, field := range spec.RequiredFields {
if _, ok := object[field]; !ok {
+98
View File
@@ -0,0 +1,98 @@
package agentexec
import (
"encoding/json"
"strings"
"testing"
"github.com/samcharles93/archie-core/internal/domain/agentrun"
)
// resultSchema is the shape of a shipped planner result (triage's), with a
// nested object and an array so every constraint kind the agent can violate
// is exercised.
const resultSchema = `{
"type": "object",
"properties": {
"needs_code_change": {"type": "boolean"},
"workflow": {"type": "string", "enum": ["implement", "tdd", "feasibility"]},
"reasons": {"type": "string"},
"labels": {"type": "array", "items": {"type": "string"}, "maxItems": 2},
"plan": {"type": "object", "properties": {"risk": {"type": "integer"}}, "required": ["risk"]}
},
"required": ["needs_code_change", "workflow", "reasons"]
}`
// TestValidateCaptureArgsEnforcesSchema pins that the advertised schema is
// enforced, not merely offered to the agent: an enum, a numeric type, an
// array's item type and length, and a nested object's shape all reject with a
// message naming what was wrong, so the agent can correct it.
func TestValidateCaptureArgsEnforcesSchema(t *testing.T) {
base := `"needs_code_change": true, "workflow": "tdd", "reasons": "why"`
tests := []struct {
name string
args string
wantOK bool
wantText string
}{
{name: "valid", args: `{` + base + `}`, wantOK: true},
{name: "enum violation", args: `{` + base + `, "workflow": "banana"}`, wantText: "workflow"},
{name: "wrong boolean type", args: `{"needs_code_change": "yes", "workflow": "tdd", "reasons": "why"}`, wantText: "needs_code_change"},
{name: "string where integer expected", args: `{` + base + `, "plan": {"risk": "high"}}`, wantText: "plan"},
{name: "array item type", args: `{` + base + `, "labels": [1, 2]}`, wantText: "labels"},
{name: "array too long", args: `{` + base + `, "labels": ["a", "b", "c"]}`, wantText: "labels"},
{name: "nested required missing", args: `{` + base + `, "plan": {}}`, wantText: "plan"},
}
spec := agentrun.CaptureTool{Name: "result", Parameters: json.RawMessage(resultSchema)}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rejection, ok := ValidateCaptureArgs(spec, json.RawMessage(tt.args))
if ok != tt.wantOK {
t.Fatalf("ok = %v, rejection = %q; want ok %v", ok, rejection, tt.wantOK)
}
if ok {
return
}
if !strings.HasPrefix(rejection, "result rejected: ") {
t.Fatalf("rejection %q does not name the tool", rejection)
}
if !strings.Contains(rejection, tt.wantText) {
t.Fatalf("rejection %q does not mention %q", rejection, tt.wantText)
}
})
}
}
// TestValidateCaptureArgsWithoutSchema pins that a tool with no schema, or one
// whose schema cannot resolve, still accepts a call: the derived constraint
// lists remain the whole check, so the agent is never blocked by a tool
// definition the schema check cannot read.
func TestValidateCaptureArgsWithoutSchema(t *testing.T) {
tests := []struct {
name string
spec agentrun.CaptureTool
args string
wantOK bool
}{
{name: "no schema", spec: agentrun.CaptureTool{Name: "note"}, args: `{"anything": 1}`, wantOK: true},
{
name: "unresolvable schema is not the agent's problem",
spec: agentrun.CaptureTool{Name: "note", Parameters: json.RawMessage(`{"$ref": "#/definitions/missing"}`)},
args: `{"anything": 1}`,
wantOK: true,
},
{
name: "derived checks still apply without a schema",
spec: agentrun.CaptureTool{Name: "note", NonEmptyStrings: []string{"body"}},
args: `{"body": " "}`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, ok := ValidateCaptureArgs(tt.spec, json.RawMessage(tt.args))
if ok != tt.wantOK {
t.Fatalf("ok = %v, want %v", ok, tt.wantOK)
}
})
}
}
+17 -5
View File
@@ -3,16 +3,23 @@ package agentworker
import (
"context"
"errors"
"log/slog"
"strconv"
"time"
"github.com/samcharles93/archie-core/internal/agentexec"
"github.com/samcharles93/archie-core/internal/domain/health"
"github.com/samcharles93/archie-core/internal/domain/presence"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
"github.com/samcharles93/archie-core/internal/infrastructure/agentboot"
"github.com/samcharles93/archie-core/internal/infrastructure/agentgit"
agentnats "github.com/samcharles93/archie-core/internal/infrastructure/agenttransport/nats"
"github.com/samcharles93/archie-core/internal/infrastructure/readiness"
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
"github.com/samcharles93/archie-core/internal/installtype"
"github.com/samcharles93/archie-core/internal/storage"
"github.com/samcharles93/archie-core/internal/taskrun"
"github.com/samcharles93/archie-core/internal/tools"
@@ -44,14 +51,12 @@ type Settings struct {
}
type workerTransport interface {
taskServiceTransport
Close()
LogPublisher() agentexec.LogPublisher
EventPublisher() agentexec.EventPublisher
SubscribeTasks(context.Context, int64, agentnats.TaskHandler, *slog.Logger) (agentnats.Subscription, error)
Forger(identity, credential string, timeout time.Duration) workflow.Forger
Store(time.Duration) workflow.Store
Calls(time.Duration) task.Caller
Trees(string, string, time.Duration) agentnats.RemoteTrees
PutPresence(context.Context, storecontract.Presence) error
BrokerReady() bool
}
type workerDependencies struct {
@@ -148,6 +153,13 @@ func run(ctx context.Context, settings Settings, log *slog.Logger, dependencies
}
}()
registry := health.NewRegistry(readiness.NewContractProbe("nats", 0, func(context.Context) error {
if !transport.BrokerReady() {
return errors.New("broker disconnected")
}
return nil
}))
go presence.Run(ctx, presence.Agent, presence.Build{InstanceID: strconv.FormatInt(taskID, 10), Version: Version(), InstallType: installtype.Type()}, transport, registry, log)
log.Info("archie-agent ready", "nats", settings.NATSURL)
dependencies.wait(ctx, log)
return nil
+30 -79
View File
@@ -48,6 +48,8 @@ import (
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
"github.com/samcharles93/archie-core/internal/infrastructure/eventbus/nats"
"github.com/samcharles93/archie-core/internal/infrastructure/extension"
"github.com/samcharles93/archie-core/internal/infrastructure/kitrun"
"github.com/samcharles93/archie-core/internal/infrastructure/logrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/messagingrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/modelcatalog"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
@@ -158,8 +160,11 @@ type boot struct {
containerPool *container.Pool
// kitLauncher starts Kit profile tasks; nil when the egress path cannot
// be built on this host.
kitLauncher daemon.KitLauncher
storeBackend storage.Backend
kitLauncher *kitrun.Launcher
// sessionGrants records live setup sessions' granted credential services
// for the egress proxy's resolver.
sessionGrants *sessionGrants
storeBackend storage.Backend
// runtimeVersions records the version of every control-plane kind boot's
// layering applied. Set once at boot before the runtime-resource watches
@@ -178,14 +183,6 @@ type boot struct {
// schedulingEngine is the cron/scheduling ticker engine (setupScheduling).
schedulingEngine *scheduling.Engine
// kindWorkflows/labelWorkflows are the resolved kind/label -> workflow
// routing bindings loaded by loadWorkflowRouting. They are handed to the
// daemon so runViaAgent can carry them in taskrun.Request: workflow.Route
// runs in the archie-agent process, which never sees the daemon's
// package-level workflow state.
kindWorkflows workflow.KindWorkflows
labelWorkflows workflow.LabelWorkflows
// agentStatus records the most recent version/install-type an
// archie-agent worker reported about itself. Allocated up front, before
// any gateway goroutine (Telegram, webui) that might read it via
@@ -340,7 +337,7 @@ func (b *boot) setupObservability(ctx context.Context) {
const reactionStreamMaxAge = 24 * time.Hour
func (b *boot) connectNATS(ctx context.Context) error { //nolint:nestif // embedded broker discovery and startup require explicit fallback branches
func (b *boot) connectNATS(ctx context.Context) error { // embedded broker discovery and startup require explicit fallback branches
cfg, log := b.cfg, b.log
url := cfg.NATS.URL
var natsToken string
@@ -480,52 +477,6 @@ func (b *boot) setupGatewayClient() error {
return nil
}
func (b *boot) loadWorkflows() error {
cfg, log := b.cfg, b.log
if err := b.loadWorkflowRouting(cfg, log); err != nil {
return err
}
log.Info("workflow step registry built", "shipped_workflows", len(workflow.ShippedDefinitions().Definitions))
return nil
}
func (b *boot) loadWorkflowRouting(cfg config.Config, log *slog.Logger) error {
kindWorkflows, err := workflow.LoadKindWorkflowsYAML(cfg.WorkflowRoutingFile)
if err != nil {
log.Error("workflow routing file load failed", "path", cfg.WorkflowRoutingFile, "err", err)
return err
}
labelWorkflows, err := workflow.LoadLabelWorkflowsYAML(cfg.WorkflowLabelsFile)
if err != nil {
log.Error("workflow labels file load failed", "path", cfg.WorkflowLabelsFile, "err", err)
return err
}
dirKindWorkflows, dirLabelWorkflows, err := workflow.LoadPlaybookDirs(cfg.PlaybookDirs)
if err != nil {
log.Error("playbook dirs load failed", "dirs", cfg.PlaybookDirs, "err", err)
return err
}
kindWorkflows, err = workflow.MergeKindWorkflows(kindWorkflows, dirKindWorkflows)
if err != nil {
log.Error("workflow binding collision between routing file and playbook dir", "err", err)
return err
}
labelWorkflows, err = workflow.MergeLabelWorkflows(labelWorkflows, dirLabelWorkflows)
if err != nil {
log.Error("workflow binding collision between labels file and playbook dir", "err", err)
return err
}
workflow.SetKindWorkflows(kindWorkflows)
workflow.SetLabelWorkflows(labelWorkflows)
b.kindWorkflows = kindWorkflows
b.labelWorkflows = labelWorkflows
return nil
}
func (b *boot) buildWorktreeManager() {
cfg := b.cfg
b.trees = &worktree.Manager{
@@ -578,6 +529,11 @@ func (b *boot) registerNATSRPC() error {
}
b.addCleanup(unsubscribe)
stopLogs, err := logrpc.RegisterDaemon(coreConn, b.logFeed, log)
if err != nil {
return fmt.Errorf("register daemon logs: %w", err)
}
b.addCleanup(stopLogs)
unsubscribeSystemLogs, err := subscribeSystemLogs(coreConn, b.taskLogs, log)
if err != nil {
log.Error("system log subscribe failed", "err", err)
@@ -622,11 +578,9 @@ func (b *boot) buildDaemon() {
RootIdentityID: identity.StableID(servicekit.IdentityNames(b.cfg)[0]),
TaskLogs: b.taskLogs,
AgentStatus: b.agentStatus,
KindWorkflows: b.kindWorkflows,
LabelWorkflows: b.labelWorkflows,
WorkflowDefinitions: b.workflowDefinitions,
WorkflowEnablement: b.controlPlane,
Playbooks: controlplane.NewLivePlaybooks(b.controlPlaneRPC, b.log),
Playbooks: b.livePlaybooks(),
}
if identities, ok := b.stateStore.(identity.Repository); ok {
@@ -720,26 +674,11 @@ func (b *boot) publishConfig(ctx context.Context, cfg config.Config) {
}
}
func (b *boot) configOrigins() []webui.ConfigOrigin {
provenance := b.currentProvenance.Load()
if provenance == nil {
return nil
}
origins := make([]webui.ConfigOrigin, 0, len(provenance.Origins))
for _, origin := range provenance.Origins {
origins = append(origins, webui.ConfigOrigin{
Path: origin.Path, Role: string(origin.Role), Layer: string(origin.Layer), Feature: string(origin.Feature),
})
}
return origins
}
func (b *boot) configViewInput() webui.ConfigViewInput {
catalog, _ := b.catalogState()
in := webui.ConfigViewInput{
Config: b.cfgHolder.Get(),
Provenance: b.configOrigins(),
Catalog: catalogView(catalog),
Config: b.cfgHolder.Get(),
Catalog: catalogView(catalog),
}
if b.lastReload != nil {
status := b.lastReload()
@@ -805,12 +744,14 @@ func (b *boot) startServices(ctx context.Context) error {
func (b *boot) setupBackends(ctx context.Context) error {
closeContainers := b.setupContainers(ctx)
// Registered before the Kit launcher so its cleanups (the harness
// listener and the egress relay) run first: cleanups unwind LIFO, and a
// session teardown needs the Docker client the pool still owns.
b.addCleanup(closeContainers)
err := b.connectNATS(ctx)
if err == nil {
b.setupKitLauncher(ctx)
}
b.addCleanup(closeContainers)
return err
}
@@ -826,6 +767,10 @@ func (b *boot) runLoop(ctx context.Context, once bool) error {
b.announceReady()
if once {
b.d.Cycle(ctx)
// A drain pass no longer waits for the tasks it submitted, so a
// single-cycle invocation waits for them here: exiting while their
// containers still run would kill work this pass accepted.
b.d.WaitForTasks()
return nil
}
b.log.Info("archied running", "repos", len(b.cfg.Repos), "poll", b.cfg.PollInterval.Std().String(), "label", b.cfg.Label)
@@ -894,3 +839,9 @@ func (b *boot) startModelCatalogRefresh(ctx context.Context) {
}
})
}
func (b *boot) livePlaybooks() *controlplane.LivePlaybooks {
live := controlplane.NewLivePlaybooks(b.controlPlaneRPC, b.log)
live.ApplyStatus = b.applyStatus
return live
}
+5 -5
View File
@@ -63,15 +63,17 @@ func (b *boot) reloadConfig(ctx context.Context, doc *configuration.Document) er
if err != nil {
return fmt.Errorf("runtime settings unavailable: %w", err)
}
doc.Config = cfg
old := b.cfgHolder.Get()
restartFields := changedNonReloadableFields(old, cfg)
cfg = preserveForgeIdentity(old, cfg)
doc.Config = cfg
b.currentProvenance.Store(&doc.Provenance)
b.publishConfig(ctx, cfg)
// The container pool reads the published config on every acquire, but the
// dispatcher holds its limit; a reloaded containers.max_concurrency must
// reach it the same way a control-plane update does.
b.resizeTaskDispatcher(cfg.Containers.MaxConcurrency)
if fields := changedNonReloadableFields(old, cfg); len(fields) > 0 {
if fields := restartFields; len(fields) > 0 {
b.log.Warn("config reloaded; some changes require a restart",
"fields", fields, "paths", doc.Provenance.Paths())
} else {
@@ -210,9 +212,6 @@ func applyExecutionBudgets(cfg *config.Config, settings workflow.ExecutionSettin
MaxSteps: settings.MaxModelToolSteps, WallClock: config.Duration(settings.MaxRuntime), GateMaxFailures: settings.MaxConsecutiveGateFailures,
TaskWallClock: config.Duration(settings.MaxTaskRuntime),
}
for i := range cfg.Identities {
cfg.Identities[i].Budgets = cfg.Budgets
}
}
// startLiveSettings starts everything a live control-plane update needs
@@ -263,6 +262,7 @@ func (b *boot) startLiveSettings(ctx context.Context) error {
// restart-required. Each kind is watched in its own goroutine, exactly the
// shape the workflow-execution-settings watch established.
var runtimeResourceKinds = []string{
controlplane.TaskPolicyKind,
controlplane.ProviderSettingsKind,
controlplane.ModelRoleAssignmentsKind,
controlplane.RepositoryPoliciesKind,
+224
View File
@@ -0,0 +1,224 @@
package archied
import (
"context"
"errors"
"io"
"log/slog"
"net"
"os"
"slices"
"sync"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"github.com/samcharles93/archie-core/internal/app/servicekit"
"github.com/samcharles93/archie-core/internal/config"
pb "github.com/samcharles93/archie-core/internal/contracts/harness/v1"
"github.com/samcharles93/archie-core/internal/infrastructure/harnessrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/harnesssession"
"github.com/samcharles93/archie-core/internal/infrastructure/kitrun"
)
// setupSessionTTL bounds a setup container's life independently of the pool's
// max-uptime reaper: a login that never finishes must not hold a container
// until the reaper's (longer) cap.
const setupSessionTTL = 30 * time.Minute
// sessionGrants records the credential services each live setup session is
// bound for. A session's proxy token names no task, so the egress proxy's run
// resolver consults this registry: without it, the token endpoint is never
// intercepted and no OAuth token is ever captured.
type sessionGrants struct {
mu sync.RWMutex
m map[string]sessionGrant
}
type sessionGrant struct {
org string
services []string
}
func (g *sessionGrants) Grant(token, org string, services []string) {
g.mu.Lock()
defer g.mu.Unlock()
if g.m == nil {
g.m = map[string]sessionGrant{}
}
g.m[token] = sessionGrant{org: org, services: slices.Clone(services)}
}
func (g *sessionGrants) Revoke(token string) {
g.mu.Lock()
defer g.mu.Unlock()
delete(g.m, token)
}
// GrantedFor reports the org and granted services a session token carries.
func (g *sessionGrants) GrantedFor(token string) (string, []string, bool) {
g.mu.RLock()
defer g.mu.RUnlock()
grant, ok := g.m[token]
return grant.org, grant.services, ok
}
// harnessServer serves the daemon's harness session contract.
type harnessServer struct {
pb.UnimplementedHarnessServiceServer
sessions *harnesssession.Manager
log *slog.Logger
}
// Open dispatches on the first frame's target. Setup sessions are served; the
// run-scoped target is reserved for archie-core-qi91.
func (h *harnessServer) Open(stream pb.HarnessService_OpenServer) error {
first, err := stream.Recv()
if err != nil {
return err
}
target := first.GetTarget()
if target == nil || target.GetOrg() == "" {
return status.Error(codes.InvalidArgument, "the first frame must name a session target with an org")
}
switch t := target.GetTarget().(type) {
case *pb.OpenTarget_Setup:
if t.Setup.GetProfile() == "" {
return status.Error(codes.InvalidArgument, "the setup target names no profile")
}
return h.openSetup(stream, target.GetOrg(), t.Setup.GetProfile(), first.GetSize())
case *pb.OpenTarget_Run:
return status.Error(codes.Unimplemented, "run-scoped sessions are not served yet")
default:
return status.Error(codes.InvalidArgument, "the target names no session")
}
}
// openSetup opens one setup session and pumps frames until the shell exits or
// either side closes. Teardown runs on every return.
func (h *harnessServer) openSetup(stream pb.HarnessService_OpenServer, org, profile string, size *pb.ConsoleSize) error {
session, err := h.sessions.Open(stream.Context(), org, profile, int(size.GetRows()), int(size.GetCols()))
if err != nil {
h.log.Warn("setup session failed to open", "org", org, "profile", profile, "err", err)
// The failure rides the stream as a frame, not only as a status: the
// dashboard's terminal shows the frame's text, which is the operator's
// only honest account of why the session did not open.
return stream.Send(&pb.OpenResponse{Frame: &pb.OpenResponse_Error{Error: err.Error()}})
}
defer func() {
_ = session.Close()
h.log.Info("setup session closed", "org", org, "profile", profile)
}()
h.log.Info("setup session opened", "org", org, "profile", profile)
output := make(chan error, 1)
go func() { output <- h.pumpOutput(stream, session) }()
input := make(chan error, 1)
go func() { input <- h.pumpInput(stream, session) }()
select {
case err := <-output:
if err != nil {
return status.Error(codes.Internal, err.Error())
}
code, codeErr := session.ExitCode(context.WithoutCancel(stream.Context()))
if codeErr != nil {
return nil
}
return stream.Send(&pb.OpenResponse{Frame: &pb.OpenResponse_ExitCode{ExitCode: int32(code)}})
case err := <-input:
if errors.Is(err, io.EOF) {
return nil
}
return err
}
}
// pumpOutput copies terminal output to the stream until the shell ends.
func (h *harnessServer) pumpOutput(stream pb.HarnessService_OpenServer, session *harnesssession.Session) error {
buf := make([]byte, 32<<10)
for {
n, err := session.Read(buf)
if n > 0 {
frame := &pb.OpenResponse{Frame: &pb.OpenResponse_Stdout{Stdout: slices.Clone(buf[:n])}}
if sendErr := stream.Send(frame); sendErr != nil {
return sendErr
}
}
if err != nil {
if errors.Is(err, io.EOF) {
return nil
}
return err
}
}
}
// pumpInput copies client frames to the session until the client closes.
func (h *harnessServer) pumpInput(stream pb.HarnessService_OpenServer, session *harnesssession.Session) error {
for {
req, err := stream.Recv()
if err != nil {
return err
}
if stdin, ok := req.GetFrame().(*pb.OpenRequest_Stdin); ok {
if _, err := session.Write(stdin.Stdin); err != nil {
return err
}
}
}
}
// setupHarness serves the harness session contract on the configured listen
// address. It is skipped, with the reason logged, when no service token is
// configured: a session is a shell in a container that holds an org's
// credential, so the token is required even on loopback.
func (b *boot) setupHarness(ctx context.Context) {
conn := b.cfg.Services.Get(config.ServiceNameHarness)
listen, err := servicekit.ResolveListen(config.ServiceNameHarness, "", conn.Listen)
if err != nil {
b.log.Warn("harness sessions disabled", "err", err)
return
}
token := b.cfg.Services.ResolvedToken(config.ServiceNameHarness, os.Getenv)
if token == "" {
b.log.Warn("harness sessions disabled: [services.harness].target_token or HARNESS_TOKEN is required, even on loopback")
return
}
ln, err := (&net.ListenConfig{}).Listen(ctx, "tcp", listen)
if err != nil {
b.log.Warn("harness sessions disabled: listen", "err", err)
return
}
manager := &harnesssession.Manager{
Launcher: b.kitLauncher,
Client: b.containerPool.Client(),
Profile: func(name string) (config.AgentProfile, error) { return b.cfgHolder.Get().Containers.Profile(name) },
TTL: setupSessionTTL,
}
srv := grpc.NewServer(
//nolint:contextcheck // grpc.StreamServerInterceptor has no context.Context parameter; the interceptor derives its context from stream.Context()
grpc.StreamInterceptor(harnessrpc.StreamServerInterceptor(token)),
)
pb.RegisterHarnessServiceServer(srv, &harnessServer{sessions: manager, log: b.log})
go func() {
if err := srv.Serve(ln); err != nil && !errors.Is(err, grpc.ErrServerStopped) {
b.log.Error("harness listener stopped", "err", err)
}
}()
b.addCleanup(func() {
// An open session stream would hold GracefulStop until its TTL; give
// in-flight frames a moment, then stop. Mirrors the State Store's
// shutdown so a session cannot hold shutdown to the watchdog leash.
timer := time.AfterFunc(10*time.Second, srv.Stop)
srv.GracefulStop()
timer.Stop()
_ = ln.Close()
})
b.log.Info("harness sessions enabled", "addr", ln.Addr().String())
}
var _ kitrun.GrantRegistry = (*sessionGrants)(nil)
+8 -3
View File
@@ -60,8 +60,10 @@ func (b *boot) setupKitLauncher(ctx context.Context) {
return
}
// The proxy resolves each credential from the run credential the
// container logs in with, against the live grants and bindings.
resolver := runCredentialResolver{runs: b.stateStoreGrants.Client, config: b.cfgHolder, secrets: b.secrets}
// container logs in with, against the live grants and bindings; a setup
// session's token resolves through the session grants instead.
b.sessionGrants = &sessionGrants{}
resolver := runCredentialResolver{runs: b.stateStoreGrants.Client, sessions: b.sessionGrants, config: b.cfgHolder, secrets: b.secrets}
// Without a harness secret store, Register refuses a Kit with a
// required OAuth credential.
oauthStore, _ := b.stateStore.(egress.OAuthStore)
@@ -77,11 +79,14 @@ func (b *boot) setupKitLauncher(ctx context.Context) {
Entrypoint: []string{"archie-agent"}, Cmd: cmd, Network: pool.NetworkName(),
})
b.addCleanup(shutdownEgress(networks, srv, log))
b.kitLauncher = &kitrun.Launcher{
kitLauncher := &kitrun.Launcher{
Pool: pool, Fetch: fetcher, Proxy: proxy, Networks: networks,
AgentBinary: agentBinary, CAFile: egress.CACertPath(caDir),
Config: b.cfgHolder, Secrets: b.secrets, OAuth: oauthStore,
Grants: b.sessionGrants,
}
b.kitLauncher = kitLauncher
b.setupHarness(ctx)
log.Info("kit harness runs enabled", "proxy", ln.Addr().String())
}
+21 -3
View File
@@ -8,6 +8,7 @@ import (
"context"
"crypto/sha256"
"encoding/json"
"errors"
"flag"
"fmt"
"log/slog"
@@ -29,6 +30,7 @@ import (
"github.com/samcharles93/archie-core/internal/container"
"github.com/samcharles93/archie-core/internal/daemon"
"github.com/samcharles93/archie-core/internal/domain/applystatus"
"github.com/samcharles93/archie-core/internal/domain/health"
"github.com/samcharles93/archie-core/internal/domain/presence"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/workflow"
@@ -36,6 +38,7 @@ import (
"github.com/samcharles93/archie-core/internal/forge"
"github.com/samcharles93/archie-core/internal/forgerpc"
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
"github.com/samcharles93/archie-core/internal/infrastructure/readiness"
"github.com/samcharles93/archie-core/internal/logging"
"github.com/samcharles93/archie-core/internal/secret"
"github.com/samcharles93/archie-core/internal/storage"
@@ -187,6 +190,7 @@ func Run() int { //nolint:cyclop,funlen // the composition root's setup sequence
}
if ps, ok := b.stateStore.(storecontract.PresenceStore); ok {
go presence.Run(ctx, b.processName, servicekit.Build(), ps, b.healthRegistry, b.log)
b.reportBrokerPresence(ctx, ps)
}
// The daemon publishes the configuration view, which carries the catalog's
// model limits.
@@ -194,9 +198,6 @@ func Run() int { //nolint:cyclop,funlen // the composition root's setup sequence
if err := b.buildTreesAndIdentities(ctx); err != nil {
return 1
}
if err := b.loadWorkflows(); err != nil {
return 1
}
if err := b.registerNATSRPC(); err != nil {
return 1
@@ -288,6 +289,11 @@ func subscribeSystemLogs(nc *natsio.Conn, taskLogs *logging.TaskRegistry, log *s
}
// The NATS callback has no context of its own; the write is an append to
// an already-open sink, so there is nothing to cancel.
if entry.Fields == nil {
entry.Fields = map[string]any{}
}
entry.Fields["service"] = presence.Agent
entry.Fields["task_id"] = taskID
taskLogs.Write(context.Background(), taskID, entry)
})
if err != nil {
@@ -526,3 +532,15 @@ func resolveRegistryAuth(ref secret.SecretRef, secrets *secret.Registry, log *sl
}
return value
}
func (b *boot) reportBrokerPresence(ctx context.Context, ps storecontract.PresenceStore) {
if conn, err := b.natsClient.CoreConn(); err == nil {
registry := health.NewRegistry(readiness.NewContractProbe("connection", 0, func(context.Context) error {
if !conn.IsConnected() {
return errors.New("broker disconnected")
}
return nil
}))
go presence.Run(ctx, presence.Broker, presence.Build{Version: conn.ConnectedServerVersion(), InstallType: b.cfg.NATS.Mode}, ps, registry, b.log)
}
}
-32
View File
@@ -1,32 +0,0 @@
package archied
import (
"flag"
"fmt"
"io"
"strings"
"github.com/samcharles93/archie-core/internal/releaseupdate"
)
func RunReleaseChannel(args []string, in io.Reader, out, stderr io.Writer) int {
flags := flag.NewFlagSet("select-release", flag.ContinueOnError)
flags.SetOutput(stderr)
channel := flags.String("channel", "stable", "stable, next, or exact-pin")
pin := flags.String("pin", "", "exact release version")
if err := flags.Parse(args); err != nil {
return 1
}
data, err := io.ReadAll(in)
if err != nil {
fmt.Fprintln(stderr, err)
return 1
}
version, err := releaseupdate.SelectRelease(strings.Fields(string(data)), *channel, *pin)
if err != nil {
fmt.Fprintln(stderr, err)
return 1
}
fmt.Fprintln(out, version)
return 0
}
+8 -16
View File
@@ -117,21 +117,8 @@ var reloadableFields = map[string]bool{
"ModelLimits": true,
"Notify": true,
"Review": true,
// BotUser/BotEmail used to be poll-path only; ForTask now carries both
// into the task snapshot as well, because the sandboxed worker builds
// its own worktree.Manager and signs its own commits (agentworker
// task_execution.go, config.go ForTask). The commit identity is
// therefore per-dispatch and does reload. The startup-built daemon
// trees (bootstrap.go:654) still capture both, but only to record
// user.name/user.email in the clone's own .git/config
// (worktree.go setIdentity) -- cosmetic, not the signature go-git
// commits with -- so a reload applies where it counts and this stays
// reloadable rather than requires-restart.
"BotEmail": true,
// cfg.Label additionally feeds IssuesWithLabel per poll cycle
// (daemon.go:359,390) and cfg.BotUser feeds AssignedIssues (:364,377).
"Label": true,
"BotUser": true,
// Poll loops read the dispatch label each cycle.
"Label": true,
// MaxRetries has two consumers, both re-reading it: the webui maxRetriesFor
// handler (api_tasks.go:380,383) reads it per request, and ForTask carries
// it into every dispatched TaskConfig, where the container's remediation
@@ -148,7 +135,6 @@ var reloadableSubFields = map[string]map[string]bool{
"VolumeTTL": true, "Profiles": true, "Credentials": true,
"Image": true, "MaxConcurrency": true, "MaxUptime": true, "PullPolicy": true, "Network": true,
},
"Forge": {"Host": true},
// Policy is carried into TaskConfig by ForTask (config.go); MCPServers,
// WebFetch and Minimax are not and stay requires-restart.
"Tools": {"Policy": true},
@@ -188,3 +174,9 @@ func changedNonReloadableFields(old, next config.Config) []string {
}
return changed
}
// Account credentials, identity namespaces and forge listeners survive a reload.
func preserveForgeIdentity(old, next config.Config) config.Config {
next.BotUser, next.BotEmail, next.Forge = old.BotUser, old.BotEmail, old.Forge
return next
}
@@ -0,0 +1,31 @@
package archied
import (
"slices"
"testing"
"github.com/samcharles93/archie-core/internal/config"
)
func TestReloadPreservesForgeIdentity(t *testing.T) {
for _, tt := range []struct {
name, field string
change func(*config.Config)
}{
{"forge host", "Forge", func(c *config.Config) { c.Forge.Host = "other.example" }},
{"bot user", "BotUser", func(c *config.Config) { c.BotUser = "other-account" }},
{"commit email", "BotEmail", func(c *config.Config) { c.BotEmail = "other@example.test" }},
} {
t.Run(tt.name, func(t *testing.T) {
old := config.Config{BotUser: "archie", BotEmail: "archie@example.test", Forge: config.Forge{Host: "forge.example"}}
next := old.Clone()
tt.change(&next)
if restored := preserveForgeIdentity(old, next); restored.BotUser != old.BotUser || restored.BotEmail != old.BotEmail || restored.Forge != old.Forge {
t.Fatal("reload changed the running forge identity")
}
if changed := changedNonReloadableFields(old, next); !slices.Contains(changed, tt.field) {
t.Fatalf("identity change was reloadable: %v", changed)
}
})
}
}
@@ -12,10 +12,18 @@ import (
// credential for the run a run credential names, when the request is made,
// from the live grants and bindings. A revoked credential, a grant removed
// mid-run or a binding in another org stops resolving at once.
//
// A setup session's token names no task, so the resolver consults the session
// grants first: the session is bound for the org's bindings of the Kit's
// declared services, which is the intersection the operator's update-on-secret
// authorisation covers.
type runCredentialResolver struct {
runs interface {
TaskForCredential(ctx context.Context, token string) (*workflow.Task, error)
}
sessions interface {
GrantedFor(token string) (org string, services []string, ok bool)
}
config *config.Holder
secrets interface {
Resolve(config.SecretRef) (string, error)
@@ -23,19 +31,34 @@ type runCredentialResolver struct {
}
func (r runCredentialResolver) Resolve(ctx context.Context, credential, service string) (string, error) {
if r.sessions != nil {
if org, services, ok := r.sessions.GrantedFor(credential); ok {
return r.resolveBinding(org, services, service)
}
}
task, err := r.runs.TaskForCredential(ctx, credential)
if err != nil {
return "", egress.ErrUnbound
}
cfg := r.config.Get()
org, granted := cfg.CredentialAccess(task.Identity)
return r.resolveBinding(org, granted, service)
}
// resolveBinding resolves service from the org's binding when the run or
// session is granted it. An OAuth binding resolves to the empty string: the
// proxy supplies the org's token set itself.
func (r runCredentialResolver) resolveBinding(org string, granted []string, service string) (string, error) {
cfg := r.config.Get()
binding, ok := cfg.Containers.BoundCredentials(org, granted, []string{service})[service]
if !ok {
return "", egress.ErrUnbound
}
if binding.Secret == (config.SecretRef{}) {
// An OAuth binding: the proxy supplies the org's token set itself.
return "", nil
}
if r.secrets == nil {
return "", egress.ErrUnbound
}
return r.secrets.Resolve(binding.Secret)
}
@@ -65,3 +65,47 @@ func TestEgressResolvesOnlyGrantedCredentials(t *testing.T) {
})
}
}
// TestEgressResolvesASessionGrant pins that a setup session's token, which
// names no task, resolves only the granted services of its own org. A grant
// that widened to another org's binding would hand the container a credential
// the operator's update-on-secret authorisation never covered.
func TestEgressResolvesASessionGrant(t *testing.T) {
cfg := config.Config{Containers: config.ContainerConfig{Credentials: []config.CredentialBinding{
{Service: "github", Org: "acme", Secret: config.SecretRef{Engine: "env", Key: "ACME_GH"}},
{Service: "linear", Org: "acme"},
}}}
sessions := &sessionGrants{}
sessions.Grant("acme-session", "acme", []string{"github", "linear"})
sessions.Grant("other-session", "other", []string{"github"})
r := runCredentialResolver{
runs: runsByToken{"acme-run": {ID: 1, Identity: "acme-bot"}},
sessions: sessions,
config: config.NewHolder(cfg),
secrets: secretValues{"ACME_GH": "acme-gh"},
}
tests := []struct {
name, credential, service, want string
bound bool
}{
{name: "a bound api key resolves", credential: "acme-session", service: "github", want: "acme-gh", bound: true},
{name: "a bound oauth service resolves to the proxy's own token set", credential: "acme-session", service: "linear", bound: true},
{name: "a service outside the grant is unbound", credential: "acme-session", service: "nope"},
{name: "another org's binding is unbound", credential: "other-session", service: "github"},
{name: "an unknown token falls through to the task path", credential: "acme-run", service: "github"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := r.Resolve(context.Background(), tt.credential, tt.service)
if !tt.bound {
if !errors.Is(err, egress.ErrUnbound) {
t.Fatalf("got %q, %v; want ErrUnbound", got, err)
}
return
}
if err != nil || got != tt.want {
t.Fatalf("got %q, %v; want %q", got, err, tt.want)
}
})
}
}
-2
View File
@@ -174,8 +174,6 @@ func (s scheduleResourceStore) MarkRun(ctx context.Context, id string, runAt tim
// The stop path derives its own timeout budget from Background rather than
// the boot context, which is already cancelled by the time shutdown runs --
// the same reasoning as shutdownCuratorRuntime.
//
//nolint:contextcheck
func shutdownSchedulingEngine(e *scheduling.Engine, log *slog.Logger) func() {
return func() {
stopCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+2 -13
View File
@@ -72,18 +72,7 @@ func (b *boot) startWatchdog(ctx context.Context) {
// beginning a fresh poll pass, and therefore how stale the progress marker may
// be before the loop counts as stalled.
//
// It is the root poll interval, or the slowest configured identity in
// multi-identity mode: each identity polls its own repos, identity poll
// intervals are frozen at boot, and the marker is stamped by whichever identity
// polled most recently -- so the worst case between stamps is the slowest
// identity, not the fastest. The root value is re-read per sample, so a
// reloaded poll_interval is honoured.
// The shared interval is re-read so live polling changes are honoured.
func (b *boot) loopCadence() time.Duration {
cadence := b.cfgHolder.Get().PollInterval.Std()
for _, id := range b.d.Identities {
if override := id.Cfg.PollInterval.Std(); override > cadence {
cadence = override
}
}
return cadence
return b.cfgHolder.Get().PollInterval.Std()
}
+2
View File
@@ -48,6 +48,8 @@ func (b *boot) openDaemonWorkflowDefinitions() error {
if err != nil {
return fmt.Errorf("build workflow definitions client: %w", err)
}
definitions.ApplyStatus = b.applyStatus
b.controlPlane.ApplyStatus = b.applyStatus
b.workflowDefinitions = definitions
return nil
}
+48
View File
@@ -0,0 +1,48 @@
package archied
import (
"context"
"encoding/json"
"flag"
"fmt"
"io"
"os"
"github.com/samcharles93/archie-core/internal/releaseupdate"
)
// RunUpdate serves `archied update check`, printing the release snapshot as
// JSON. The channel defaults to the environment the installer passes on.
func RunUpdate(args []string, out, stderr io.Writer) int {
if len(args) == 0 || args[0] != "check" {
fmt.Fprintln(stderr, "usage: archied update check [-channel stable|next|exact-pin] [-pin version]")
return 2
}
flags := flag.NewFlagSet("update check", flag.ContinueOnError)
flags.SetOutput(stderr)
channel := flags.String("channel", envOr("ARCHIE_UPDATE_RELEASE_CHANNEL", "stable"), "stable, next, or exact-pin")
pin := flags.String("pin", os.Getenv("ARCHIE_UPDATE_PIN"), "exact release version")
if err := flags.Parse(args[1:]); err != nil {
return 2
}
catalog := &releaseupdate.GitHubCatalog{Channel: func(context.Context) (string, string, error) { return *channel, *pin, nil }}
snapshot, err := catalog.Check(context.Background())
if err != nil {
fmt.Fprintln(stderr, err)
return 1
}
encoder := json.NewEncoder(out)
encoder.SetIndent("", " ")
if err := encoder.Encode(snapshot); err != nil {
fmt.Fprintln(stderr, err)
return 1
}
return 0
}
func envOr(key, fallback string) string {
if value := os.Getenv(key); value != "" {
return value
}
return fallback
}
+16
View File
@@ -2,9 +2,13 @@ package archiegateway
import (
"context"
"fmt"
"slices"
"github.com/samcharles93/archie-core/internal/domain/presence"
"github.com/samcharles93/archie-core/internal/infrastructure/gatewayrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/logrpc"
"github.com/samcharles93/archie-core/internal/logging"
"github.com/samcharles93/archie-core/internal/skill"
)
@@ -21,6 +25,7 @@ func (b *server) setSkillList(entries []skill.CatalogEntry) {
// channels, so channel reload is left unset and answers Unavailable.
func (b *server) chatCatalog() gatewayrpc.Catalog {
return gatewayrpc.Catalog{
Logs: b.recentLogs,
Skills: b.skillEntries,
Curators: b.curatorStates,
}
@@ -86,3 +91,14 @@ func skillSource(entry skill.CatalogEntry, workDir, sharedDir string) string {
return "user-global skills"
}
}
func (b *server) recentLogs(ctx context.Context, service string, q logging.Query) (logging.Result, error) {
switch service {
case "", presence.Gateway:
return b.logFeed.Read(q)
case presence.Daemon:
return logrpc.ReadDaemon(ctx, b.taskActionsConn, q)
default:
return logging.Result{}, fmt.Errorf("unknown log service %q", service)
}
}
+4 -17
View File
@@ -17,6 +17,7 @@ import (
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
"github.com/samcharles93/archie-core/internal/installtype"
"github.com/samcharles93/archie-core/internal/releaseupdate"
"github.com/samcharles93/archie-core/internal/tools"
@@ -95,23 +96,9 @@ func componentInstallTypeEnricher(nats config.NATSConfig) func(string) (string,
}
}
func makeUpdateService(s chatSetup) *releaseupdate.Service {
cfg := s.Cfg.Get()
if len(cfg.Chat.Telegram.UpdateCheckCommand) == 0 {
return nil
}
updates := &releaseupdate.Service{
Catalog: releaseupdate.CommandCatalog{Command: cfg.Chat.Telegram.UpdateCheckCommand},
StatePath: filepath.Join(cfg.WorkDir, "telegram-update-deferrals.json"),
InstallType: installtype.Type(),
Enrich: componentInstallTypeEnricher(cfg.NATS),
}
if len(cfg.Chat.Telegram.UpdateInstallCommand) != 0 {
updates.Installer = releaseupdate.CommandInstaller{
Command: cfg.Chat.Telegram.UpdateInstallCommand,
HealthURL: cfg.Health.URL(),
}
}
func makeUpdateService(reader controlplanerpc.ResourceReader, cfg config.Config) *releaseupdate.Service {
updates := controlplanerpc.UpdateService(reader, filepath.Join(cfg.WorkDir, "telegram-update-deferrals.json"), cfg.Health.URL())
updates.Enrich = componentInstallTypeEnricher(cfg.NATS)
return updates
}
+5 -14
View File
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/taskactions"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/gateway"
@@ -115,7 +116,7 @@ func (a chatTaskListerAdapter) ListChatTasks(ctx context.Context, identity strin
// identity through the tool input.
type chatTaskLogReaderAdapter struct {
tasks func(context.Context, int64) (*workflow.Task, error)
taskLogs *logging.TaskRegistry
taskLogs storecontract.TaskLogStore
}
func (a chatTaskLogReaderAdapter) ReadChatTaskLogs(
@@ -141,19 +142,9 @@ func (a chatTaskLogReaderAdapter) ReadChatTaskLogs(
if attempt <= 0 {
attempt = task.Attempt
}
path := a.taskLogs.Path(taskID, attempt)
if path == "" {
return gateway.ChatTaskLogResult{Attempt: attempt, Entries: []gateway.ChatTaskLogEntry{}}, nil
}
page, err := logging.Page(path, logging.Query{
Component: q.Component,
Contains: q.Contains,
Levels: q.Levels,
Since: q.Since,
Until: q.Until,
Limit: q.Limit,
}, q.AfterID)
page, err := a.taskLogs.TaskLog(ctx, taskID, attempt, logging.Query{
Component: q.Component, Contains: q.Contains, Levels: q.Levels, Since: q.Since, Until: q.Until, Limit: q.Limit, BeforeID: q.AfterID,
})
if err != nil {
return gateway.ChatTaskLogResult{}, err
}
+22 -24
View File
@@ -11,6 +11,8 @@ import (
"github.com/samcharles93/ai-sdk/runtime"
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
"github.com/samcharles93/archie-core/internal/app/controlplane"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/curator"
"github.com/samcharles93/archie-core/internal/events"
infraMemory "github.com/samcharles93/archie-core/internal/infrastructure/memory"
@@ -44,30 +46,6 @@ func (b *server) setupCurators(ctx context.Context) {
log.Error("session-memory curator registration failed", "err", err)
}
for _, def := range b.cfg.Curators {
if !def.Enabled {
continue
}
engine := curator.NewDefinitionEngine(curator.Definition{
Name: def.Name,
Enabled: def.Enabled,
Instructions: def.Instructions,
Manifest: curator.Manifest{
Interval: def.Interval.Std(),
Cooldown: def.Cooldown.Std(),
OnInput: def.OnInput,
Tools: def.Tools,
Skills: def.Skills,
MemoryEngine: def.MemoryEngine,
Conversations: def.Conversations,
Model: def.Model,
},
})
if err := b.curatorRegistry.Register(engine); err != nil {
log.Error("config curator registration failed", "curator", def.Name, "err", err)
}
}
b.curatorRuntime = curator.NewRuntime(b.curatorRegistry, curator.RuntimeConfig{})
curator.WakeOnPrimaryInput(ctx, b.bus, b.curatorRuntime, events.KindTurnCompleted)
rt := b.curatorRuntime
@@ -168,3 +146,23 @@ func (r curatorLLMRunner) Chat(ctx context.Context, req curator.ChatRequest) (cu
}
return curator.ChatResult{Text: res.Text, ToolCalls: calls}, nil
}
func (b *server) applyCurators(ctx context.Context, defs []config.CuratorDefinition) error {
definitions := make([]curator.Definition, 0, len(defs))
for _, def := range defs {
definitions = append(definitions, controlplane.CuratorDefinition(def))
}
return b.curatorRuntime.ReplaceDefinitions(ctx, definitions)
}
func (b *server) watchCurators(ctx context.Context, version int64) error {
updates, err := b.controlPlane.WatchCurators(ctx, version)
if err != nil {
return err
}
watchStatus(ctx, b.applyStatus.Report, b.log, controlplane.CuratorsKind, "curators", version, updates, b.controlPlane.WatchCurators,
func(update controlplane.AppliedCurators) int64 { return update.Version },
func(update controlplane.AppliedCurators) error { return update.Err },
func(update controlplane.AppliedCurators) error { return b.applyCurators(ctx, update.Definitions) })
return nil
}
+20 -4
View File
@@ -8,10 +8,13 @@ import (
"google.golang.org/grpc"
"github.com/samcharles93/archie-core/internal/app/controlplane"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/gatewayrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/rpcidentity"
"github.com/samcharles93/archie-core/internal/plugin"
)
@@ -66,6 +69,18 @@ func (b *server) startGatewayRuntime(ctx context.Context, actor gateway.ChatTask
if err := b.curatorRuntime.Start(ctx); err != nil {
b.log.Error("curator runtime startup", "err", err)
}
defs, version, err := b.controlPlane.Curators(ctx)
if err == nil {
err = b.applyCurators(ctx, defs)
}
b.applyStatus.Report(ctx, controlplane.CuratorsKind, version, err)
if err != nil {
b.log.Error("curator definitions apply failed", "err", err)
}
if err := b.watchCurators(ctx, version); err != nil {
return nil, fmt.Errorf("watch curators: %w", err)
}
return contract, nil
}
@@ -85,14 +100,15 @@ const inboundMessageHeadroomBytes = 1 << 20
// request carries a channel attachment's bytes: the channel frontend
// downloads the file, and this process -- which runs the turn -- holds no
// platform credential to fetch it again.
func gatewayServerOpts(listen, token string) (opts []grpc.ServerOption, loopback bool, err error) {
func gatewayServerOpts(listen, token string, principals access.PrincipalSource) (opts []grpc.ServerOption, loopback bool, err error) {
callers := rpcidentity.Callers{Principals: principals}
recv := grpc.MaxRecvMsgSize(messaging.MaxInboundAttachmentBytes + inboundMessageHeadroomBytes)
loopback, err = gatewayrpc.TargetIsLoopback(listen)
if err != nil {
return nil, false, err
}
if loopback {
return []grpc.ServerOption{recv}, true, nil
return []grpc.ServerOption{recv, grpc.ChainUnaryInterceptor(callers.Unary()), grpc.ChainStreamInterceptor(callers.Stream())}, true, nil
}
if token == "" {
return nil, false, fmt.Errorf(
@@ -102,8 +118,8 @@ func gatewayServerOpts(listen, token string) (opts []grpc.ServerOption, loopback
}
return []grpc.ServerOption{
recv,
grpc.ChainUnaryInterceptor(gatewayrpc.UnaryServerInterceptor(token)),
grpc.ChainStreamInterceptor(gatewayrpc.StreamServerInterceptor(token)),
grpc.ChainUnaryInterceptor(gatewayrpc.UnaryServerInterceptor(token), callers.Unary()),
grpc.ChainStreamInterceptor(gatewayrpc.StreamServerInterceptor(token), callers.Stream()),
}, false, nil
}
+13 -8
View File
@@ -12,8 +12,10 @@ import (
"github.com/samcharles93/archie-core/internal/buildinfo"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/agent"
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/domain/taskactions"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
"github.com/samcharles93/archie-core/internal/taskstate"
"github.com/samcharles93/archie-core/internal/tools"
)
@@ -86,7 +88,7 @@ func (b *server) setupChatRuntime(ctx context.Context, cfg config.Config, actor
},
cancelExecution: b.stateStore.CancelExecution,
})
b.updateService = makeUpdateService(chatSetup{Cfg: config.NewHolder(cfg)})
b.updateService = makeUpdateService(b.controlPlane, cfg)
return nil
}
@@ -115,7 +117,7 @@ func watchStatus[T any](
open func(ctx context.Context, after int64) (<-chan T, error),
versionOf func(T) int64,
errOf func(T) error,
deliver func(T),
deliver func(T) error,
) {
var lastApplyErr, streamErr error
watchOpen := func(ctx context.Context, after int64) (<-chan T, error) {
@@ -140,7 +142,8 @@ func watchStatus[T any](
log.Error("control-plane watch failed", "kind", kind, "err", err)
return
}
deliver(update)
lastApplyErr = deliver(update)
report(ctx, kind, versionOf(update), lastApplyErr)
},
func() {
// A refusal already on the record outranks the outage; it is the
@@ -171,9 +174,9 @@ func (b *server) watchPersonas(ctx context.Context, version int64) error {
b.controlPlane.WatchPersonas,
func(update controlplane.AppliedPersonas) int64 { return update.Version },
func(update controlplane.AppliedPersonas) error { return update.Err },
func(update controlplane.AppliedPersonas) {
func(update controlplane.AppliedPersonas) error {
b.applyPersonas(update.Collection, update.Version)
b.applyStatus.Report(ctx, controlplane.PersonasKind, update.Version, nil)
return nil
})
return nil
}
@@ -200,9 +203,9 @@ func (b *server) watchSoul(ctx context.Context, version int64) error {
b.controlPlane.WatchSoul,
func(update controlplane.AppliedSoul) int64 { return update.Version },
func(update controlplane.AppliedSoul) error { return update.Err },
func(update controlplane.AppliedSoul) {
func(update controlplane.AppliedSoul) error {
b.applySoul(update.Soul, update.Version)
b.applyStatus.Report(ctx, controlplane.SoulKind, update.Version, nil)
return nil
})
return nil
}
@@ -220,6 +223,7 @@ func (b *server) setupGatewayChat(ctx context.Context, actor gateway.ChatTaskAct
}
cfg := b.cfg
router := gateway.NewRouter(b.stateStore, nil, "web")
router.Settings = messaging.NewSettingsCommand(controlplanerpc.NewSettingsClient(b.stateStore.ControlPlane())).WithIdentities(b.stateStore)
router.Limiter = b.rateLimiter
router.Version = fmt.Sprintf("Archie\nGateway: %s\nRuntime: %s", buildinfo.Version, buildinfo.Runtime)
router.Models = b.chatModels
@@ -237,7 +241,7 @@ func (b *server) setupGatewayChat(ctx context.Context, actor gateway.ChatTaskAct
LLM: b.chatLLM, ChatModels: b.chatModels, ToolReg: b.toolReg,
Soul: b.soul, ChatTasks: b.chatTasks,
ChatTaskLister: chatTaskListerAdapter{tasks: b.stateStore.Tasks},
ChatTaskLogs: chatTaskLogReaderAdapter{tasks: b.stateStore.TaskByID, taskLogs: b.taskLogs},
ChatTaskLogs: chatTaskLogReaderAdapter{tasks: b.stateStore.TaskByID, taskLogs: b.stateStore},
ChatTaskActor: actor,
DefaultChatIdentity: b.defaultChatIdentity,
Bus: b.bus, Log: b.log,
@@ -256,5 +260,6 @@ func (b *server) setupGatewayChat(ctx context.Context, actor gateway.ChatTaskAct
return &gateway.LocalChatAdapter{
Router: router, Sessions: b.chatSessionStore, Turns: gateway.NewTurns(b.log),
Models: b.chatModels, Personas: b.personas, TaskActor: actor, Tasks: b.stateStore,
VoiceAvailable: b.transcriber != nil,
}, nil
}
+27 -5
View File
@@ -21,6 +21,7 @@ import (
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/applystatus"
"github.com/samcharles93/archie-core/internal/domain/curator"
"github.com/samcharles93/archie-core/internal/domain/health"
domainmemory "github.com/samcharles93/archie-core/internal/domain/memory"
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/domain/presence"
@@ -28,6 +29,7 @@ import (
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
"github.com/samcharles93/archie-core/internal/infrastructure/modelcatalog"
"github.com/samcharles93/archie-core/internal/infrastructure/readiness"
"github.com/samcharles93/archie-core/internal/infrastructure/secretengine"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/infrastructure/taskactions"
@@ -61,9 +63,9 @@ type server struct {
cfgHolder *config.Holder
cfgPath string
log *slog.Logger
taskLogs *logging.TaskRegistry
secrets *secret.Registry
engines *secretengine.Runner
stateStore *staterpc.Client
controlPlane *controlplane.Client
applyStatus *applystatus.Reporter
@@ -71,6 +73,7 @@ type server struct {
catalog *modelcatalog.Catalog
chatPool *pgxpool.Pool
chatSessionStore gateway.SessionStore
logFeed *logging.Feed
taskActionsConn *natsio.Conn
bus *events.Bus
@@ -131,7 +134,7 @@ func Run(ctx context.Context, options Options) error {
token = b.cfg.Services.ResolvedToken(config.ServiceNameGateway, b.secrets.Getenv)
}
//nolint:contextcheck // grpc.StreamServerInterceptor has no context.Context parameter; gatewayrpc.StreamServerInterceptor derives its context from stream.Context() instead
opts, loopback, err := gatewayServerOpts(listen, token)
opts, loopback, err := gatewayServerOpts(listen, token, b.stateStore)
if err != nil {
return err
}
@@ -164,7 +167,7 @@ func (b *server) loadConfig(cfgPath, overlayPath string) error {
b.catalog = modelcatalog.NewCatalog(cfgPath)
b.cfgHolder = config.NewHolder(b.cfg)
logs := servicekit.Logging(b.cfg, "gateway")
b.log, b.taskLogs = logs.Log, logs.TaskLogs
b.log, b.logFeed = logs.Log, logs.Feed
b.addCleanup(func() { _ = logs.Closer.Close() })
b.secrets = secret.NewRegistry()
return nil
@@ -191,7 +194,8 @@ func (b *server) openState(ctx context.Context) error {
report := func(ctx context.Context, version int64, err error) {
b.applyStatus.Report(ctx, controlplanerpc.ExtensionSettingsKind, version, err)
}
b.addCleanup(secretengine.Supervise(ctx, b.secrets, source, applystatus.Gateway, report, b.log).Close)
b.engines = secretengine.Supervise(ctx, b.secrets, source, applystatus.Gateway, report, b.log)
b.addCleanup(b.engines.Close)
cfg, versions, err := b.runtimeConfig(ctx, b.cfg)
if err != nil {
return fmt.Errorf("load runtime settings: %w", err)
@@ -199,7 +203,7 @@ func (b *server) openState(ctx context.Context) error {
b.cfg, b.runtimeVersions = cfg, versions
b.cfgHolder.Set(cfg)
go b.applyStatus.Run(ctx)
go presence.Run(ctx, presence.Gateway, servicekit.Build(), client, nil, b.log)
return nil
}
@@ -230,6 +234,16 @@ func (b *server) connectNATS(ctx context.Context) (*natsio.Conn, error) {
}
b.addCleanup(nc.Close)
b.taskActionsConn = nc
registry := health.NewRegistry(
readiness.NewContractProbe("state_store", b.cfg.Health.DependencyTimeout.Std(), func(ctx context.Context) error { _, err := b.stateStore.StatusCounts(ctx); return err }),
readiness.NewContractProbe("nats", b.cfg.Health.DependencyTimeout.Std(), func(context.Context) error {
if !nc.IsConnected() {
return fmt.Errorf("broker disconnected")
}
return nil
}),
)
go presence.Run(ctx, presence.Gateway, servicekit.Build(), b.stateStore, registry, b.log)
return nc, nil
}
@@ -286,6 +300,14 @@ var gatewayRuntimeKinds = []string{
}
func (b *server) startRuntimeWatches(ctx context.Context) error {
// An engine that starts after boot answers references that failed then, so
// the providers resolve again.
b.engines.OnChange(func() {
base := b.cfgHolder.Get()
catalog, _ := b.catalogState()
modelcatalog.Apply(&base, catalog)
_ = b.relayer(ctx, base, controlplane.ProviderSettingsKind)
})
for _, kind := range gatewayRuntimeKinds {
updates, err := b.controlPlane.WatchResource(ctx, kind, b.runtimeVersions[kind])
if err != nil {
+15 -7
View File
@@ -40,10 +40,18 @@ func (b *server) refreshSkills(ctx context.Context, applied *string) {
cfg, log := b.cfg, b.log
catalog, err := skill.CatalogRoots(skill.DefaultRoots(cfg.WorkDir, cfg.SkillsDir)...)
if err != nil {
b.applyStatus.Report(ctx, controlplane.SkillsKind, 0, err)
log.Warn("skill catalog load failed", "err", err)
return
}
catalog = append(catalog, b.storedSkills(ctx, catalog)...)
stored, version, err := b.storedSkills(ctx, catalog)
if err != nil {
b.applyStatus.Report(ctx, controlplane.SkillsKind, version, err)
return
}
var applyErr error
defer func() { b.applyStatus.Report(ctx, controlplane.SkillsKind, version, applyErr) }()
catalog = append(catalog, stored...)
b.setSkillList(catalog)
signature := catalogSignature(catalog)
if signature == *applied {
@@ -52,6 +60,7 @@ func (b *server) refreshSkills(ctx context.Context, applied *string) {
b.toolReg.Unregister(skillActivateTool)
if entry := skill.ActivateTool(cfg.WorkDir, catalog); entry != nil {
if err := b.toolReg.Register(*entry); err != nil {
applyErr = err
log.Warn("skill_activate registration failed", "err", err)
return
}
@@ -61,10 +70,10 @@ func (b *server) refreshSkills(ctx context.Context, applied *string) {
}
// storedSkills reads the skills resource, leaving out any name already in
// have. An unreadable resource contributes nothing this round.
func (b *server) storedSkills(ctx context.Context, have []skill.CatalogEntry) []skill.CatalogEntry {
// have. An unreadable resource leaves the running catalog in place.
func (b *server) storedSkills(ctx context.Context, have []skill.CatalogEntry) ([]skill.CatalogEntry, int64, error) {
var out []skill.CatalogEntry
_, _, err := b.controlPlane.Query(ctx, controlplane.SkillsKind, func(value []byte) error {
version, _, err := b.controlPlane.Query(ctx, controlplane.SkillsKind, func(value []byte) error {
var collection controlplane.SkillCollection
if err := json.Unmarshal(value, &collection); err != nil {
return err
@@ -72,8 +81,7 @@ func (b *server) storedSkills(ctx context.Context, have []skill.CatalogEntry) []
for _, held := range collection.Skills {
entry, err := skill.EntryFromContent([]byte(held.Content))
if err != nil {
b.log.Warn("stored skill skipped", "skill", held.Name, "err", err)
continue
return err
}
if !slices.ContainsFunc(have, func(e skill.CatalogEntry) bool { return e.Name == entry.Name }) {
out = append(out, entry)
@@ -84,7 +92,7 @@ func (b *server) storedSkills(ctx context.Context, have []skill.CatalogEntry) []
if err != nil {
b.log.Warn("skills resource unavailable", "err", err)
}
return out
return out, version, err
}
func catalogSignature(catalog []skill.CatalogEntry) string {
+10 -2
View File
@@ -8,11 +8,13 @@ import (
"strings"
"time"
"github.com/samcharles93/ai-sdk/provider/minimax"
"github.com/samcharles93/archie-core/internal/agentexec/modelloop"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/plugin"
"github.com/samcharles93/archie-core/internal/tools/mcp"
"github.com/samcharles93/archie-core/internal/tools/minimax"
minimaxtool "github.com/samcharles93/archie-core/internal/tools/minimax"
toolprovider "github.com/samcharles93/archie-core/internal/tools/provider"
builtintoolprovider "github.com/samcharles93/archie-core/internal/tools/provider/builtin"
mcptoolprovider "github.com/samcharles93/archie-core/internal/tools/provider/mcp"
@@ -123,7 +125,13 @@ func (b *server) registerMinimaxTool(cfg config.Config, log *slog.Logger) {
return
}
entry := minimax.Tool(minimax.Config{Enabled: true, APIKey: apiKey, BaseURL: cfg.Tools.Minimax.BaseURL})
provider, err := minimax.New(minimax.Config{APIKey: apiKey, BaseURL: cfg.Tools.Minimax.BaseURL})
if err != nil {
log.Warn("minimax video generation enabled but the provider could not be built; tool not registered", "err", err)
return
}
entry := minimaxtool.Tool(provider)
if entry == nil {
return
}
@@ -21,7 +21,7 @@ func channelDescriptors(instances []*channelInstance) []status.Descriptor {
descriptors = append(descriptors, status.Descriptor{
ID: instance.name,
Name: instance.name,
Configured: true,
Configured: instance.current() != nil,
// The capability comes from the same place the action does, so a
// channel cannot declare reload support and then answer nothing.
ReloadSupported: reloadable(instance),
@@ -35,7 +35,14 @@ func channelDescriptors(instances []*channelInstance) []status.Descriptor {
// a reload performs; deriving them separately is how a dashboard ends up offering
// a button that reports success and changes nothing.
func reloadable(instance *channelInstance) bool {
gateway, ok := instance.current().(*telegram.Gateway)
return channelReloadable(instance.current())
}
// channelReloadable reports whether a built channel can re-read its
// configuration in place. A disabled channel (nil) cannot, which is what a
// channel that enablement turned off should declare.
func channelReloadable(ch channels.Channel) bool {
gateway, ok := ch.(*telegram.Gateway)
return ok && gateway.Reload != nil
}
+27 -11
View File
@@ -19,6 +19,7 @@ import (
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/infrastructure/messagingrpc"
"github.com/samcharles93/archie-core/internal/releaseupdate"
"github.com/samcharles93/archie-core/internal/sdnotify"
"github.com/samcharles93/archie-core/internal/secret"
)
@@ -43,6 +44,9 @@ type deps struct {
// SettingsSource enables live channel-settings reconciliation. Nil disables
// it.
SettingsSource chatSettingsSource
// Updates serves /update from the stored update settings. Nil without a
// State Store.
Updates *releaseupdate.Service
// ApplyReporter records what this process applied. Nil reports nothing.
ApplyReporter *applystatus.Reporter
// AppliedVersion is the channel-settings version already running when the
@@ -54,7 +58,7 @@ type deps struct {
}
// channelInstance is one composed channel and what is needed to restart it.
// mu guards channel and cancel.
// mu guards every field below.
type channelInstance struct {
name string
rebuild func(ResolvedConfig) (channels.Channel, error)
@@ -64,6 +68,9 @@ type channelInstance struct {
cancel context.CancelFunc
supervised bool
restartPending bool
// notify wakes a supervisor that is between runs, so an explicit restart
// starts the replacement instead of waiting out a backoff or a park.
notify chan struct{}
}
// current returns the channel instance currently serving, under the lock.
@@ -104,6 +111,9 @@ type Service struct {
appliedVersion atomic.Int64
// reconcileInterval is how often the stored settings are re-read.
reconcileInterval time.Duration
// retry bounds how a supervisor retries a channel that fails on its own
// before parking it for an explicit restart.
retry channelRetryPolicy
mu sync.Mutex
running bool
@@ -131,6 +141,7 @@ func compose(ctx context.Context, d deps) (*Service, error) {
if srv.reconcileInterval <= 0 {
srv.reconcileInterval = applystatus.RestampInterval
}
srv.retry = defaultChannelRetry()
instances, err := composeChannels(ctx, d)
if err != nil {
@@ -149,13 +160,15 @@ func compose(ctx context.Context, d deps) (*Service, error) {
// still restarted alone rather than through a process restart.
func composeChannels(ctx context.Context, d deps) ([]*channelInstance, error) {
instances := make([]*channelInstance, 0, 3)
if d.Config.TelegramToken != "" {
instance, err := composeTelegram(ctx, d)
if err != nil {
return nil, err
}
instances = append(instances, instance)
// Telegram is composed whether or not it is enabled: a channel that
// enablement can turn on has to exist for the supervisor to start. Its
// channel is nil until a token is configured (composeTelegram). The other
// channels are fixed by what the process can build at startup.
telegramInstance, err := composeTelegram(ctx, d)
if err != nil {
return nil, err
}
instances = append(instances, telegramInstance)
for _, instance := range d.ExtensionChannels {
if slices.ContainsFunc(instances, func(have *channelInstance) bool { return have.name == instance.name }) {
d.Log.Warn("channel extension skipped: a built-in channel has the same name", "channel", instance.name)
@@ -163,18 +176,21 @@ func composeChannels(ctx context.Context, d deps) ([]*channelInstance, error) {
}
instances = append(instances, instance)
}
for _, instance := range instances {
instance.notify = make(chan struct{}, 1)
}
return instances, nil
}
func composeTelegram(ctx context.Context, d deps) (*channelInstance, error) {
if len(d.Config.Telegram.AllowedUserIDs) == 0 {
d.Log.Warn("chat.telegram has no allowed_user_ids: every sender will be rejected. " +
"Add your Telegram user id to chat.telegram.allowed_user_ids to enable the bot.")
}
build := func(cfg ResolvedConfig) (channels.Channel, error) {
if cfg.TelegramToken == "" {
return nil, nil
}
if len(cfg.Telegram.AllowedUserIDs) == 0 {
d.Log.Warn("chat.telegram has no allowed_user_ids: every sender will be rejected. " +
"Add your Telegram user id to chat.telegram.allowed_user_ids to enable the bot.")
}
tg := telegram.New(cfg.TelegramToken, cfg.Telegram.AllowedUserIDs, d.Log)
tg.Settings = d.Settings
configureTelegram(ctx, tg, cfg, d)
@@ -0,0 +1,241 @@
package archiemessaging
import (
"context"
"encoding/json"
"log/slog"
"sync"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/channels"
"github.com/samcharles93/archie-core/internal/channels/status"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/secret"
)
// servingChannel reports itself running once it starts, which is what an
// enable is reported active on: a channel that only reaches "starting" is not
// serving yet.
type servingChannel struct {
mu sync.Mutex
starts int
}
func (c *servingChannel) Name() string { return "telegram" }
func (c *servingChannel) Start(ctx context.Context, _ messaging.ChatContract, lifecycle channels.Lifecycle) error {
c.mu.Lock()
c.starts++
c.mu.Unlock()
if lifecycle.Starting != nil {
lifecycle.Starting()
}
if lifecycle.Running != nil {
lifecycle.Running()
}
<-ctx.Done()
return nil
}
func (c *servingChannel) Stop(context.Context) error { return nil }
func (c *servingChannel) ConfigSchema() json.RawMessage { return nil }
func (c *servingChannel) ValidateConfig(map[string]any) error { return nil }
func (c *servingChannel) count() int {
c.mu.Lock()
defer c.mu.Unlock()
return c.starts
}
// waitCount blocks until count reaches want, so an async start is observed
// rather than slept for.
func waitCount(t *testing.T, count func() int, want int) {
t.Helper()
deadline := time.Now().Add(5 * time.Second)
for time.Now().Before(deadline) {
if count() >= want {
return
}
time.Sleep(time.Millisecond)
}
t.Fatalf("channel started %d times, want at least %d", count(), want)
}
// startEnablementService composes and starts a service with the Telegram
// channel's factory replaced by a scripted channel, so enablement can be driven
// without a real bot.
func startEnablementService(t *testing.T, ch *servingChannel) *Service {
t.Helper()
srv, err := compose(context.Background(), deps{Log: slog.Default()})
if err != nil {
t.Fatalf("compose: %v", err)
}
inst := srv.instanceByID("telegram")
if inst == nil {
t.Fatal("telegram channel was not composed")
}
inst.rebuild = func(ResolvedConfig) (channels.Channel, error) { return ch, nil }
srv.retry = channelRetryPolicy{attempts: 3, backoff: time.Millisecond, maxBackoff: time.Millisecond}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() {
defer close(done)
_ = srv.Start(ctx)
}()
t.Cleanup(func() {
cancel()
select {
case <-done:
case <-time.After(5 * time.Second):
t.Error("service did not stop after cancel")
}
})
waitSupervised(t, srv, "telegram")
return srv
}
// waitSupervised blocks until the channel's supervisor exists, so an enable
// issued right after Start is not refused for racing it.
func waitSupervised(t *testing.T, srv *Service, id string) {
t.Helper()
deadline := time.Now().Add(5 * time.Second)
for time.Now().Before(deadline) {
inst := srv.instanceByID(id)
inst.mu.Lock()
supervised := inst.supervised
inst.mu.Unlock()
if supervised {
return
}
time.Sleep(time.Millisecond)
}
t.Fatalf("channel %q was never supervised", id)
}
func channelByID(t *testing.T, srv *Service, id string) status.Status {
t.Helper()
for _, st := range srv.ChannelStatus() {
if st.ID == id {
return st
}
}
t.Fatalf("no channel %q in status", id)
return status.Status{}
}
// TestChannelEnablementAppliesLive pins that enabling and disabling a channel
// starts and stops it without a service restart, and that its declaration
// follows: a channel that was never configured is not reported as configured,
// and one that enablement stopped is reported stopped and unconfigured.
func TestChannelEnablementAppliesLive(t *testing.T) {
ch := &servingChannel{}
srv := startEnablementService(t, ch)
if before := channelByID(t, srv, "telegram"); before.Configured || before.State != status.StateStopped {
t.Fatalf("telegram before enable = %+v, want not configured and stopped", before)
}
if err := srv.enableChannel("telegram", ResolvedConfig{TelegramToken: "token"}); err != nil {
t.Fatalf("enable: %v", err)
}
waitCount(t, ch.count, 1)
if enabled := channelByID(t, srv, "telegram"); !enabled.Configured || enabled.State != status.StateRunning {
t.Fatalf("telegram after enable = %+v, want configured and running", enabled)
}
if err := srv.disableChannel("telegram"); err != nil {
t.Fatalf("disable: %v", err)
}
if disabled := channelByID(t, srv, "telegram"); disabled.Configured || disabled.State != status.StateStopped {
t.Fatalf("telegram after disable = %+v, want not configured and stopped", disabled)
}
}
// fakeSettings serves a stored chat config at a version, the way the
// control-plane client does.
type fakeSettings struct {
chat config.ChatConfig
version int64
}
func (f fakeSettings) RuntimeChatConfig(context.Context, config.ChatConfig) (config.ChatConfig, int64, error) {
return f.chat, f.version, nil
}
// fakeEngine resolves every key, so a stored token reference is resolvable.
type fakeEngine struct{}
func (fakeEngine) Name() string { return "test" }
func (fakeEngine) Resolve(key string) (string, error) { return "token-" + key, nil }
// TestReconcileOnceEnablesAndDisables drives the whole stored-settings path:
// a settings version that configures a token enables the channel and is
// recorded as applied only once the channel is running it; a later version
// that clears the token disables it.
func TestReconcileOnceEnablesAndDisables(t *testing.T) {
ch := &servingChannel{}
srv := startEnablementService(t, ch)
srv.secrets = secret.NewRegistry()
srv.secrets.Register(fakeEngine{})
srv.settingsSource = fakeSettings{
chat: config.ChatConfig{Telegram: config.TelegramConfig{Token: secret.SecretRef{Engine: "test", Key: "bot"}}},
version: 1,
}
if err := srv.reconcileOnce(context.Background()); err != nil {
t.Fatalf("reconcile enable: %v", err)
}
waitCount(t, ch.count, 1)
if got := channelByID(t, srv, "telegram"); !got.Configured {
t.Fatalf("telegram not enabled by reconcile: %+v", got)
}
if srv.appliedVersion.Load() != 1 {
t.Fatalf("appliedVersion = %d after enable, want 1", srv.appliedVersion.Load())
}
srv.settingsSource = fakeSettings{version: 2}
if err := srv.reconcileOnce(context.Background()); err != nil {
t.Fatalf("reconcile disable: %v", err)
}
if got := channelByID(t, srv, "telegram"); got.Configured || got.State != status.StateStopped {
t.Fatalf("telegram not disabled by reconcile: %+v", got)
}
if srv.appliedVersion.Load() != 2 {
t.Fatalf("appliedVersion = %d after disable, want 2", srv.appliedVersion.Load())
}
}
// TestApplyChannelSettingsEnablesAndDisables pins the reconcile entry point:
// a settings change that turns a channel on or off starts or stops it and
// reports applied, and a tick that changes nothing is a no-op.
func TestApplyChannelSettingsEnablesAndDisables(t *testing.T) {
ch := &servingChannel{}
srv := startEnablementService(t, ch)
applied, err := srv.applyChannelSettings(ResolvedConfig{TelegramToken: "token"})
if err != nil || !applied {
t.Fatalf("enable: applied=%v err=%v", applied, err)
}
waitCount(t, ch.count, 1)
if got := channelByID(t, srv, "telegram"); !got.Configured || got.State == status.StateStopped {
t.Fatalf("telegram not enabled: %+v", got)
}
applied, err = srv.applyChannelSettings(ResolvedConfig{})
if err != nil || !applied {
t.Fatalf("disable: applied=%v err=%v", applied, err)
}
if got := channelByID(t, srv, "telegram"); got.Configured || got.State != status.StateStopped {
t.Fatalf("telegram not disabled: %+v", got)
}
applied, err = srv.applyChannelSettings(ResolvedConfig{})
if err != nil || !applied {
t.Fatalf("no-op tick: applied=%v err=%v", applied, err)
}
}
+36 -30
View File
@@ -85,30 +85,32 @@ func resolveChatSecrets(base ResolvedConfig, layered config.ChatConfig, secrets
return next, nil
}
// applyChannelSettings restarts only channels whose transport settings
// changed and reports whether every restart succeeded. Enabling or
// disabling a channel or changing a listen address needs a process restart.
// applyChannelSettings applies the changed channel settings: channels whose
// enablement changed are started or stopped, channels whose transport settings
// changed are restarted, and the report is success only once every change is in
// effect.
func (s *Service) applyChannelSettings(next ResolvedConfig) (bool, error) {
current := s.currentConfig()
effective, outstanding := pinProcessBindings(current, next)
if next.ShowToolCalls != current.ShowToolCalls {
s.applyShowToolCalls(next.ShowToolCalls)
}
applied := true
for _, id := range channelChanges(current, effective) {
if err := s.restartChannel(id, effective); err != nil {
outstanding := s.applyChannelEnablement(current, next)
if len(outstanding) > 0 {
applied = false
}
for _, id := range channelChanges(current, next) {
if err := s.restartChannel(id, next); err != nil {
outstanding = append(outstanding, err.Error())
applied = false
}
}
// A failed restart leaves the running configuration in place, so the next
// tick sees the same change and retries it. A change that was carried out
// (including a binding change that was refused and reported) is recorded, so
// it is not applied again on every tick.
// A failed change leaves the running configuration in place, so the next
// tick sees the same difference and retries it.
if applied {
s.setConfig(effective)
s.setConfig(next)
}
if len(outstanding) == 0 {
return applied, nil
@@ -116,6 +118,24 @@ func (s *Service) applyChannelSettings(next ResolvedConfig) (bool, error) {
return applied, errors.New(strings.Join(outstanding, "; "))
}
// applyChannelEnablement starts or stops the channels whose enablement the
// stored settings changed, and reports only once the requested state is in
// effect. A channel whose enablement did not change is left alone.
func (s *Service) applyChannelEnablement(current, next ResolvedConfig) []string {
var outstanding []string
switch {
case current.TelegramToken != "" && next.TelegramToken == "":
if err := s.disableChannel("telegram"); err != nil {
outstanding = append(outstanding, err.Error())
}
case current.TelegramToken == "" && next.TelegramToken != "":
if err := s.enableChannel("telegram", next); err != nil {
outstanding = append(outstanding, err.Error())
}
}
return outstanding
}
// applyShowToolCalls updates the running Telegram gateways in place. Nothing
// else consumes the setting, and it is read per reply, so it needs no restart.
func (s *Service) applyShowToolCalls(show bool) {
@@ -126,28 +146,14 @@ func (s *Service) applyShowToolCalls(show bool) {
}
}
// pinProcessBindings holds the values a running process cannot change in place:
// the set of channels it composed.
// The returned configuration keeps the running value for each, and the returned
// problems name what the operator changed and when it takes effect.
func pinProcessBindings(current, next ResolvedConfig) (ResolvedConfig, []string) {
effective := next
var problems []string
if (current.TelegramToken != "") != (next.TelegramToken != "") {
problems = append(problems, "chat.telegram enabled/disabled; takes effect on restart")
effective.Telegram, effective.TelegramToken = current.Telegram, current.TelegramToken
}
return effective, problems
}
// channelChanges returns the composed channels whose transport settings differ
// between the running and the effective configuration. A channel absent from
// current (not composed) never appears, and an unchanged channel is not
// restarted.
// between the running and the next configuration while remaining enabled. A
// channel absent from current (not composed) never appears, an unchanged
// channel is not restarted, and an enable or disable is applied as enablement
// rather than as a transport change.
func channelChanges(current, next ResolvedConfig) []string {
var changed []string
if current.TelegramToken != "" &&
if current.TelegramToken != "" && next.TelegramToken != "" &&
(current.TelegramToken != next.TelegramToken || !slices.Equal(current.Telegram.AllowedUserIDs, next.Telegram.AllowedUserIDs)) {
changed = append(changed, "telegram")
}
+11 -4
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"log/slog"
"net"
"path/filepath"
"google.golang.org/grpc"
@@ -20,6 +21,8 @@ import (
"github.com/samcharles93/archie-core/internal/infrastructure/messagingrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/secretengine"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/logging"
"github.com/samcharles93/archie-core/internal/releaseupdate"
"github.com/samcharles93/archie-core/internal/secret"
)
@@ -27,6 +30,7 @@ import (
func Run(ctx context.Context, o Options) error {
log := slog.Default().With("service", "archie-messaging")
log, feed := servicekit.Diagnostics(log, presence.Messaging)
cfg, err := Resolve(o, log)
if err != nil {
return fmt.Errorf("resolve messaging options: %w", err)
@@ -52,6 +56,7 @@ func Run(ctx context.Context, o Options) error {
var settingsSource chatSettingsSource
var applyReporter *applystatus.Reporter
var appliedVersion int64
var updates *releaseupdate.Service
var extensionChannels []*channelInstance
if cfg.Options.StateStore.Target != "" {
stateStore, closeClient, dialErr := staterpc.Dial(cfg.Options.StateStore.Target, presence.Messaging, cfg.Options.StateStore.Token, staterpc.WaitForPeer)
@@ -78,9 +83,10 @@ func Run(ctx context.Context, o Options) error {
return fmt.Errorf("resolve database telegram token: %w", settingsErr)
}
extensionChannels = openExtensionChannels(ctx, extensions, secrets, log)
settings = messaging.NewSettingsCommand(messagingControlPlane{client: stateStore.ControlPlane()}).WithIdentities(stateStore)
settings = messaging.NewSettingsCommand(controlplanerpc.NewSettingsClient(stateStore.ControlPlane())).WithIdentities(stateStore)
channelStatusStore, presenceStore = stateStore, stateStore
settingsSource, applyReporter, appliedVersion = controlPlaneClient, reporter, channelVersion
updates = controlplanerpc.UpdateService(controlPlaneClient, filepath.Join(cfg.WorkDir, "telegram-update-deferrals.json"), cfg.HealthURL)
}
if cfg.Options.StateStore.Target == "" {
@@ -101,15 +107,16 @@ func Run(ctx context.Context, o Options) error {
SettingsSource: settingsSource,
ApplyReporter: applyReporter,
AppliedVersion: appliedVersion,
Updates: updates,
})
if err != nil {
return err
}
return serve(ctx, cfg.Options, srv, log)
return serve(ctx, cfg.Options, srv, log, feed)
}
// serve runs srv and serves MessagingService over it until ctx is cancelled.
func serve(ctx context.Context, o Options, srv *Service, log *slog.Logger) error {
func serve(ctx context.Context, o Options, srv *Service, log *slog.Logger, feed *logging.Feed) error {
opts, err := messagingrpc.ServerOptions(o.Listen, o.Token)
if err != nil {
return err
@@ -119,7 +126,7 @@ func serve(ctx context.Context, o Options, srv *Service, log *slog.Logger) error
return fmt.Errorf("listen messaging (%s): %w", o.Listen, err)
}
server := grpc.NewServer(opts...)
messagingrpc.RegisterServer(server, srv)
messagingrpc.RegisterServer(server, srv, feed)
go func() {
if err := server.Serve(listener); err != nil {
log.Error("messaging gRPC server stopped", "err", err)
+283 -26
View File
@@ -4,27 +4,46 @@ import (
"context"
"errors"
"fmt"
"time"
"github.com/samcharles93/archie-core/internal/channels"
"github.com/samcharles93/archie-core/internal/channels/status"
)
// runChannel runs one channel and starts its replacement after a requested
// restart. A channel that exits on its own is not restarted.
// runChannel supervises one channel for the service's lifetime. It starts the
// channel and, when the run ends, decides what happens next: a requested
// restart starts the replacement, a run that failed is retried with bounded
// backoff, and a channel that keeps failing is parked until an explicit
// restart. Only shutdown ends the supervisor, so a channel that stops is never
// left without one to start it again.
func (s *Service) runChannel(ctx context.Context, c *channelInstance) {
defer s.wg.Done()
defer c.supervisorExits()
attempts := 0
for {
c.mu.Lock()
ch := c.channel
if ch == nil {
// Disabled: there is no channel to run until an enable builds one.
c.mu.Unlock()
s.status.MarkStopped(c.name, "")
if s.awaitChannel(ctx, c, 0) == channelShutdown {
return
}
attempts = 0
continue
}
runCtx, cancel := context.WithCancel(ctx)
c.cancel = cancel
c.mu.Unlock()
s.log.Info("starting channel", "name", c.name)
err := ch.Start(runCtx, s.chat, s.lifecycleFor(c.name))
attempts++
// Decide the next state under the same lock a restart takes, so a
// restart racing this exit is either seen here (restartPending) or
// observed as a channel that is no longer supervised.
// delivered to the wait below (notify).
c.mu.Lock()
c.cancel = nil
shutdown := ctx.Err() != nil
@@ -32,9 +51,6 @@ func (s *Service) runChannel(ctx context.Context, c *channelInstance) {
if !shutdown && restart {
c.restartPending = false
}
if shutdown || !restart {
c.supervised = false
}
c.mu.Unlock()
cancel()
@@ -43,32 +59,168 @@ func (s *Service) runChannel(ctx context.Context, c *channelInstance) {
// idempotent tail for the paths that leave a listener behind.
s.stopChannel(ctx, c.name, ch)
if shutdown {
if err != nil && !errors.Is(err, context.Canceled) {
s.status.MarkFailed(c.name, err.Error())
} else {
s.status.MarkStopped(c.name, "")
}
next, wait := s.afterChannelRun(c, err, shutdown, restart, attempts)
if next == nextStop {
return
}
if restart {
// The replacement is already in c.channel (see restartChannel); the
// next pass starts it.
if next == nextReplaced {
attempts = 0
continue
}
// The channel stopped on its own.
if err != nil {
s.status.MarkFailed(c.name, err.Error())
s.log.Error("channel stopped with error", "name", c.name, "err", err)
} else {
s.status.MarkStopped(c.name, "")
if attempts, next = s.holdAfterRun(ctx, c, next, wait, attempts); next == nextStop {
return
}
return
}
}
// restartChannel builds a replacement for channel id from next, then stops
// the old one and starts it. A channel not running is not restarted.
// channelNext is what a supervisor does after one channel run ended.
type channelNext int
const (
// nextStop: the service is stopping; the supervisor returns.
nextStop channelNext = iota
// nextReplaced: a replacement is already in the instance; run it now.
nextReplaced
// nextPark: hold, with no timer, until an explicit restart or shutdown.
nextPark
// nextRetry: wait the backoff, then run the same channel again.
nextRetry
)
// afterChannelRun records the ended run's state and decides what the supervisor
// does next. shutdown is the service stopping; restart is an explicit
// replacement already waiting in the instance.
func (s *Service) afterChannelRun(c *channelInstance, err error, shutdown, restart bool, attempts int) (channelNext, time.Duration) {
switch {
case shutdown:
if err != nil && !errors.Is(err, context.Canceled) {
s.status.MarkFailed(c.name, err.Error())
} else {
s.status.MarkStopped(c.name, "")
}
return nextStop, 0
case restart:
return nextReplaced, 0
case err == nil:
// The channel stopped itself. Hold it stopped rather than spending the
// failure retry budget on a run that chose to end.
s.status.MarkStopped(c.name, "")
s.log.Info("channel stopped", "name", c.name)
return nextPark, 0
case attempts >= s.retry.attempts:
s.status.MarkFailed(c.name, err.Error())
s.log.Error("channel failed repeatedly; parked until an explicit restart",
"name", c.name, "attempts", attempts, "err", err)
return nextPark, 0
default:
s.status.MarkFailed(c.name, err.Error())
wait := s.retry.backoffFor(attempts)
s.log.Warn("channel failed; restarting", "name", c.name,
"attempt", attempts, "of", s.retry.attempts, "in", wait, "err", err)
return nextRetry, wait
}
}
// holdAfterRun waits out a park or a retry backoff, and reports the attempt
// count to carry into the next run and whether the supervisor keeps going. A
// restart delivered during a backoff starts the replacement with a fresh
// budget; a shutdown ends the supervisor.
func (s *Service) holdAfterRun(ctx context.Context, c *channelInstance, next channelNext, wait time.Duration, attempts int) (int, channelNext) {
if next == nextRetry {
switch s.awaitChannel(ctx, c, wait) {
case channelShutdown:
return attempts, nextStop
case channelRestart:
return 0, nextReplaced
default:
return attempts, nextRetry
}
}
if s.awaitChannel(ctx, c, 0) == channelShutdown {
return attempts, nextStop
}
return 0, nextReplaced
}
// channelRetry bounds how a supervisor retries a channel that fails on its own
// before parking it for an explicit restart.
type channelRetryPolicy struct {
attempts int
backoff time.Duration
maxBackoff time.Duration
}
// defaultChannelRetry is the retry policy a composed service uses.
func defaultChannelRetry() channelRetryPolicy {
return channelRetryPolicy{attempts: 5, backoff: time.Second, maxBackoff: 30 * time.Second}
}
// backoffFor is the wait before attempt n (1-based): doubling from backoff,
// capped at maxBackoff so a long outage does not become a long silence between
// retries.
func (r channelRetryPolicy) backoffFor(attempt int) time.Duration {
backoff := r.backoff << (attempt - 1)
if backoff <= 0 || backoff > r.maxBackoff {
return r.maxBackoff
}
return backoff
}
// channelWait is how a supervisor's wait between runs ended.
type channelWait int
const (
// channelRetry: the backoff elapsed; run the channel again.
channelRetry channelWait = iota
// channelRestart: an explicit restart arrived; start the replacement.
channelRestart
// channelShutdown: the service is stopping.
channelShutdown
)
// awaitChannel blocks until an explicit restart arrives, the service stops, or
// the backoff elapses. A delay of zero waits with no timer: the parked state
// after a channel stopped or gave up, which only a restart or shutdown ends.
func (s *Service) awaitChannel(ctx context.Context, c *channelInstance, delay time.Duration) channelWait {
if delay <= 0 {
select {
case <-ctx.Done():
return channelShutdown
case <-c.notify:
return channelRestart
}
}
timer := time.NewTimer(delay)
defer timer.Stop()
select {
case <-ctx.Done():
return channelShutdown
case <-c.notify:
return channelRestart
case <-timer.C:
// A restart delivered as the timer fired is read here rather than
// lost between the two branches.
select {
case <-c.notify:
return channelRestart
default:
return channelRetry
}
}
}
// supervisorExits records that a channel's supervisor is leaving, so a restart
// after shutdown is refused rather than signalling a goroutine that is gone.
func (c *channelInstance) supervisorExits() {
c.mu.Lock()
c.supervised = false
c.mu.Unlock()
}
// restartChannel builds a replacement for channel id from next and starts it in
// place of the running one. A supervisor that is between runs -- backing off a
// failure, or parked after giving up -- is woken to start the replacement; a
// channel with no supervisor (never started, or shut down) is refused.
func (s *Service) restartChannel(id string, next ResolvedConfig) error {
c := s.instanceByID(id)
if c == nil {
@@ -84,21 +236,126 @@ func (s *Service) restartChannel(id string, next ResolvedConfig) error {
if replacement == nil {
return fmt.Errorf("channel %q has no configuration", id)
}
return s.setChannelState(c, replacement)
}
// setChannelState swaps the channel a supervisor runs; a nil replacement stops
// it. It returns once the supervisor has been told, not once the channel is up:
// a caller that needs the observable state awaits it (awaitChannelActive).
func (s *Service) setChannelState(c *channelInstance, replacement channels.Channel) error {
c.mu.Lock()
if !c.supervised {
c.mu.Unlock()
return fmt.Errorf("channel %q is not running", id)
return fmt.Errorf("channel %q is not running", c.name)
}
c.channel = replacement
c.restartPending = true
cancel := c.cancel
if cancel != nil {
// A run is in flight. Cancelling its context ends Start, and the
// supervisor sees restartPending and starts the replacement.
c.restartPending = true
} else {
// The supervisor is between runs. Wake it so it starts the replacement
// now, rather than waiting out its backoff or its park.
c.restartPending = false
}
c.mu.Unlock()
if cancel != nil {
cancel()
return nil
}
select {
case c.notify <- struct{}{}:
default:
}
return nil
}
// enableChannel builds and starts a channel that a settings change turned on,
// and reports only once it is no longer stopped.
func (s *Service) enableChannel(id string, next ResolvedConfig) error {
c := s.instanceByID(id)
if c == nil {
return fmt.Errorf("no channel %q", id)
}
if c.rebuild == nil {
return fmt.Errorf("channel %q cannot be rebuilt", id)
}
replacement, err := c.rebuild(next)
if err != nil {
return fmt.Errorf("build %q: %w", id, err)
}
if replacement == nil {
return fmt.Errorf("channel %q has no configuration", id)
}
if err := s.setChannelState(c, replacement); err != nil {
return err
}
s.status.Declare(id, true, channelReloadable(replacement))
s.signalPublish()
return s.awaitChannelActive(id, true)
}
// disableChannel stops a channel that a settings change turned off, and reports
// only once it has stopped.
func (s *Service) disableChannel(id string) error {
c := s.instanceByID(id)
if c == nil || c.current() == nil {
return nil
}
if err := s.setChannelState(c, nil); err != nil {
return err
}
s.status.Declare(id, false, false)
s.signalPublish()
return s.awaitChannelActive(id, false)
}
// channelApplyTimeout bounds how long reconcile waits for an enable or disable
// to take effect before reporting it unapplied, which leaves the next tick to
// retry it.
const channelApplyTimeout = 5 * time.Second
// channelApplyPoll is how often that wait re-reads the channel's state.
const channelApplyPoll = 10 * time.Millisecond
// awaitChannelActive waits, up to channelApplyTimeout, for a channel to leave
// or reach the stopped state, so an enable or disable is reported applied only
// once the channel is actually running (or not) rather than merely queued. A
// failed channel is never active: reporting it so would mark the settings
// version applied and retire a failure the next tick must retry. Only a
// serving state counts, and a failed enable fails fast instead of burning the
// timeout while no restart can succeed without one.
func (s *Service) awaitChannelActive(id string, active bool) error {
deadline := time.Now().Add(channelApplyTimeout)
for {
for _, st := range s.status.Snapshot() {
if st.ID != id {
continue
}
if active && st.State == status.StateFailed {
return fmt.Errorf("channel %q failed to start", id)
}
// Serving counts, impaired or not; anything else is still in
// flight, except a stopped channel answering a disable.
if st.State == status.StateRunning || st.State == status.StateDegraded {
if active {
return nil
}
} else if !active && st.State == status.StateStopped {
return nil
}
}
if time.Now().After(deadline) {
if active {
return fmt.Errorf("channel %q did not start", id)
}
return fmt.Errorf("channel %q did not stop", id)
}
time.Sleep(channelApplyPoll)
}
}
// stopChannel closes one channel's listener with its own bounded context, so a
// stop that hangs cannot hold the shutdown past the configured timeout. The
// parent context is stripped of its cancellation because this runs on the
@@ -0,0 +1,209 @@
package archiemessaging
import (
"context"
"encoding/json"
"errors"
"log/slog"
"sync"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/channels"
"github.com/samcharles93/archie-core/internal/channels/status"
"github.com/samcharles93/archie-core/internal/domain/messaging"
)
// fakeChannel is a channel whose runs a test scripts: it counts starts and
// either fails immediately or blocks until its run context ends.
type fakeChannel struct {
mu sync.Mutex
starts int
runErr error
}
func (f *fakeChannel) Name() string { return "fake" }
func (f *fakeChannel) Start(ctx context.Context, _ messaging.ChatContract, lifecycle channels.Lifecycle) error {
f.mu.Lock()
f.starts++
f.mu.Unlock()
if lifecycle.Starting != nil {
lifecycle.Starting()
}
if f.runErr != nil {
return f.runErr
}
<-ctx.Done()
return nil
}
func (f *fakeChannel) Stop(context.Context) error { return nil }
func (f *fakeChannel) ConfigSchema() json.RawMessage { return nil }
func (f *fakeChannel) ValidateConfig(map[string]any) error { return nil }
func (f *fakeChannel) count() int {
f.mu.Lock()
defer f.mu.Unlock()
return f.starts
}
// startService composes and starts a service over one scripted channel, with a
// retry policy the test drives without waiting out real backoff.
func startService(t *testing.T, ch *fakeChannel, retry channelRetryPolicy) *Service {
t.Helper()
inst := &channelInstance{
name: "fake",
channel: ch,
rebuild: func(ResolvedConfig) (channels.Channel, error) { return ch, nil },
}
srv, err := compose(context.Background(), deps{Log: slog.Default(), ExtensionChannels: []*channelInstance{inst}})
if err != nil {
t.Fatalf("compose: %v", err)
}
srv.retry = retry
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() {
defer close(done)
_ = srv.Start(ctx)
}()
t.Cleanup(func() {
cancel()
select {
case <-done:
case <-time.After(5 * time.Second):
t.Error("service did not stop after cancel")
}
})
return srv
}
func waitStarts(t *testing.T, ch *fakeChannel, want int) {
t.Helper()
deadline := time.Now().Add(5 * time.Second)
for time.Now().Before(deadline) {
if ch.count() >= want {
return
}
time.Sleep(time.Millisecond)
}
t.Fatalf("channel started %d times, want at least %d", ch.count(), want)
}
// TestAwaitChannelActiveStates pins what counts as up: running and degraded
// satisfy an enable, only stopped satisfies a disable, and a failed enable
// fails fast instead of reporting success the next tick would never retry.
func TestAwaitChannelActiveStates(t *testing.T) {
mark := map[string]func(*status.Manager){
"running": func(m *status.Manager) { m.MarkRunning("fake") },
"degraded": func(m *status.Manager) { m.MarkDegraded("fake", "slow") },
"failed": func(m *status.Manager) { m.MarkFailed("fake", "boom") },
"starting": func(m *status.Manager) { m.MarkStarting("fake") },
}
// Refusals that wait out the 5s timeout (disable while failed or
// running) are pre-existing behavior, unchanged here, and too slow to
// pin; every case below answers at once.
tests := []struct {
name string
state string
active bool
wantErr bool
}{
{name: "enable accepts running", state: "running", active: true},
{name: "enable accepts degraded", state: "degraded", active: true},
{name: "enable refuses failed", state: "failed", active: true, wantErr: true},
{name: "disable accepts stopped", state: "", active: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
manager := status.NewManager([]status.Descriptor{{ID: "fake"}})
if tt.state == "" {
manager.MarkStopped("fake", "")
} else {
mark[tt.state](manager)
}
srv := &Service{status: manager}
// Every case above answers at once: success and failed-enable
// refusal never reach the timeout wait.
done := make(chan error, 1)
go func() { done <- srv.awaitChannelActive("fake", tt.active) }()
select {
case err := <-done:
if (err != nil) != tt.wantErr {
t.Fatalf("err = %v, wantErr %v", err, tt.wantErr)
}
case <-time.After(6 * time.Second):
t.Fatal("await did not answer")
}
})
}
}
// TestChannelSupervisorRetriesAndRestarts pins that a channel that fails on its
// own is retried with bounded backoff and, once the budget is spent, parked so
// an explicit restart brings it back rather than being refused.
func TestChannelSupervisorRetriesAndRestarts(t *testing.T) {
ch := &fakeChannel{runErr: errors.New("transport down")}
srv := startService(t, ch, channelRetryPolicy{attempts: 3, backoff: time.Millisecond, maxBackoff: time.Millisecond})
// The supervisor retries the failed run instead of abandoning the channel.
waitStarts(t, ch, 3)
if err := srv.restartChannel("fake", ResolvedConfig{}); err != nil {
t.Fatalf("restart after the retry budget was spent: %v", err)
}
waitStarts(t, ch, 4)
}
// TestChannelSupervisorRestartReplacesRunningChannel pins the in-flight path: a
// restart of a channel whose run is live cancels the run and starts the
// replacement, with no backoff.
func TestChannelSupervisorRestartReplacesRunningChannel(t *testing.T) {
ch := &fakeChannel{} // blocks until its run context ends
srv := startService(t, ch, channelRetryPolicy{attempts: 3, backoff: time.Hour, maxBackoff: time.Hour})
waitStarts(t, ch, 1)
if err := srv.restartChannel("fake", ResolvedConfig{}); err != nil {
t.Fatalf("restart running channel: %v", err)
}
waitStarts(t, ch, 2)
}
// TestChannelRestartRefusedWithoutSupervisor pins that a channel with no
// supervisor -- never started, or already shut down -- is refused rather than
// signalling a goroutine that is gone.
func TestChannelRestartRefusedWithoutSupervisor(t *testing.T) {
ch := &fakeChannel{}
inst := &channelInstance{
name: "fake",
channel: ch,
rebuild: func(ResolvedConfig) (channels.Channel, error) { return ch, nil },
}
srv, err := compose(context.Background(), deps{Log: slog.Default(), ExtensionChannels: []*channelInstance{inst}})
if err != nil {
t.Fatalf("compose: %v", err)
}
if err := srv.restartChannel("fake", ResolvedConfig{}); err == nil {
t.Fatal("restart before the service started should be refused")
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() {
defer close(done)
_ = srv.Start(ctx)
}()
waitStarts(t, ch, 1)
cancel()
select {
case <-done:
case <-time.After(5 * time.Second):
t.Fatal("service did not stop")
}
if err := srv.restartChannel("fake", ResolvedConfig{}); err == nil {
t.Fatal("restart after shutdown should be refused")
}
}
@@ -9,14 +9,11 @@ import (
"time"
configtemplate "github.com/samcharles93/archie-core"
"github.com/samcharles93/archie-core/internal/app/servicekit"
"github.com/samcharles93/archie-core/internal/buildinfo"
"github.com/samcharles93/archie-core/internal/channels/telegram"
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/domain/presence"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/installtype"
"github.com/samcharles93/archie-core/internal/releaseannounce"
"github.com/samcharles93/archie-core/internal/releaseupdate"
"github.com/samcharles93/archie-core/internal/secret"
)
@@ -26,10 +23,10 @@ func configureTelegram(ctx context.Context, g *telegram.Gateway, cfg ResolvedCon
g.Version = gatewayVersionReporter(ctx, d.Chat, cfg.Options.DependencyTimeout)
g.SetShowToolCalls(cfg.ShowToolCalls)
g.Reload = telegramReloader(cfg.Options, d.Secrets, d.Log)
g.RunningVersions = runningVersions(ctx, d.Presence, cfg.Options.DependencyTimeout)
g.RunningVersions = servicekit.RunningVersions(ctx, d.Presence, cfg.Options.DependencyTimeout)
if updates := updateService(cfg); updates != nil {
g.Updates = updates
if d.Updates != nil {
g.Updates = d.Updates
}
if cfg.WorkDir != "" {
g.UpdateReportPath = identityStatePath(cfg.WorkDir, "update-report", cfg.BotUser)
@@ -42,34 +39,6 @@ func configureTelegram(ctx context.Context, g *telegram.Gateway, cfg ResolvedCon
}
}
// runningVersions reports the daemon version archied's own presence record
// carries, so an update report is checked against what archied compiled in
// rather than this process's build. A down or unreachable daemon, or a
// missing store, leaves the daemon unverified. The agent stays unverified:
// only archied observes it.
func runningVersions(ctx context.Context, store storecontract.PresenceStore, timeout time.Duration) func() map[string]string {
return func() map[string]string {
if store == nil {
return nil
}
if timeout <= 0 {
timeout = 5 * time.Second
}
callCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
records, err := store.ListPresence(callCtx)
if err != nil {
return nil
}
for _, service := range presence.Mesh(records, time.Now()) {
if service.Service == presence.Daemon && service.State != presence.StateDown {
return map[string]string{releaseupdate.ComponentDaemon: service.Version}
}
}
return nil
}
}
// gatewayVersionReporter renders /version from the Gateway's build info,
// fetched per call with a timeout.
func gatewayVersionReporter(ctx context.Context, chat messaging.ChatContract, timeout time.Duration) func() string {
@@ -91,26 +60,6 @@ func gatewayVersionReporter(ctx context.Context, chat messaging.ChatContract, ti
}
}
// updateService builds /update from [chat.telegram] commands, or nil when
// none are configured.
func updateService(cfg ResolvedConfig) *releaseupdate.Service {
if len(cfg.Telegram.UpdateCheckCommand) == 0 {
return nil
}
updates := &releaseupdate.Service{
Catalog: releaseupdate.CommandCatalog{Command: cfg.Telegram.UpdateCheckCommand},
StatePath: filepath.Join(cfg.WorkDir, "telegram-update-deferrals.json"),
InstallType: installtype.Type(),
}
if len(cfg.Telegram.UpdateInstallCommand) != 0 {
updates.Installer = releaseupdate.CommandInstaller{
Command: cfg.Telegram.UpdateInstallCommand,
HealthURL: cfg.HealthURL,
}
}
return updates
}
// telegramReloader re-resolves the token and allowlist from this service's own
// configuration on /restart. A reload that cannot produce a token is refused so
// the running bot keeps the credential it is already authenticated with.
+6 -2
View File
@@ -5,6 +5,7 @@ import (
"log/slog"
"time"
"github.com/samcharles93/archie-core/internal/app/servicekit"
controlpb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/health"
@@ -17,6 +18,7 @@ import (
infraaccess "github.com/samcharles93/archie-core/internal/infrastructure/access"
"github.com/samcharles93/archie-core/internal/infrastructure/captureintake"
"github.com/samcharles93/archie-core/internal/infrastructure/gatewayrpc"
"github.com/samcharles93/archie-core/internal/releaseupdate"
"github.com/samcharles93/archie-core/internal/webhookguard"
"github.com/samcharles93/archie-core/internal/webui"
)
@@ -29,6 +31,7 @@ type deps struct {
Log *slog.Logger
Store storecontract.TaskStore
Chat messaging.ChatContract
Updates *releaseupdate.Service
Health *health.Registry
ControlPlane controlpb.ControlPlaneServiceClient
Identities identity.Repository
@@ -48,7 +51,7 @@ type deps struct {
// compose builds the dashboard server from contract-backed dependencies.
// Unset fields answer 501/503 or empty.
func compose(d deps) *webui.Server {
func compose(ctx context.Context, d deps) *webui.Server {
srv := &webui.Server{
Store: d.Store,
Log: d.Log,
@@ -88,6 +91,7 @@ func compose(d deps) *webui.Server {
wireOrgSurface(srv, d.Store)
if presence, ok := d.Store.(storecontract.PresenceStore); ok {
srv.Presence = presence
srv.RunningVersions = servicekit.RunningVersions(ctx, presence, 0)
}
if applyStatus, ok := d.Store.(storecontract.ApplyStatusStore); ok {
srv.ApplyStatus = applyStatus
@@ -107,7 +111,7 @@ func compose(d deps) *webui.Server {
srv.EventTypes = eventTypes
}
if d.Chat != nil {
srv.Chat = &webui.ChatService{Contract: d.Chat}
srv.Chat = &webui.ChatService{Contract: d.Chat, Updates: d.Updates}
}
wireCatalog(d, srv)
// The policy chain: wired together or not at all.
+28 -12
View File
@@ -58,6 +58,8 @@ type projection struct {
trustForwardedHeaders bool
gateway ServiceTarget
state ServiceTarget
messaging ServiceTarget
harness ServiceTarget
capture CaptureOptions
}
@@ -66,7 +68,9 @@ func project(cfg config.Config) projection {
listen: cfg.Web.Listen,
trustForwardedHeaders: cfg.Web.TrustForwardedHeaders,
gateway: ServiceTarget{Target: cfg.Services.Get(config.ServiceNameGateway).Target, Token: cfg.Services.Get(config.ServiceNameGateway).TargetToken},
messaging: ServiceTarget{Target: cfg.Services.Get(config.ServiceNameMessaging).Target, Token: cfg.Services.Get(config.ServiceNameMessaging).TargetToken},
state: ServiceTarget{Target: cfg.Services.Get(config.ServiceNameState).Target, Token: cfg.Services.Get(config.ServiceNameState).TargetToken},
harness: ServiceTarget{Target: cfg.Services.Get(config.ServiceNameHarness).Target, Token: cfg.Services.Get(config.ServiceNameHarness).TargetToken},
capture: CaptureOptions{
Retention: cfg.Capture.Retention.Std(),
MaxEvents: cfg.Capture.MaxEvents,
@@ -85,6 +89,12 @@ func withEnvTokens(o Options) Options {
if o.State.Token == "" {
o.State.Token = os.Getenv("STATE_STORE_TOKEN")
}
if o.Messaging.Token == "" {
o.Messaging.Token = os.Getenv("MESSAGING_TOKEN")
}
if o.Harness.Token == "" {
o.Harness.Token = os.Getenv("HARNESS_TOKEN")
}
return o
}
@@ -97,18 +107,10 @@ func merge(o Options, p projection) Options {
trust := p.trustForwardedHeaders
o.TrustForwardedHeaders = &trust
}
if o.Gateway.Target == "" {
o.Gateway.Target = p.gateway.Target
}
if o.Gateway.Token == "" {
o.Gateway.Token = p.gateway.Token
}
if o.State.Target == "" {
o.State.Target = p.state.Target
}
if o.State.Token == "" {
o.State.Token = p.state.Token
}
o.Gateway = mergeTarget(o.Gateway, p.gateway)
o.Messaging = mergeTarget(o.Messaging, p.messaging)
o.State = mergeTarget(o.State, p.state)
o.Harness = mergeTarget(o.Harness, p.harness)
if o.Capture.Retention <= 0 {
o.Capture.Retention = p.capture.Retention
}
@@ -128,6 +130,10 @@ func merge(o Options, p projection) Options {
}
func withDefaults(o Options) Options {
if o.Messaging.Target == "" {
spec, _ := config.LookupService(config.ServiceNameMessaging)
o.Messaging.Target = spec.Target
}
// "off" disables the daemon's dashboard but means the default listener here.
if o.Listen == "" || o.Listen == "off" {
o.Listen = defaultListen
@@ -176,3 +182,13 @@ func (o Options) validate() error {
}
return nil
}
func mergeTarget(o, p ServiceTarget) ServiceTarget {
if o.Target == "" {
o.Target = p.Target
}
if o.Token == "" {
o.Token = p.Token
}
return o
}
+32
View File
@@ -0,0 +1,32 @@
package archieui
import (
"context"
"github.com/samcharles93/archie-core/internal/domain/presence"
"github.com/samcharles93/archie-core/internal/infrastructure/gatewayrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/messagingrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/logging"
"github.com/samcharles93/archie-core/internal/webui"
)
func wireServiceLogs(opts Options, srv *webui.Server, tasks *staterpc.Client, chat *gatewayrpc.Client, feed *logging.Feed) (func(), error) {
srv.LogFeed = feed
srv.LogSources = map[string]webui.LogSource{
presence.UI: func(_ context.Context, q logging.Query) (logging.Result, error) { return feed.Read(q) },
presence.StateStore: tasks.RecentLogs,
presence.Gateway: func(ctx context.Context, q logging.Query) (logging.Result, error) {
return chat.RecentLogs(ctx, presence.Gateway, q)
},
presence.Daemon: func(ctx context.Context, q logging.Query) (logging.Result, error) {
return chat.RecentLogs(ctx, presence.Daemon, q)
},
}
messaging, closeMessaging, dialErr := messagingrpc.Dial(opts.Messaging.Target, opts.Messaging.Token)
if dialErr != nil {
return nil, dialErr
}
srv.LogSources[presence.Messaging] = messaging.RecentLogs
return closeMessaging, nil
}
+7 -2
View File
@@ -67,8 +67,13 @@ type Options struct {
Capture CaptureOptions
// Gateway and State are the two remote contracts the dashboard consumes.
Gateway ServiceTarget
State ServiceTarget
Gateway ServiceTarget
State ServiceTarget
Messaging ServiceTarget
// Harness is the daemon's session contract. Optional: empty leaves the
// setup terminal unwired, and /api/harness reports terminal:false rather
// than offering a control that would fail.
Harness ServiceTarget
// DependencyTimeout bounds each readiness probe's call to a dependency.
DependencyTimeout time.Duration
+40 -17
View File
@@ -8,6 +8,7 @@ import (
"net"
"net/http"
"os"
"path/filepath"
"slices"
"time"
@@ -18,7 +19,10 @@ import (
"github.com/samcharles93/archie-core/internal/domain/applystatus"
"github.com/samcharles93/archie-core/internal/domain/presence"
infraaccess "github.com/samcharles93/archie-core/internal/infrastructure/access"
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
"github.com/samcharles93/archie-core/internal/infrastructure/gatewayrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/harnessrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/webui"
)
@@ -44,18 +48,17 @@ func buildAccessChain(ctx context.Context, store *staterpc.Client, log *slog.Log
// on shutdown.
func Run(ctx context.Context, options Options) error {
log := slog.New(slog.NewJSONHandler(os.Stderr, nil)).With("component", "ui")
log, feed := servicekit.Diagnostics(log, presence.UI)
opts, err := Resolve(options, log)
if err != nil {
return err
}
var cleanups []func()
defer func() {
for _, cleanup := range slices.Backward(cleanups) {
cleanup()
}
}()
tasks, closeState, err := staterpc.Dial(opts.State.Target, presence.UI, opts.State.Token)
if err != nil {
return err
@@ -66,12 +69,10 @@ func Run(ctx context.Context, options Options) error {
return err
}
cleanups = append(cleanups, closeGateway)
chain, err := buildAccessChain(ctx, tasks, log)
if err != nil {
return err
}
provider, err := signInProvider(opts)
if err != nil {
return err
@@ -81,12 +82,12 @@ func Run(ctx context.Context, options Options) error {
if err != nil {
return err
}
srv := compose(deps{
srv := compose(ctx, deps{
Options: opts,
Log: log,
Store: tasks,
Chat: chat,
Updates: controlplanerpc.UpdateService(controlplanerpc.NewRPCClient(tasks.ControlPlane()), filepath.Join(configuration.DefaultWorkDir(), "ui-update-deferrals.json"), ""),
Health: newReadinessRegistry(opts, tasks, chat, chain, provider),
ControlPlane: tasks.ControlPlane(),
Identities: tasks,
@@ -96,33 +97,55 @@ func Run(ctx context.Context, options Options) error {
Principals: tasks,
Denials: tasks,
})
// Live activity has no in-process bus in this process: the pump reads
// events back out of the State Store over the same cursor SSE catch-up
// uses. Backgrounded so priming past a large events table cannot delay the
// listener.
stopLogs, err := wireServiceLogs(opts, srv, tasks, chat, feed)
if err != nil {
return err
}
cleanups = append(cleanups, stopLogs)
if err := wireHarness(opts, srv, &cleanups); err != nil {
return err
}
// Priming event history must not delay the listener.
go pumpEvents(ctx, srv, opts.EventPollInterval, log)
go presence.Run(ctx, presence.UI, servicekit.Build(), tasks, srv.Health, log)
liveCtx, stopLive := context.WithCancel(ctx)
defer stopLive()
go srv.RunLive(liveCtx)
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", opts.Listen)
if err != nil {
return fmt.Errorf("listen for ui: %w", err)
}
cleanups = append(cleanups, func() { _ = listener.Close() })
log.Info(
"archie-ui running",
attrs := []any{
"addr", listener.Addr().String(),
"open", webui.DashboardURL(listener.Addr().String(), opts.Token),
"open", webui.DashboardURL(listener.Addr().String()),
"gateway", opts.Gateway.Target,
"state", opts.State.Target,
)
}
// Name the file holding the token, never the token itself: this line
// reaches the journal and any log an operator pastes.
if opts.TokenFile != "" {
attrs = append(attrs, "token_file", opts.TokenFile)
}
log.Info("archie-ui running", attrs...)
return serve(ctx, listener, srv.Handler(), opts)
}
// wireHarness attaches the setup terminal to the daemon's session contract
// when the operator configured one, and registers its cleanup.
func wireHarness(opts Options, srv *webui.Server, cleanups *[]func()) error {
if opts.Harness.Target == "" {
return nil
}
harness, closeHarness, err := harnessrpc.Dial(opts.Harness.Target, opts.Harness.Token)
if err != nil {
return err
}
srv.HarnessTerminal = harness
*cleanups = append(*cleanups, closeHarness)
return nil
}
// serve runs handler until ctx ends, then shuts down within
// ShutdownTimeout.
func serve(ctx context.Context, listener net.Listener, handler http.Handler, opts Options) error {
@@ -0,0 +1,73 @@
package controlplane_test
import (
"encoding/json"
"testing"
"github.com/samcharles93/archie-core/internal/app/controlplane"
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
)
func TestStoredChannelSettingsCanBeEdited(t *testing.T) {
ctx := t.Context()
db := pgstore.Open(t)
if err := db.BootstrapIdentities(ctx, nil); err != nil {
t.Fatal(err)
}
if err := db.UpgradeDefaultOrg(ctx); err != nil {
t.Fatal(err)
}
steps, err := workflowsteps.NewManager()
if err != nil {
t.Fatal(err)
}
server, err := controlplane.NewServer(db, steps)
if err != nil {
t.Fatal(err)
}
client := workflowRPC(t, server, db).ControlPlane()
stored, err := db.PutResource(ctx, storecontract.ResourceWrite{
OrgID: storecontract.DefaultOrgID, Kind: controlplane.ChannelSettingsKind,
Value: []byte(`{"email":{"listen_addr":""},"webhook":{},"webhook_addr":"","telegram":{"allowed_user_ids":[42],"token_ref":{"engine":"bws","key":"telegram"}},"workspace":"/workspace","max_steps":17,"rate_limit":{"window":"30s","max_requests":5}}`),
Actor: "operator", Source: "test", RequestID: "old-settings",
})
if err != nil {
t.Fatal(err)
}
got, err := client.Query(ctx, &pb.QueryRequest{Kind: controlplane.ChannelSettingsKind})
if err != nil {
t.Fatal(err)
}
var settings controlplanerpc.ChannelSettings
if err := json.Unmarshal(got.Resource.ValueJson, &settings); err != nil {
t.Fatal(err)
}
var document map[string]json.RawMessage
if err := json.Unmarshal(got.Resource.ValueJson, &document); err != nil {
t.Fatal(err)
}
for _, retired := range []string{"email", "webhook", "webhook_addr"} {
if _, exists := document[retired]; exists {
t.Fatalf("served retired %s field in editable settings", retired)
}
}
if settings.Telegram.Token.Key != "telegram" || len(settings.Telegram.AllowedUserIDs) != 1 || settings.Telegram.AllowedUserIDs[0] != 42 || settings.Workspace != "/workspace" || settings.MaxSteps != 17 || settings.RateLimit.MaxRequests != 5 {
t.Fatalf("lost supported settings: %+v", settings)
}
settings.Telegram.AllowedUserIDs = []int64{42, 43}
value, err := json.Marshal(settings)
if err != nil {
t.Fatal(err)
}
changed, err := client.Command(ctx, &pb.CommandRequest{Kind: controlplane.ChannelSettingsKind, Command: "replace", ValueJson: value, ExpectedVersion: got.Resource.Version, Actor: "operator", Source: "test", RequestId: "edit"})
if err != nil {
t.Fatalf("saving edited channel settings: %v", err)
}
if changed.Resource.Version != stored.Version+1 {
t.Fatalf("saved version %d", changed.Resource.Version)
}
}
@@ -0,0 +1,143 @@
package controlplane_test
import (
"context"
"net"
"strings"
"testing"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/status"
"github.com/samcharles93/archie-core/internal/app/controlplane"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/identity"
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/domain/org"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
"github.com/samcharles93/archie-core/internal/infrastructure/gatewayrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
"github.com/samcharles93/archie-core/internal/infrastructure/rpcidentity"
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
)
func TestWebChatSettingsActor(t *testing.T) {
ctx := t.Context()
db := pgstore.Open(t)
if err := db.BootstrapIdentities(ctx, nil); err != nil {
t.Fatal(err)
}
if err := db.UpgradeDefaultOrg(ctx); err != nil {
t.Fatal(err)
}
steps, err := workflowsteps.NewManager()
if err != nil {
t.Fatal(err)
}
cp, err := controlplane.NewServer(db, steps)
if err != nil {
t.Fatal(err)
}
if _, _, err := cp.ImportConfig(ctx, config.Config{}); err != nil {
t.Fatal(err)
}
state := workflowRPC(t, cp, db)
router := gateway.NewRouter(state, nil, "web")
router.Settings = messaging.NewSettingsCommand(controlplanerpc.NewSettingsClient(state.ControlPlane())).WithIdentities(state)
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
callers := rpcidentity.Callers{Principals: state}
srv := grpc.NewServer(grpc.ChainUnaryInterceptor(gatewayrpc.UnaryServerInterceptor("gateway-secret"), callers.Unary()), grpc.ChainStreamInterceptor(gatewayrpc.StreamServerInterceptor("gateway-secret"), callers.Stream()))
gatewayrpc.RegisterServer(srv, &gateway.LocalChatAdapter{Router: router}, nil, gatewayrpc.Catalog{})
go func() { _ = srv.Serve(listener) }()
t.Cleanup(srv.Stop)
person, err := db.PrincipalFor(ctx, identity.OperatorID())
if err != nil {
t.Fatal(err)
}
tests := []struct {
name, token, sender string
person, write, stream bool
code codes.Code
}{
{name: "streamed signed-in person", stream: true, token: "gateway-secret", person: true, write: true},
{name: "streamed wrong service credential", stream: true, token: "wrong", person: true, code: codes.Unauthenticated},
{name: "signed-in person", token: "gateway-secret", person: true, write: true},
{name: "message cannot spoof actor", token: "gateway-secret", sender: string(identity.SystemID), person: true, write: true},
{name: "message alone cannot name a person", token: "gateway-secret", sender: string(identity.OperatorID())},
{name: "wrong service credential cannot forward a principal", token: "wrong", person: true, code: codes.Unauthenticated},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
conn, err := grpc.NewClient(listener.Addr().String(), grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithUnaryInterceptor(gatewayrpc.UnaryClientTokenInterceptor(tt.token)), grpc.WithStreamInterceptor(gatewayrpc.StreamClientTokenInterceptor(tt.token)))
if err != nil {
t.Fatal(err)
}
defer conn.Close()
requestCtx := ctx
if tt.person {
requestCtx = access.WithPrincipal(ctx, person)
}
before, err := db.Resource(ctx, string(org.DefaultOrgID), controlplane.ReviewSettingsKind)
if err != nil {
t.Fatal(err)
}
in := messaging.Inbound{Platform: "web", Message: messaging.Message{SenderID: tt.sender, Text: "/settings set review-settings precision_gate=true"}}
client := gatewayrpc.NewClient(conn)
reply, err := settingsReply(t, client, requestCtx, in, tt.stream, tt.code)
if status.Code(err) != tt.code {
t.Fatalf("route error = %v, want %s", err, tt.code)
}
after, err := db.Resource(ctx, string(org.DefaultOrgID), controlplane.ReviewSettingsKind)
if err != nil {
t.Fatal(err)
}
if !tt.write {
if after.Version != before.Version {
t.Fatal("unauthenticated command changed settings")
}
return
}
history, err := db.ResourceHistory(ctx, string(org.DefaultOrgID), controlplane.ReviewSettingsKind, 1)
if err != nil {
t.Fatal(err)
}
if len(history) != 1 {
t.Fatal("settings audit missing")
}
if !strings.Contains(reply.Text, "Updated review-settings") || after.Version != before.Version+1 || history[0].Actor != string(person.IdentityID) {
t.Fatalf("reply %q, version %d -> %d, actor %q", reply.Text, before.Version, after.Version, history[0].Actor)
}
})
}
}
func settingsReply(t *testing.T, client *gatewayrpc.Client, ctx context.Context, in messaging.Inbound, stream bool, want codes.Code) (messaging.ChatReply, error) {
t.Helper()
if !stream {
return client.Route(ctx, in)
}
events, err := client.Stream(ctx, in)
if err != nil {
return messaging.ChatReply{}, err
}
var reply messaging.ChatReply
for event := range events {
if event.Kind == "error" {
if want == codes.OK || !strings.Contains(event.Text, "code = "+want.String()) {
t.Fatalf("stream: %s", event.Text)
}
return messaging.ChatReply{}, status.Error(want, event.Text)
}
if event.Kind == "done" {
reply.Text = event.Text
}
}
return reply, nil
}
+11 -3
View File
@@ -11,6 +11,7 @@ import (
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/agent"
"github.com/samcharles93/archie-core/internal/domain/applystatus"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
@@ -20,7 +21,8 @@ import (
// Service's client and adds resource-specific reads.
type Client struct {
*controlplanerpc.Client
rpc pb.ControlPlaneServiceClient
rpc pb.ControlPlaneServiceClient
ApplyStatus *applystatus.Reporter
}
// NewClient dials the control plane over an existing connection.
@@ -36,8 +38,9 @@ func NewRPCClient(client pb.ControlPlaneServiceClient) *Client {
// WorkflowDefinitionsClient reads and replaces the workflow-definitions
// resource, resolving step types against the given vocabulary.
type WorkflowDefinitionsClient struct {
rpc pb.ControlPlaneServiceClient
steps workflow.StepRegistry
rpc pb.ControlPlaneServiceClient
steps workflow.StepRegistry
ApplyStatus *applystatus.Reporter
}
func NewWorkflowDefinitionsClient(client pb.ControlPlaneServiceClient, steps *workflow.Manager) (*WorkflowDefinitionsClient, error) {
@@ -59,9 +62,11 @@ func (c *Client) Catalog(ctx context.Context) ([]*pb.ResourceDescriptor, error)
func (c *WorkflowDefinitionsClient) WorkflowDefinitions(ctx context.Context) (workflow.WorkflowDefinitionCollection, int64, error) {
response, err := c.rpc.Query(ctx, &pb.QueryRequest{Kind: WorkflowDefinitionsKind})
if err != nil {
c.ApplyStatus.Report(ctx, WorkflowDefinitionsKind, 0, err)
return workflow.WorkflowDefinitionCollection{}, 0, controlplanerpc.ClientError(err)
}
definitions, err := workflow.DecodeDefinitionCollection(response.Resource.ValueJson, c.steps)
c.ApplyStatus.Report(ctx, WorkflowDefinitionsKind, response.Resource.Version, err)
return definitions, response.Resource.Version, err
}
@@ -238,12 +243,15 @@ func sendUpdate[T any](ctx context.Context, out chan<- T, update T) bool {
func (c *Client) Schedules(ctx context.Context) ([]scheduling.JobSpec, int64, error) {
response, err := c.rpc.Query(ctx, &pb.QueryRequest{Kind: SchedulesKind})
if err != nil {
c.ApplyStatus.Report(ctx, SchedulesKind, 0, err)
return nil, 0, controlplanerpc.ClientError(err)
}
var jobs []scheduling.JobSpec
if err := json.Unmarshal(response.Resource.ValueJson, &jobs); err != nil {
c.ApplyStatus.Report(ctx, SchedulesKind, response.Resource.Version, err)
return nil, 0, err
}
c.ApplyStatus.Report(ctx, SchedulesKind, response.Resource.Version, nil)
return jobs, response.Resource.Version, nil
}
+22 -3
View File
@@ -11,6 +11,7 @@ import (
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/timestamppb"
"github.com/samcharles93/archie-core/internal/config"
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/org"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
@@ -33,6 +34,7 @@ var (
// root asserts the State Store against it, so this is the single definition of
// what a control-plane backing store must offer.
type ResourceStore interface {
ListResources(context.Context) ([]storecontract.Resource, error)
Resource(ctx context.Context, orgID, kind string) (storecontract.Resource, error)
ResourceHistory(ctx context.Context, orgID, kind string, limit int) ([]storecontract.Resource, error)
Audit(ctx context.Context, table string, keys []string, limit int) ([]storecontract.AuditEntry, error)
@@ -80,7 +82,7 @@ func (s *Server) Query(ctx context.Context, request *pb.QueryRequest) (*pb.Query
if _, ok := s.definitions[request.GetKind()]; !ok {
return nil, status.Error(codes.NotFound, "resource not found")
}
resource, err := s.store.Resource(ctx, requestOrg(ctx), request.Kind)
resource, err := s.resource(ctx, requestOrg(ctx), request.Kind)
if err != nil {
return nil, mapError(err)
}
@@ -182,7 +184,7 @@ func (s *Server) Watch(request *pb.WatchRequest, stream pb.ControlPlaneService_W
ticker := time.NewTicker(250 * time.Millisecond)
defer ticker.Stop()
for {
resource, err := s.store.Resource(stream.Context(), requestOrg(stream.Context()), request.Kind)
resource, err := s.resource(stream.Context(), requestOrg(stream.Context()), request.Kind)
if err == nil && resource.Version > version {
if err := stream.Send(&pb.WatchResponse{Resource: resourceProto(resource)}); err != nil {
return err
@@ -199,12 +201,29 @@ func (s *Server) Watch(request *pb.WatchRequest, stream pb.ControlPlaneService_W
}
}
// resource seeds an org's workflow documents on first use, including orgs
// created after this process started. File-derived settings are never copied.
func (s *Server) resource(ctx context.Context, orgID, kind string) (storecontract.Resource, error) {
resource, err := s.store.Resource(ctx, orgID, kind)
if !errors.Is(err, storecontract.ErrResourceNotFound) || (kind != WorkflowDefinitionsKind && kind != WorkflowEnablementKind) {
// Serve the current channel schema so stored settings remain editable.
if err == nil && kind == ChannelSettingsKind {
resource.Value, err = normalizeChannels(resource.Value)
}
return resource, err
}
if _, err := s.seedKind(ctx, orgID, s.definitions[kind], config.Config{}); err != nil && !errors.Is(err, storecontract.ErrResourceVersionConflict) {
return storecontract.Resource{}, err
}
return s.store.Resource(ctx, orgID, kind)
}
func resourceProto(resource storecontract.Resource) *pb.Resource {
return &pb.Resource{OrgId: resource.OrgID, Kind: resource.Kind, Version: resource.Version, ValueJson: resource.Value, UpdatedAt: timestamp(resource.At)}
}
// requestOrg is the org a request acts in, derived from the caller's
// principal when the call carried one (staterpc.Callers).
// principal when the call carried one (rpcidentity.Callers).
func requestOrg(ctx context.Context) string {
return string(org.OrgFromContext(ctx))
}
+70
View File
@@ -0,0 +1,70 @@
package controlplane
import (
"context"
"encoding/json"
"fmt"
"strings"
"github.com/samcharles93/archie-core/internal/config"
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/curator"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
)
const CuratorsKind = "curators"
func curatorDefinition() Definition {
return Definition{
Kind: CuratorsKind, Title: "Curators", Document: []config.CuratorDefinition{}, ApplyMode: "live",
Seed: func(cfg config.Config) any { return cfg.Curators },
Validate: func(input []byte) error {
return validateAs(input, func(defs []config.CuratorDefinition) error {
names := map[string]bool{"skill": true, "session-memory": true}
for _, def := range defs {
if strings.TrimSpace(def.Name) == "" || names[def.Name] {
return fmt.Errorf("curator name %q is empty, reserved or repeated", def.Name)
}
names[def.Name] = true
if def.Interval <= 0 {
return fmt.Errorf("curator %s: interval must be positive", def.Name)
}
if err := CuratorDefinition(def).Manifest.Validate(); err != nil {
return fmt.Errorf("curator %s: %w", def.Name, err)
}
}
return nil
})
},
}
}
func CuratorDefinition(def config.CuratorDefinition) curator.Definition {
return curator.Definition{Name: def.Name, Enabled: def.Enabled, Instructions: def.Instructions, Manifest: curator.Manifest{
Interval: def.Interval.Std(), Cooldown: def.Cooldown.Std(), OnInput: def.OnInput, Tools: def.Tools, Skills: def.Skills, MemoryEngine: def.MemoryEngine, Conversations: def.Conversations, Model: def.Model,
}}
}
func (c *Client) Curators(ctx context.Context) ([]config.CuratorDefinition, int64, error) {
var defs []config.CuratorDefinition
version, _, err := c.Query(ctx, CuratorsKind, func(value []byte) error { return json.Unmarshal(value, &defs) })
return defs, version, err
}
type AppliedCurators struct {
Definitions []config.CuratorDefinition
Version int64
Err error
}
func (c *Client) WatchCurators(ctx context.Context, after int64) (<-chan AppliedCurators, error) {
stream, err := c.rpc.Watch(ctx, &pb.WatchRequest{Kind: CuratorsKind, AfterVersion: after})
if err != nil {
return nil, controlplanerpc.ClientError(err)
}
return watchUpdates(ctx, stream, func(resource *pb.Resource) AppliedCurators {
update := AppliedCurators{Version: resource.Version}
update.Err = json.Unmarshal(resource.ValueJson, &update.Definitions)
return update
}, func(err error) AppliedCurators { return AppliedCurators{Err: err} }), nil
}
+1 -1
View File
@@ -8,7 +8,7 @@ import (
// Schemas are not declared here: each Definition names its document type, and
// Definition.Descriptor derives the JSON Schema from it (internal/app/controlplane/schema.go).
func builtinDefinitions(steps workflow.StepRegistry, settings map[string]any) []Definition {
definitions := []Definition{workflowDefinition(), workflowDefinitionsDefinition(steps, settings), workflowEnablementDefinition(), personaDefinition(), soulDefinition(), scheduleDefinition(), agentProfileDefinition(), playbooksDefinition(), skillsDefinition(), credentialBindingsDefinition(), identityGrantsDefinition(), reviewSettingsDefinition()}
definitions := []Definition{taskPolicyDefinition(), workflowDefinition(), workflowDefinitionsDefinition(steps, settings), workflowEnablementDefinition(), personaDefinition(), curatorDefinition(), soulDefinition(), scheduleDefinition(), agentProfileDefinition(), playbooksDefinition(), skillsDefinition(), credentialBindingsDefinition(), identityGrantsDefinition(), reviewSettingsDefinition()}
definitions = append(definitions, modelDefinitions()...)
definitions = append(definitions, operationalDefinitions()...)
return definitions
@@ -9,6 +9,7 @@ import (
"github.com/samcharles93/archie-core/internal/domain/workintake"
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
"github.com/samcharles93/archie-core/internal/releaseupdate"
)
const (
@@ -18,6 +19,7 @@ const (
PluginSettingsKind = "plugin-settings"
ContainerRuntimePoliciesKind = "container-runtime-policies"
ExtensionSettingsKind = controlplanerpc.ExtensionSettingsKind
UpdateSettingsKind = controlplanerpc.UpdateSettingsKind
)
type schedulingPolicy struct {
@@ -34,7 +36,7 @@ type pluginSettings struct {
func operationalDefinitions() []Definition {
return []Definition{
{Kind: RepositoryPoliciesKind, Title: "Repository policies", ApplyMode: "live", Document: []config.Repo{}, Seed: func(cfg config.Config) any { return cfg.Repos }, Validate: validateRepositories},
{Kind: RepositoryPoliciesKind, Title: "Repository policies", ApplyMode: "live", Document: []config.Repo{}, Seed: func(cfg config.Config) any { return append([]config.Repo{}, cfg.Repos...) }, Validate: validateRepositories},
// Restart-required: the Messaging Service, which owns the channel
// transports, reconciles them live and archied re-layers each update, but
// the Gateway's chat surfaces (workspace tools, rate limit, model
@@ -59,6 +61,8 @@ func operationalDefinitions() []Definition {
{Kind: ExtensionSettingsKind, Title: "Extension settings", ApplyMode: "live", Document: controlplanerpc.ExtensionSettings{}, Seed: func(config.Config) any { return controlplanerpc.ExtensionSettings{} }, Validate: validateExtensions},
// Live: the container pool reads these on every acquire and the dispatcher
// is resized on publish.
// Live: every check and install reads the channel afresh.
{Kind: UpdateSettingsKind, Title: "Update settings", ApplyMode: "live", Document: controlplanerpc.UpdateSettings{}, Seed: func(config.Config) any { return controlplanerpc.UpdateSettings{Channel: "stable"} }, Validate: validateUpdateSettings},
{Kind: ContainerRuntimePoliciesKind, Title: "Container runtime policies", ApplyMode: "live", Document: containerRuntimePolicies{}, Seed: seedContainerPolicies, Validate: validateContainers, Normalize: normalizeContainerPolicies},
}
}
@@ -119,3 +123,10 @@ func validateScheduling(input []byte) error {
return nil
})
}
// validateUpdateSettings refuses a channel the update scripts cannot select.
func validateUpdateSettings(input []byte) error {
return validateAs(input, func(doc controlplanerpc.UpdateSettings) error {
return releaseupdate.ValidateChannel(doc.Channel, doc.Pin)
})
}
@@ -8,6 +8,7 @@ import (
"time"
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/applystatus"
"github.com/samcharles93/archie-core/internal/domain/eda/playbook"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
@@ -21,8 +22,9 @@ const playbookQueryTimeout = 5 * time.Second
// cannot be reached, or a document that no longer compiles, leaves the last
// good set in force.
type LivePlaybooks struct {
rpc pb.ControlPlaneServiceClient
log *slog.Logger
rpc pb.ControlPlaneServiceClient
log *slog.Logger
ApplyStatus *applystatus.Reporter
mu sync.Mutex
version int64
@@ -40,23 +42,28 @@ func (l *LivePlaybooks) current(ctx context.Context) *playbook.Store {
defer cancel()
response, err := l.rpc.Query(ctx, &pb.QueryRequest{Kind: PlaybooksKind})
if err != nil {
l.ApplyStatus.Report(ctx, PlaybooksKind, 0, err)
l.log.Warn("eda playbooks unavailable; keeping the last good set", "err", controlplanerpc.ClientError(err))
return l.store
}
if l.store != nil && response.Resource.Version == l.version {
l.ApplyStatus.Report(ctx, PlaybooksKind, l.version, nil)
return l.store
}
var collection PlaybookCollection
if err := json.Unmarshal(response.Resource.ValueJson, &collection); err != nil {
l.ApplyStatus.Report(ctx, PlaybooksKind, response.Resource.Version, err)
l.log.Error("eda playbooks undecodable; keeping the last good set", "err", err)
return l.store
}
store, err := CompilePlaybooks(collection)
if err != nil {
l.ApplyStatus.Report(ctx, PlaybooksKind, response.Resource.Version, err)
l.log.Error("eda playbooks do not compile; keeping the last good set", "err", err)
return l.store
}
l.version, l.store = response.Resource.Version, store
l.ApplyStatus.Report(ctx, PlaybooksKind, l.version, nil)
l.log.Info("eda playbooks loaded", "version", l.version, "playbooks", len(store.Playbooks))
return store
}
+29 -16
View File
@@ -89,7 +89,7 @@ func (s *Server) ImportConfig(ctx context.Context, cfg config.Config) (map[strin
versions := make(map[string]int64, len(s.ordered))
var skipped []SeedSkip
for _, definition := range s.ordered {
version, err := s.seedKind(ctx, definition, cfg)
version, err := s.seedKind(ctx, storecontract.DefaultOrgID, definition, cfg)
var refusal *seedRefusal
switch {
case errors.As(err, &refusal):
@@ -100,6 +100,18 @@ func (s *Server) ImportConfig(ctx context.Context, cfg config.Config) (map[strin
versions[definition.Kind] = version
}
}
resources, err := s.store.ListResources(ctx)
if err != nil {
return nil, nil, err
}
for _, resource := range resources {
if resource.OrgID == storecontract.DefaultOrgID || resource.Kind != WorkflowDefinitionsKind {
continue
}
if _, err := s.seedKind(ctx, resource.OrgID, s.definitions[resource.Kind], config.Config{}); err != nil {
return nil, nil, err
}
}
return versions, skipped, nil
}
@@ -115,8 +127,8 @@ func (r *seedRefusal) Unwrap() error { return r.err }
// seedKind seeds, refreshes or leaves one kind and returns its version. A
// *seedRefusal means the seed failed validation and nothing was written.
func (s *Server) seedKind(ctx context.Context, definition Definition, cfg config.Config) (int64, error) {
stored, storedErr := s.store.Resource(ctx, storecontract.DefaultOrgID, definition.Kind)
func (s *Server) seedKind(ctx context.Context, orgID string, definition Definition, cfg config.Config) (int64, error) {
stored, storedErr := s.store.Resource(ctx, orgID, definition.Kind)
absent := errors.Is(storedErr, storecontract.ErrResourceNotFound)
if storedErr != nil && !absent {
return 0, storedErr
@@ -133,7 +145,7 @@ func (s *Server) seedKind(ctx context.Context, definition Definition, cfg config
return stored.Version, nil
}
if !absent {
refresh, err := s.shippedValueIsStale(ctx, definition)
refresh, err := s.shippedValueIsStale(ctx, orgID, definition)
if err != nil {
return 0, err
}
@@ -142,7 +154,7 @@ func (s *Server) seedKind(ctx context.Context, definition Definition, cfg config
}
}
write := storecontract.ResourceWrite{
OrgID: storecontract.DefaultOrgID, Kind: definition.Kind, Value: value,
OrgID: orgID, Kind: definition.Kind, Value: value,
Actor: seedActor, Source: seedSource,
RequestID: seedRequestID(definition.Kind, value),
ExpectedVersion: expectedVersion(stored, absent), At: time.Now().UTC(),
@@ -165,18 +177,18 @@ func expectedVersion(stored storecontract.Resource, absent bool) int64 {
// shippedValueIsStale reports whether the newest stored revision of a
// shipped document was written by the seed.
func (s *Server) shippedValueIsStale(ctx context.Context, definition Definition) (bool, error) {
func (s *Server) shippedValueIsStale(ctx context.Context, orgID string, definition Definition) (bool, error) {
if definition.Defaults == nil {
return false, nil
}
return s.seedWroteNewest(ctx, definition.Kind)
return s.seedWroteNewest(ctx, orgID, definition.Kind)
}
// seedWroteNewest reports whether the newest revision of kind was written by a
// seed. It reads the ledger rather than the current row because the ledger is
// what records who wrote a value, and the row does not.
func (s *Server) seedWroteNewest(ctx context.Context, kind string) (bool, error) {
history, err := s.store.ResourceHistory(ctx, storecontract.DefaultOrgID, kind, 1)
func (s *Server) seedWroteNewest(ctx context.Context, orgID, kind string) (bool, error) {
history, err := s.store.ResourceHistory(ctx, orgID, kind, 1)
if err != nil {
return false, fmt.Errorf("read %s history: %w", kind, err)
}
@@ -231,19 +243,20 @@ func (s *Server) Owns(kind string) bool {
// ValidateStored decodes every stored resource with its definition and
// returns how many it checked. Errors name the kind and revision.
func (s *Server) ValidateStored(ctx context.Context) (int, error) {
resources, err := s.store.ListResources(ctx)
if err != nil {
return 0, err
}
checked := 0
var failures []error
for _, definition := range s.ordered {
resource, err := s.store.Resource(ctx, storecontract.DefaultOrgID, definition.Kind)
if errors.Is(err, storecontract.ErrResourceNotFound) {
for _, resource := range resources {
definition, ok := s.definitions[resource.Kind]
if !ok {
continue
}
if err != nil {
return checked, fmt.Errorf("read %s: %w", definition.Kind, err)
}
checked++
if _, err := definition.Decode(resource.Value); err != nil {
failures = append(failures, fmt.Errorf("%s at version %d: %w", definition.Kind, resource.Version, err))
failures = append(failures, fmt.Errorf("org %s: %s at version %d: %w", resource.OrgID, definition.Kind, resource.Version, err))
}
}
return checked, errors.Join(failures...)
@@ -140,6 +140,13 @@ func runtimeConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReade
}
func runtimeToolConfigFrom(ctx context.Context, reader controlplanerpc.ResourceReader, versions map[string]int64, out config.Config) (config.Config, map[string]int64, error) {
policy := taskPolicy{DiffCapLines: out.DiffCap(), NotifyWebhook: out.Notify.Webhook}
if err := layerResourceJSON(ctx, reader, versions, TaskPolicyKind, &policy); err != nil {
return config.Config{}, nil, err
}
out.DiffCapLines = new(policy.DiffCapLines)
out.Notify.Webhook = policy.NotifyWebhook
if err := layerResource(ctx, reader, versions, ToolSettingsKind, func(value []byte) error {
var settings toolSettings
if err := json.Unmarshal(value, &settings); err != nil {
+3
View File
@@ -29,6 +29,9 @@ func scheduleDefinition() Definition {
func validateSchedules(jobs []scheduling.JobSpec) error {
seen := make(map[string]struct{}, len(jobs))
for _, job := range jobs {
if err := scheduling.RefuseDangerousJob(job); err != nil {
return err
}
if strings.TrimSpace(job.ID) == "" {
return fmt.Errorf("schedule ID is required")
}
+34
View File
@@ -0,0 +1,34 @@
package controlplane
import (
"fmt"
"net/url"
"github.com/samcharles93/archie-core/internal/config"
)
const TaskPolicyKind = "task-policy"
type taskPolicy struct {
DiffCapLines int `json:"diff_cap_lines"`
NotifyWebhook string `json:"notify_webhook"`
}
func taskPolicyDefinition() Definition {
return Definition{Kind: TaskPolicyKind, Title: "Task policy", ApplyMode: "live", Document: taskPolicy{}, Seed: func(cfg config.Config) any {
return taskPolicy{DiffCapLines: cfg.DiffCap(), NotifyWebhook: cfg.Notify.Webhook}
}, Validate: func(input []byte) error {
return validateAs(input, func(doc taskPolicy) error {
if doc.DiffCapLines < 0 {
return fmt.Errorf("diff_cap_lines must not be negative")
}
if doc.NotifyWebhook != "" {
parsed, err := url.Parse(doc.NotifyWebhook)
if err != nil || parsed.Host == "" || (parsed.Scheme != "http" && parsed.Scheme != "https") {
return fmt.Errorf("notify_webhook must be an HTTP or HTTPS URL")
}
}
return nil
})
}}
}
@@ -30,9 +30,11 @@ func workflowEnablementDefinition() Definition {
func (c *Client) WorkflowEnablement(ctx context.Context) (task.WorkflowEnablement, error) {
response, err := c.rpc.Query(ctx, &controlpb.QueryRequest{Kind: WorkflowEnablementKind})
if err != nil {
c.ApplyStatus.Report(ctx, WorkflowEnablementKind, 0, err)
return task.WorkflowEnablement{}, controlplanerpc.ClientError(err)
}
var enablement task.WorkflowEnablement
err = json.Unmarshal(response.Resource.ValueJson, &enablement)
c.ApplyStatus.Report(ctx, WorkflowEnablementKind, response.Resource.Version, err)
return enablement, err
}
@@ -0,0 +1,212 @@
package controlplane_test
import (
"context"
"encoding/json"
"net"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
"google.golang.org/grpc"
"github.com/samcharles93/archie-core/internal/app/controlplane"
"github.com/samcharles93/archie-core/internal/config"
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/identity"
"github.com/samcharles93/archie-core/internal/domain/org"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/workflow"
infraaccess "github.com/samcharles93/archie-core/internal/infrastructure/access"
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgtest"
"github.com/samcharles93/archie-core/internal/infrastructure/rpcidentity"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
"github.com/samcharles93/archie-core/internal/webui"
)
func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
func TestOrgWorkflowResources(t *testing.T) {
for _, kind := range []string{controlplane.WorkflowDefinitionsKind, controlplane.WorkflowEnablementKind} {
t.Run(kind, func(t *testing.T) {
ctx := t.Context()
db := pgstore.Open(t)
if err := db.BootstrapIdentities(ctx, nil); err != nil {
t.Fatal(err)
}
if err := db.UpgradeDefaultOrg(ctx); err != nil {
t.Fatal(err)
}
steps, err := workflowsteps.NewManager()
if err != nil {
t.Fatal(err)
}
server, err := controlplane.NewServer(db, steps)
if err != nil {
t.Fatal(err)
}
if _, _, err := server.ImportConfig(ctx, config.Config{}); err != nil {
t.Fatal(err)
}
owner, err := db.Create(ctx, identity.Identity{ID: "acme-owner", Kind: identity.KindUser, DisplayName: "Owner", Lifecycle: identity.LifecycleActive}, identity.Audit{ActorID: identity.SystemID, Source: "test", RequestID: "owner", At: time.Now()})
if err != nil {
t.Fatal(err)
}
if _, err := db.CreateOrg(ctx, org.Org{ID: "acme", Name: "Acme"}, owner.ID); err != nil {
t.Fatal(err)
}
client := workflowRPC(t, server, db)
principal, err := db.PrincipalFor(ctx, owner.ID)
if err != nil {
t.Fatal(err)
}
other := access.WithPrincipal(ctx, principal)
initial, err := client.ControlPlane().Query(other, &pb.QueryRequest{Kind: kind})
if err != nil {
t.Fatalf("new org cannot read its shipped %s: %v", kind, err)
}
if initial.Resource.OrgId != "acme" {
t.Fatalf("read org %q", initial.Resource.OrgId)
}
value := []byte(`{"orgs":{"acme":{"disabled":["tdd"]}}}`)
if kind == controlplane.WorkflowDefinitionsKind {
collection := workflow.ShippedDefinitions()
collection.Definitions = append(collection.Definitions, workflow.WorkflowDefinitionEntry{ID: "acme-only", YAML: "id: acme-only\nrepository: none\nsteps:\n - type: workflow.finish\n"})
value, err = json.Marshal(collection)
if err != nil {
t.Fatal(err)
}
}
changed, err := client.ControlPlane().Command(other, &pb.CommandRequest{Kind: kind, Command: "replace", ValueJson: value, ExpectedVersion: initial.Resource.Version, Actor: "owner", Source: "test", RequestId: "edit"})
if err != nil {
t.Fatal(err)
}
if _, _, err := server.ImportConfig(ctx, config.Config{}); err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
ctxOrg org.OrgID
want bool
}{{"acme", true}, {org.DefaultOrgID, false}} {
queryCtx := ctx
if tc.ctxOrg == "acme" {
queryCtx = other
}
got, err := client.ControlPlane().Query(queryCtx, &pb.QueryRequest{Kind: kind})
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(got.Resource.ValueJson), "acme") != tc.want {
t.Fatalf("org %q got %s", tc.ctxOrg, got.Resource.ValueJson)
}
if tc.want && got.Resource.Version != changed.Resource.Version {
t.Fatalf("boot replaced operator's document")
}
}
if kind == controlplane.WorkflowDefinitionsKind {
checkWorkflowDashboard(t, db, client)
}
})
}
}
func workflowRPC(t *testing.T, control *controlplane.Server, db *pgstore.TaskDB) *staterpc.Client {
t.Helper()
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = listener.Close() })
grants := &staterpc.TaskGrants{}
callers := rpcidentity.Callers{Principals: db}
srv := grpc.NewServer(grpc.ChainUnaryInterceptor(grants.UnaryInterceptor("test-token"), callers.Unary()), grpc.ChainStreamInterceptor(grants.StreamInterceptor("test-token"), callers.Stream()))
staterpc.RegisterServer(srv, staterpc.Deps{ControlPlane: control, Tasks: db, Principals: db, Identities: db})
t.Cleanup(srv.Stop)
go func() { _ = srv.Serve(listener) }()
client, closeClient, err := staterpc.Dial(listener.Addr().String(), "archie-ui", "test-token")
if err != nil {
t.Fatal(err)
}
t.Cleanup(closeClient)
return client
}
func TestValidateStoredOrgWorkflows(t *testing.T) {
ctx := t.Context()
db := pgstore.Open(t)
if err := db.BootstrapIdentities(ctx, nil); err != nil {
t.Fatal(err)
}
if err := db.UpgradeDefaultOrg(ctx); err != nil {
t.Fatal(err)
}
steps, err := workflowsteps.NewManager()
if err != nil {
t.Fatal(err)
}
server, err := controlplane.NewServer(db, steps)
if err != nil {
t.Fatal(err)
}
if _, err := db.CreateOrg(ctx, org.Org{ID: "acme", Name: "Acme"}, identity.SystemID); err != nil {
t.Fatal(err)
}
if _, err := db.PutResource(ctx, storecontract.ResourceWrite{OrgID: "acme", Kind: controlplane.WorkflowDefinitionsKind, Value: []byte(`{"definitions":[{"id":"broken","yaml":"id: broken\nsteps:\n - type: nonexistent\n"}]}`), Actor: "owner", Source: "test", RequestID: "invalid"}); err != nil {
t.Fatal(err)
}
if _, err := server.ValidateStored(ctx); err == nil || !strings.Contains(err.Error(), "acme") {
t.Fatalf("invalid org workflow was not reported: %v", err)
}
}
func checkWorkflowDashboard(t *testing.T, db *pgstore.TaskDB, client *staterpc.Client) {
t.Helper()
ctx := t.Context()
policies := append(access.ShippedOrgPolicies("acme"), access.ShippedOrgPolicies(org.DefaultOrgID)...)
chain, err := infraaccess.New(policies)
if err != nil {
t.Fatal(err)
}
srv := &webui.Server{Store: client, ControlPlane: client.ControlPlane(), Access: chain, Principals: client, Authenticate: func(ctx context.Context, token string) (identity.Identity, error) {
return db.Get(ctx, identity.IdentityID(token))
}}
handler := srv.Handler()
for _, tc := range []struct {
method, path, body string
status int
}{
{"GET", "/api/workflows", "", http.StatusOK},
{"POST", "/api/work-requests", `{"identity":"acme-owner","workflow":"acme-only","title":"Org work","instructions":"Finish"}`, http.StatusCreated},
} {
r := httptest.NewRequestWithContext(ctx, tc.method, tc.path, strings.NewReader(tc.body))
r.Header.Set("Authorization", "Bearer acme-owner")
r.Header.Set("Content-Type", "application/json")
r.Header.Set("X-Archie-CSRF", "1")
w := httptest.NewRecorder()
handler.ServeHTTP(w, r)
if w.Code != tc.status {
t.Fatalf("%s %s: %d %s", tc.method, tc.path, w.Code, w.Body.String())
}
if tc.method == "GET" && !strings.Contains(w.Body.String(), "acme-only") {
t.Fatalf("dashboard omitted the org's workflow: %s", w.Body.String())
}
if tc.method == "POST" {
var reply struct {
TaskID int64 `json:"task_id"`
}
if err := json.Unmarshal(w.Body.Bytes(), &reply); err != nil {
t.Fatal(err)
}
run, err := db.TaskByID(ctx, reply.TaskID)
if err != nil || run.Org != "acme" || run.Workflow != "acme-only" {
t.Fatalf("admitted task %#v err %v", run, err)
}
}
}
}
+6
View File
@@ -58,3 +58,9 @@ func Logging(cfg config.Config, component string) Logs {
}
return Logs{Log: log, Feed: feed, TaskLogs: taskLogs, Closer: closer}
}
// Diagnostics captures a service's stderr log for its recent-log contract.
func Diagnostics(log *slog.Logger, service string) (*slog.Logger, *logging.Feed) {
feed := logging.NewFeed(1000)
return slog.New(logging.NewFeedHandler(log.Handler(), feed)).With("service", service), feed
}
+2 -8
View File
@@ -15,11 +15,6 @@ func ResolveProviderSecrets(cfg *config.Config, registry *secret.Registry, log *
if err := resolveProviderMap("root", cfg.Providers, registry, log); err != nil {
return err
}
for i := range cfg.Identities {
if err := resolveProviderMap(cfg.Identities[i].Name, cfg.Identities[i].Providers, registry, log); err != nil {
return fmt.Errorf("identity %q: %w", cfg.Identities[i].Name, err)
}
}
return nil
}
@@ -69,12 +64,11 @@ func resolveProviderMap(scope string, providers map[string]config.Provider, regi
}
// disableProvider clears a provider's credential so every consumer sees it as
// unconfigured, and says why. The reference is cleared rather than left in
// place: webui's ProviderView derives Configured from it, so keeping an
// unresolvable reference would advertise a provider that cannot be used.
// unconfigured, and says why.
func disableProvider(providers map[string]config.Provider, id string, provider config.Provider, log *slog.Logger, reason string, cause error) {
provider.APIKey = secret.SecretRef{}
provider.APIKeyEnv = ""
provider.Disabled = true
providers[id] = provider
if cause != nil {
log.Warn("provider disabled: "+reason, "provider", id, "err", cause)
+38
View File
@@ -0,0 +1,38 @@
package servicekit
import (
"context"
"time"
"github.com/samcharles93/archie-core/internal/domain/presence"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/releaseupdate"
)
// RunningVersions reports the daemon version archied's own presence record
// carries, so an update report is checked against what archied compiled in
// rather than this process's build. A down or unreachable daemon, or a
// missing store, leaves the daemon unverified. The agent stays unverified:
// only archied observes it.
func RunningVersions(ctx context.Context, store storecontract.PresenceStore, timeout time.Duration) func() map[string]string {
return func() map[string]string {
if store == nil {
return nil
}
if timeout <= 0 {
timeout = 5 * time.Second
}
callCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
records, err := store.ListPresence(callCtx)
if err != nil {
return nil
}
for _, service := range presence.Mesh(records, time.Now()) {
if service.Service == presence.Daemon && service.State != presence.StateDown {
return map[string]string{releaseupdate.ComponentDaemon: service.Version}
}
}
return nil
}
}
+4 -3
View File
@@ -20,8 +20,9 @@ import (
// server is the State Store process: the one owner of the task and
// control-plane tables.
type server struct {
cfg config.Config
log *slog.Logger
cfg config.Config
log *slog.Logger
logFeed *logging.Feed
// stderrLog keeps an offline command off cfg.Log.File: a diagnosis must
// not create, append to or rotate the deployment's log.
stderrLog bool
@@ -57,7 +58,7 @@ func (b *server) loadConfig(_ context.Context, cfgPath, overlayPath string) erro
return nil
}
logs := servicekit.Logging(b.cfg, "state-store")
b.log, b.taskLogs = logs.Log, logs.TaskLogs
b.log, b.taskLogs, b.logFeed = logs.Log, logs.TaskLogs, logs.Feed
b.addCleanup(func() { _ = logs.Closer.Close() })
return nil
}
+6 -4
View File
@@ -29,6 +29,7 @@ import (
infraaccess "github.com/samcharles93/archie-core/internal/infrastructure/access"
"github.com/samcharles93/archie-core/internal/infrastructure/postgres"
"github.com/samcharles93/archie-core/internal/infrastructure/readiness"
"github.com/samcharles93/archie-core/internal/infrastructure/rpcidentity"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
registry "github.com/samcharles93/archie-core/internal/infrastructure/storepkg"
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
@@ -265,7 +266,8 @@ func (b *server) openStateStore(ctx context.Context) error {
// degrades that group rather than aborting boot.
func (b *server) stateStoreDeps(grants *staterpc.TaskGrants) staterpc.Deps {
deps := staterpc.Deps{
Tasks: b.st, Log: b.log, Grants: grants,
LogFeed: b.logFeed,
Tasks: b.st, Log: b.log, Grants: grants,
// The same root archied dispatches as (servicekit.IdentityNames).
RootIdentity: identity.StableID(servicekit.IdentityNames(b.cfg)[0]),
}
@@ -403,9 +405,9 @@ func (b *server) runCredentials(ctx context.Context) (*staterpc.TaskGrants, erro
// callers resolves the service and principal of each call against this
// store's principals.
func (b *server) callers() staterpc.Callers {
func (b *server) callers() rpcidentity.Callers {
ps, _ := b.st.(access.PrincipalSource)
return staterpc.Callers{Principals: ps}
return rpcidentity.Callers{Principals: ps}
}
// accessDeps wires the policy chain and its denial records from the same
@@ -511,7 +513,7 @@ func (b *server) startOptionalSurfaces(ctx context.Context, options Options) err
// as a remote one (staterpc.Dial installs the client keepalive on every target,
// loopback included), and grpc's default policy would answer their idle
// watches with GOAWAY too_many_pings. See staterpc.ServerKeepaliveOption.
func stateStoreServerOpts(listen, token string, grants *staterpc.TaskGrants, callers staterpc.Callers) (opts []grpc.ServerOption, loopback bool, err error) {
func stateStoreServerOpts(listen, token string, grants *staterpc.TaskGrants, callers rpcidentity.Callers) (opts []grpc.ServerOption, loopback bool, err error) {
loopback, err = staterpc.TargetIsLoopback(listen)
if err != nil {
return nil, false, err
+18
View File
@@ -74,6 +74,24 @@ func (m *Manager) MarkDegraded(id, detail string) {
func (m *Manager) MarkFailed(id, detail string) { m.set(id, StateFailed, detail) }
func (m *Manager) MarkStopped(id, detail string) { m.set(id, StateStopped, detail) }
// Declare records whether a composed channel is currently configured and
// reloadable. A live enable or disable changes both without changing the
// channel's identity, so the operator-facing declaration follows the channel
// rather than being fixed at composition.
func (m *Manager) Declare(id string, configured, reloadSupported bool) {
if m == nil {
return
}
m.mu.Lock()
defer m.mu.Unlock()
i, ok := m.index[id]
if !ok {
return
}
m.channels[i].Configured = configured
m.channels[i].ReloadSupported = reloadSupported
}
func (m *Manager) set(id string, state State, detail string) {
if m == nil {
return
+24 -45
View File
@@ -9,6 +9,8 @@ import (
"net"
"strings"
"time"
orgdomain "github.com/samcharles93/archie-core/internal/domain/org"
)
// Duration is a time.Duration that unmarshals from TOML strings ("60s").
@@ -155,6 +157,12 @@ type Provider struct {
APIKeyEnv string `toml:"api_key_env" yaml:"api_key_env" json:"api_key_env"`
APIKey SecretRef `toml:"api_key" yaml:"api_key" json:"-"`
BaseURL string `toml:"base_url" yaml:"base_url" json:"base_url"`
// Disabled marks a configured provider whose api_key could not be
// resolved at boot. It is runtime state, not a setting: without it a
// disabled provider is indistinguishable from one the operator never
// configured, and the model catalog would serve it from an ambient
// environment variable.
Disabled bool `toml:"-" yaml:"-" json:"-"`
}
// Forge intake modes for Forge.Intake. Poll is the default; webhook reacts to
@@ -307,17 +315,6 @@ type Config struct {
// SkillsDir is an optional directory of */SKILL.md skills. Its workflows
// override built-ins, and Kits mount it read-only.
SkillsDir string `toml:"skills_dir" yaml:"skills_dir"`
// WorkflowRoutingFile is an optional path to a YAML file rebinding which
// registered workflow an intake Kind (bug/feature) prefers.
WorkflowRoutingFile string `toml:"workflow_routing_file" yaml:"workflow_routing_file"`
// WorkflowLabelsFile is an optional path to a YAML file binding forge issue
// labels outside the closed bug/feature kind set to registered
// workflows (e.g. A label already owned by the kind set, or a duplicate
// binding, is a load failure per the design doc's collision rule.
WorkflowLabelsFile string `toml:"workflow_labels_file" yaml:"workflow_labels_file"`
// PlaybookDirs are optional directories of kind/label-to-workflow binding
// files. A key bound in more than one file is a load error.
PlaybookDirs []string `toml:"playbook_dirs" yaml:"playbook_dirs"`
// MaxRetries caps how many times a parked task is retried before
// being permanently parked (status "dead"). Defaults to 3.
MaxRetries int `toml:"max_retries" yaml:"max_retries"`
@@ -382,16 +379,11 @@ type Config struct {
// Curators are curator definitions seeded from [[curators]].
Curators []CuratorDefinition `toml:"curators" yaml:"curators" json:"curators,omitempty"`
// Extra holds additional feature configuration from conf.d/ files that
// don't match a known feature name. Keys are the filename stem (e.g.
// "custom-tool" for conf.d/custom-tool.yaml).
Extra map[string]any `toml:"-" yaml:"-" json:"extra,omitempty"`
}
// IdentityConfig is a per-identity configuration subset. Each identity
// gets its own poll loop goroutine with isolated forge, repo, model,
// and dispatch configuration.
// gets its own forge account and repository membership. Runtime settings
// come from the shared control plane.
type IdentityConfig struct {
// Name identifies this identity in logs, events, and NATS subject
// namespaces (archie.<name>.task.<type>). Required when Identities
@@ -407,22 +399,11 @@ type IdentityConfig struct {
Org string `toml:"org" yaml:"org"`
// GrantedCredentials overrides the shared GrantedCredentials for this
// identity. Nil (absent) inherits it; a present, empty list grants none
// -- the same explicit-empty-means-off shape DiffCapLines uses, needed
// -- an explicit empty list turns grants off, needed
// because a plain nil already means "not set" here.
GrantedCredentials *[]string `toml:"granted_credentials" yaml:"granted_credentials"`
// DiffCapLines overrides the shared cap for this identity. Nil (absent)
// inherits it; an explicit 0 switches the cap off for this identity only.
DiffCapLines *int `toml:"diff_cap_lines" yaml:"diff_cap_lines"`
// PollInterval overrides the shared PollInterval. When 0, the shared
// interval is used.
PollInterval Duration `toml:"poll_interval" yaml:"poll_interval"`
Forge Forge `toml:"forge" yaml:"forge"`
Dispatch Dispatch `toml:"dispatch" yaml:"dispatch"`
Models map[string]string `toml:"models" yaml:"models"`
Providers map[string]Provider `toml:"providers" yaml:"providers"`
Budgets Budgets `toml:"budgets" yaml:"budgets"`
Notify Notify `toml:"notify" yaml:"notify"`
Repos []Repo `toml:"repos" yaml:"repos"`
Forge Forge `toml:"forge" yaml:"forge"`
Repos []Repo `toml:"repos" yaml:"repos"`
}
type ModelLimits struct {
@@ -534,8 +515,6 @@ func (c Config) Clone() Config {
c.Identities = cloneIdentities(c.Identities)
c.Chat.Models = append([]string(nil), c.Chat.Models...)
c.Chat.Telegram.AllowedUserIDs = append([]int64(nil), c.Chat.Telegram.AllowedUserIDs...)
c.Chat.Telegram.UpdateCheckCommand = append([]string(nil), c.Chat.Telegram.UpdateCheckCommand...)
c.Chat.Telegram.UpdateInstallCommand = append([]string(nil), c.Chat.Telegram.UpdateInstallCommand...)
c.Bindings.PreviousEncryptionKeys = append([]SecretRef(nil), c.Bindings.PreviousEncryptionKeys...)
c.Tools.MCPServers = cloneMCPServers(c.Tools.MCPServers)
// Services is a map of structs, so the header is shared by the value copy
@@ -548,7 +527,6 @@ func (c Config) Clone() Config {
c.Tools.WebFetch.Enabled = &v
}
c.DiffCapLines = cloneIntPtr(c.DiffCapLines)
c.Extra = maps.Clone(c.Extra)
return c
}
@@ -583,10 +561,7 @@ func cloneIdentities(ids []IdentityConfig) []IdentityConfig {
}
out := make([]IdentityConfig, len(ids))
for i, id := range ids {
id.Models = cloneStringMap(id.Models)
id.Providers = maps.Clone(id.Providers)
id.Repos = cloneRepos(id.Repos)
id.DiffCapLines = cloneIntPtr(id.DiffCapLines)
out[i] = id
}
return out
@@ -720,6 +695,7 @@ type CredentialBinding struct {
// BoundCredentials returns the bindings a run may use: services that are
// declared, granted, and belong to org.
func (c ContainerConfig) BoundCredentials(org string, granted, declared []string) map[string]CredentialBinding {
org = credentialOrg(org)
grantedSet := make(map[string]bool, len(granted))
for _, g := range granted {
grantedSet[g] = true
@@ -730,6 +706,7 @@ func (c ContainerConfig) BoundCredentials(org string, granted, declared []string
}
out := make(map[string]CredentialBinding)
for _, b := range c.Credentials {
b.Org = credentialOrg(b.Org)
if b.Org != org || !grantedSet[b.Service] || !declaredSet[b.Service] {
continue
}
@@ -754,7 +731,14 @@ func (c Config) CredentialAccess(name string) (org string, granted []string) {
granted = *id.GrantedCredentials
}
}
return org, granted
return credentialOrg(org), granted
}
func credentialOrg(value string) string {
if value == "" {
return string(orgdomain.DefaultOrgID)
}
return value
}
// ValidateCredentialBindings rejects a binding with an empty service name or
@@ -766,7 +750,7 @@ func (c ContainerConfig) ValidateCredentialBindings() error {
if strings.TrimSpace(b.Service) == "" {
return fmt.Errorf("containers.credentials: a service name must not be empty")
}
key := b.Org + "\x00" + b.Service
key := credentialOrg(b.Org) + "\x00" + b.Service
if seen[key] {
return fmt.Errorf("containers.credentials: service %q is bound more than once for org %q", b.Service, b.Org)
}
@@ -918,11 +902,6 @@ type TelegramConfig struct {
// Token references the bot token from @BotFather through the configured
// secret engine. Empty disables the Telegram channel.
Token SecretRef `toml:"token" yaml:"token"`
// UpdateCheckCommand writes a releaseupdate.Snapshot JSON document to
// stdout. UpdateInstallCommand applies an already-approved update. Both
// are argv arrays, never shell snippets.
UpdateCheckCommand []string `toml:"update_check_command" yaml:"update_check_command"`
UpdateInstallCommand []string `toml:"update_install_command" yaml:"update_install_command"`
}
// Notify configures outbound notifications (n8n webhook → email etc.).
+42
View File
@@ -0,0 +1,42 @@
package config
import (
"testing"
"github.com/samcharles93/archie-core/internal/domain/org"
)
func TestCredentialOrgBoundary(t *testing.T) {
tests := []struct {
name, binding, requested, identity, root string
bound bool
want string
}{
{name: "empty binding matches dashboard default", requested: string(org.DefaultOrgID), bound: true, want: string(org.DefaultOrgID)},
{name: "explicit default matches root", binding: string(org.DefaultOrgID), bound: true, want: string(org.DefaultOrgID)},
{name: "empty binding and root", bound: true, want: string(org.DefaultOrgID)},
{name: "named tenant remains isolated", binding: "acme", requested: "other", identity: "acme", root: "other", want: "acme"},
{name: "identity inherits root tenant", binding: "acme", requested: "acme", root: "acme", bound: true, want: "acme"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := Config{Org: tt.root, GrantedCredentials: []string{"codex"}, Identities: []IdentityConfig{{Name: "bot", Org: tt.identity}}, Containers: ContainerConfig{Credentials: []CredentialBinding{{Service: "codex", Org: tt.binding}}}}
got, granted := cfg.CredentialAccess("bot")
if got != tt.want {
t.Fatalf("run org %q, want %q", got, tt.want)
}
bindings := cfg.Containers.BoundCredentials(tt.requested, granted, []string{"codex"})
b, ok := bindings["codex"]
if ok != tt.bound {
t.Fatalf("bound=%v, want %v", ok, tt.bound)
}
if ok && b.Org == "" {
t.Fatal("returned binding retains empty org")
}
})
}
cfg := ContainerConfig{Credentials: []CredentialBinding{{Service: "codex"}, {Service: "codex", Org: string(org.DefaultOrgID)}}}
if err := cfg.ValidateCredentialBindings(); err == nil {
t.Fatal("empty and explicit default bindings must be duplicates")
}
}
+6
View File
@@ -47,6 +47,7 @@ const (
ServiceNameGateway = "gateway"
ServiceNameState = "state"
ServiceNameMessaging = "messaging"
ServiceNameHarness = "harness"
)
var (
@@ -115,4 +116,9 @@ func init() {
RegisterService(ServiceBoth, ServiceNameGateway, "127.0.0.1:8585", "127.0.0.1:8585", "GATEWAY_TOKEN")
RegisterService(ServiceClient, ServiceNameState, "", "127.0.0.1:9090", "STATE_STORE_TOKEN")
RegisterService(ServiceBoth, ServiceNameMessaging, "127.0.0.1:8586", "127.0.0.1:8586", "MESSAGING_TOKEN")
// The daemon hosts the harness session contract in this repository; the
// dashboard dials it, and its target is the operator's to supply, so a
// deployment with no [services.harness].target leaves the setup terminal
// unwired rather than dialling a default nothing owns.
RegisterService(ServiceBoth, ServiceNameHarness, "", "127.0.0.1:8587", "HARNESS_TOKEN")
}
+5
View File
@@ -49,6 +49,10 @@ func StartKit(ctx context.Context, cli *client.Client, s KitSpec) (string, error
}
mounts = append(mounts, mount.Mount{Type: mount.TypeVolume, Source: v.Name, Target: v.Path})
}
// Docker's own init becomes PID 1 and reaps orphans. The worker is PID 1
// otherwise, and a detached hook that forks and exits leaves zombies the
// worker never waits on, until the container's PID table is exhausted.
hostInit := true
created, err := cli.ContainerCreate(ctx, client.ContainerCreateOptions{
Name: s.Name,
Config: &container.Config{
@@ -64,6 +68,7 @@ func StartKit(ctx context.Context, cli *client.Client, s KitSpec) (string, error
NetworkMode: container.NetworkMode(s.Network),
Binds: s.Binds,
Mounts: mounts,
Init: &hostInit,
},
})
if err != nil {
+61
View File
@@ -0,0 +1,61 @@
package container
import (
"bufio"
"context"
"net"
"github.com/moby/moby/client"
"github.com/samcharles93/archie-core/internal/infrastructure/kit"
)
// PTY is an interactive exec attached to a container's TTY: writes are
// terminal input, reads are terminal output, and Close closes the hijacked
// socket so a blocked Read unblocks. Docker cannot reattach to a disconnected
// exec, so the caller holds this for the life of the session.
type PTY struct {
execID string
conn net.Conn
reader *bufio.Reader
cli *client.Client
}
// ExecPTY starts argv in a container as user with exactly env, on a TTY. It
// mirrors ExecIn's explicit user and env: the exec must never inherit the
// container's worker environment.
func ExecPTY(ctx context.Context, cli *client.Client, id, user string, env, argv []string, rows, cols uint) (*PTY, error) {
exec, err := cli.ExecCreate(ctx, id, client.ExecCreateOptions{
User: user, Env: env, Cmd: argv, WorkingDir: kit.WorkspaceDir,
TTY: true, AttachStdin: true, AttachStdout: true, AttachStderr: true,
ConsoleSize: client.ConsoleSize{Height: rows, Width: cols},
})
if err != nil {
return nil, err
}
attached, err := cli.ExecAttach(ctx, exec.ID, client.ExecAttachOptions{TTY: true})
if err != nil {
return nil, err
}
return &PTY{execID: exec.ID, conn: attached.Conn, reader: attached.Reader, cli: cli}, nil
}
// Read reads terminal output. A TTY stream is raw, so there is no
// stdout/stderr to demultiplex.
func (p *PTY) Read(b []byte) (int, error) { return p.reader.Read(b) }
// Write writes terminal input.
func (p *PTY) Write(b []byte) (int, error) { return p.conn.Write(b) }
// ExitCode reports the exec's exit status. It is meaningful only once the
// output stream has ended.
func (p *PTY) ExitCode(ctx context.Context) (int, error) {
inspected, err := p.cli.ExecInspect(ctx, p.execID, client.ExecInspectOptions{})
if err != nil {
return 0, err
}
return inspected.ExitCode, nil
}
// Close closes the hijacked socket, which unblocks a blocked Read.
func (p *PTY) Close() error { return p.conn.Close() }
File diff suppressed because it is too large. Load diff
@@ -19,12 +19,14 @@ import (
const _ = grpc.SupportPackageIsVersion9
const (
ChatService_RecentLogs_FullMethodName = "/gateway.v1.ChatService/RecentLogs"
ChatService_Snapshot_FullMethodName = "/gateway.v1.ChatService/Snapshot"
ChatService_GetSession_FullMethodName = "/gateway.v1.ChatService/GetSession"
ChatService_RecentMessages_FullMethodName = "/gateway.v1.ChatService/RecentMessages"
ChatService_RecentTurns_FullMethodName = "/gateway.v1.ChatService/RecentTurns"
ChatService_Route_FullMethodName = "/gateway.v1.ChatService/Route"
ChatService_Stream_FullMethodName = "/gateway.v1.ChatService/Stream"
ChatService_Answer_FullMethodName = "/gateway.v1.ChatService/Answer"
ChatService_Cancel_FullMethodName = "/gateway.v1.ChatService/Cancel"
ChatService_StopTasks_FullMethodName = "/gateway.v1.ChatService/StopTasks"
ChatService_SetPersona_FullMethodName = "/gateway.v1.ChatService/SetPersona"
@@ -50,12 +52,14 @@ const (
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type ChatServiceClient interface {
RecentLogs(ctx context.Context, in *RecentLogsRequest, opts ...grpc.CallOption) (*RecentLogsResponse, error)
Snapshot(ctx context.Context, in *SnapshotRequest, opts ...grpc.CallOption) (*SnapshotResponse, error)
GetSession(ctx context.Context, in *GetSessionRequest, opts ...grpc.CallOption) (*GetSessionResponse, error)
RecentMessages(ctx context.Context, in *RecentMessagesRequest, opts ...grpc.CallOption) (*RecentMessagesResponse, error)
RecentTurns(ctx context.Context, in *RecentTurnsRequest, opts ...grpc.CallOption) (*RecentTurnsResponse, error)
Route(ctx context.Context, in *RouteRequest, opts ...grpc.CallOption) (*RouteResponse, error)
Stream(ctx context.Context, in *StreamRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[StreamResponse], error)
Answer(ctx context.Context, in *AnswerRequest, opts ...grpc.CallOption) (*AnswerResponse, error)
Cancel(ctx context.Context, in *CancelRequest, opts ...grpc.CallOption) (*CancelResponse, error)
StopTasks(ctx context.Context, in *StopTasksRequest, opts ...grpc.CallOption) (*StopTasksResponse, error)
SetPersona(ctx context.Context, in *SetPersonaRequest, opts ...grpc.CallOption) (*SetPersonaResponse, error)
@@ -85,6 +89,16 @@ func NewChatServiceClient(cc grpc.ClientConnInterface) ChatServiceClient {
return &chatServiceClient{cc}
}
func (c *chatServiceClient) RecentLogs(ctx context.Context, in *RecentLogsRequest, opts ...grpc.CallOption) (*RecentLogsResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(RecentLogsResponse)
err := c.cc.Invoke(ctx, ChatService_RecentLogs_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *chatServiceClient) Snapshot(ctx context.Context, in *SnapshotRequest, opts ...grpc.CallOption) (*SnapshotResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(SnapshotResponse)
@@ -154,6 +168,16 @@ func (c *chatServiceClient) Stream(ctx context.Context, in *StreamRequest, opts
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type ChatService_StreamClient = grpc.ServerStreamingClient[StreamResponse]
func (c *chatServiceClient) Answer(ctx context.Context, in *AnswerRequest, opts ...grpc.CallOption) (*AnswerResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(AnswerResponse)
err := c.cc.Invoke(ctx, ChatService_Answer_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *chatServiceClient) Cancel(ctx context.Context, in *CancelRequest, opts ...grpc.CallOption) (*CancelResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(CancelResponse)
@@ -348,12 +372,14 @@ func (c *chatServiceClient) ReloadChannel(ctx context.Context, in *ReloadChannel
// All implementations must embed UnimplementedChatServiceServer
// for forward compatibility.
type ChatServiceServer interface {
RecentLogs(context.Context, *RecentLogsRequest) (*RecentLogsResponse, error)
Snapshot(context.Context, *SnapshotRequest) (*SnapshotResponse, error)
GetSession(context.Context, *GetSessionRequest) (*GetSessionResponse, error)
RecentMessages(context.Context, *RecentMessagesRequest) (*RecentMessagesResponse, error)
RecentTurns(context.Context, *RecentTurnsRequest) (*RecentTurnsResponse, error)
Route(context.Context, *RouteRequest) (*RouteResponse, error)
Stream(*StreamRequest, grpc.ServerStreamingServer[StreamResponse]) error
Answer(context.Context, *AnswerRequest) (*AnswerResponse, error)
Cancel(context.Context, *CancelRequest) (*CancelResponse, error)
StopTasks(context.Context, *StopTasksRequest) (*StopTasksResponse, error)
SetPersona(context.Context, *SetPersonaRequest) (*SetPersonaResponse, error)
@@ -383,6 +409,9 @@ type ChatServiceServer interface {
// pointer dereference when methods are called.
type UnimplementedChatServiceServer struct{}
func (UnimplementedChatServiceServer) RecentLogs(context.Context, *RecentLogsRequest) (*RecentLogsResponse, error) {
return nil, status.Error(codes.Unimplemented, "method RecentLogs not implemented")
}
func (UnimplementedChatServiceServer) Snapshot(context.Context, *SnapshotRequest) (*SnapshotResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Snapshot not implemented")
}
@@ -401,6 +430,9 @@ func (UnimplementedChatServiceServer) Route(context.Context, *RouteRequest) (*Ro
func (UnimplementedChatServiceServer) Stream(*StreamRequest, grpc.ServerStreamingServer[StreamResponse]) error {
return status.Error(codes.Unimplemented, "method Stream not implemented")
}
func (UnimplementedChatServiceServer) Answer(context.Context, *AnswerRequest) (*AnswerResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Answer not implemented")
}
func (UnimplementedChatServiceServer) Cancel(context.Context, *CancelRequest) (*CancelResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Cancel not implemented")
}
@@ -479,6 +511,24 @@ func RegisterChatServiceServer(s grpc.ServiceRegistrar, srv ChatServiceServer) {
s.RegisterService(&ChatService_ServiceDesc, srv)
}
func _ChatService_RecentLogs_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(RecentLogsRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ChatServiceServer).RecentLogs(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: ChatService_RecentLogs_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ChatServiceServer).RecentLogs(ctx, req.(*RecentLogsRequest))
}
return interceptor(ctx, in, info, handler)
}
func _ChatService_Snapshot_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(SnapshotRequest)
if err := dec(in); err != nil {
@@ -580,6 +630,24 @@ func _ChatService_Stream_Handler(srv interface{}, stream grpc.ServerStream) erro
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type ChatService_StreamServer = grpc.ServerStreamingServer[StreamResponse]
func _ChatService_Answer_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(AnswerRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ChatServiceServer).Answer(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: ChatService_Answer_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ChatServiceServer).Answer(ctx, req.(*AnswerRequest))
}
return interceptor(ctx, in, info, handler)
}
func _ChatService_Cancel_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(CancelRequest)
if err := dec(in); err != nil {
@@ -929,6 +997,10 @@ var ChatService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "gateway.v1.ChatService",
HandlerType: (*ChatServiceServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "RecentLogs",
Handler: _ChatService_RecentLogs_Handler,
},
{
MethodName: "Snapshot",
Handler: _ChatService_Snapshot_Handler,
@@ -949,6 +1021,10 @@ var ChatService_ServiceDesc = grpc.ServiceDesc{
MethodName: "Route",
Handler: _ChatService_Route_Handler,
},
{
MethodName: "Answer",
Handler: _ChatService_Answer_Handler,
},
{
MethodName: "Cancel",
Handler: _ChatService_Cancel_Handler,
+575
View File
@@ -0,0 +1,575 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc (unknown)
// source: harness/v1/harness.proto
package harnessv1
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
unsafe "unsafe"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
// OpenRequest is one client frame. The first frame must be target; a frame
// with no field set is a keepalive the server ignores.
type OpenRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
// Types that are valid to be assigned to Frame:
//
// *OpenRequest_Target
// *OpenRequest_Stdin
Frame isOpenRequest_Frame `protobuf_oneof:"frame"`
// size is the terminal size the client wants, in character cells. The
// dashboard sends it with the target frame so the PTY is created at the
// right size; the daemon falls back to its own default when it is absent.
Size *ConsoleSize `protobuf:"bytes,3,opt,name=size,proto3" json:"size,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *OpenRequest) Reset() {
*x = OpenRequest{}
mi := &file_harness_v1_harness_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *OpenRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*OpenRequest) ProtoMessage() {}
func (x *OpenRequest) ProtoReflect() protoreflect.Message {
mi := &file_harness_v1_harness_proto_msgTypes[0]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use OpenRequest.ProtoReflect.Descriptor instead.
func (*OpenRequest) Descriptor() ([]byte, []int) {
return file_harness_v1_harness_proto_rawDescGZIP(), []int{0}
}
func (x *OpenRequest) GetFrame() isOpenRequest_Frame {
if x != nil {
return x.Frame
}
return nil
}
func (x *OpenRequest) GetTarget() *OpenTarget {
if x != nil {
if x, ok := x.Frame.(*OpenRequest_Target); ok {
return x.Target
}
}
return nil
}
func (x *OpenRequest) GetStdin() []byte {
if x != nil {
if x, ok := x.Frame.(*OpenRequest_Stdin); ok {
return x.Stdin
}
}
return nil
}
func (x *OpenRequest) GetSize() *ConsoleSize {
if x != nil {
return x.Size
}
return nil
}
type isOpenRequest_Frame interface {
isOpenRequest_Frame()
}
type OpenRequest_Target struct {
// target names the session and must be the first frame.
Target *OpenTarget `protobuf:"bytes,1,opt,name=target,proto3,oneof"`
}
type OpenRequest_Stdin struct {
// stdin is terminal input bytes.
Stdin []byte `protobuf:"bytes,2,opt,name=stdin,proto3,oneof"`
}
func (*OpenRequest_Target) isOpenRequest_Frame() {}
func (*OpenRequest_Stdin) isOpenRequest_Frame() {}
// OpenResponse is one server frame.
type OpenResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
// Types that are valid to be assigned to Frame:
//
// *OpenResponse_Stdout
// *OpenResponse_ExitCode
// *OpenResponse_Error
Frame isOpenResponse_Frame `protobuf_oneof:"frame"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *OpenResponse) Reset() {
*x = OpenResponse{}
mi := &file_harness_v1_harness_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *OpenResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*OpenResponse) ProtoMessage() {}
func (x *OpenResponse) ProtoReflect() protoreflect.Message {
mi := &file_harness_v1_harness_proto_msgTypes[1]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use OpenResponse.ProtoReflect.Descriptor instead.
func (*OpenResponse) Descriptor() ([]byte, []int) {
return file_harness_v1_harness_proto_rawDescGZIP(), []int{1}
}
func (x *OpenResponse) GetFrame() isOpenResponse_Frame {
if x != nil {
return x.Frame
}
return nil
}
func (x *OpenResponse) GetStdout() []byte {
if x != nil {
if x, ok := x.Frame.(*OpenResponse_Stdout); ok {
return x.Stdout
}
}
return nil
}
func (x *OpenResponse) GetExitCode() int32 {
if x != nil {
if x, ok := x.Frame.(*OpenResponse_ExitCode); ok {
return x.ExitCode
}
}
return 0
}
func (x *OpenResponse) GetError() string {
if x != nil {
if x, ok := x.Frame.(*OpenResponse_Error); ok {
return x.Error
}
}
return ""
}
type isOpenResponse_Frame interface {
isOpenResponse_Frame()
}
type OpenResponse_Stdout struct {
// stdout is terminal output bytes.
Stdout []byte `protobuf:"bytes,1,opt,name=stdout,proto3,oneof"`
}
type OpenResponse_ExitCode struct {
// exit_code ends the session: the command's exit status.
ExitCode int32 `protobuf:"varint,2,opt,name=exit_code,json=exitCode,proto3,oneof"`
}
type OpenResponse_Error struct {
// error ends the session when it could not be opened or was aborted.
Error string `protobuf:"bytes,3,opt,name=error,proto3,oneof"`
}
func (*OpenResponse_Stdout) isOpenResponse_Frame() {}
func (*OpenResponse_ExitCode) isOpenResponse_Frame() {}
func (*OpenResponse_Error) isOpenResponse_Frame() {}
// OpenTarget addresses one session. A setup session is addressed by Kit
// profile and is ephemeral; a run session is addressed by task and attempt and
// is bounded by the task.
type OpenTarget struct {
state protoimpl.MessageState `protogen:"open.v1"`
Org string `protobuf:"bytes,1,opt,name=org,proto3" json:"org,omitempty"`
// Types that are valid to be assigned to Target:
//
// *OpenTarget_Setup
// *OpenTarget_Run
Target isOpenTarget_Target `protobuf_oneof:"target"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *OpenTarget) Reset() {
*x = OpenTarget{}
mi := &file_harness_v1_harness_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *OpenTarget) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*OpenTarget) ProtoMessage() {}
func (x *OpenTarget) ProtoReflect() protoreflect.Message {
mi := &file_harness_v1_harness_proto_msgTypes[2]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use OpenTarget.ProtoReflect.Descriptor instead.
func (*OpenTarget) Descriptor() ([]byte, []int) {
return file_harness_v1_harness_proto_rawDescGZIP(), []int{2}
}
func (x *OpenTarget) GetOrg() string {
if x != nil {
return x.Org
}
return ""
}
func (x *OpenTarget) GetTarget() isOpenTarget_Target {
if x != nil {
return x.Target
}
return nil
}
func (x *OpenTarget) GetSetup() *SetupTarget {
if x != nil {
if x, ok := x.Target.(*OpenTarget_Setup); ok {
return x.Setup
}
}
return nil
}
func (x *OpenTarget) GetRun() *RunTarget {
if x != nil {
if x, ok := x.Target.(*OpenTarget_Run); ok {
return x.Run
}
}
return nil
}
type isOpenTarget_Target interface {
isOpenTarget_Target()
}
type OpenTarget_Setup struct {
Setup *SetupTarget `protobuf:"bytes,2,opt,name=setup,proto3,oneof"`
}
type OpenTarget_Run struct {
Run *RunTarget `protobuf:"bytes,3,opt,name=run,proto3,oneof"`
}
func (*OpenTarget_Setup) isOpenTarget_Target() {}
func (*OpenTarget_Run) isOpenTarget_Target() {}
// SetupTarget runs the CLI's own login in an ephemeral container built from
// the named agent profile's Kit.
type SetupTarget struct {
state protoimpl.MessageState `protogen:"open.v1"`
Profile string `protobuf:"bytes,1,opt,name=profile,proto3" json:"profile,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *SetupTarget) Reset() {
*x = SetupTarget{}
mi := &file_harness_v1_harness_proto_msgTypes[3]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *SetupTarget) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*SetupTarget) ProtoMessage() {}
func (x *SetupTarget) ProtoReflect() protoreflect.Message {
mi := &file_harness_v1_harness_proto_msgTypes[3]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use SetupTarget.ProtoReflect.Descriptor instead.
func (*SetupTarget) Descriptor() ([]byte, []int) {
return file_harness_v1_harness_proto_rawDescGZIP(), []int{3}
}
func (x *SetupTarget) GetProfile() string {
if x != nil {
return x.Profile
}
return ""
}
// RunTarget is a running task's container. It is reserved for the run-scoped
// live view; this version answers Unimplemented for it.
type RunTarget struct {
state protoimpl.MessageState `protogen:"open.v1"`
TaskId int64 `protobuf:"varint,1,opt,name=task_id,json=taskId,proto3" json:"task_id,omitempty"`
Attempt int64 `protobuf:"varint,2,opt,name=attempt,proto3" json:"attempt,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RunTarget) Reset() {
*x = RunTarget{}
mi := &file_harness_v1_harness_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *RunTarget) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*RunTarget) ProtoMessage() {}
func (x *RunTarget) ProtoReflect() protoreflect.Message {
mi := &file_harness_v1_harness_proto_msgTypes[4]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use RunTarget.ProtoReflect.Descriptor instead.
func (*RunTarget) Descriptor() ([]byte, []int) {
return file_harness_v1_harness_proto_rawDescGZIP(), []int{4}
}
func (x *RunTarget) GetTaskId() int64 {
if x != nil {
return x.TaskId
}
return 0
}
func (x *RunTarget) GetAttempt() int64 {
if x != nil {
return x.Attempt
}
return 0
}
// ConsoleSize is a terminal size in character cells.
type ConsoleSize struct {
state protoimpl.MessageState `protogen:"open.v1"`
Rows uint32 `protobuf:"varint,1,opt,name=rows,proto3" json:"rows,omitempty"`
Cols uint32 `protobuf:"varint,2,opt,name=cols,proto3" json:"cols,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *ConsoleSize) Reset() {
*x = ConsoleSize{}
mi := &file_harness_v1_harness_proto_msgTypes[5]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *ConsoleSize) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*ConsoleSize) ProtoMessage() {}
func (x *ConsoleSize) ProtoReflect() protoreflect.Message {
mi := &file_harness_v1_harness_proto_msgTypes[5]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use ConsoleSize.ProtoReflect.Descriptor instead.
func (*ConsoleSize) Descriptor() ([]byte, []int) {
return file_harness_v1_harness_proto_rawDescGZIP(), []int{5}
}
func (x *ConsoleSize) GetRows() uint32 {
if x != nil {
return x.Rows
}
return 0
}
func (x *ConsoleSize) GetCols() uint32 {
if x != nil {
return x.Cols
}
return 0
}
var File_harness_v1_harness_proto protoreflect.FileDescriptor
const file_harness_v1_harness_proto_rawDesc = "" +
"\n" +
"\x18harness/v1/harness.proto\x12\n" +
"harness.v1\"\x8d\x01\n" +
"\vOpenRequest\x120\n" +
"\x06target\x18\x01 \x01(\v2\x16.harness.v1.OpenTargetH\x00R\x06target\x12\x16\n" +
"\x05stdin\x18\x02 \x01(\fH\x00R\x05stdin\x12+\n" +
"\x04size\x18\x03 \x01(\v2\x17.harness.v1.ConsoleSizeR\x04sizeB\a\n" +
"\x05frame\"h\n" +
"\fOpenResponse\x12\x18\n" +
"\x06stdout\x18\x01 \x01(\fH\x00R\x06stdout\x12\x1d\n" +
"\texit_code\x18\x02 \x01(\x05H\x00R\bexitCode\x12\x16\n" +
"\x05error\x18\x03 \x01(\tH\x00R\x05errorB\a\n" +
"\x05frame\"\x84\x01\n" +
"\n" +
"OpenTarget\x12\x10\n" +
"\x03org\x18\x01 \x01(\tR\x03org\x12/\n" +
"\x05setup\x18\x02 \x01(\v2\x17.harness.v1.SetupTargetH\x00R\x05setup\x12)\n" +
"\x03run\x18\x03 \x01(\v2\x15.harness.v1.RunTargetH\x00R\x03runB\b\n" +
"\x06target\"'\n" +
"\vSetupTarget\x12\x18\n" +
"\aprofile\x18\x01 \x01(\tR\aprofile\">\n" +
"\tRunTarget\x12\x17\n" +
"\atask_id\x18\x01 \x01(\x03R\x06taskId\x12\x18\n" +
"\aattempt\x18\x02 \x01(\x03R\aattempt\"5\n" +
"\vConsoleSize\x12\x12\n" +
"\x04rows\x18\x01 \x01(\rR\x04rows\x12\x12\n" +
"\x04cols\x18\x02 \x01(\rR\x04cols2O\n" +
"\x0eHarnessService\x12=\n" +
"\x04Open\x12\x17.harness.v1.OpenRequest\x1a\x18.harness.v1.OpenResponse(\x010\x01B\xb4\x01\n" +
"\x0ecom.harness.v1B\fHarnessProtoP\x01ZKgithub.com/samcharles93/archie-core/internal/contracts/harness/v1;harnessv1\xa2\x02\x03HXX\xaa\x02\n" +
"Harness.V1\xca\x02\n" +
"Harness\\V1\xe2\x02\x16Harness\\V1\\GPBMetadata\xea\x02\vHarness::V1b\x06proto3"
var (
file_harness_v1_harness_proto_rawDescOnce sync.Once
file_harness_v1_harness_proto_rawDescData []byte
)
func file_harness_v1_harness_proto_rawDescGZIP() []byte {
file_harness_v1_harness_proto_rawDescOnce.Do(func() {
file_harness_v1_harness_proto_rawDescData = protoimpl.X.CompressGZIP(unsafe.Slice(unsafe.StringData(file_harness_v1_harness_proto_rawDesc), len(file_harness_v1_harness_proto_rawDesc)))
})
return file_harness_v1_harness_proto_rawDescData
}
var file_harness_v1_harness_proto_msgTypes = make([]protoimpl.MessageInfo, 6)
var file_harness_v1_harness_proto_goTypes = []any{
(*OpenRequest)(nil), // 0: harness.v1.OpenRequest
(*OpenResponse)(nil), // 1: harness.v1.OpenResponse
(*OpenTarget)(nil), // 2: harness.v1.OpenTarget
(*SetupTarget)(nil), // 3: harness.v1.SetupTarget
(*RunTarget)(nil), // 4: harness.v1.RunTarget
(*ConsoleSize)(nil), // 5: harness.v1.ConsoleSize
}
var file_harness_v1_harness_proto_depIdxs = []int32{
2, // 0: harness.v1.OpenRequest.target:type_name -> harness.v1.OpenTarget
5, // 1: harness.v1.OpenRequest.size:type_name -> harness.v1.ConsoleSize
3, // 2: harness.v1.OpenTarget.setup:type_name -> harness.v1.SetupTarget
4, // 3: harness.v1.OpenTarget.run:type_name -> harness.v1.RunTarget
0, // 4: harness.v1.HarnessService.Open:input_type -> harness.v1.OpenRequest
1, // 5: harness.v1.HarnessService.Open:output_type -> harness.v1.OpenResponse
5, // [5:6] is the sub-list for method output_type
4, // [4:5] is the sub-list for method input_type
4, // [4:4] is the sub-list for extension type_name
4, // [4:4] is the sub-list for extension extendee
0, // [0:4] is the sub-list for field type_name
}
func init() { file_harness_v1_harness_proto_init() }
func file_harness_v1_harness_proto_init() {
if File_harness_v1_harness_proto != nil {
return
}
file_harness_v1_harness_proto_msgTypes[0].OneofWrappers = []any{
(*OpenRequest_Target)(nil),
(*OpenRequest_Stdin)(nil),
}
file_harness_v1_harness_proto_msgTypes[1].OneofWrappers = []any{
(*OpenResponse_Stdout)(nil),
(*OpenResponse_ExitCode)(nil),
(*OpenResponse_Error)(nil),
}
file_harness_v1_harness_proto_msgTypes[2].OneofWrappers = []any{
(*OpenTarget_Setup)(nil),
(*OpenTarget_Run)(nil),
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_harness_v1_harness_proto_rawDesc), len(file_harness_v1_harness_proto_rawDesc)),
NumEnums: 0,
NumMessages: 6,
NumExtensions: 0,
NumServices: 1,
},
GoTypes: file_harness_v1_harness_proto_goTypes,
DependencyIndexes: file_harness_v1_harness_proto_depIdxs,
MessageInfos: file_harness_v1_harness_proto_msgTypes,
}.Build()
File_harness_v1_harness_proto = out.File
file_harness_v1_harness_proto_goTypes = nil
file_harness_v1_harness_proto_depIdxs = nil
}
@@ -0,0 +1,133 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.2
// - protoc (unknown)
// source: harness/v1/harness.proto
package harnessv1
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
HarnessService_Open_FullMethodName = "/harness.v1.HarnessService/Open"
)
// HarnessServiceClient is the client API for HarnessService service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
//
// HarnessService opens one duplex PTY session into a Kit container the daemon
// owns. The dashboard's setup terminal consumes the setup target; the
// run-scoped live view consumes the run target (archie-core-qi91), which this
// version does not serve. See docs/prds/external-agent-harness.md,
// "Session contract".
type HarnessServiceClient interface {
// Open carries a session: the first request frame names the target, later
// frames carry terminal input and sizing, and the response stream carries
// terminal output until the command exits or the session ends.
Open(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[OpenRequest, OpenResponse], error)
}
type harnessServiceClient struct {
cc grpc.ClientConnInterface
}
func NewHarnessServiceClient(cc grpc.ClientConnInterface) HarnessServiceClient {
return &harnessServiceClient{cc}
}
func (c *harnessServiceClient) Open(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[OpenRequest, OpenResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &HarnessService_ServiceDesc.Streams[0], HarnessService_Open_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[OpenRequest, OpenResponse]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type HarnessService_OpenClient = grpc.BidiStreamingClient[OpenRequest, OpenResponse]
// HarnessServiceServer is the server API for HarnessService service.
// All implementations must embed UnimplementedHarnessServiceServer
// for forward compatibility.
//
// HarnessService opens one duplex PTY session into a Kit container the daemon
// owns. The dashboard's setup terminal consumes the setup target; the
// run-scoped live view consumes the run target (archie-core-qi91), which this
// version does not serve. See docs/prds/external-agent-harness.md,
// "Session contract".
type HarnessServiceServer interface {
// Open carries a session: the first request frame names the target, later
// frames carry terminal input and sizing, and the response stream carries
// terminal output until the command exits or the session ends.
Open(grpc.BidiStreamingServer[OpenRequest, OpenResponse]) error
mustEmbedUnimplementedHarnessServiceServer()
}
// UnimplementedHarnessServiceServer must be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedHarnessServiceServer struct{}
func (UnimplementedHarnessServiceServer) Open(grpc.BidiStreamingServer[OpenRequest, OpenResponse]) error {
return status.Error(codes.Unimplemented, "method Open not implemented")
}
func (UnimplementedHarnessServiceServer) mustEmbedUnimplementedHarnessServiceServer() {}
func (UnimplementedHarnessServiceServer) testEmbeddedByValue() {}
// UnsafeHarnessServiceServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to HarnessServiceServer will
// result in compilation errors.
type UnsafeHarnessServiceServer interface {
mustEmbedUnimplementedHarnessServiceServer()
}
func RegisterHarnessServiceServer(s grpc.ServiceRegistrar, srv HarnessServiceServer) {
// If the following call panics, it indicates UnimplementedHarnessServiceServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&HarnessService_ServiceDesc, srv)
}
func _HarnessService_Open_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(HarnessServiceServer).Open(&grpc.GenericServerStream[OpenRequest, OpenResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type HarnessService_OpenServer = grpc.BidiStreamingServer[OpenRequest, OpenResponse]
// HarnessService_ServiceDesc is the grpc.ServiceDesc for HarnessService service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var HarnessService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "harness.v1.HarnessService",
HandlerType: (*HarnessServiceServer)(nil),
Methods: []grpc.MethodDesc{},
Streams: []grpc.StreamDesc{
{
StreamName: "Open",
Handler: _HarnessService_Open_Handler,
ServerStreams: true,
ClientStreams: true,
},
},
Metadata: "harness/v1/harness.proto",
}
+209
View File
@@ -0,0 +1,209 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc (unknown)
// source: logging/v1/logging.proto
package loggingv1
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
unsafe "unsafe"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type LogQuery struct {
state protoimpl.MessageState `protogen:"open.v1"`
Limit int32 `protobuf:"varint,1,opt,name=limit,proto3" json:"limit,omitempty"`
Levels []string `protobuf:"bytes,2,rep,name=levels,proto3" json:"levels,omitempty"`
Component string `protobuf:"bytes,3,opt,name=component,proto3" json:"component,omitempty"`
Contains string `protobuf:"bytes,4,opt,name=contains,proto3" json:"contains,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *LogQuery) Reset() {
*x = LogQuery{}
mi := &file_logging_v1_logging_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *LogQuery) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*LogQuery) ProtoMessage() {}
func (x *LogQuery) ProtoReflect() protoreflect.Message {
mi := &file_logging_v1_logging_proto_msgTypes[0]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use LogQuery.ProtoReflect.Descriptor instead.
func (*LogQuery) Descriptor() ([]byte, []int) {
return file_logging_v1_logging_proto_rawDescGZIP(), []int{0}
}
func (x *LogQuery) GetLimit() int32 {
if x != nil {
return x.Limit
}
return 0
}
func (x *LogQuery) GetLevels() []string {
if x != nil {
return x.Levels
}
return nil
}
func (x *LogQuery) GetComponent() string {
if x != nil {
return x.Component
}
return ""
}
func (x *LogQuery) GetContains() string {
if x != nil {
return x.Contains
}
return ""
}
type LogSnapshot struct {
state protoimpl.MessageState `protogen:"open.v1"`
EntriesJson []byte `protobuf:"bytes,1,opt,name=entries_json,json=entriesJson,proto3" json:"entries_json,omitempty"`
Truncated bool `protobuf:"varint,2,opt,name=truncated,proto3" json:"truncated,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *LogSnapshot) Reset() {
*x = LogSnapshot{}
mi := &file_logging_v1_logging_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *LogSnapshot) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*LogSnapshot) ProtoMessage() {}
func (x *LogSnapshot) ProtoReflect() protoreflect.Message {
mi := &file_logging_v1_logging_proto_msgTypes[1]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use LogSnapshot.ProtoReflect.Descriptor instead.
func (*LogSnapshot) Descriptor() ([]byte, []int) {
return file_logging_v1_logging_proto_rawDescGZIP(), []int{1}
}
func (x *LogSnapshot) GetEntriesJson() []byte {
if x != nil {
return x.EntriesJson
}
return nil
}
func (x *LogSnapshot) GetTruncated() bool {
if x != nil {
return x.Truncated
}
return false
}
var File_logging_v1_logging_proto protoreflect.FileDescriptor
const file_logging_v1_logging_proto_rawDesc = "" +
"\n" +
"\x18logging/v1/logging.proto\x12\n" +
"logging.v1\"r\n" +
"\bLogQuery\x12\x14\n" +
"\x05limit\x18\x01 \x01(\x05R\x05limit\x12\x16\n" +
"\x06levels\x18\x02 \x03(\tR\x06levels\x12\x1c\n" +
"\tcomponent\x18\x03 \x01(\tR\tcomponent\x12\x1a\n" +
"\bcontains\x18\x04 \x01(\tR\bcontains\"N\n" +
"\vLogSnapshot\x12!\n" +
"\fentries_json\x18\x01 \x01(\fR\ventriesJson\x12\x1c\n" +
"\ttruncated\x18\x02 \x01(\bR\ttruncatedB\xb4\x01\n" +
"\x0ecom.logging.v1B\fLoggingProtoP\x01ZKgithub.com/samcharles93/archie-core/internal/contracts/logging/v1;loggingv1\xa2\x02\x03LXX\xaa\x02\n" +
"Logging.V1\xca\x02\n" +
"Logging\\V1\xe2\x02\x16Logging\\V1\\GPBMetadata\xea\x02\vLogging::V1b\x06proto3"
var (
file_logging_v1_logging_proto_rawDescOnce sync.Once
file_logging_v1_logging_proto_rawDescData []byte
)
func file_logging_v1_logging_proto_rawDescGZIP() []byte {
file_logging_v1_logging_proto_rawDescOnce.Do(func() {
file_logging_v1_logging_proto_rawDescData = protoimpl.X.CompressGZIP(unsafe.Slice(unsafe.StringData(file_logging_v1_logging_proto_rawDesc), len(file_logging_v1_logging_proto_rawDesc)))
})
return file_logging_v1_logging_proto_rawDescData
}
var file_logging_v1_logging_proto_msgTypes = make([]protoimpl.MessageInfo, 2)
var file_logging_v1_logging_proto_goTypes = []any{
(*LogQuery)(nil), // 0: logging.v1.LogQuery
(*LogSnapshot)(nil), // 1: logging.v1.LogSnapshot
}
var file_logging_v1_logging_proto_depIdxs = []int32{
0, // [0:0] is the sub-list for method output_type
0, // [0:0] is the sub-list for method input_type
0, // [0:0] is the sub-list for extension type_name
0, // [0:0] is the sub-list for extension extendee
0, // [0:0] is the sub-list for field type_name
}
func init() { file_logging_v1_logging_proto_init() }
func file_logging_v1_logging_proto_init() {
if File_logging_v1_logging_proto != nil {
return
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_logging_v1_logging_proto_rawDesc), len(file_logging_v1_logging_proto_rawDesc)),
NumEnums: 0,
NumMessages: 2,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_logging_v1_logging_proto_goTypes,
DependencyIndexes: file_logging_v1_logging_proto_depIdxs,
MessageInfos: file_logging_v1_logging_proto_msgTypes,
}.Build()
File_logging_v1_logging_proto = out.File
file_logging_v1_logging_proto_goTypes = nil
file_logging_v1_logging_proto_depIdxs = nil
}
+117 -14
View File
@@ -7,6 +7,7 @@
package messagingv1
import (
v1 "github.com/samcharles93/archie-core/internal/contracts/logging/v1"
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
@@ -119,17 +120,111 @@ func (*DeliverResponse) Descriptor() ([]byte, []int) {
return file_messaging_v1_messaging_proto_rawDescGZIP(), []int{1}
}
type RecentLogsRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
Query *v1.LogQuery `protobuf:"bytes,1,opt,name=query,proto3" json:"query,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RecentLogsRequest) Reset() {
*x = RecentLogsRequest{}
mi := &file_messaging_v1_messaging_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *RecentLogsRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*RecentLogsRequest) ProtoMessage() {}
func (x *RecentLogsRequest) ProtoReflect() protoreflect.Message {
mi := &file_messaging_v1_messaging_proto_msgTypes[2]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use RecentLogsRequest.ProtoReflect.Descriptor instead.
func (*RecentLogsRequest) Descriptor() ([]byte, []int) {
return file_messaging_v1_messaging_proto_rawDescGZIP(), []int{2}
}
func (x *RecentLogsRequest) GetQuery() *v1.LogQuery {
if x != nil {
return x.Query
}
return nil
}
type RecentLogsResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
Snapshot *v1.LogSnapshot `protobuf:"bytes,1,opt,name=snapshot,proto3" json:"snapshot,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RecentLogsResponse) Reset() {
*x = RecentLogsResponse{}
mi := &file_messaging_v1_messaging_proto_msgTypes[3]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *RecentLogsResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*RecentLogsResponse) ProtoMessage() {}
func (x *RecentLogsResponse) ProtoReflect() protoreflect.Message {
mi := &file_messaging_v1_messaging_proto_msgTypes[3]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use RecentLogsResponse.ProtoReflect.Descriptor instead.
func (*RecentLogsResponse) Descriptor() ([]byte, []int) {
return file_messaging_v1_messaging_proto_rawDescGZIP(), []int{3}
}
func (x *RecentLogsResponse) GetSnapshot() *v1.LogSnapshot {
if x != nil {
return x.Snapshot
}
return nil
}
var File_messaging_v1_messaging_proto protoreflect.FileDescriptor
const file_messaging_v1_messaging_proto_rawDesc = "" +
"\n" +
"\x1cmessaging/v1/messaging.proto\x12\fmessaging.v1\"W\n" +
"\x1cmessaging/v1/messaging.proto\x12\fmessaging.v1\x1a\x18logging/v1/logging.proto\"W\n" +
"\x0eDeliverRequest\x12\x18\n" +
"\achannel\x18\x01 \x01(\tR\achannel\x12\x17\n" +
"\achat_id\x18\x02 \x01(\tR\x06chatId\x12\x12\n" +
"\x04text\x18\x03 \x01(\tR\x04text\"\x11\n" +
"\x0fDeliverResponse2Z\n" +
"\x10MessagingService\x12F\n" +
"\x0fDeliverResponse\"?\n" +
"\x11RecentLogsRequest\x12*\n" +
"\x05query\x18\x01 \x01(\v2\x14.logging.v1.LogQueryR\x05query\"I\n" +
"\x12RecentLogsResponse\x123\n" +
"\bsnapshot\x18\x01 \x01(\v2\x17.logging.v1.LogSnapshotR\bsnapshot2\xab\x01\n" +
"\x10MessagingService\x12O\n" +
"\n" +
"RecentLogs\x12\x1f.messaging.v1.RecentLogsRequest\x1a .messaging.v1.RecentLogsResponse\x12F\n" +
"\aDeliver\x12\x1c.messaging.v1.DeliverRequest\x1a\x1d.messaging.v1.DeliverResponseB\xc4\x01\n" +
"\x10com.messaging.v1B\x0eMessagingProtoP\x01ZOgithub.com/samcharles93/archie-core/internal/contracts/messaging/v1;messagingv1\xa2\x02\x03MXX\xaa\x02\fMessaging.V1\xca\x02\fMessaging\\V1\xe2\x02\x18Messaging\\V1\\GPBMetadata\xea\x02\rMessaging::V1b\x06proto3"
@@ -145,19 +240,27 @@ func file_messaging_v1_messaging_proto_rawDescGZIP() []byte {
return file_messaging_v1_messaging_proto_rawDescData
}
var file_messaging_v1_messaging_proto_msgTypes = make([]protoimpl.MessageInfo, 2)
var file_messaging_v1_messaging_proto_msgTypes = make([]protoimpl.MessageInfo, 4)
var file_messaging_v1_messaging_proto_goTypes = []any{
(*DeliverRequest)(nil), // 0: messaging.v1.DeliverRequest
(*DeliverResponse)(nil), // 1: messaging.v1.DeliverResponse
(*DeliverRequest)(nil), // 0: messaging.v1.DeliverRequest
(*DeliverResponse)(nil), // 1: messaging.v1.DeliverResponse
(*RecentLogsRequest)(nil), // 2: messaging.v1.RecentLogsRequest
(*RecentLogsResponse)(nil), // 3: messaging.v1.RecentLogsResponse
(*v1.LogQuery)(nil), // 4: logging.v1.LogQuery
(*v1.LogSnapshot)(nil), // 5: logging.v1.LogSnapshot
}
var file_messaging_v1_messaging_proto_depIdxs = []int32{
0, // 0: messaging.v1.MessagingService.Deliver:input_type -> messaging.v1.DeliverRequest
1, // 1: messaging.v1.MessagingService.Deliver:output_type -> messaging.v1.DeliverResponse
1, // [1:2] is the sub-list for method output_type
0, // [0:1] is the sub-list for method input_type
0, // [0:0] is the sub-list for extension type_name
0, // [0:0] is the sub-list for extension extendee
0, // [0:0] is the sub-list for field type_name
4, // 0: messaging.v1.RecentLogsRequest.query:type_name -> logging.v1.LogQuery
5, // 1: messaging.v1.RecentLogsResponse.snapshot:type_name -> logging.v1.LogSnapshot
2, // 2: messaging.v1.MessagingService.RecentLogs:input_type -> messaging.v1.RecentLogsRequest
0, // 3: messaging.v1.MessagingService.Deliver:input_type -> messaging.v1.DeliverRequest
3, // 4: messaging.v1.MessagingService.RecentLogs:output_type -> messaging.v1.RecentLogsResponse
1, // 5: messaging.v1.MessagingService.Deliver:output_type -> messaging.v1.DeliverResponse
4, // [4:6] is the sub-list for method output_type
2, // [2:4] is the sub-list for method input_type
2, // [2:2] is the sub-list for extension type_name
2, // [2:2] is the sub-list for extension extendee
0, // [0:2] is the sub-list for field type_name
}
func init() { file_messaging_v1_messaging_proto_init() }
@@ -171,7 +274,7 @@ func file_messaging_v1_messaging_proto_init() {
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_messaging_v1_messaging_proto_rawDesc), len(file_messaging_v1_messaging_proto_rawDesc)),
NumEnums: 0,
NumMessages: 2,
NumMessages: 4,
NumExtensions: 0,
NumServices: 1,
},
@@ -19,7 +19,8 @@ import (
const _ = grpc.SupportPackageIsVersion9
const (
MessagingService_Deliver_FullMethodName = "/messaging.v1.MessagingService/Deliver"
MessagingService_RecentLogs_FullMethodName = "/messaging.v1.MessagingService/RecentLogs"
MessagingService_Deliver_FullMethodName = "/messaging.v1.MessagingService/Deliver"
)
// MessagingServiceClient is the client API for MessagingService service.
@@ -30,6 +31,7 @@ const (
// the chat channel transports. Other services send outbound messages through
// it rather than holding channel credentials themselves.
type MessagingServiceClient interface {
RecentLogs(ctx context.Context, in *RecentLogsRequest, opts ...grpc.CallOption) (*RecentLogsResponse, error)
// Deliver sends text to one chat on one channel. FAILED_PRECONDITION means
// the channel is not running or cannot send; the caller may retry later.
Deliver(ctx context.Context, in *DeliverRequest, opts ...grpc.CallOption) (*DeliverResponse, error)
@@ -43,6 +45,16 @@ func NewMessagingServiceClient(cc grpc.ClientConnInterface) MessagingServiceClie
return &messagingServiceClient{cc}
}
func (c *messagingServiceClient) RecentLogs(ctx context.Context, in *RecentLogsRequest, opts ...grpc.CallOption) (*RecentLogsResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(RecentLogsResponse)
err := c.cc.Invoke(ctx, MessagingService_RecentLogs_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *messagingServiceClient) Deliver(ctx context.Context, in *DeliverRequest, opts ...grpc.CallOption) (*DeliverResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(DeliverResponse)
@@ -61,6 +73,7 @@ func (c *messagingServiceClient) Deliver(ctx context.Context, in *DeliverRequest
// the chat channel transports. Other services send outbound messages through
// it rather than holding channel credentials themselves.
type MessagingServiceServer interface {
RecentLogs(context.Context, *RecentLogsRequest) (*RecentLogsResponse, error)
// Deliver sends text to one chat on one channel. FAILED_PRECONDITION means
// the channel is not running or cannot send; the caller may retry later.
Deliver(context.Context, *DeliverRequest) (*DeliverResponse, error)
@@ -74,6 +87,9 @@ type MessagingServiceServer interface {
// pointer dereference when methods are called.
type UnimplementedMessagingServiceServer struct{}
func (UnimplementedMessagingServiceServer) RecentLogs(context.Context, *RecentLogsRequest) (*RecentLogsResponse, error) {
return nil, status.Error(codes.Unimplemented, "method RecentLogs not implemented")
}
func (UnimplementedMessagingServiceServer) Deliver(context.Context, *DeliverRequest) (*DeliverResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Deliver not implemented")
}
@@ -98,6 +114,24 @@ func RegisterMessagingServiceServer(s grpc.ServiceRegistrar, srv MessagingServic
s.RegisterService(&MessagingService_ServiceDesc, srv)
}
func _MessagingService_RecentLogs_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(RecentLogsRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(MessagingServiceServer).RecentLogs(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: MessagingService_RecentLogs_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(MessagingServiceServer).RecentLogs(ctx, req.(*RecentLogsRequest))
}
return interceptor(ctx, in, info, handler)
}
func _MessagingService_Deliver_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DeliverRequest)
if err := dec(in); err != nil {
@@ -123,6 +157,10 @@ var MessagingService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "messaging.v1.MessagingService",
HandlerType: (*MessagingServiceServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "RecentLogs",
Handler: _MessagingService_RecentLogs_Handler,
},
{
MethodName: "Deliver",
Handler: _MessagingService_Deliver_Handler,
File diff suppressed because it is too large. Load diff
@@ -19,6 +19,7 @@ import (
const _ = grpc.SupportPackageIsVersion9
const (
StateStoreService_RecentLogs_FullMethodName = "/state.v1.StateStoreService/RecentLogs"
StateStoreService_InstallPackage_FullMethodName = "/state.v1.StateStoreService/InstallPackage"
StateStoreService_ListCatalogue_FullMethodName = "/state.v1.StateStoreService/ListCatalogue"
StateStoreService_InstallFromCatalogue_FullMethodName = "/state.v1.StateStoreService/InstallFromCatalogue"
@@ -156,6 +157,7 @@ const (
// service over internal/infrastructure/postgres; the proto bypasses the Go
// interfacebloat cap (max 8), like gateway.v1.ChatService does.
type StateStoreServiceClient interface {
RecentLogs(ctx context.Context, in *RecentLogsRequest, opts ...grpc.CallOption) (*RecentLogsResponse, error)
// Installed Archie packages. Administrative until org-scoped principals land.
InstallPackage(ctx context.Context, in *InstallPackageRequest, opts ...grpc.CallOption) (*InstallPackageResponse, error)
// ListCatalogue returns the packages the instance's verified catalogue
@@ -386,6 +388,16 @@ func NewStateStoreServiceClient(cc grpc.ClientConnInterface) StateStoreServiceCl
return &stateStoreServiceClient{cc}
}
func (c *stateStoreServiceClient) RecentLogs(ctx context.Context, in *RecentLogsRequest, opts ...grpc.CallOption) (*RecentLogsResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(RecentLogsResponse)
err := c.cc.Invoke(ctx, StateStoreService_RecentLogs_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *stateStoreServiceClient) InstallPackage(ctx context.Context, in *InstallPackageRequest, opts ...grpc.CallOption) (*InstallPackageResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(InstallPackageResponse)
@@ -1655,6 +1667,7 @@ func (c *stateStoreServiceClient) AssignAgent(ctx context.Context, in *AssignAge
// service over internal/infrastructure/postgres; the proto bypasses the Go
// interfacebloat cap (max 8), like gateway.v1.ChatService does.
type StateStoreServiceServer interface {
RecentLogs(context.Context, *RecentLogsRequest) (*RecentLogsResponse, error)
// Installed Archie packages. Administrative until org-scoped principals land.
InstallPackage(context.Context, *InstallPackageRequest) (*InstallPackageResponse, error)
// ListCatalogue returns the packages the instance's verified catalogue
@@ -1885,6 +1898,9 @@ type StateStoreServiceServer interface {
// pointer dereference when methods are called.
type UnimplementedStateStoreServiceServer struct{}
func (UnimplementedStateStoreServiceServer) RecentLogs(context.Context, *RecentLogsRequest) (*RecentLogsResponse, error) {
return nil, status.Error(codes.Unimplemented, "method RecentLogs not implemented")
}
func (UnimplementedStateStoreServiceServer) InstallPackage(context.Context, *InstallPackageRequest) (*InstallPackageResponse, error) {
return nil, status.Error(codes.Unimplemented, "method InstallPackage not implemented")
}
@@ -2275,6 +2291,24 @@ func RegisterStateStoreServiceServer(s grpc.ServiceRegistrar, srv StateStoreServ
s.RegisterService(&StateStoreService_ServiceDesc, srv)
}
func _StateStoreService_RecentLogs_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(RecentLogsRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(StateStoreServiceServer).RecentLogs(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: StateStoreService_RecentLogs_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(StateStoreServiceServer).RecentLogs(ctx, req.(*RecentLogsRequest))
}
return interceptor(ctx, in, info, handler)
}
func _StateStoreService_InstallPackage_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(InstallPackageRequest)
if err := dec(in); err != nil {
@@ -4475,6 +4509,10 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "state.v1.StateStoreService",
HandlerType: (*StateStoreServiceServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "RecentLogs",
Handler: _StateStoreService_RecentLogs_Handler,
},
{
MethodName: "InstallPackage",
Handler: _StateStoreService_InstallPackage_Handler,
+53
View File
@@ -0,0 +1,53 @@
package daemon
import (
"context"
"github.com/samcharles93/archie-core/internal/events"
)
// callWaitBuffer is the watcher's subscription buffer. The bus carries the
// whole agent-event stream and drops for a subscriber whose buffer is full, so
// this has to absorb a burst rather than overflow on one.
const callWaitBuffer = 1024
// watchCallWaits gives a run's execution capacity back while it waits on a
// callee, and takes it again when the wait ends. The events arrive on the
// daemon's own bus, republished from the container running the workflow, and
// carry the caller's task id.
func (d *Daemon) watchCallWaits(ctx context.Context) {
if d.Bus == nil {
return
}
sub := d.Bus.Subscribe(callWaitBuffer)
dispatcher := d.taskDispatcher()
go func() {
defer sub.Close()
for {
select {
case <-ctx.Done():
return
case e, ok := <-sub.C:
if !ok {
return
}
applyCallWait(dispatcher, e)
}
}
}()
}
// applyCallWait suspends the run a call start names and resumes it when the
// call finishes. It stays non-blocking: a full subscriber buffer drops events,
// and a dropped start leaves a waiting run holding the capacity its callee
// needs, so this must never wait on anything.
func applyCallWait(dispatcher *taskDispatcher, e events.Event) {
switch e.Kind {
case events.KindWorkflowCallStarted:
if waiting, _ := e.Data["wait"].(bool); waiting {
dispatcher.Suspend(e.TaskID)
}
case events.KindWorkflowCallFinished:
dispatcher.Resume(e.TaskID)
}
}
+284 -74
View File
@@ -170,7 +170,7 @@ type Daemon struct {
// dispatcher bounds task execution globally across poll passes. Built once,
// on first use, so ResizeTaskDispatcher can apply a live
// containers.max_concurrency change without a restart; drainNATS submits
// each pass through it and waits for that pass.
// each pass through it and leaves its tasks running.
dispatcherOnce sync.Once
dispatcher *taskDispatcher
// Storage is the pluggable storage backend for container mounts.
@@ -241,8 +241,7 @@ type IdentityRunner struct {
Forge forge.Forge
Trees *worktree.Manager
Repos []config.Repo
// Cfg is the identity-scoped config subset (forge, models, dispatch,
// budgets, etc.).
// Cfg holds the identity's forge account and repository membership.
Cfg config.IdentityConfig
Log *slog.Logger
}
@@ -345,6 +344,10 @@ func (d *Daemon) Run(ctx context.Context) error {
if len(d.Identities) > 0 {
return d.runIdentities(ctx)
}
// A pass no longer waits for the tasks it submitted, so the run does, once
// the ctx is cancelled.
defer d.WaitForTasks()
d.watchCallWaits(ctx)
interval := d.Cfg.Get().PollInterval.Std()
ticker := time.NewTicker(interval)
defer ticker.Stop()
@@ -370,6 +373,7 @@ func (d *Daemon) Run(ctx context.Context) error {
func (d *Daemon) runIdentities(ctx context.Context) error {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
d.watchCallWaits(ctx)
var wg sync.WaitGroup
@@ -380,9 +384,6 @@ func (d *Daemon) runIdentities(ctx context.Context) error {
go func(id *IdentityRunner) {
defer wg.Done()
interval := d.Cfg.Get().PollInterval.Std()
if id.Cfg.PollInterval > 0 {
interval = id.Cfg.PollInterval.Std()
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
@@ -391,12 +392,8 @@ func (d *Daemon) runIdentities(ctx context.Context) error {
case <-ctx.Done():
return
case <-ticker.C:
// The identity's own override is frozen (identity config is
// startup-built); the root fallback may have been reloaded.
// Pick up the current shared polling interval.
iv := d.Cfg.Get().PollInterval.Std()
if id.Cfg.PollInterval > 0 {
iv = id.Cfg.PollInterval.Std()
}
if iv != interval {
interval = iv
ticker.Reset(iv)
@@ -428,6 +425,7 @@ func (d *Daemon) runIdentities(ctx context.Context) error {
})
wg.Wait()
d.WaitForTasks()
return ctx.Err()
}
@@ -441,7 +439,7 @@ func (d *Daemon) pollForIdentity(ctx context.Context, id *IdentityRunner) {
}
d.markPoll()
cfg := configForIdentity(d.Cfg.Get(), id.Cfg)
for _, repo := range id.Repos {
for _, repo := range identityRepositories(cfg.Repos, id.Repos) {
issues, complete := d.pollIssuesWithConfig(ctx, id.Forge, cfg, repo)
if complete {
d.withdrawUnpolled(ctx, repo, string(id.ID), dispatchRule(cfg), issues)
@@ -473,8 +471,8 @@ func (d *Daemon) maintainAndDrain(ctx context.Context) {
// pollIssuesWithConfig returns the repo's issues the dispatch rule makes
// work, using the given forge client and dispatch config: single-identity
// mode passes d.Forge/d.Cfg, identity poll loops their own forge and
// configForIdentity, so each identity polls with its own bot user, label and
// trigger. complete is false when any part of the poll failed or was skipped, so
// configForIdentity, so each identity uses its own bot user with the shared
// dispatch settings. complete is false when any part of the poll failed or was skipped, so
// the result must not be read as the whole eligible set.
func (d *Daemon) pollIssuesWithConfig(ctx context.Context, fg forge.Forge, cfg config.Config, repo config.Repo) (issues []forge.Issue, complete bool) {
// Webhook intake hears about issues as they change; polling as well would
@@ -571,7 +569,32 @@ func (d *Daemon) dispatchBindings(ctx context.Context) {
if d.Bindings == nil || d.BindingDispatcher == nil || d.BindingTaskCreator == nil {
return
}
if d.Principals == nil {
d.dispatchOrgBindings(ctx)
return
}
identities := []identity.IdentityID{d.RootIdentityID}
for _, runner := range d.Identities {
if runner != nil {
identities = append(identities, runner.ID)
}
}
seen := make(map[org.OrgID]bool)
for _, id := range identities {
principal, err := d.Principals.PrincipalFor(ctx, id)
if err != nil {
d.Log.Warn("binding dispatch: principal unavailable", "identity", id, "err", err)
continue
}
if seen[principal.Org] {
continue
}
seen[principal.Org] = true
d.dispatchOrgBindings(org.WithOrg(access.WithPrincipal(ctx, principal), principal.Org))
}
}
func (d *Daemon) dispatchOrgBindings(ctx context.Context) {
// Gather armed sources first so the captures query filters down to
// only relevant rows.
bindings, err := d.Bindings.ListBindings(ctx)
@@ -581,7 +604,7 @@ func (d *Daemon) dispatchBindings(ctx context.Context) {
}
sources := make([]string, 0, len(bindings))
for _, b := range bindings {
if b.Status == binding.StatusArmed && b.Matcher.Source != "" {
if b.OrgID == org.OrgFromContext(ctx) && b.Status == binding.StatusArmed && b.Matcher.Source != "" {
sources = append(sources, b.Matcher.Source)
}
}
@@ -619,7 +642,7 @@ func (d *Daemon) dispatchCapture(ctx context.Context, c storecontract.CapturedEv
return
}
for _, b := range armed {
if b.Matcher.Matches(c.Source, c.Dispatchable()) && enablement.Enabled(b.OrgID, b.Workflow) {
if b.OrgID == org.OrgFromContext(ctx) && b.Matcher.Matches(c.Source, c.Dispatchable()) && enablement.Enabled(b.OrgID, b.Workflow) {
d.dispatchOneBinding(ctx, b, c, workflows)
}
}
@@ -703,7 +726,11 @@ func (d *Daemon) claimAndEnqueue(ctx context.Context, b binding.Binding, c store
if target.declared {
body = fmt.Sprintf("Started by binding %q from capture %q; the workflow's inputs travel with the task.", b.Name, c.ID)
}
task, err := d.BindingTaskCreator.EnqueueBindingTask(ctx, target.owner, target.repo, title, body, b.Workflow, "", b.ID, b.Version, target.inputs)
taskIdentity := ""
if principal, ok := access.PrincipalFromContext(ctx); ok && (principal.IdentityID != d.RootIdentityID || principal.Org != org.DefaultOrgID) {
taskIdentity = string(principal.IdentityID)
}
task, err := d.BindingTaskCreator.EnqueueBindingTask(ctx, target.owner, target.repo, title, body, b.Workflow, taskIdentity, b.ID, b.Version, target.inputs)
if err != nil {
d.Log.Warn("binding dispatch: enqueue", "binding", b.ID, "capture", c.ID, "error", err)
return
@@ -720,7 +747,11 @@ func (d *Daemon) authorizeDispatch(ctx context.Context, b binding.Binding, c sto
if d.Access == nil || d.Principals == nil {
return true
}
principal, err := d.Principals.PrincipalFor(ctx, d.RootIdentityID)
principal, ok := access.PrincipalFromContext(ctx)
var err error
if !ok {
principal, err = d.Principals.PrincipalFor(ctx, d.RootIdentityID)
}
if err != nil {
d.Log.Warn("binding dispatch: principal unavailable; dispatch parked for the next cycle",
"binding", b.ID, "capture", c.ID, "error", err)
@@ -890,7 +921,10 @@ func (d *Daemon) botUserForTask(task *workflowtask.Task) string {
// drainNATS processes tasks from NATS, falling back to the task store's ClaimNext for
// requeued tasks (waiting_human approval, retry-parked) that didn't come
// through a NATS publish.
// through a NATS publish. It returns once the stream and the claim fallback
// are both empty: in-flight tasks keep running, so a task waiting on a callee
// does not hold up the pass that would claim it. Shutdown waits for them
// through WaitForTasks.
func (d *Daemon) drainNATS(ctx context.Context) {
dispatcher := d.taskDispatcher()
for ctx.Err() == nil {
@@ -918,13 +952,11 @@ func (d *Daemon) drainNATS(ctx context.Context) {
}
dispatcher.Submit(ctx, task, d.process)
}
dispatcher.Wait()
}
// taskDispatcher returns the daemon's long-lived global dispatcher, building
// it from the running config on first use. It outlives a single drain pass so
// ResizeTaskDispatcher can resize the running dispatcher; drainNATS still
// waits for each pass's work before returning.
// ResizeTaskDispatcher can resize the running dispatcher.
func (d *Daemon) taskDispatcher() *taskDispatcher {
d.dispatcherOnce.Do(func() {
d.dispatcher = newTaskDispatcher(d.Cfg.Get().Containers.MaxConcurrency, d.allowConcurrentForTask)
@@ -932,6 +964,13 @@ func (d *Daemon) taskDispatcher() *taskDispatcher {
return d.dispatcher
}
// WaitForTasks blocks until every submitted task's process has returned. A
// drain pass no longer waits for its own tasks, so shutdown is where in-flight
// work is waited for.
func (d *Daemon) WaitForTasks() {
d.taskDispatcher().Wait()
}
// ResizeTaskDispatcher applies a new containers.max_concurrency to the
// running global task dispatcher without a restart. Running tasks keep their
// slots; a lowered limit is honoured as they finish, and a raised limit
@@ -951,8 +990,8 @@ func (d *Daemon) submitNATSTask(ctx context.Context, dispatcher *taskDispatcher,
return
}
task := &workflow.Task{Owner: envelope.Owner, Repo: envelope.Repo, Identity: envelope.Identity}
dispatcher.Submit(ctx, task, func(ctx context.Context, _ *workflow.Task) {
d.processNATSTask(ctx, msg)
dispatcher.Submit(ctx, task, func(ctx context.Context, _ *workflow.Task, hold *taskHold) {
d.processNATSTask(ctx, msg, hold)
})
}
@@ -967,6 +1006,47 @@ type taskDispatcher struct {
slotsFree *sync.Cond
repoTail map[string]chan struct{}
pending int
// holds maps a task id to the capacity its run holds, so a run waiting on
// a callee can give that capacity back and take it again by task id.
holds map[int64]*taskHold
}
// taskHold is one submitted task's claim on execution capacity: its slot under
// the global limit, and its place at the tail of its repo's chain, which the
// next task for that repo is blocked on. A run that waits on a callee gives
// both back so the callee can run, and takes them again when the wait ends.
type taskHold struct {
dispatcher *taskDispatcher
repo string
// chain is true when the repo runs one task at a time, so the run holds a
// place in that repo's chain.
chain bool
// baton is the run's place in its repo's chain, closed when it gives that
// place up; nil while it holds none.
baton chan struct{}
// slot is true while the run counts against the global limit.
slot bool
// waits counts the callees the run is waiting on: the first Suspend gives
// capacity back and the last Resume takes it again.
waits int
// finished is true once the run's process returned.
finished bool
// id is the task this hold belongs to, named by the run once it knows the
// task's id.
id int64
}
// name records which task's capacity this hold is, so a wait on a callee can
// give it back by task id. Zero names nothing: a task arriving on the NATS
// stream has no id until the store claims its row inside the run.
func (h *taskHold) name(id int64) {
if id == 0 {
return
}
h.dispatcher.mu.Lock()
h.id = id
h.dispatcher.holds[id] = h
h.dispatcher.mu.Unlock()
}
// SetMaxConcurrency accepts a new global limit without a restart. Running
@@ -987,42 +1067,176 @@ func newTaskDispatcher(maxConcurrency int, allowConcurrent func(task *workflow.T
limit: maxConcurrency,
allowConcurrent: allowConcurrent,
repoTail: make(map[string]chan struct{}),
holds: make(map[int64]*taskHold),
}
d.slotsFree = sync.NewCond(&d.mu)
return d
}
// acquireSlot blocks until the global limit has room and returns the release
// function. limit <= 0 means unlimited. The limit is re-read on every wake, so
// joinRepoLocked appends baton to the tail of repo's chain and returns the
// baton ahead of it, which the joining task is blocked on. Callers hold d.mu.
func (d *taskDispatcher) joinRepoLocked(repo string, baton chan struct{}) chan struct{} {
previous := d.repoTail[repo]
d.repoTail[repo] = baton
return previous
}
// takeSlotLocked blocks until the global limit has room, then counts the run
// in. limit <= 0 means unlimited. The limit is re-read on every wake, so
// SetMaxConcurrency reaches waiting tasks without recreating the dispatcher.
func (d *taskDispatcher) acquireSlot() func() {
d.mu.Lock()
defer d.mu.Unlock()
// Callers hold d.mu.
func (d *taskDispatcher) takeSlotLocked() {
for d.limit > 0 && d.active >= d.limit {
d.slotsFree.Wait()
}
d.active++
return func() {
d.mu.Lock()
}
// giveBackLocked gives back everything the run holds: its slot under the
// global limit, and its place in its repo's chain, which releases the task
// queued behind it. Callers hold d.mu.
func (d *taskDispatcher) giveBackLocked(hold *taskHold) {
if hold.slot {
d.active--
hold.slot = false
}
if hold.baton == nil {
return
}
close(hold.baton)
if d.repoTail[hold.repo] == hold.baton {
delete(d.repoTail, hold.repo)
}
hold.baton = nil
}
// finishRun retires a returned run: it holds no capacity any more, and no
// Resume may take any for it. Whatever an in-flight resume takes later is
// given back by reacquire, which sees finished.
func (d *taskDispatcher) finishRun(hold *taskHold) {
d.mu.Lock()
hold.finished = true
if hold.id != 0 && d.holds[hold.id] == hold {
delete(d.holds, hold.id)
}
d.giveBackLocked(hold)
d.pending--
d.mu.Unlock()
d.slotsFree.Broadcast()
}
// Suspend gives a run's capacity back while it waits on a callee, so the
// callee can start. Calls are refcounted per task: a run waiting on callees
// from parallel branches releases once. An unknown task, or one whose run has
// already returned, is a no-op.
func (d *taskDispatcher) Suspend(id int64) {
d.mu.Lock()
hold := d.holds[id]
if hold == nil || hold.finished {
d.mu.Unlock()
return
}
hold.waits++
if hold.waits == 1 {
d.giveBackLocked(hold)
}
d.mu.Unlock()
d.slotsFree.Broadcast()
}
// Resume takes a run's capacity back when one of its waits ends. Only the last
// wait does: a run waiting on callees from parallel branches keeps nothing
// until all of them are done. An unknown task, one whose run has returned, and
// one that never suspended are all no-ops.
func (d *taskDispatcher) Resume(id int64) {
d.mu.Lock()
hold := d.holds[id]
if hold == nil || hold.finished || hold.waits == 0 {
d.mu.Unlock()
return
}
hold.waits--
if hold.waits > 0 {
d.mu.Unlock()
return
}
d.mu.Unlock()
// The two waits below can block, and neither may hold up the bus
// subscriber calling this or the resumed run, which is already executing
// inside its container: the execution cap can be exceeded by at most the
// number of simultaneous resumes, and re-converges as tasks finish.
go d.reacquire(hold)
}
// reacquire returns a resumed run's capacity: its place in its repo's chain
// and a slot under the global limit. The run keeps executing meanwhile -- a
// wait on a callee is not a pause. A hold records one slot and one place, so a
// re-acquire queued while another is still waiting gives back whichever of the
// two it finds already recorded instead of taking a second.
func (d *taskDispatcher) reacquire(hold *taskHold) {
d.mu.Lock()
if hold.finished || hold.waits > 0 {
// The run returned, or started waiting again, after the Resume that
// queued this: it needs no capacity.
d.mu.Unlock()
return
}
var previous chan struct{}
// A place another re-acquire already queued is the run's one place; joining
// behind it would queue a second, which the task behind it would wait on
// forever.
if hold.chain && hold.baton == nil {
hold.baton = make(chan struct{})
previous = d.joinRepoLocked(hold.repo, hold.baton)
}
d.mu.Unlock()
if previous != nil {
<-previous
}
d.mu.Lock()
d.takeSlotLocked()
if hold.slot {
// Another re-acquire took the run's one slot while this waited: give
// back the slot this one just took. The place is the other's, so it is
// left alone.
d.active--
d.mu.Unlock()
d.slotsFree.Broadcast()
return
}
hold.slot = true
if hold.finished || hold.waits > 0 {
// The run ended, or started waiting again, while this was queued for
// capacity: give back what it took, so no slot or place in the repo
// chain leaks.
d.giveBackLocked(hold)
d.mu.Unlock()
d.slotsFree.Broadcast()
return
}
d.mu.Unlock()
}
// Submit queues a task's process behind the global limit and, unless the repo
// allows concurrency, behind the task already holding that repo. process is
// handed the run's capacity so it can name it once it knows the task's id and
// give it back while it waits on a callee.
func (d *taskDispatcher) Submit(
ctx context.Context,
task *workflow.Task,
process func(context.Context, *workflow.Task),
process func(context.Context, *workflow.Task, *taskHold),
) {
repo := task.Owner + "/" + task.Repo
hold := &taskHold{dispatcher: d, repo: repo}
var previous, done chan struct{}
var previous chan struct{}
if !d.allowConcurrent(task) {
done = make(chan struct{})
hold.chain = true
hold.baton = make(chan struct{})
d.mu.Lock()
previous = d.repoTail[repo]
d.repoTail[repo] = done
previous = d.joinRepoLocked(repo, hold.baton)
d.mu.Unlock()
}
@@ -1030,28 +1244,15 @@ func (d *taskDispatcher) Submit(
d.pending++
d.mu.Unlock()
go func() {
defer func() {
d.mu.Lock()
d.pending--
d.mu.Unlock()
d.slotsFree.Broadcast()
}()
defer d.finishRun(hold)
if previous != nil {
<-previous
}
releaseSlot := d.acquireSlot()
defer releaseSlot()
if done != nil {
defer func() {
close(done)
d.mu.Lock()
if d.repoTail[repo] == done {
delete(d.repoTail, repo)
}
d.mu.Unlock()
}()
}
process(ctx, task)
d.mu.Lock()
d.takeSlotLocked()
hold.slot = true
d.mu.Unlock()
process(ctx, task, hold)
}()
}
@@ -1265,7 +1466,7 @@ func (d *Daemon) acknowledge(ctx context.Context, fg forge.Forge, cfg config.Con
// processNATSTask decodes a NATS message, writes it to SQLite, claims it,
// and runs the workflow. The message is ack'd on terminal (park is a valid
// outcome). Nak on transient errors so NATS redelivers.
func (d *Daemon) processNATSTask(ctx context.Context, msg eventbus.Message) {
func (d *Daemon) processNATSTask(ctx context.Context, msg eventbus.Message, hold *taskHold) {
tm, err := workintake.DecodeTask(msg.Data())
if err != nil {
d.Log.Error("nats decode failed", "err", err)
@@ -1316,7 +1517,7 @@ func (d *Daemon) processNATSTask(ctx context.Context, msg eventbus.Message) {
return
}
d.process(ctx, task)
d.process(ctx, task, hold)
if err := msg.Ack(); err != nil {
d.Log.Warn("ack failed", "err", err)
}
@@ -1389,7 +1590,10 @@ func (d *Daemon) reconcilePRs(ctx context.Context) {
}
}
func (d *Daemon) process(ctx context.Context, task *workflow.Task) {
func (d *Daemon) process(ctx context.Context, task *workflow.Task, hold *taskHold) {
// A task off the NATS stream only has an id once the store has claimed its
// row, inside this call, so its capacity is named here.
hold.name(task.ID)
// The workflow may name the identity the run acts as, which everything
// below is chosen by, so it is settled before anything else.
if !d.adoptWorkflowIdentity(ctx, task) {
@@ -1858,6 +2062,10 @@ func validateProfileMeetsNeeds(profile config.AgentProfile, needs workflowtask.W
// passes through here, whatever started it, so this is the one enablement
// check for issue labels, handoffs, approvals, calls and playbooks alike.
func (d *Daemon) pinWorkflowDefinition(ctx context.Context, task *workflow.Task) error {
ctx, err := d.workflowContext(ctx, task)
if err != nil {
return err
}
if err := d.pinDefinition(ctx, task); err != nil {
return err
}
@@ -2149,22 +2357,9 @@ func (d *Daemon) captureAttemptConfig(ctx context.Context, task *workflow.Task,
}
func configForIdentity(root config.Config, identity config.IdentityConfig) config.Config {
root.BotUser = identity.BotUser
root.BotEmail = identity.BotEmail
// Only an identity that set the cap overrides the shared one. Assigning
// unconditionally made an identity that omits the key inherit a nil cap,
// which reads as "no cap" and silently disabled the safety rail.
if identity.DiffCapLines != nil {
root.DiffCapLines = identity.DiffCapLines
}
root.BotUser, root.BotEmail = identity.BotUser, identity.BotEmail
root.Org, root.GrantedCredentials = root.CredentialAccess(identity.Name)
root.Forge = identity.Forge
root.Dispatch = identity.Dispatch
root.Models = identity.Models
root.Providers = identity.Providers
root.Budgets = identity.Budgets
root.Notify = identity.Notify
root.Repos = identity.Repos
return root
}
@@ -2188,7 +2383,7 @@ func (d *Daemon) identityFor(task *workflow.Task) *IdentityRunner {
// class if not. Empty is the root identity. An unknown or inactive identity
// never falls back to the root forge.
func (d *Daemon) identityMayAct(ctx context.Context, name string) (bool, string, taskstate.ParkClass) {
if name == "" {
if name == "" || name == string(d.RootIdentityID) {
return true, "", ""
}
runner := d.identityFor(&workflow.Task{Identity: name})
@@ -2253,7 +2448,7 @@ func (d *Daemon) treesFor(task *workflow.Task) *worktree.Manager {
func (d *Daemon) repoFor(t *workflow.Task) (config.Repo, bool) {
repos := d.Cfg.Get().Repos
if id := d.identityFor(t); id != nil {
repos = id.Repos
repos = identityRepositories(repos, id.Repos)
}
for _, r := range repos {
if r.Owner == t.Owner && r.Name == t.Repo {
@@ -2339,3 +2534,18 @@ func (d *Daemon) InFlight() int {
defer dispatcher.mu.Unlock()
return dispatcher.pending
}
// identityRepositories keeps account membership while taking live settings
// for repositories present in the shared control-plane configuration.
func identityRepositories(shared, owned []config.Repo) []config.Repo {
repos := append([]config.Repo(nil), owned...)
for i, repo := range repos {
for _, current := range shared {
if current.Owner == repo.Owner && current.Name == repo.Name {
repos[i] = current
break
}
}
}
return repos
}
+544
View File
@@ -0,0 +1,544 @@
package daemon
import (
"context"
"encoding/json"
"log/slog"
"runtime"
"sync"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/eventbus"
"github.com/samcharles93/archie-core/internal/events"
)
// runDeadline bounds a wait for something that must happen. Only a broken
// dispatcher ever reaches it, so it is generous rather than tight.
const runDeadline = 5 * time.Second
// startWindow is the window a test gives the dispatcher to show that something
// does not happen. The window only ever fails a broken dispatcher: a run that
// was going to take capacity does it within microseconds of the event that
// wakes it.
const startWindow = 100 * time.Millisecond
// awaitStart waits for a task's process to report that it started.
func awaitStart(t *testing.T, started <-chan int64, want int64) {
t.Helper()
select {
case got := <-started:
if got != want {
t.Fatalf("task %d started, want %d", got, want)
}
case <-time.After(runDeadline):
t.Fatalf("task %d did not start", want)
}
}
// awaitNoStart fails if any task starts within startWindow.
func awaitNoStart(t *testing.T, started <-chan int64, why string) {
t.Helper()
select {
case got := <-started:
t.Fatalf("task %d started %s", got, why)
case <-time.After(startWindow):
}
}
// awaitState waits until want holds. Taking capacity back is asynchronous by
// design -- it must not block the bus subscriber that resumed the run, and the
// resumed run keeps executing meanwhile -- so there is no channel to wait on.
func awaitState(t *testing.T, why string, want func() bool) {
t.Helper()
deadline := time.Now().Add(runDeadline)
for {
if want() {
return
}
if time.Now().After(deadline) {
t.Fatal(why)
}
runtime.Gosched()
}
}
// awaitNoState fails if want becomes true within startWindow. An assertion that
// something stays false needs a window: state that never changes signals on
// nothing.
func awaitNoState(t *testing.T, why string, want func() bool) {
t.Helper()
deadline := time.Now().Add(startWindow)
for time.Now().Before(deadline) {
if want() {
t.Fatal(why)
}
runtime.Gosched()
}
}
// capacity reads one task's execution capacity under the dispatcher's lock:
// slot is true while it counts against the global limit, place while it holds
// its repo's chain, and known is false once the run has retired.
func capacity(d *taskDispatcher, id int64) (slot, place, known bool) {
d.mu.Lock()
defer d.mu.Unlock()
hold := d.holds[id]
if hold == nil {
return false, false, false
}
return hold.slot, hold.baton != nil, true
}
// holdsCapacity reports whether a task's run holds both its slot and its place.
func holdsCapacity(d *taskDispatcher, id int64) bool {
slot, place, _ := capacity(d, id)
return slot && place
}
// activeRuns counts the runs held against the global limit.
func activeRuns(d *taskDispatcher) int {
d.mu.Lock()
defer d.mu.Unlock()
return d.active
}
// taskHoldOf reads the capacity record for a task.
func taskHoldOf(d *taskDispatcher, id int64) *taskHold {
d.mu.Lock()
defer d.mu.Unlock()
return d.holds[id]
}
// placeClaim reads the place the run holds in its repo's chain, as the
// dispatcher records it.
func placeClaim(d *taskDispatcher, id int64) chan struct{} {
d.mu.Lock()
defer d.mu.Unlock()
if hold := d.holds[id]; hold != nil {
return hold.baton
}
return nil
}
// slotHolders counts the runs the dispatcher knows that record a slot. Every
// run a test submits is named before its process starts, and no test submits
// while it checks this, so the two counts describe the same set of runs.
func slotHolders(d *taskDispatcher) int {
d.mu.Lock()
defer d.mu.Unlock()
held := 0
for _, hold := range d.holds {
if hold.slot {
held++
}
}
return held
}
// submitRun queues a task whose process reports its start on started and then
// blocks until released. Naming the run's capacity is what the daemon's own
// process does, where the task's id first exists.
func submitRun(d *taskDispatcher, id int64, repo string, started chan<- int64, released <-chan struct{}) {
d.Submit(context.Background(), &workflow.Task{ID: id, Owner: "acme", Repo: repo}, func(_ context.Context, task *workflow.Task, hold *taskHold) {
hold.name(task.ID)
started <- task.ID
<-released
})
}
// A run waiting on a callee holds no capacity: the task next in its repo, and
// behind it in the chain, runs while the wait lasts, and the run takes its slot
// and its place back once the wait ends.
func TestSuspendReleasesCapacityForTheTaskBehindIt(t *testing.T) {
d := newTaskDispatcher(1, nil)
started := make(chan int64, 4)
a, b, c := make(chan struct{}), make(chan struct{}), make(chan struct{})
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
submitRun(d, 2, "app", started, b)
awaitNoStart(t, started, "while the waiting run held the only slot and the repo's place")
d.Suspend(1)
awaitStart(t, started, 2)
close(b)
d.Resume(1)
awaitState(t, "task 1 did not take its slot and its place back", func() bool { return holdsCapacity(d, 1) })
submitRun(d, 3, "app", started, c)
awaitNoStart(t, started, "while the resumed run held the only slot and the repo's place")
close(a)
awaitStart(t, started, 3)
close(c)
d.Wait()
}
// A resumed run keeps its repo's place, not just its slot: with slots to spare,
// a task queued behind it still waits.
func TestResumedRunKeepsItsPlaceInTheReposChain(t *testing.T) {
d := newTaskDispatcher(3, nil)
started := make(chan int64, 4)
a, b, c := make(chan struct{}), make(chan struct{}), make(chan struct{})
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
submitRun(d, 2, "app", started, b)
awaitNoStart(t, started, "while the repo's place was another run's")
d.Suspend(1)
awaitStart(t, started, 2)
close(b)
d.Resume(1)
awaitState(t, "task 1 did not take its slot and its place back", func() bool { return holdsCapacity(d, 1) })
submitRun(d, 3, "app", started, c)
awaitNoStart(t, started, "while the resumed run held the repo's place, with slots free")
close(a)
awaitStart(t, started, 3)
close(c)
d.Wait()
}
// A run waiting on callees from parallel branches gives its capacity back once,
// and takes none of it again until the last wait ends.
func TestResumeTakesCapacityBackOnlyAfterTheLastWait(t *testing.T) {
d := newTaskDispatcher(1, nil)
started := make(chan int64, 4)
a, b := make(chan struct{}), make(chan struct{})
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
d.Suspend(1)
d.Suspend(1)
// The callee queued behind it runs, and returns.
submitRun(d, 2, "app", started, b)
awaitStart(t, started, 2)
close(b)
awaitState(t, "the callee's slot was not given back", func() bool { return activeRuns(d) == 0 })
// One wait outstanding: the run keeps no capacity while its other callee
// runs.
d.Resume(1)
awaitNoState(t, "task 1 took capacity back with a callee still running", func() bool { return holdsCapacity(d, 1) })
// With the last wait done it takes capacity back.
d.Resume(1)
awaitState(t, "task 1 did not take its capacity back", func() bool { return holdsCapacity(d, 1) })
close(a)
d.Wait()
}
// A hold records one slot and one place in its repo's chain, in whatever order
// the re-acquires a run queues take them: a second taker gives back the slot it
// took, and a re-acquire that finds a place already queued does not queue a
// second. Otherwise each occurrence permanently over-counts the global limit
// and the dispatcher throttles harder than containers.max_concurrency says.
func TestResumeTwiceDoesNotCountASecondSlot(t *testing.T) {
// Two slots: the task holding the repo's place must be able to take one
// while the run's second re-acquire takes another.
d := newTaskDispatcher(2, nil)
started := make(chan int64, 4)
a, x, z := make(chan struct{}), make(chan struct{}), make(chan struct{})
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
d.Suspend(1)
// X takes the repo's place while A waits on its callee, so A's first
// re-acquire queues its next place behind X.
submitRun(d, 2, "app", started, x)
awaitStart(t, started, 2)
d.Resume(1)
awaitState(t, "task 1 did not queue its place back", func() bool {
_, place, _ := capacity(d, 1)
return place
})
// That callee ends and the run starts its next call at once: the place the
// first re-acquire claimed is given back with it, and the second re-acquire
// takes a slot immediately.
d.Suspend(1)
d.Resume(1)
awaitState(t, "task 1 did not take its slot back", func() bool { return holdsCapacity(d, 1) })
// X leaves, so the first re-acquire can take a slot as well.
close(x)
awaitState(t, "task 2 did not retire", func() bool {
_, _, known := capacity(d, 2)
return !known
})
// Nothing may be counted against the limit that no run records.
awaitNoState(t, "the dispatcher counts a slot no run holds", func() bool {
return activeRuns(d) != slotHolders(d)
})
close(a)
awaitState(t, "task 1 did not retire", func() bool {
_, _, known := capacity(d, 1)
return !known
})
awaitState(t, "a slot is still counted with no run in flight", func() bool { return activeRuns(d) == 0 })
// A later task for another repo still gets a slot at the configured limit.
submitRun(d, 3, "other", started, z)
awaitStart(t, started, 3)
close(z)
d.Wait()
}
// A re-acquire that finds the run's place already queued leaves it at one: a
// second place in the same repo's chain is a claim nothing closes, and the
// re-acquire waiting on it is parked for good.
func TestSecondReacquireDoesNotQueueASecondPlace(t *testing.T) {
d := newTaskDispatcher(2, nil)
started := make(chan int64, 4)
a, x, z := make(chan struct{}), make(chan struct{}), make(chan struct{})
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
d.Suspend(1)
// X holds the repo's place, so the run's re-acquire queues behind it and
// stays there.
submitRun(d, 2, "app", started, x)
awaitStart(t, started, 2)
d.Resume(1)
awaitState(t, "task 1 did not queue its place back", func() bool {
_, place, _ := capacity(d, 1)
return place
})
queued := placeClaim(d, 1)
// A second re-acquire for the same run, with the first still waiting. A
// Resume cannot queue this one: queueing two takes a Suspend between them,
// and that Suspend would clear the place queued above.
go d.reacquire(taskHoldOf(d, 1))
awaitNoState(t, "a second place was queued for the run", func() bool { return placeClaim(d, 1) != queued })
awaitState(t, "task 1 did not take its slot back", func() bool { return holdsCapacity(d, 1) })
// X leaves: the first re-acquire takes a slot as well, and gives it back,
// because the hold records one.
close(x)
awaitState(t, "task 2 did not retire", func() bool {
_, _, known := capacity(d, 2)
return !known
})
awaitNoState(t, "the dispatcher counts a slot no run holds", func() bool {
return activeRuns(d) != slotHolders(d)
})
close(a)
awaitState(t, "task 1 did not retire", func() bool {
_, _, known := capacity(d, 1)
return !known
})
awaitState(t, "a slot is still counted with no run in flight", func() bool { return activeRuns(d) == 0 })
submitRun(d, 3, "other", started, z)
awaitStart(t, started, 3)
close(z)
d.Wait()
}
// A run that returns while the resume it queued is still waiting for capacity
// leaves nothing behind: a later task still gets a slot, a Resume arriving
// afterwards is a no-op, and shutdown's wait returns.
func TestResumeAfterTheRunReturnedLeavesNoCapacity(t *testing.T) {
d := newTaskDispatcher(1, nil)
started := make(chan int64, 4)
a, b, c := make(chan struct{}), make(chan struct{}), make(chan struct{})
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
d.Suspend(1)
submitRun(d, 2, "app", started, b)
awaitStart(t, started, 2)
// The resume is queued for a slot and a place in the chain, and overtaken by
// the run returning before either comes free.
d.Resume(1)
awaitState(t, "task 1 did not take its place in the chain back", func() bool {
_, place, _ := capacity(d, 1)
return place
})
close(a)
awaitState(t, "task 1 did not retire", func() bool {
_, _, known := capacity(d, 1)
return !known
})
close(b)
awaitState(t, "task 2 did not retire", func() bool {
_, _, known := capacity(d, 2)
return !known
})
// Neither the returned run nor the resume it queued took the slot: it is
// free for the task queued behind it.
awaitNoState(t, "a returned run's slot was taken", func() bool { return activeRuns(d) > 0 })
submitRun(d, 3, "app", started, c)
awaitStart(t, started, 3)
close(c)
// A Resume arriving after the run returned is a no-op.
d.Resume(1)
d.Wait()
}
// callEvent builds one call step's event as the daemon receives it: the event
// the container's workflow engine published, JSON-encoded and decoded again by
// the agent-event bridge, so the wait flag arrives as a decoded bool.
func callEvent(t *testing.T, kind string, taskID int64, wait bool) events.Event {
t.Helper()
raw, err := json.Marshal(events.Event{Kind: kind, TaskID: taskID, Data: map[string]any{"wait": wait}})
if err != nil {
t.Fatal(err)
}
var e events.Event
if err := json.Unmarshal(raw, &e); err != nil {
t.Fatal(err)
}
return e
}
// Only a call the caller waits on releases its capacity, and the call
// finishing takes it back.
func TestCallWaitEventsReleaseCapacity(t *testing.T) {
d := newTaskDispatcher(1, nil)
started := make(chan int64, 4)
a, b := make(chan struct{}), make(chan struct{})
submitRun(d, 1, "app", started, a)
awaitStart(t, started, 1)
submitRun(d, 2, "app", started, b)
// A call that does not wait is not waited on: nothing is released.
applyCallWait(d, callEvent(t, events.KindWorkflowCallStarted, 1, false))
awaitNoStart(t, started, "while the call that started does not wait")
applyCallWait(d, callEvent(t, events.KindWorkflowCallStarted, 1, true))
awaitStart(t, started, 2)
close(b)
applyCallWait(d, callEvent(t, events.KindWorkflowCallFinished, 1, false))
awaitState(t, "task 1 did not take its capacity back", func() bool { return holdsCapacity(d, 1) })
close(a)
d.Wait()
}
// A daemon with no bus still starts and stops: the watcher is optional.
func TestCallWaitsWithoutABus(t *testing.T) {
d := &Daemon{Cfg: config.NewHolder(config.Config{}), Log: slog.New(slog.DiscardHandler)}
d.watchCallWaits(t.Context())
d.WaitForTasks()
}
// stubTasks is a task bus with an empty stream, so a drain pass falls through
// to the store's ClaimNext.
type stubTasks struct{}
func (stubTasks) PublishUnique(context.Context, string, string, []byte) error { return nil }
func (stubTasks) Fetch(context.Context) (eventbus.Message, error) {
return nil, eventbus.ErrNoMessage
}
func (stubTasks) Request(context.Context, string, []byte) ([]byte, error) { return nil, nil }
// stubStore hands out one claimed task, then nothing, and blocks in ParkTask
// for as long as the test wants a task held in flight. Every path a task takes
// through the daemon without a container parks, so ParkTask is where a test
// task waits.
type stubStore struct {
storecontract.TaskStore
claim *workflow.Task
park func()
}
func (s *stubStore) ClaimNext(context.Context) (*workflow.Task, error) {
task := s.claim
s.claim = nil
return task, nil
}
func (s *stubStore) Update(context.Context, *workflow.Task) error { return nil }
func (s *stubStore) ParkTask(context.Context, int64, string, string, string) error {
s.park()
return nil
}
// A drain pass returns as soon as the stream and the store's claim fallback are
// both empty; its tasks keep running, and shutdown waits for them. A pass that
// waits for them blocks every later pass, so a run waiting on a callee whose
// callee is queued by that same pass would never be released.
func TestDrainPassDoesNotWaitForItsTasks(t *testing.T) {
inFlight := make(chan struct{}, 1)
release := make(chan struct{})
var unblockOnce sync.Once
unblock := func() { unblockOnce.Do(func() { close(release) }) }
defer unblock()
store := &stubStore{
claim: &workflow.Task{ID: 7, Owner: "acme", Repo: "app"},
park: func() {
select {
case inFlight <- struct{}{}:
default:
}
<-release
},
}
d := &Daemon{
Cfg: config.NewHolder(config.Config{}),
Log: slog.New(slog.DiscardHandler),
Store: store,
Tasks: stubTasks{},
}
drained := make(chan struct{})
go func() {
d.drainNATS(context.Background())
close(drained)
}()
select {
case <-inFlight:
case <-time.After(runDeadline):
t.Fatal("no task reached execution")
}
select {
case <-drained:
case <-time.After(runDeadline):
t.Fatal("the drain pass waited for a task still in flight")
}
// Shutdown is where that wait now lives.
waited := make(chan struct{})
go func() {
d.WaitForTasks()
close(waited)
}()
select {
case <-waited:
t.Fatal("WaitForTasks returned with a task still in flight")
default:
}
unblock()
select {
case <-waited:
case <-time.After(runDeadline):
t.Fatal("WaitForTasks did not return after the task finished")
}
}
+3 -2
View File
@@ -66,6 +66,7 @@ func (d *Daemon) runKitTask(ctx context.Context, task *workflow.Task, repo confi
// config.Config d.configFor already resolves for every other identity
// override (Forge, Budgets, ...).
taskCfg := d.configFor(task)
credentialOrg, grantedServices := taskCfg.CredentialAccess("")
run, err := d.KitLauncher.Launch(ctx, kitrun.Request{
Execution: fmt.Sprintf("task-%d", task.ID),
Kit: profile.Kit,
@@ -73,8 +74,8 @@ func (d *Daemon) runKitTask(ctx context.Context, task *workflow.Task, repo confi
WorkDir: workDir,
WorkerEnv: env,
GateRetries: iface.Needs().GateRetries,
Org: taskCfg.Org,
GrantedServices: taskCfg.GrantedCredentials,
Org: credentialOrg,
GrantedServices: grantedServices,
RunCredential: credential,
})
if err != nil {
+1 -1
View File
@@ -14,7 +14,7 @@ func TestDispatcherWaitIncludesQueuedWork(t *testing.T) {
release := make(chan struct{})
process := func() { started <- struct{}{}; <-release }
for range 2 {
dispatcher.Submit(t.Context(), &workflow.Task{Owner: "o", Repo: "r"}, func(_ context.Context, _ *workflow.Task) { process() })
dispatcher.Submit(t.Context(), &workflow.Task{Owner: "o", Repo: "r"}, func(_ context.Context, _ *workflow.Task, _ *taskHold) { process() })
}
<-started
dispatcher.mu.Lock()
+27
View File
@@ -3,8 +3,10 @@ package daemon
import (
"context"
"log/slog"
"reflect"
"testing"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/identity"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
@@ -93,3 +95,28 @@ func TestWorkflowIdentity(t *testing.T) {
})
}
}
func TestIdentityTasksUseLiveSettings(t *testing.T) {
root := config.Config{Models: map[string]string{"builder": "old/model"}, Providers: map[string]config.Provider{"old": {}}, Repos: []config.Repo{{Owner: "acme", Name: "app", Base: "main"}}}
id := config.IdentityConfig{Name: "bot", BotUser: "bot", BotEmail: "bot@example.test", Forge: config.Forge{Type: "github"}, Repos: []config.Repo{{Owner: "acme", Name: "app", Base: "stale"}}}
d := &Daemon{Cfg: config.NewHolder(root), Identities: []*IdentityRunner{{Name: "bot", Cfg: id, Repos: id.Repos}}}
task := &workflow.Task{Identity: "bot", Owner: "acme", Repo: "app"}
for _, model := range []string{"old/model", "new/model"} {
root.Models = map[string]string{"builder": model}
root.Budgets.MaxSteps++
root.Dispatch.Trigger = "label"
root.Notify.Webhook = "https://example.test/notify"
d.Cfg.Set(root)
got := d.configFor(task)
if !reflect.DeepEqual(got.Models, root.Models) || !reflect.DeepEqual(got.Providers, root.Providers) || got.Budgets != root.Budgets || got.Dispatch != root.Dispatch || got.Notify != root.Notify {
t.Fatalf("identity settings replaced the current runtime settings: %+v", got)
}
repo, ok := d.repoFor(task)
if !ok || repo.Base != "main" {
t.Fatalf("repository = %+v, found %v; want current base main", repo, ok)
}
if got.BotUser != id.BotUser || got.Forge != id.Forge {
t.Fatal("identity lost its forge account")
}
}
}
+38
View File
@@ -0,0 +1,38 @@
package daemon
import (
"context"
"fmt"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/org"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/taskstate"
)
// workflowContext forwards a task's identity so the State Store selects its
// org's definitions. An identity assigned elsewhere cannot supply the pin.
func (d *Daemon) workflowContext(ctx context.Context, task *workflow.Task) (context.Context, error) {
owner := task.Org
if owner == "" {
owner = org.DefaultOrgID
}
if d.Principals == nil {
principal, forwarded := access.PrincipalFromContext(ctx)
if owner != org.DefaultOrgID || org.OrgFromContext(ctx) != owner || (forwarded && principal.Org != "" && principal.Org != owner) {
return nil, pinFailure{taskstate.ParkNeedsHuman, fmt.Errorf("workflow org %s: principal unavailable", owner)}
}
return ctx, nil
}
principal, err := d.Principals.PrincipalFor(ctx, d.publisherIdentity(task.Identity))
if err != nil {
return nil, pinFailure{taskstate.ParkTransient, fmt.Errorf("workflow principal: %w", err)}
}
if principal.Org == "" {
principal.Org = org.DefaultOrgID
}
if principal.Org != owner {
return nil, pinFailure{taskstate.ParkNeedsHuman, fmt.Errorf("workflow identity belongs to org %s, task belongs to %s", principal.Org, owner)}
}
return org.WithOrg(access.WithPrincipal(ctx, principal), owner), nil
}
+155
View File
@@ -0,0 +1,155 @@
package daemon
import (
"context"
"fmt"
"log/slog"
"slices"
"testing"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/access"
"github.com/samcharles93/archie-core/internal/domain/binding"
"github.com/samcharles93/archie-core/internal/domain/identity"
"github.com/samcharles93/archie-core/internal/domain/mapping"
"github.com/samcharles93/archie-core/internal/domain/org"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/domain/workflow"
workflowtask "github.com/samcharles93/archie-core/internal/domain/workflow/task"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/taskstate"
)
type workflowPrincipals map[identity.IdentityID]org.OrgID
func (p workflowPrincipals) PrincipalFor(_ context.Context, id identity.IdentityID) (access.Principal, error) {
owner, ok := p[id]
if !ok {
return access.Principal{}, fmt.Errorf("principal %q unavailable", id)
}
return access.Principal{IdentityID: id, Org: owner}, nil
}
type orgBindingStore struct {
storecontract.TaskStore
storecontract.BindingStore
storecontract.MappingStore
bindings []binding.Binding
started map[org.OrgID]*workflow.Task
}
func (s *orgBindingStore) ListBindings(context.Context) ([]binding.Binding, error) {
return s.bindings, nil
}
func (s *orgBindingStore) ArmedBindingsForSource(_ context.Context, source string) ([]binding.Binding, error) {
return slices.DeleteFunc(slices.Clone(s.bindings), func(b binding.Binding) bool { return b.Matcher.Source != source }), nil
}
func (*orgBindingStore) ListUndispatchedCaptures(_ context.Context, sources []string, _ int) ([]storecontract.CapturedEvent, error) {
var captures []storecontract.CapturedEvent
for _, source := range sources {
captures = append(captures, storecontract.CapturedEvent{ID: source, Source: source, EventType: source, Body: "{}", Authenticated: true})
}
return captures, nil
}
func (*orgBindingStore) RecordDispatch(context.Context, string, int64, string, int64, string) error {
return nil
}
func (*orgBindingStore) SetDispatchTask(context.Context, string, string, int64) error { return nil }
func (*orgBindingStore) InsertEvent(context.Context, events.Event) (int64, error) { return 0, nil }
func (*orgBindingStore) GetMapping(_ context.Context, id string) (*mapping.Mapping, error) {
return &mapping.Mapping{EventTypeID: id}, nil
}
func (s *orgBindingStore) EnqueueBindingTask(ctx context.Context, owner, repo, _, _, wf, identity, bindingID string, _ int, _ map[string]any) (*workflow.Task, error) {
run := &workflow.Task{ID: 1, Owner: owner, Repo: repo, Workflow: wf, Identity: identity, Org: org.OrgFromContext(ctx), BindingID: bindingID}
s.started[run.Org] = run
return run, nil
}
func TestDispatchOrgWorkflows(t *testing.T) {
for _, disabled := range []bool{false, true} {
t.Run(fmt.Sprintf("acme disabled %v", disabled), func(t *testing.T) {
store := &orgBindingStore{started: map[org.OrgID]*workflow.Task{}, bindings: []binding.Binding{
{ID: "root", MappingID: "root", Matcher: binding.Matcher{Source: "root"}, OrgID: org.DefaultOrgID, Workflow: "deploy", Status: binding.StatusArmed, Owner: "owner", Repo: "repo"},
{ID: "acme", MappingID: "acme", Matcher: binding.Matcher{Source: "acme"}, OrgID: "acme", Workflow: "deploy", Status: binding.StatusArmed},
}}
d := &Daemon{Cfg: config.NewHolder(config.Config{}), Store: store, Log: slog.New(slog.DiscardHandler), Bindings: store, BindingDispatcher: store, BindingTaskCreator: store, Mappings: store, RootIdentityID: "root", Identities: []*IdentityRunner{{ID: "acme-bot", Name: "bot"}}, Principals: workflowPrincipals{"root": org.DefaultOrgID, "acme-bot": "acme"}, WorkflowDefinitions: orgDefinitions{
org.DefaultOrgID: "id: deploy\nrepository: required\nsteps:\n - type: workflow.finish\n",
"acme": "id: deploy\nrepository: none\nsteps:\n - type: workflow.finish\n",
}}
if disabled {
d.WorkflowEnablement = enablementOf(workflowtask.WorkflowEnablement{}.SetEnabled("acme", "deploy", false))
}
d.dispatchBindings(t.Context())
root := store.started[org.DefaultOrgID]
if root == nil || root.Repo != "repo" || root.Identity != "" {
t.Fatalf("root dispatch %#v", root)
}
acme := store.started["acme"]
if disabled {
if acme != nil {
t.Fatalf("disabled org dispatched %#v", acme)
}
return
}
if acme == nil || acme.Repo != "" || acme.Identity != "acme-bot" {
t.Fatalf("org's repo-free workflow did not dispatch as its identity: %#v", acme)
}
})
}
}
type orgDefinitions map[org.OrgID]string
func (d orgDefinitions) WorkflowDefinitions(ctx context.Context) (workflow.WorkflowDefinitionCollection, int64, error) {
owner := org.DefaultOrgID
if p, ok := access.PrincipalFromContext(ctx); ok && p.Org != "" {
owner = p.Org
}
return workflow.WorkflowDefinitionCollection{Definitions: []workflow.WorkflowDefinitionEntry{{ID: "deploy", YAML: d[owner]}}}, 3, nil
}
func TestPinWorkflowInTaskOrg(t *testing.T) {
const shared = "id: deploy\nsteps:\n - type: workflow.finish\n"
const custom = "id: deploy\nrepository: none\nsteps:\n - type: workflow.finish\n"
for _, tc := range []struct {
name string
owner org.OrgID
principals workflowPrincipals
want string
refused bool
contextOrg org.OrgID
}{
{"legacy default org", "", nil, shared, false, ""},
{"non-default org definition", "acme", workflowPrincipals{"bot": "acme"}, custom, false, ""},
{"another org cannot supply the definition", "acme", workflowPrincipals{"bot": org.DefaultOrgID}, "", true, ""},
{"unresolved org is refused", "acme", nil, "", true, ""},
{"unresolved context cannot select another org", "", nil, "", true, "acme"},
} {
t.Run(tc.name, func(t *testing.T) {
d := &Daemon{Store: &identityStore{}, RootIdentityID: "bot", WorkflowDefinitions: orgDefinitions{org.DefaultOrgID: shared, "acme": custom}}
if tc.principals != nil {
d.Principals = tc.principals
}
task := &workflow.Task{ID: 1, Org: tc.owner, Workflow: "deploy"}
ctx := t.Context()
if tc.contextOrg != "" {
ctx = org.WithOrg(access.WithPrincipal(ctx, access.Principal{IdentityID: "foreign", Org: tc.contextOrg}), tc.contextOrg)
}
err := d.pinWorkflowDefinition(ctx, task)
if tc.refused {
if err == nil || pinParkClass(err) != taskstate.ParkNeedsHuman || task.WorkflowDefinitionYAML != "" {
t.Fatalf("cross-org pin accepted: task %#v err %v", task, err)
}
return
}
if err != nil || task.WorkflowDefinitionYAML != tc.want {
t.Fatalf("pinned %q, want %q; err %v", task.WorkflowDefinitionYAML, tc.want, err)
}
})
}
}
+16 -1
View File
@@ -27,6 +27,21 @@ const (
// for the test that binds writer to reader.
func Processes() []string { return []string{Daemon, Gateway, Messaging} }
// Applicability names the consumers of each resource, including boot-only settings.
func Applicability() map[string][]string {
kinds := map[string][]string{}
for _, kind := range []string{"task-policy", "provider-settings", "model-role-assignments", "repository-policies", "scheduling-policy", "tool-settings", "plugin-settings", "review-settings", "channel-settings", "container-runtime-policies", "credential-bindings", "identity-grants", "agent-profiles", "workflow-execution-settings", "extension-settings", "workflow-definitions", "workflow-enablement", "schedules", "eda-playbooks"} {
kinds[kind] = []string{Daemon}
}
for _, kind := range []string{"provider-settings", "model-role-assignments", "tool-settings", "channel-settings", "plugin-settings", "extension-settings", "personas", "soul", "skills", "curators"} {
kinds[kind] = append(kinds[kind], Gateway)
}
for _, kind := range []string{"channel-settings", "extension-settings"} {
kinds[kind] = append(kinds[kind], Messaging)
}
return kinds
}
const (
// RestampInterval is how often a process rewrites its records. It is the
// only thing that distinguishes a live process from one that applied a
@@ -77,7 +92,7 @@ func New(process string, store storecontract.ApplyStatusStore, log *slog.Logger)
// Report records that this process applied version of kind, or failed to. A
// failure keeps the previous version. Errors are logged.
func (r *Reporter) Report(ctx context.Context, kind string, version int64, applyErr error) {
if r == nil {
if r == nil || !slices.Contains(Applicability()[kind], r.process) {
return
}
status := storecontract.ApplyStatus{
+18 -1
View File
@@ -199,7 +199,7 @@ func (r *Registry) Names() []string {
// error. This is the family's failure-isolation boundary for lifecycle.
func (r *Registry) Start(ctx context.Context) error {
r.mu.Lock()
if r.state != stateIdle {
if r.state == stateRunning {
r.mu.Unlock()
return ErrStarted
}
@@ -312,3 +312,20 @@ func isNilEngine(c CuratorEngine) bool {
}
return false
}
func (r *Registry) replaceDefinitions(candidate *Registry) {
r.mu.Lock()
defer r.mu.Unlock()
r.order = slices.DeleteFunc(r.order, func(name string) bool {
if _, custom := r.curators[name].(*DefinitionEngine); !custom {
return false
}
delete(r.curators, name)
delete(r.status, name)
return true
})
for _, name := range candidate.order {
r.curators[name] = candidate.curators[name]
r.order = append(r.order, name)
}
}
Loaded 100 of 340 files, more files were not shown because too many files have changed in this diff. Show more