Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
829fc44cbc | ||
|
|
6b4049c9f5 | ||
|
|
934ba16c72 | ||
|
|
9f1dc44673 | ||
|
|
24258e215f | ||
|
|
793eec6e51 | ||
|
|
6785750729 | ||
|
|
72fa043f47 | ||
|
|
cc7397787b | ||
|
|
cef476aca3 | ||
|
|
f50444594f | ||
|
|
275d337032 | ||
|
|
b314470281 | ||
|
|
eb3add5b89 | ||
|
|
c126d5a24a | ||
|
|
1a3e79502d | ||
|
|
fcc6a7974e | ||
|
|
f4ed542b00 | ||
|
|
d06be5ca32 | ||
|
|
8d48a08b13 | ||
|
|
f6ee43bf55 | ||
|
|
268f99051a | ||
|
|
60a09e7def | ||
|
|
bf1dc2e33e | ||
|
|
f34714ea07 | ||
|
|
c2cfba1129 | ||
|
|
cea8bfee98 | ||
|
|
2f8107d5ca | ||
|
|
a67a142139 | ||
|
|
48416af5bc | ||
|
|
cca80ac2e8 | ||
|
|
52d267683b | ||
|
|
0b00682fac | ||
|
|
d1b8bff5ce | ||
|
|
ae8f0d42ef | ||
|
|
39b61a0340 | ||
|
|
06334ca3f9 | ||
|
|
f9e97b561f | ||
|
|
d7f6f502a8 | ||
|
|
54e36ff8b7 | ||
|
|
a1c69c5b28 | ||
|
|
bf68866dca | ||
|
|
87d52938b5 | ||
|
|
90a4189db4 | ||
|
|
013fcd0919 | ||
|
|
00209ebe87 | ||
|
|
96e54e963f | ||
|
|
2f05ed81b1 | ||
|
|
822f8b63d4 | ||
|
|
e0ec9b87e6 | ||
|
|
27bad9f1fa | ||
|
|
403ace1cef | ||
|
|
761a1271a4 | ||
|
|
c5a8942976 | ||
|
|
fb0424b729 | ||
|
|
52b8b6dcca | ||
|
|
d7178250ae | ||
|
|
9b282177e0 | ||
|
|
59273c9544 | ||
|
|
86e706dff7 | ||
|
|
ad5caf88a6 | ||
|
|
a7efb9bb00 | ||
|
|
34f47a4bd0 | ||
|
|
d60fcad18f | ||
|
|
8124c5af54 | ||
|
|
bfc8d4b1b7 | ||
|
|
0b8e9ae7f1 | ||
|
|
662a5ee336 | ||
|
|
8cbbdb20b5 | ||
|
|
cadbc6d1ca | ||
|
|
64128645c2 | ||
|
|
8155cc915a | ||
|
|
150dde5a7d |
No files matched your search
@@ -66,6 +66,14 @@ packages the moment it merged; formatting and lint drifted the same way, and onl
|
||||
the repository gate saw it. Both failures are paid for inside the lane when the
|
||||
gate runs the build and the linter, and at merge when it does not.
|
||||
|
||||
Gate the **candidate** before `main` moves. Check the merge candidate out
|
||||
detached — or gate the lane branch — and advance the ref only on a green result:
|
||||
gating after a fast-forward lands the failure on the branch of record and leaves
|
||||
only fix-forward or reset as exits. The lane brief must name the linter as well,
|
||||
`golangci-lint run ./<package>/...` beside the scoped tests, because a scoped
|
||||
test run cannot see a complexity or style limit and `task check` is where that
|
||||
failure otherwise surfaces — once it is already merged.
|
||||
|
||||
A copy of the script under `.pi/control-plane-run/` is that run's disposable
|
||||
state and may lag the skill's version. The skill's copy is the one to trust.
|
||||
|
||||
@@ -126,6 +134,36 @@ ln -sfn /work/apps/archie-core/.pi/agents /work/apps/cp-<lane>/.pi/agents
|
||||
`.pi/` is gitignored, so the worktree stays clean and the symlink cannot reach a
|
||||
commit. This costs an hour to rediscover.
|
||||
|
||||
## Peer sessions over intercom
|
||||
|
||||
A fleet here can also run as **sibling sessions** coordinated over `intercom` —
|
||||
a peer per lane — rather than pi-subagents children under one orchestrator.
|
||||
These facts are peer-specific, and each has cost a real run.
|
||||
|
||||
- **Name the channel in the brief.** A peer's knowledge of how to report lives in
|
||||
its context, not its configuration. Write "report to <peer> over intercom" as
|
||||
an instruction; a convention that is only assumed evaporates.
|
||||
- **A compaction erases conventions, not just facts.** A peer that compacts
|
||||
mid-task keeps its task state and loses its transport: it reverts to
|
||||
addressing its operator, and the coordinator reads that as idleness or death.
|
||||
**Peer silence is not evidence of either** — ask for a one-line ack over the
|
||||
channel before concluding anything. Re-priming restores the channel; the
|
||||
transport itself survives compaction intact.
|
||||
- **Match the task's shape to the session's window, not its strength.** A sweep
|
||||
over hundreds of items needs the largest window available; a bounded hard
|
||||
artifact (a PRD, a conformance review, a design decision) belongs in a smaller,
|
||||
reasoning-strong session. A wide task in a small window stalls the task — too
|
||||
wide is not the same as too hard, and it is the coordinator's assignment error,
|
||||
not the peer's failure.
|
||||
- **Checkpoint at ~70%, never compact mid-task.** A terse ledger sent to the
|
||||
coordinator before the window fills survives; one reconstructed afterwards
|
||||
loses exactly the identifiers — bead IDs, exact counts — that made it worth
|
||||
having. Instruct every long-running peer to report and stop there.
|
||||
- **One writer per resource.** Exactly one session writes the tracker. Two
|
||||
writers produce double-closes, contradictory verdicts, and counts nobody can
|
||||
reconcile. When a sweep changes hands, the successor re-derives them rather
|
||||
than inheriting the predecessor's ledger as fact.
|
||||
|
||||
## Traps worth carrying
|
||||
|
||||
- **`touch` does not invalidate a Task source.** Task hashes file _content_, so a
|
||||
@@ -143,11 +181,41 @@ build` still reports the binary up to date, shipping the old value. Tracked as
|
||||
reaches, or changes a wire type, passes its own gate and fails
|
||||
`task check` on a missing `tools/go.sum` entry or a stale schema. Such a
|
||||
lane runs `go -C tools mod tidy` and `task docs:generate` and commits both.
|
||||
Adding or editing a page under `docs/` — a PRD included — changes
|
||||
`docs/data/generated/dev-docs.json`, which gains or loses a page: regenerate it
|
||||
and commit it in the same commit, or `docs:artifact:check` fails. That failure
|
||||
lands on `main`, not in the lane, if the ref moves first.
|
||||
- **Untracked files in the shared checkout are other sessions'.** prdlint
|
||||
scratch (`docs/prds/rules.jsonl`), editor state, and gate-written artifacts
|
||||
appear and vanish under a lane. Never chase, add, stage or commit them: a
|
||||
merge train's scope is its own lanes' files, and nothing else.
|
||||
|
||||
## codegraph answers symbol questions cheaply
|
||||
|
||||
The maintainer's `codegraph` MCP server indexes this repository. Prefer it to
|
||||
grep or whole-file reading for "who calls X", "is this dead", "what does this
|
||||
change touch": it enumerates call sites that a grep only summarises, and labels
|
||||
each caller's confidence so interface fan-out is not mistaken for direct calls.
|
||||
|
||||
- **`pr_impact <base> <head>`** lists changed symbols with per-symbol caller and
|
||||
test counts. Read `interface_dispatch` and the inferred tests alongside
|
||||
`test_count`: a method reached through an interface reports a low direct test
|
||||
count while many indirect tests reach it, and the compact line alone invites
|
||||
the wrong conclusion.
|
||||
- **It is a derived cache.** `scip-go` and tree-sitter rebuild it, and a query
|
||||
indexes the revisions it names, so a cold query is slow and losing the index
|
||||
costs a rebuild rather than data. `list_repositories` reports the indexed
|
||||
revision — compare it with the tree before trusting a default-revision answer.
|
||||
- **Stagger heavy queries.** Concurrent sessions share one instance, and
|
||||
**name-based** `symbol_context` is what locks: it returns `database is locked`
|
||||
or times out, while the same symbol **by id** returns immediately. Query
|
||||
serially and retry rather than assuming a hang.
|
||||
- **Report problems one line at a time** in
|
||||
`/work/apps/codegraph/data/feedback.jsonl` — one JSON object per line, keys
|
||||
`date`, `repo`, `task`, `tools`, `helped`, `missed`, `wrong`, `slow`. A raw
|
||||
newline inside a string makes the line invalid and breaks every reader; a
|
||||
multi-line entry corrupted that file once. Validate, append, never rewrite.
|
||||
|
||||
## Tracker conventions
|
||||
|
||||
`bd` is the only tracker; no markdown TODO lists, no local trackers.
|
||||
|
||||
@@ -5,6 +5,47 @@ release's per-component sections are labelled beneath its heading.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [1.45.0] - 2026-09-30
|
||||
|
||||
### archied
|
||||
|
||||
- fix(eda): reject a result map missing a field or needing a conversion
|
||||
- feat(eda): per-org workflow enable/disable gating bindings
|
||||
- feat(telegram): carry inbound media through the chat turn
|
||||
- fix(prsource): use a CodeQL-recognised containment guard when extracting archives
|
||||
- feat(store): key control-plane resources by (org, kind)
|
||||
- fix(store): refuse resource keys the audit key cannot disambiguate
|
||||
- fix(daemon): resume the task branch before container acquisition
|
||||
- feat(webui): show attachment metadata in the session transcript
|
||||
- feat(webui): name a workflow's declared inputs on the work-request form
|
||||
- feat(controlplane): carry the caller org from the request principal
|
||||
- fix(webui): the workflow handlers act in the caller's org
|
||||
- feat(kitrun): render the OAuth credential file with sentinels
|
||||
- feat(store): record accepted package authority against the pin
|
||||
- fix(workflow): the own-PR flow no longer parks on the operator-approval gate
|
||||
- feat(gateway): a TTL-bounded dedup gate for redelivered platform messages
|
||||
- feat(egress): capture the OAuth scope set and carry it to the harness secret
|
||||
- feat(kitrun): render the OAuth scopes and omit an uncaptured primary key
|
||||
- feat(store): project installed package contributions into org resources
|
||||
- feat(gateway): coalesce rapid-fire text events into one turn
|
||||
- fix(config): expand plugin, module and skills dirs like every other operator path
|
||||
- fix(webui): drop the config schema's editability advert
|
||||
- feat(webui): events counts, YAML reading, log rows and dashboard filters
|
||||
|
||||
### archie-agent
|
||||
|
||||
- feat(eda): per-org workflow enable/disable gating bindings
|
||||
- feat(telegram): carry inbound media through the chat turn
|
||||
- fix(prsource): use a CodeQL-recognised containment guard when extracting archives
|
||||
- feat(store): key control-plane resources by (org, kind)
|
||||
- feat(controlplane): carry the caller org from the request principal
|
||||
- feat(kitrun): render the OAuth credential file with sentinels
|
||||
- feat(store): record accepted package authority against the pin
|
||||
- fix(workflow): the own-PR flow no longer parks on the operator-approval gate
|
||||
- feat(egress): capture the OAuth scope set and carry it to the harness secret
|
||||
- feat(kitrun): render the OAuth scopes and omit an uncaptured primary key
|
||||
- feat(store): project installed package contributions into org resources
|
||||
|
||||
## [1.44.1] - 2026-09-29
|
||||
|
||||
### archied
|
||||
|
||||
@@ -338,6 +338,7 @@ func TestStateStoreRealProcessSmoke(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("real-process smoke test builds and execs the binary; skip under -short")
|
||||
}
|
||||
t.Parallel()
|
||||
dir := t.TempDir()
|
||||
bin := buildBinary(t, dir)
|
||||
cfg := writeConfigWithServiceListen(t, dir)
|
||||
@@ -453,6 +454,7 @@ func TestStateStoreRealProcessDeadline(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("real-process smoke test builds and execs the binary; skip under -short")
|
||||
}
|
||||
t.Parallel()
|
||||
dir := t.TempDir()
|
||||
bin := buildBinary(t, dir)
|
||||
cfg := writeMinimalConfig(t, dir)
|
||||
@@ -479,6 +481,7 @@ func TestStateStoreRealProcessRestartRecovery(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("real-process smoke test builds and execs the binary; skip under -short")
|
||||
}
|
||||
t.Parallel()
|
||||
dir := t.TempDir()
|
||||
bin := buildBinary(t, dir)
|
||||
cfg := writeMinimalConfig(t, dir)
|
||||
@@ -525,6 +528,7 @@ func TestStateStoreRealProcessTokenProtectedCaptureStream(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("real-process smoke test builds and execs the binary; skip under -short")
|
||||
}
|
||||
t.Parallel()
|
||||
const token = "operator-secret-token"
|
||||
dir := t.TempDir()
|
||||
bin := buildBinary(t, dir)
|
||||
@@ -572,6 +576,7 @@ func TestStateStoreRealProcessRemoteSurfaces(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("real-process smoke test builds and execs the binary; skip under -short")
|
||||
}
|
||||
t.Parallel()
|
||||
dir := t.TempDir()
|
||||
bin := buildBinary(t, dir)
|
||||
cfg := writeMinimalConfig(t, dir)
|
||||
|
||||
@@ -259,7 +259,7 @@ func TestRecoveryRollbackReplaysTheRevisionThroughReplace(t *testing.T) {
|
||||
if !strings.Contains(stdout, controlplane.ModelRoleAssignmentsKind) {
|
||||
t.Errorf("rollback must name the resource it replayed; stdout = %q", stdout)
|
||||
}
|
||||
resource, err := st.Resource(t.Context(), controlplane.ModelRoleAssignmentsKind)
|
||||
resource, err := st.Resource(t.Context(), storecontract.DefaultOrgID, controlplane.ModelRoleAssignmentsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -269,7 +269,7 @@ func TestRecoveryRollbackReplaysTheRevisionThroughReplace(t *testing.T) {
|
||||
if resource.Version != 3 {
|
||||
t.Errorf("resource version = %d, want a new revision (3) rather than a rewrite of the old one", resource.Version)
|
||||
}
|
||||
history, err := st.ResourceHistory(t.Context(), controlplane.ModelRoleAssignmentsKind, 0)
|
||||
history, err := st.ResourceHistory(t.Context(), storecontract.DefaultOrgID, controlplane.ModelRoleAssignmentsKind, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -293,7 +293,7 @@ func TestRecoveryRollbackRestoresANamedRevision(t *testing.T) {
|
||||
if code != 0 {
|
||||
t.Fatalf("rollback exited %d: %s", code, stderr)
|
||||
}
|
||||
resource, err := st.Resource(t.Context(), controlplane.ModelRoleAssignmentsKind)
|
||||
resource, err := st.Resource(t.Context(), storecontract.DefaultOrgID, controlplane.ModelRoleAssignmentsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -463,7 +463,7 @@ func storedResourceCount(t *testing.T, stdout string) int {
|
||||
// a silent no-op.
|
||||
func seedResource(t *testing.T, st *pgstore.TaskDB, kind, value, label string) {
|
||||
t.Helper()
|
||||
current, err := st.Resource(t.Context(), kind)
|
||||
current, err := st.Resource(t.Context(), storecontract.DefaultOrgID, kind)
|
||||
expected := int64(0)
|
||||
if err == nil {
|
||||
expected = current.Version
|
||||
@@ -477,7 +477,8 @@ func seedResource(t *testing.T, st *pgstore.TaskDB, kind, value, label string) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := st.PutResource(t.Context(), storecontract.ResourceWrite{
|
||||
Kind: kind, Value: encoded, Actor: label, Source: "test",
|
||||
OrgID: storecontract.DefaultOrgID,
|
||||
Kind: kind, Value: encoded, Actor: label, Source: "test",
|
||||
RequestID: fmt.Sprintf("%s-%d", label, expected+1), ExpectedVersion: expected,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed resource %s: %v", kind, err)
|
||||
|
||||
@@ -1,6 +1,29 @@
|
||||
# Adversarial self-review
|
||||
|
||||
Archie runs a fresh adversarial review of the diff before it opens a pull
|
||||
> **Removed, not shipped (2026-09-28, `d73e0748`). This page records a system
|
||||
> that no longer exists in the tree.**
|
||||
>
|
||||
> `d73e0748` ("remove the superseded adversarial-self-review system") deleted the
|
||||
> stage and its contract: `workflow.Reviewer`, `ReviewRequest`, `ReviewReport`
|
||||
> and `StageReview` are gone from `internal/domain/workflow`, so everything under
|
||||
> "Contract" below describes types that cannot be found.
|
||||
>
|
||||
> The authority is `docs/prds/pr-review-agent.md`, which says so itself: it
|
||||
> "supersedes the reviewer executor and blocking rule in
|
||||
> `docs/architecture/adversarial-review.md`". archied now runs its PR review
|
||||
> pipeline before it opens a pull request; there is no separate adversarial
|
||||
> self-review pass and no `repo.review_enabled` setting.
|
||||
>
|
||||
> The page is kept rather than deleted because PRDs link to it
|
||||
> (`docs/prds/adversarial-self-review.md`, `docs/prds/inline-review.md`,
|
||||
> `docs/prds/pr-review-agent.md`). Read it as history.
|
||||
>
|
||||
> One section is stale in a second, older way: "Operator-triggered review" below
|
||||
> describes the synchronous `review_pr` (fetch the head, snapshot it, run the
|
||||
> reviewer, return findings), which `archie-core-afbk.14` replaced with a queued
|
||||
> pr-review task.
|
||||
|
||||
Archie ran a fresh adversarial review of the diff before it opened a pull
|
||||
request, so a PR arrives with a findings record rather than an unreviewed
|
||||
claim. The reviewer provably cannot see the implementer's context: it reviews a
|
||||
`.git`-free snapshot of the committed tree, not the worktree, the commit
|
||||
|
||||
@@ -312,6 +312,11 @@
|
||||
"additionalProperties": false,
|
||||
"description": "MediaAttachment describes a file attached to a message.",
|
||||
"properties": {
|
||||
"data": {
|
||||
"contentEncoding": "base64",
|
||||
"description": "Data carries the attachment bytes in-process. An inbound path sets\nit after downloading the file; outbound senders never set it and\nread Path or URL instead. It never crosses a wire or persistence\nboundary -- it is a turn-scoped value, not a stored one.",
|
||||
"type": "string"
|
||||
},
|
||||
"duration": {
|
||||
"description": "Duration is the play time in seconds for audio and video.",
|
||||
"type": "integer"
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
"url": "/dev/architecture/adversarial-review/",
|
||||
"title": "Adversarial self-review",
|
||||
"section": "Architecture",
|
||||
"body": "# Adversarial self-review\n\nArchie runs a fresh adversarial review of the diff before it opens a pull\nrequest, so a PR arrives with a findings record rather than an unreviewed\nclaim. The reviewer provably cannot see the implementer's context: it reviews a\n`.git`-free snapshot of the committed tree, not the worktree, the commit\nhistory, or the implementer's transcript.\n\nPromoted from `docs/prds/adversarial-self-review.md` (the design decision;\n`archie-core-h019`). Describes the stage as shipped.\n\n## Contract (`internal/domain/workflow`)\n\nThe workflow domain owns the contract and the blocking decision; it never names\nthe executor.\n\n- `Reviewer.Review(ctx, ReviewRequest) ReviewReport` — the interface the\n workflow stage consumes.\n- `ReviewRequest` is the reviewer's entire input: `SnapshotDir` (a `.git`-free\n export of the reviewed commit), `Diff` (base...head), `IssueText` (the\n originating issue title/body), and `MaxSteps`.\n- `ReviewFinding` / `ReviewReport` are the structured result. The load-bearing\n rule is `Blocking() == (Verdict == confirmed \u0026\u0026 Level == error)`: only a\n confirmed error-level finding blocks; a plausible finding never blocks\n however severe it claims to be.\n\n`ReviewReport` distinguishes `not_run`, `skipped`, and `completed` (zero\nfindings) structurally, so \"ran and found nothing\" is not the same as \"never\nran\" — collapsing them would make an outage indistinguishable from a clean\nreview.\n\n## Execution (`internal/app/agentworker`)\n\nThe executor is an ai-sdk `agent.Subagent`: a nested synchronous generation in\nthe same worker process, with its own model, system prompt, toolset and step\nbudget.\n\n- **Conversation isolation is structural.** `Subagent.Run` sends only a prompt\n string, never the implementer's message history.\n- **Workspace isolation is the `.git`-free snapshot.** The reviewer's toolset is\n read-only (`read`, `grep`, `find`) and rooted at `SnapshotDir` — never the\n worker's own toolset, which carries the `worktreerpc` publication grant.\n- **Findings return through a typed capture tool** (`record_finding`), validated\n on the way in, so the reviewer reports through the contract rather than free\n text the caller regexes.\n- **Fail-closed.** A reviewer that errors, truncates, or reaches no conclusion\n yields `ReviewStatusNotRun`, which the stage treats as a park — never a silent\n \"PR opened unreviewed\".\n\n## Lifecycle effect\n\n`StageReview` runs in the implement workflow after the deterministic gates and\nbefore `StageOpenPR`, gated by `repo.review_enabled` (opt-in, default off). A\nsurviving blocking finding — or a review that failed to run — parks the task\n(`StatusParked`, not `waiting_human`) with the findings rendered into the park\nreason; `StageOpenPR` never runs. Warn/plausible findings and the zero-finding\ncase leave the outcome unset, so the PR opens with a findings section rendered\nonto its body. `StagePostReviewComments` then runs after `StageOpenPR` and posts\nthe **line-anchored** findings as inline review comments, carrying a one-click\nsuggestion on a confirmed finding the reviewer could state a fix for. That stage\nis best-effort — a comment that fails to post is logged, never a park — because\nthe body section already lists every finding, and it is the record a whole-file\nfinding (no line to anchor to) only ever appears in.\n\n## Model and budget\n\nThe reviewer model is `models.reviewer`, falling back to `models.builder`.\nEffort is the per-stage `MaxSteps` budget, mapped onto `Subagent.MaxSteps`\n(default 10).\n\n## Operator-triggered review\n\nThe same reviewer runs against an _existing_ PR via the channel-neutral\n`review_pr` chat command: fetch the PR's head, snapshot it, run the isolated\nreviewer, and return the structured findings. Authorisation is identity-scoped\nto the operator's configured repositories; duplicate or concurrent reviews of\nthe same PR are deduplicated.\n"
|
||||
"body": "# Adversarial self-review\n\n\u003e **Removed, not shipped (2026-09-28, `d73e0748`). This page records a system\n\u003e that no longer exists in the tree.**\n\u003e\n\u003e `d73e0748` (\"remove the superseded adversarial-self-review system\") deleted the\n\u003e stage and its contract: `workflow.Reviewer`, `ReviewRequest`, `ReviewReport`\n\u003e and `StageReview` are gone from `internal/domain/workflow`, so everything under\n\u003e \"Contract\" below describes types that cannot be found.\n\u003e\n\u003e The authority is `docs/prds/pr-review-agent.md`, which says so itself: it\n\u003e \"supersedes the reviewer executor and blocking rule in\n\u003e `docs/architecture/adversarial-review.md`\". archied now runs its PR review\n\u003e pipeline before it opens a pull request; there is no separate adversarial\n\u003e self-review pass and no `repo.review_enabled` setting.\n\u003e\n\u003e The page is kept rather than deleted because PRDs link to it\n\u003e (`docs/prds/adversarial-self-review.md`, `docs/prds/inline-review.md`,\n\u003e `docs/prds/pr-review-agent.md`). Read it as history.\n\u003e\n\u003e One section is stale in a second, older way: \"Operator-triggered review\" below\n\u003e describes the synchronous `review_pr` (fetch the head, snapshot it, run the\n\u003e reviewer, return findings), which `archie-core-afbk.14` replaced with a queued\n\u003e pr-review task.\n\nArchie ran a fresh adversarial review of the diff before it opened a pull\nrequest, so a PR arrives with a findings record rather than an unreviewed\nclaim. The reviewer provably cannot see the implementer's context: it reviews a\n`.git`-free snapshot of the committed tree, not the worktree, the commit\nhistory, or the implementer's transcript.\n\nPromoted from `docs/prds/adversarial-self-review.md` (the design decision;\n`archie-core-h019`). Describes the stage as shipped.\n\n## Contract (`internal/domain/workflow`)\n\nThe workflow domain owns the contract and the blocking decision; it never names\nthe executor.\n\n- `Reviewer.Review(ctx, ReviewRequest) ReviewReport` — the interface the\n workflow stage consumes.\n- `ReviewRequest` is the reviewer's entire input: `SnapshotDir` (a `.git`-free\n export of the reviewed commit), `Diff` (base...head), `IssueText` (the\n originating issue title/body), and `MaxSteps`.\n- `ReviewFinding` / `ReviewReport` are the structured result. The load-bearing\n rule is `Blocking() == (Verdict == confirmed \u0026\u0026 Level == error)`: only a\n confirmed error-level finding blocks; a plausible finding never blocks\n however severe it claims to be.\n\n`ReviewReport` distinguishes `not_run`, `skipped`, and `completed` (zero\nfindings) structurally, so \"ran and found nothing\" is not the same as \"never\nran\" — collapsing them would make an outage indistinguishable from a clean\nreview.\n\n## Execution (`internal/app/agentworker`)\n\nThe executor is an ai-sdk `agent.Subagent`: a nested synchronous generation in\nthe same worker process, with its own model, system prompt, toolset and step\nbudget.\n\n- **Conversation isolation is structural.** `Subagent.Run` sends only a prompt\n string, never the implementer's message history.\n- **Workspace isolation is the `.git`-free snapshot.** The reviewer's toolset is\n read-only (`read`, `grep`, `find`) and rooted at `SnapshotDir` — never the\n worker's own toolset, which carries the `worktreerpc` publication grant.\n- **Findings return through a typed capture tool** (`record_finding`), validated\n on the way in, so the reviewer reports through the contract rather than free\n text the caller regexes.\n- **Fail-closed.** A reviewer that errors, truncates, or reaches no conclusion\n yields `ReviewStatusNotRun`, which the stage treats as a park — never a silent\n \"PR opened unreviewed\".\n\n## Lifecycle effect\n\n`StageReview` runs in the implement workflow after the deterministic gates and\nbefore `StageOpenPR`, gated by `repo.review_enabled` (opt-in, default off). A\nsurviving blocking finding — or a review that failed to run — parks the task\n(`StatusParked`, not `waiting_human`) with the findings rendered into the park\nreason; `StageOpenPR` never runs. Warn/plausible findings and the zero-finding\ncase leave the outcome unset, so the PR opens with a findings section rendered\nonto its body. `StagePostReviewComments` then runs after `StageOpenPR` and posts\nthe **line-anchored** findings as inline review comments, carrying a one-click\nsuggestion on a confirmed finding the reviewer could state a fix for. That stage\nis best-effort — a comment that fails to post is logged, never a park — because\nthe body section already lists every finding, and it is the record a whole-file\nfinding (no line to anchor to) only ever appears in.\n\n## Model and budget\n\nThe reviewer model is `models.reviewer`, falling back to `models.builder`.\nEffort is the per-stage `MaxSteps` budget, mapped onto `Subagent.MaxSteps`\n(default 10).\n\n## OperatorLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/architecture/agent-system/",
|
||||
@@ -106,7 +106,7 @@
|
||||
"url": "/dev/development/",
|
||||
"title": "Development guides",
|
||||
"section": "Development",
|
||||
"body": "# Development guides\n\nChecklists for changing Archie, one page per area. Each names every place a\nchange must reach, so a feature that works in one layer does not silently stop\nat the next. A change that spans areas starts on the page for the area it\nbegins in, which points to the others. The pages point at the documents that\nown the design (`docs/architecture/`, the PRDs, `CLAUDE.md`) rather than\nrestating them.\n\n| Area | Covers |\n| ---------------------------------------------- | -------------------------------------------------------------------------------- |\n| [Frontend UI](/dev/development/frontend-ui/) | Anything under `ui/`, and the dashboard's API routes |\n| [Agent](/dev/development/agent/) | The agent container, its image, profiles, tools and workspace |\n| [Workflows](/dev/development/workflows/) | Workflow definitions and step types |\n| [Daemon](/dev/development/daemon/) | Task statuses, fields and the brief; sources, event types, mappings and bindings |\n| [State Store](/dev/development/state-store/) | Stored fields, tables, RPCs and errors |\n| [Control plane and settings](/dev/development/control-plane/) | Settings in `config.toml` or the dashboard |\n| [Processes and wiring](/dev/development/wiring/) | Where code goes, processes, plugin engines, channels, NATS subjects |\n| [Merging work](/dev/development/merging-work/) | Merging a lane or a branch |\n\n## Every change\n\n- Write the failing test first for anything with logic, branching or a\n contract, and check it fails on an assertion, not a compile error.\n- Test through the path production uses. A test that sets a value directly on\n the consumer proves nothing about the layers that normally deliver it.\n- Run `task check` and let its exit status decide the commit:\n `task check \u0026\u0026 git commit ...`. `task test` passes `-short`, which skips the\n process and Postgres tests, so run `go test -count=1 ./cmd/... ./internal/...`\n too when a change crosses a process or the database.\n- Regenerate, never hand-edit, generated output: `task proto:generate`,\n `go -C tools tool sqlc generate -f $PWD/internal/infrastructure/postgres/sqlc.yaml`,\n `task ui`, `task docs:artifact`. `task proto:check` compares against the git\n index, so stage regenerated contracts before running it.\n- Stage by path, or read `git status` before `git add -A`. Tests that start\n embedded NATS or write logs must use `t.TempDir()`; anything they leave in the\n tree gets committed. `task check` also adds linter-tool lines to `go.sum`\n (bead 1e37); restore `go.sum` before committing unless you changed a\n dependency.\n- A change to `docs/prds/`, `docs/architecture/` **or `docs/development/`**\n carries the regenerated `docs/data/generated/dev-docs.json`: run\n `task docs:artifact` (not `task docs:check`, which checks contracts only).\n PRD edits also follow `docs/prds/RULES.md` and pass `task docs:prds`.\n"
|
||||
"body": "# Development guides\n\nChecklists for changing Archie, one page per area. Each names every place a\nchange must reach, so a feature that works in one layer does not silently stop\nat the next. A change that spans areas starts on the page for the area it\nbegins in, which points to the others. The pages point at the documents that\nown the design (`docs/architecture/`, the PRDs, `CLAUDE.md`) rather than\nrestating them.\n\n| Area | Covers |\n| ---------------------------------------------- | -------------------------------------------------------------------------------- |\n| [Frontend UI](/dev/development/frontend-ui/) | Anything under `ui/`, and the dashboard's API routes |\n| [Agent](/dev/development/agent/) | The agent container, its image, profiles, tools and workspace |\n| [Workflows](/dev/development/workflows/) | Workflow definitions and step types |\n| [Daemon](/dev/development/daemon/) | Task statuses, fields and the brief; sources, event types, mappings and bindings |\n| [State Store](/dev/development/state-store/) | Stored fields, tables, RPCs and errors |\n| [Control plane and settings](/dev/development/control-plane/) | Settings in `config.toml` or the dashboard |\n| [Processes and wiring](/dev/development/wiring/) | Where code goes, processes, plugin engines, channels, NATS subjects |\n| [Merging work](/dev/development/merging-work/) | Merging a lane or a branch |\n\n## Every change\n\n- Write the failing test first for anything with logic, branching or a\n contract, and check it fails on an assertion, not a compile error.\n- Test through the path production uses. A test that sets a value directly on\n the consumer proves nothing about the layers that normally deliver it.\n- Run `task check` and let its exit status decide the commit:\n `task check \u0026\u0026 git commit ...`. `task test` passes `-short`, which skips the\n process and Postgres tests, so run `go test -count=1 ./cmd/... ./internal/...`\n too when a change crosses a process or the database.\n- Regenerate, never hand-edit, generated output: `task proto:generate`,\n `go -C tools tool sqlc generate -f $PWD/internal/infrastructure/postgres/sqlc.yaml`,\n `task ui`, `task docs:artifact`. `task proto:check` compares against the git\n index, so stage regenerated contracts before running it.\n- Stage by path, or read `git status` before `git add -A`. Tests that start\n embedded NATS or write logs must use `t.TempDir()`; anything they leave in the\n tree gets committed. `task check` writes no module file on a consistent tree,\n root or `tools/` (verified at HEAD with `GOFLAGS` unset and with `-mod=mod`):\n a stale `go.mod` or `go.sum` fails it, with `missing go.sum entry` or\n `updates to go.mod needed`, rather than being repaired. Only a global\n `GOFLAGS=-mod=mod` in `go env` re-enables that repair, which is why the\n `go -C tools` steps pass `-mod=readonly`; if `go.sum` goes dirty, run\n `go mod tidy` and commit the result on its own.\n- A change to `docs/prds/`, `docs/architecture/` **or `docs/development/`**\n carries the regenerated `docs/data/generated/dev-docs.json`: run\n `task docs:artifact` (not `task docs:check`, which checks contracts only).\n PRD edits also follow `docs/prds/RULES.md` and pass `task docs:prds`.\n"
|
||||
},
|
||||
{
|
||||
"url": "/dev/development/agent/",
|
||||
@@ -130,7 +130,7 @@
|
||||
"url": "/dev/development/frontend-ui/",
|
||||
"title": "Frontend UI",
|
||||
"section": "Development",
|
||||
"body": "# Frontend UI\n\nAuthority: `.agents/skills/archie-web-ui` (stack facts, design-system rules,\ntone and accessibility). This page covers what a change must reach, not how it\nshould look.\n\n## Adding or changing a feature\n\n1. The feature lives in `ui/src/\u003cfeature\u003e/`. Move a helper to `ui/src/base/`\n only when a second feature needs it.\n2. Keep wire shapes, draft state and payload building in a plain `.ts` module\n beside the components (`ui/src/bindings/binding-draft.ts` is the pattern),\n and test it in `ui/test/*.test.ts`. Components stay thin.\n3. Every mutation goes through `api.*` in `ui/src/lib/api`, which sends the\n CSRF and Content-Type headers the server's mutation guard requires. A bare\n `fetch` is refused.\n4. A value the server also describes (task statuses, actions, workflow inputs)\n comes from the server. Keep a local default only so the first render works,\n and keep it in step (`ui/src/lib/task-meta.ts`).\n5. Run `task ui`, `task test:ui` and `cd ui \u0026\u0026 npm run typecheck`. Commit the\n regenerated `ui/dist/`; the Go binaries embed it. A fresh git worktree has\n no ignored `ui/node_modules`: run `cd ui \u0026\u0026 npm ci` before these commands.\n6. **One long-lived browser connection:** `ui/src/lib/api.ts` has the only\n `new EventSource`, owned by `stores/live-updates.ts`. Feature stores consume\n topics with `live.subscribe(topic, callback)`; do not open a page-owned SSE\n stream. Logs alone opt in while mounted, replacing that same EventSource\n with `?topics=logs\u0026since=\u003clast cursor\u003e`; a browser's `Last-Event-ID` only\n survives its own automatic reconnect, not a newly constructed EventSource.\n Task and log topics resume by cursor; control-plane, identity and apply-status\n topics replay the latest snapshot. Test a fetch and same-origin navigation\n after visiting Settings and Logs: the UI process serves HTTP/1.1 and six\n persistent streams can starve every later browser request.\n\n## Adding an API route\n\n1. The handler goes in the `internal/webui/api_\u003cconcern\u003e.go` file for its\n concern. Check the request fully on the server; the UI's checks are only a\n convenience.\n2. A mutation calls `authorizeTaskMutation` first.\n3. A route whose store is not configured answers 503, and\n `internal/app/archieui/degraded_routes_test.go` pins what each route\n returns without its contract. Add the route there.\n4. The dashboard runs in its own process (`archie-ui`) and must not link the\n daemon or the workflow engine (`cmd/archie-ui/architecture_test.go`). Reach\n stored data through `storecontract` interfaces wired in\n `internal/app/archieui/compose.go`.\n5. Settings that live in a control-plane resource need no new route: the\n Advanced settings page edits every resource from its generated schema.\n6. For a new live topic, add one process-wide producer to `internal/webui`'s\n `RunLive` and fan it through the `/api/stream` hub; `internal/app/archieui/run.go`\n starts the producers with the process context. Never create an upstream\n watch or poller in an HTTP handler. A slow client must reconnect for cursor\n and snapshot repair rather than silently lose an update. Prove the producer\n with several browser clients against the real peer, not just a scripted\n stream; `live_hub_test.go` is the example.\n"
|
||||
"body": "# Frontend UI\n\nAuthority: `.agents/skills/archie-web-ui` (stack facts, design-system rules,\ntone and accessibility). This page covers what a change must reach, not how it\nshould look.\n\n## Adding or changing a feature\n\n1. The feature lives in `ui/src/\u003cfeature\u003e/`. Move a helper to `ui/src/base/`\n only when a second feature needs it.\n2. Keep wire shapes, draft state and payload building in a plain `.ts` module\n beside the components (`ui/src/bindings/binding-draft.ts` is the pattern),\n and test it in `ui/test/*.test.ts`. Components stay thin.\n3. Every mutation goes through `api.*` in `ui/src/lib/api`, which sends the\n CSRF and Content-Type headers the server's mutation guard requires. A bare\n `fetch` is refused.\n4. A value the server also describes (task statuses, actions, workflow inputs)\n comes from the server. Keep a local default only so the first render works,\n and keep it in step (`ui/src/lib/task-meta.ts`).\n5. Run `task ui`, `task test:ui` and `cd ui \u0026\u0026 npm run typecheck`. Commit the\n regenerated `ui/dist/`; the Go binaries embed it. A fresh git worktree has\n no ignored `ui/node_modules`: run `cd ui \u0026\u0026 npm ci` before these commands.\n6. **One long-lived browser connection:** `ui/src/lib/api.ts` has the only\n `new EventSource`, owned by `stores/live-updates.ts`. Feature stores consume\n topics with `live.subscribe(topic, callback)`; do not open a page-owned SSE\n stream. Logs alone opt in while mounted, replacing that same EventSource\n with `?topics=logs\u0026since=\u003clast cursor\u003e`; a browser's `Last-Event-ID` only\n survives its own automatic reconnect, not a newly constructed EventSource.\n Task and log topics resume by cursor; control-plane, identity and apply-status\n topics replay the latest snapshot. Test a fetch and same-origin navigation\n after visiting Settings and Logs: the UI process serves HTTP/1.1 and six\n persistent streams can starve every later browser request.\n\n## Testing the dashboard\n\nAuthority: the runner itself is `ui/package.json`'s `test` script: `node --test\n--experimental-strip-types test/*.test.ts` over `ui/test/*.test.ts`. It runs\npure TypeScript modules with `node`'s type stripping — no SFC loader, no DOM,\nno second dependency beyond what the app already carries. That is the standard,\nand a decision record for when it is deliberately not enough:\n\n- **Test archie's logic in a module; leave SFC wiring alone.** Wire shapes,\n draft state, copy conditions, filters and payload building live in plain\n `.ts` files beside the components and are tested in `ui/test/`. Components\n stay thin, so there is normally nothing in an SFC that is archie logic.\n- **Installed primitives are upstream-tested, not re-tested here.**\n `reka-ui` (2.10.5 here) maintains colocated tests per component in its own\n repo (Vitest/jsdom + axe: Switch, TagsInput, NumberField, Select, the\n arrow-navigation composables). The npm package ships none of those files, so\n this coverage is inherited through the version bound, not the tarball —\n `grep -l test node_modules/reka-ui` finds nothing. Duplicating them as local\n mount tests would test a dependency.\n- **A `.vue` import cannot run under the module runner.** `node --test` fails\n such a module with `ERR_UNKNOWN_FILE_EXTENSION: .vue` and the runner has no\n built-in loader for SFCs. A file under `ui/test/` that imports a `.vue` is\n therefore a broken test, not a weak one.\n- **If a DOM mount harness is ever needed** (a page-level accessibility\n contract, cross-component interaction, focus behaviour): `happy-dom` +\n `@vue/test-utils` + `vitest` scoped to a separate DOM test glob, keeping the\n module runner for everything else. Measured on the ui tree (vitest 5.0.2,\n happy-dom 20.14.5, @vue/test-utils 2.5.1): they add 24 top-level\n `node_modules` entries and 27 MB on a 296 MB install, a working 2-test mount\n suite (role=radiogroup vs role=switch, aria-label, click activation against\n the real SFCs) runs in 1.2 s on top of the current 0.5 s module suite, and\n `vue-tsc` (7.4 s) dominates `task test:ui` either way. The single-runner\n alternative (absorbing the existing suite into vitest) was measured\n infeasible as-is: all 29 existing files import `node:test`, which vitest\n cannot collect (\"No test suite found in file\" for every one), so it would be\n a rewrite of the whole suite, and its runtime was not measured. Nothing in\n `ui/dist` or CI changes either way; happy-dom is pure JS, no browser binary.\n- **What the module standard does not verify:** whether a call site passes an\n `aria-label` or which `v-if` branch shows. Those live in SFC templates and\n are only visible with a mount. If a page's accessible-name wiring is the\n risk, say so in the page's task and reach for the harness decision above\n rather than quietly growing mount tests. Never suppress a lint rule or a\n test to keep Line truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/development/merging-work/",
|
||||
@@ -154,7 +154,7 @@
|
||||
"url": "/dev/development/workflows/",
|
||||
"title": "Workflows",
|
||||
"section": "Development",
|
||||
"body": "# Workflows\n\nAuthority: `docs/architecture/agent-system.md` for the lifecycle,\n`docs/prds/event-automation.md` for inputs, repository modes and profiles.\n\nWorkflow definitions are YAML stored as a control-plane resource. The State\nStore validates them and the agent compiles them, each against the step\nvocabulary it registers, so both must see the same step types.\n\n## Adding a step type\n\n1. Write the factory in `internal/domain/workflow` as a `StepType`. It decodes\n and validates its settings and returns an error for anything it cannot\n honour. The factory runs when a definition is saved and again when it is\n compiled, so a bad definition is refused on save.\n2. Register it through a provider in\n `internal/infrastructure/workflowsteps/workflowsteps.go`. That one list is\n what every process registers, which is how the validating and executing\n sides agree.\n3. An agent-driven step builds on `AgentStage`; do not start a second path to\n the agent runner.\n4. A step that needs no repository goes in `repoFreeStepTypes` in\n `definition.go`, or workflows declaring `repository: none` or `optional`\n cannot use it. A step that needs one may assume `tc.Dir` holds a worktree\n only when the workflow's repository is `required`.\n5. A step that ends the workflow sets `tc.Outcome`. A workflow that runs out of\n steps without an outcome is parked as a definition bug.\n6. Add a parse test (settings accepted and refused) and a run test with fake\n `Trees` and agent runner.\n\n## Changing the definition format\n\n- The fields a caller needs without the engine (inputs, repository, profile)\n live in `task.WorkflowInterface`, because the dashboard process must not\n import the workflow engine (`cmd/archie-ui/architecture_test.go`). Anything a\n binding or the dashboard checks goes there.\n- The YAML decoder rejects unknown keys. Existing stored definitions must keep\n parsing, so a new key is optional with a default that preserves today's\n behaviour.\n- Examples in `examples/workflows/` are parsed against the registered\n vocabulary by `TestExampleWorkflowsParse`. Add one for a new capability.\n\n## Starting a workflow from chat\n\n`task_spawn` accepts `workflow` and an `inputs` object. The gateway forwards\ninputs through the task creation contract; it does not interpret individual\nworkflows. The daemon checks inputs against the pinned `WorkflowInterface`\nbefore acquiring a container and parks invalid requests with the reason.\n"
|
||||
"body": "# Workflows\n\nAuthority: `docs/architecture/agent-system.md` for the lifecycle,\n`docs/prds/event-automation.md` for inputs, repository modes and profiles.\n\nWorkflow definitions are YAML stored as a control-plane resource. The State\nStore validates them and the agent compiles them, each against the step\nvocabulary it registers, so both must see the same step types.\n\n## Adding a step type\n\n1. Write the factory in `internal/domain/workflow` as a `StepType`. It decodes\n and validates its settings and returns an error for anything it cannot\n honour. The factory runs when a definition is saved and again when it is\n compiled, so a bad definition is refused on save.\n2. Register it through a provider in\n `internal/infrastructure/workflowsteps/workflowsteps.go`. That one list is\n what every process registers, which is how the validating and executing\n sides agree.\n3. An agent-driven step builds on `AgentStage`; do not start a second path to\n the agent runner.\n4. A step that needs no repository goes in `repoFreeStepTypes` in\n `definition.go`, or workflows declaring `repository: none` or `optional`\n cannot use it. A step that needs one may assume `tc.Dir` holds a worktree\n only when the workflow's repository is `required`.\n5. A step that ends the workflow sets `tc.Outcome`. A workflow that runs out of\n steps without an outcome is parked as a definition bug.\n6. Add a parse test (settings accepted and refused) and a run test with fake\n `Trees` and agent runner.\n\n## Changing the definition format\n\n- The fields a caller needs without the engine (inputs, repository, profile)\n live in `task.WorkflowInterface`, because the dashboard process must not\n import the workflow engine (`cmd/archie-ui/architecture_test.go`). Anything a\n binding or the dashboard checks goes there.\n- The YAML decoder rejects unknown keys. Existing stored definitions must keep\n parsing, so a new key is optional with a default that preserves today's\n behaviour.\n- Examples in `examples/workflows/` are parsed against the registered\n vocabulary by `TestExampleWorkflowsParse`. Add one for a new capability.\n\n## Enabling and disabling\n\nThe `workflow-enablement` control-plane resource lists, per org, the workflows\nthat org has disabled; anything unlisted is enabled. Its readers are the\ndaemon's binding dispatch (a disabled workflow's binding is skipped like an\nunarmed one), the dashboard's binding save (refused) and binding list\n(`workflow_disabled`), and work requests. A new reader of \"may this org run\nthis workflow\" goes through `task.WorkflowEnablement.Enabled`, never a second\nlist. The dashboard acts in the default org until access resolves a caller's\nown.\n\n## Starting a workflow from chat\n\n`task_spawn` accepts `workflow` and an `inputs` object. The gateway forwards\ninputs through the task creation contract; it does not interpret individual\nworkflows. The daemon checks inputs against the pinned `WorkflowInterface`\nbefore acquiring a container and parks invalid requests with the reason.\n"
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/01-project-management/",
|
||||
@@ -376,7 +376,7 @@
|
||||
"url": "/dev/prds/multi-action-playbooks/",
|
||||
"title": "Multi-action playbooks and typed action results",
|
||||
"section": "PRDs",
|
||||
"body": "# Multi-action playbooks and typed action results\n\n**Status:** Approved\n**Date:** 2026-09-22\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-nx30` (unblocks `archie-core-t2db.27` and `archie-core-t2db.31`)\n\n## Why this exists\n\n`eda-playbook-engine.md`'s hard boundary rejects any playbook that is not\nexactly one `workflow` action, so the parent document's data-flow, Module and\nChannel/Forge decisions have no loadable shape to attach to. The boundary's own\nstated condition is now met: both execution-time gaps are resolved\n(`eda-playbook-engine.md`, \"Execution-time gaps\"; the `playbook_dispatches`\nledger). This document designs the relaxation.\n\n## A playbook is one of two shapes\n\n- A **workflow playbook** is exactly one `workflow` action. Unchanged: it names\n a workflow definition and is routed by the daemon's definition pin.\n- An **action playbook** is one or more `module` actions in order. Each action\n has an optional `id`, a `kind` naming a registered Module kind, `args`, and an\n optional `when`.\n\nOne playbook does not mix the two shapes.\n\n```yaml\ntrigger: { kind: bug }\nactions:\n - id: build\n position: module\n kind: log\n args: { message: '\"build started\"' }\n - id: done\n position: module\n kind: log\n args: { message: '\"done: \" + string(actions.build.result.written)' }\n when: actions.build.result.written == true\n```\n\nThe first Module kind is the side-effect-free `log` kind\n(`module-position.md`, \"Recommended first slice\"). A Module kind with an\nexternal side effect, and the Channel and Forge positions, are out of scope.\n\n## A result type is the kind's generated Result struct\n\nEach Module kind owns a hand-written `Args` and `Result` struct\n(`internal/domain/eda/module/log` is the worked example); `ModuleRegistry.Invoke`\nreturns the result as a flat `map[string]any`. `ModuleRegistry.DecodeResult`\n(`internal/domain/eda/module`) is the single site that converts that map into\nthe kind's `Result` struct before a later expression reads it, so the value the\nchecker validates and the value the run reads have one definition.\n\n`actions.\u003cid\u003e` is typed by the kind of the action that declares `\u003cid\u003e`:\n`{ result: \u003cKindResult\u003e }`. `event` stays `dyn` (J4,\n`playbook-expression-syntax.md`).\n\n## Reject-at-load, preserved\n\nRelaxing the boundary must not make a bad reference resolvable. The load fails,\nnaming the playbook and the offending action, for:\n\n- an `id` missing where a later action references it, duplicated, or outside\n its shape's grammar: a workflow action id keeps the shared\n stable-identifier grammar (J1, already enforced), while a module action id\n must be a lowercase CEL field name writable as `actions.\u003cid\u003e` (so a CEL\n keyword such as `in`, `true`, `false`, or `null` is rejected at load);\n- an `actions.\u003cid\u003e` read of an id no earlier action declares;\n- an `actions` read that is not statically resolvable to an id (any map index,\n literal or dynamic, `in`, a comprehension): the typed `actions` root is an\n object type, so only field selection `actions.\u003cid\u003e` resolves and the\n `actions[\"id\"]` spelling is not accepted;\n- a `result` field the referenced kind's `Result` does not define;\n- a reference to an earlier action that produces no result;\n- an unknown `position`, an unknown `kind`, or a `module` action with no `kind`;\n- a field belonging to the other shape (`kind` on a `workflow` action, or\n `workflow` on a `module` action);\n- an arg KEY the kind's `Args` schema does not define;\n- an arg value whose checked CEL type cannot fill its `Args` field (`'123'`\n for a string field). A `dyn` value, such as an `event` read, passes: its\n type is known only per event, and the module decode refuses a bad one at\n run.\n\nDeclaring the kind's `Result` schema to CEL at load makes a result-field typo a\nload failure rather than a dispatch failure. Matching arg keys against the\nkind's `Args` schema makes an arg-key typo or a wrongly typed value a load\nfailure.\n\n## The environment is per playbook\n\nResult typing depends on the ids a playbook declares and the kind each id runs,\nso the CEL environment is built from one playbook's action list rather than once\nper `Store`, and `expr.NewEnv` takes that list: each declared id becomes a field\nof the `actions` object typed by its kind's `Result`.\n\n## Running an action playbook is out of scope\n\nThis document makes an action playbook load and its references type-check.\nRunning one is designed in `docs/prds/action-playbook-run.md`.\n\n## Decisions\n\n**D1 — dispatch before `t2db.31`.** Superseded by\n`docs/prds/action-playbook-run.md`. Was: an action playbook loads and\nvalidates but is not routed; the definition-pin dispatch reports no match for it,\nand the daemon logs a visible warning naming it.\n\n**D2 — mixing positions.*Line truncated
|
||||
"body": "# Multi-action playbooks and typed action results\n\n**Status:** Approved\n**Date:** 2026-09-22\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-nx30` (unblocks `archie-core-t2db.27` and `archie-core-t2db.31`)\n\n## Why this exists\n\n`eda-playbook-engine.md`'s hard boundary rejects any playbook that is not\nexactly one `workflow` action, so the parent document's data-flow, Module and\nChannel/Forge decisions have no loadable shape to attach to. The boundary's own\nstated condition is now met: both execution-time gaps are resolved\n(`eda-playbook-engine.md`, \"Execution-time gaps\"; the `playbook_dispatches`\nledger). This document designs the relaxation.\n\n## A playbook is one of two shapes\n\n- A **workflow playbook** is exactly one `workflow` action. Unchanged: it names\n a workflow definition and is routed by the daemon's definition pin.\n- An **action playbook** is one or more `module` actions in order. Each action\n has an optional `id`, a `kind` naming a registered Module kind, `args`, and an\n optional `when`.\n\nOne playbook does not mix the two shapes.\n\n```yaml\ntrigger: { kind: bug }\nactions:\n - id: build\n position: module\n kind: log\n args: { message: '\"build started\"' }\n - id: done\n position: module\n kind: log\n args: { message: '\"done: \" + string(actions.build.result.written)' }\n when: actions.build.result.written == true\n```\n\nThe first Module kind is the side-effect-free `log` kind\n(`module-position.md`, \"Recommended first slice\"). A Module kind with an\nexternal side effect, and the Channel and Forge positions, are out of scope.\n\n## A result type is the kind's generated Result struct\n\nEach Module kind owns a hand-written `Args` and `Result` struct\n(`internal/domain/eda/module/log` is the worked example); `ModuleRegistry.Invoke`\nreturns the result as a flat `map[string]any`. `ModuleRegistry.DecodeResult`\n(`internal/domain/eda/module`) is the single site that converts that map into\nthe kind's `Result` struct before a later expression reads it, so the value the\nchecker validates and the value the run reads have one definition.\n\n`actions.\u003cid\u003e` is typed by the kind of the action that declares `\u003cid\u003e`:\n`{ result: \u003cKindResult\u003e }`. `event` stays `dyn` (J4,\n`playbook-expression-syntax.md`).\n\n## Reject-at-load, preserved\n\nRelaxing the boundary must not make a bad reference resolvable. The load fails,\nnaming the playbook and the offending action, for:\n\n- an `id` missing where a later action references it, duplicated, or outside\n its shape's grammar: a workflow action id keeps the shared\n stable-identifier grammar (J1, already enforced), while a module action id\n must be a lowercase CEL field name writable as `actions.\u003cid\u003e` (so a CEL\n keyword such as `in`, `true`, `false`, or `null` is rejected at load);\n- an `actions.\u003cid\u003e` read of an id no earlier action declares;\n- an `actions` read that is not statically resolvable to an id (any map index,\n literal or dynamic, `in`, a comprehension): the typed `actions` root is an\n object type, so only field selection `actions.\u003cid\u003e` resolves and the\n `actions[\"id\"]` spelling is not accepted;\n- a `result` field the referenced kind's `Result` does not define;\n- a reference to an earlier action that produces no result;\n- an unknown `position`, an unknown `kind`, or a `module` action with no `kind`;\n- a field belonging to the other shape (`kind` on a `workflow` action, or\n `workflow` on a `module` action);\n- an arg KEY the kind's `Args` schema does not define;\n- an arg value whose checked CEL type cannot fill its `Args` field (`'123'`\n for a string field). A `dyn` value, such as an `event` read, passes: its\n type is known only per event, and the module decode refuses a bad one at\n run.\n\nDeclaring the kind's `Result` schema to CEL at load makes a result-field typo a\nload failure rather than a dispatch failure. Matching arg keys against the\nkind's `Args` schema makes an arg-key typo or a wrongly typed value a load\nfailure.\n\n## The environment is per playbook\n\nResult typing depends on the ids a playbook declares and the kind each id runs,\nso the CEL environment is built from one playbook's action list rather than once\nper `Store`, and `expr.NewEnv` takes that list: each declared id becomes a field\nof the `actions` object typed by its kind's `Result`.\n\n## Running an action playbook is out of scope\n\nThis document makes an action playbook load and its references type-check.\nRunning one is designed in `docs/prds/action-playbook-run.md`.\n\n## Decisions\n\n**D1 — dispatch before `t2db.31`.** Superseded by\n`docs/prds/action-playbook-run.md`. Was: an action playbook loads and\nvalidates but is not routed; the definition-pin dispatch reports no match for it,\nand the daemon logs a visible warning naming it.\n\n**D2 — mixing positions.*Line truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/news-and-releases/",
|
||||
@@ -406,13 +406,25 @@
|
||||
"url": "/dev/prds/playbook-expression-syntax/",
|
||||
"title": "Playbook data-flow and condition syntax",
|
||||
"section": "PRDs",
|
||||
"body": "# Playbook data-flow and condition syntax\n\n**Status:** Approved\n**Date:** 2026-09-03\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-t2db.14` (expression environment).\nUnbuilt decisions: `archie-core-t2db.25` (J1 action ids),\n`archie-core-t2db.26` (J2 args as CEL),\n`archie-core-t2db.27` (Result schemas declared to CEL).\n\n**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression\nmechanism covers both an action's `when` condition and its `args` values.\n\n## The requirement, stated directly\n\nA playbook action is gated and parametrised by runtime data: (a) a condition\ndeciding whether the action runs; (b) a later action's `args` values referencing\nan earlier action's `Result` and the triggering event's fields. Both are **CEL\nexpressions**, evaluated against one evaluation context. There is no separate\ninterpolation syntax: CEL reads nested maps and fields natively, so `args`\nvalues that need data are written as CEL, and only plain literal args remain\nplain YAML scalars.\n\nBoth halves are read-only: expressions may read context values and compute a\nresult from them, never mutate, never call host functions, never perform I/O.\nCEL enforces this structurally (no side effects, linear evaluation when macros\nare bounded, cost-limited).\n\n```yaml\nactions:\n - id: build\n position: module\n kind: log\n args:\n message: '\"build finished\"' # literal\n when: 'event.label == \"bugfix\"' # gate on event field\n - position: module\n kind: log\n args:\n message: '\"done: \" + string(actions.build.result.written)'\n when: \"actions.build.result.written == true\"\n```\n\n## Trust boundary of the expression\n\nA playbook YAML is **operator-installed, in-process, daemon-privileged** -- the\nsame tier as `ModuleDir`/`PluginDir`/`SecretEngineDir` (module-position.md's\ntrust-boundary section): loaded from a configured directory at startup, never\nfrom a webhook body or task worktree. The schema-by-example flow uses live\nevents to _design against_; the saved playbook is operator-authored file content\nin an operator-configured location.\n\nTwo consequences:\n\n1. The expression string is **trusted**. Operator-authored config, not attacker\n input; no sandboxing against a hostile expression string is required. If a\n future playbook source is less trusted (auto-generated bindings from\n unauthenticated captures), that is the playbook-file trust tier moving -- the\n loader's acceptance check is the right place, not the evaluator.\n2. The **data the expression reads is NOT trusted** (decoded webhook body, prior\n results through the same pipeline). Robustness against hostile data is\n required: no panic on wrong-typed/missing fields, no unbounded cost on\n deep/cyclic maps, bounded evaluation time. CEL provides this structurally\n (cost limits, linear non-Turing-complete evaluation, error values instead of\n panics -- verified in the vetting record below).\n\n## The gate precedent is deliberately NOT copied\n\n`internal/gate/gateeval` evaluates arbitrary interpreted Go (`.archie/gate.go`)\nas the repo's conditional-logic precedent. That is the wrong precedent for\nplaybook conditions because this document's premise is: **YAML is pure\norchestration data; Yaegi is where imperative logic lives** (Problem section;\nmodule-position.md's \"no generic hook\" section). Yaegi snippets in YAML would\nmove imperative logic into the data layer -- the inversion this design exists to\nprevent -- and make playbooks un-lintable.\n\nThe line: conditions are **declarative predicates over runtime data, not\nprograms**. CEL is non-Turing-complete and side-effect-free, so it cannot become\na program; a condition that needs real logic gets a Module kind (Yaegi),\nreferenced by the action, not a stronger expression body.\n\n## Plugin-engine-rule interaction\n\nThe plugin engine rule forbids a generic `Module` interface with an `any`\npayload; it exists to keep static implementation contracts typed. Evaluating CEL\nagainst runtime `map[string]any` is **not the same problem**: the rule concerns\n_implementation contracts_ (static operations a capability family exposes), not\n_runtime data values_ (legitimately dynamic -- an event payload's shape is\nunknown until it arrives). The Module contract stays fully typed (generated\n`Args`/`Result` schemas); CEL is a read-only data reader under the playbook\nengine family, not a new capability contract. Stated once.\n\n## Minimum viable evaluation context (what expressions see)\n\nOne flat context, namespaced from the playbook run, built at dispatch time:\n\n| Name | Type | Source Line truncated
|
||||
"body": "# Playbook data-flow and condition syntax\n\n**Status:** Approved\n**Date:** 2026-09-03\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-t2db.14` (expression environment).\n\n**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression\nmechanism covers both an action's `when` condition and its `args` values.\n\n## The requirement, stated directly\n\nA playbook action is gated and parametrised by runtime data: (a) a condition\ndeciding whether the action runs; (b) a later action's `args` values referencing\nan earlier action's `Result` and the triggering event's fields. Both are **CEL\nexpressions**, evaluated against one evaluation context. There is no separate\ninterpolation syntax: CEL reads nested maps and fields natively, so `args`\nvalues that need data are written as CEL, and only plain literal args remain\nplain YAML scalars.\n\nBoth halves are read-only: expressions may read context values and compute a\nresult from them, never mutate, never call host functions, never perform I/O.\nCEL enforces this structurally (no side effects, linear evaluation when macros\nare bounded, cost-limited).\n\n```yaml\nactions:\n - id: build\n position: module\n kind: log\n args:\n message: '\"build finished\"' # literal\n when: 'event.label == \"bugfix\"' # gate on event field\n - position: module\n kind: log\n args:\n message: '\"done: \" + string(actions.build.result.written)'\n when: \"actions.build.result.written == true\"\n```\n\n## Trust boundary of the expression\n\nA playbook YAML is **operator-installed, in-process, daemon-privileged** -- the\nsame tier as `ModuleDir`/`PluginDir`/`SecretEngineDir` (module-position.md's\ntrust-boundary section): loaded from a configured directory at startup, never\nfrom a webhook body or task worktree. The schema-by-example flow uses live\nevents to _design against_; the saved playbook is operator-authored file content\nin an operator-configured location.\n\nTwo consequences:\n\n1. The expression string is **trusted**. Operator-authored config, not attacker\n input; no sandboxing against a hostile expression string is required. If a\n future playbook source is less trusted (auto-generated bindings from\n unauthenticated captures), that is the playbook-file trust tier moving -- the\n loader's acceptance check is the right place, not the evaluator.\n2. The **data the expression reads is NOT trusted** (decoded webhook body, prior\n results through the same pipeline). Robustness against hostile data is\n required: no panic on wrong-typed/missing fields, no unbounded cost on\n deep/cyclic maps, bounded evaluation time. CEL provides this structurally\n (cost limits, linear non-Turing-complete evaluation, error values instead of\n panics -- verified in the vetting record below).\n\n## The gate precedent is deliberately NOT copied\n\n`internal/gate/gateeval` evaluates arbitrary interpreted Go (`.archie/gate.go`)\nas the repo's conditional-logic precedent. That is the wrong precedent for\nplaybook conditions because this document's premise is: **YAML is pure\norchestration data; Yaegi is where imperative logic lives** (Problem section;\nmodule-position.md's \"no generic hook\" section). Yaegi snippets in YAML would\nmove imperative logic into the data layer -- the inversion this design exists to\nprevent -- and make playbooks un-lintable.\n\nThe line: conditions are **declarative predicates over runtime data, not\nprograms**. CEL is non-Turing-complete and side-effect-free, so it cannot become\na program; a condition that needs real logic gets a Module kind (Yaegi),\nreferenced by the action, not a stronger expression body.\n\n## Plugin-engine-rule interaction\n\nThe plugin engine rule forbids a generic `Module` interface with an `any`\npayload; it exists to keep static implementation contracts typed. Evaluating CEL\nagainst runtime `map[string]any` is **not the same problem**: the rule concerns\n_implementation contracts_ (static operations a capability family exposes), not\n_runtime data values_ (legitimately dynamic -- an event payload's shape is\nunknown until it arrives). The Module contract stays fully typed (generated\n`Args`/`Result` schemas); CEL is a read-only data reader under the playbook\nengine family, not a new capability contract. Stated once.\n\n## Minimum viable evaluation context (what expressions see)\n\nOne flat context, namespaced from the playbook run, built at dispatch time:\n\n| Name | Type | Source |\n| --------- | ------------------------ | ---------------------------------------------------------------------------------Line truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/plugin-settings-live/",
|
||||
"title": "Plugin settings reconcile live",
|
||||
"section": "PRDs",
|
||||
"body": "# Plugin settings reconcile live\n\n**Status:** Draft\n**Tracking:** GitHub [#1154](https://github.com/samcharles93/archie-core/issues/1154), epic [#998](https://github.com/samcharles93/archie-core/issues/998)\n**Authority:** `docs/prds/runtime-control-plane.md` decides each feature owns how\nits changes take effect; `docs/architecture/plugins-and-extensions.md` owns the\nplugin engine rule; `docs/prds/control-plane-apply-status.md` owns where apply\nreports go.\n\n## Outcome\n\nAn operator drops a plugin, module or secret-engine file into the configured\ndirectory while archied runs, and the daemon loads it without a restart. An\nedit to an installed file replaces what is running. Deleting an installed file\nnever stops running code; apply status says so. A stored change to\n`plugin-settings` applies live for additions instead of requiring a restart.\n\n## Triggers\n\n`plugin-settings` joins the kinds archied watches (`PluginSettingsKind` moves\nto the live list in `internal/app/archied/control_plane.go`). After the daemon\nlayers a new version it reconciles the three directories the document names.\nA second trigger needs no document change: a poller reconciles on an interval,\nso a file dropped into a directory loads on its own. The poll interval is the\napply-status restamp constant; no filesystem-watch dependency is added. The\npoller reads the current document's directories from the running config each\ntick, so a changed `plugin_dir`, `module_dir` or `secret_engine_dir` retargets\nreconciliation without a restart.\n\n## Rules\n\n- Adding and changing load; removing never unloads. Yaegi interpreters are not\n reclaimed while a caller still exists and there is no unload path; the\n running set can only grow.\n- Each file is loaded through a fresh interpreter (the boot pattern). A changed\n file therefore loads cleanly, and replacement switches consumers to the new\n interpretation; files are content-hashed so only new and changed files load.\n- A file that fails to load is skipped and reported; already-running code keeps\n running. This is the degrade-and-skip convention `plugin.LoadDir` and\n `Registry.LoadDir` already apply at boot. Boot keeps its reject-at-load rule\n for the module directory; a broken file at runtime skips without aborting.\n- A directory that does not exist reconciles to an empty set.\n\n## The three registries\n\n- **Secret engines** (`internal/secret`). Reconciliation re-runs\n `Registry.LoadDir` against the live directory. `Register` replaces by name,\n so an edited engine takes effect for the next resolution. A deleted engine\n stays registered: existing references keep resolving through it until\n restart, and the record says so. Engines are stateless resolvers, so a swap\n needs no lifecycle step.\n- **Modules** (`internal/domain/eda/module`): reconciliation re-runs\n `ModuleRegistry.Register` for the kind's file. The invoker maps\n last-write-wins, and playbook dispatch resolves through the registry at\n invoke time, so the swap reaches a running playbook store unchanged. A file\n deleted for a kind leaves the old invoker in place, running; reported.\n- **Plugins** (`internal/plugin`): `Host` grows running-state operations beside\n `Register`, `Start` and `Stop`. One adds a single module while the host is\n running: it is inserted in dependency order, started alone, and on failure\n rolled back alone, leaving every previously started module running. The\n other replaces a module whose manifest id matches: stop it, start the new\n one, and restart the old one when the new start fails. Replacing a module\n that another depends on is refused, not swapped. These operations obey the\n plugin engine rule: the host still invokes start and stop itself, isolates\n the failure to the one module, and keeps a rollback. A deleted file makes no\n host call: the module runs on and is reported.\n\n## `skills_dir`\n\n`skills_dir` is not part of this change. The daemon refuses a candidate\n`plugin-settings` document whose `skills_dir` differs from the value it is\nrunning, reporting that the field applies on restart, until a skill\nreconciliation exists to carry it. The refusal is the existing rejected-edit\napply-status shape: the store keeps the version, the document is not applied,\nlast-known-good keeps running.\n\n## Apply status\n\nOne record: `archied` on `plugin-settings`. The reconciliation outcome is\nreported against the version the process is running. The record's error field\ncarries the outstanding reconciliation problem: a deleted file still running, a\nfile that failed to load, or the directory failing to read. A reconciliation\nwith nothing outstanding clears the error. The version stays the version the\nprocess is running; an add that succeeded is live even while a removal is\noutstanding, and the error says so rather than showing the kind as\nrestart-required. archie-gateway keeps boot-time loading; its record simply\nstays on the vLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/pr-review-agent/",
|
||||
"title": "PR review agent",
|
||||
"section": "PRDs",
|
||||
"body": "# PR review agent\n\n**Status:** Approved\n**Authority:** `docs/architecture/agent-system.md`; supersedes the reviewer\nexecutor and blocking rule in `docs/architecture/adversarial-review.md`, and\nthe designs in `docs/prds/operator-pr-review.md` and\n`docs/prds/review-calibration.md`.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (every agent\ncall is a StepExecution), `docs/prds/external-agent-harness.md` (any agent\nstep may run on a harness), `docs/prds/inline-review.md` (posting),\n`docs/prds/pr-review-remediation.md` (consumes the findings).\n\n## Decision\n\nArchie ships a pull request reviewer as its first end-to-end product. It\nreviews any pull request on a watched repository, a PR an operator names, and\narchie's own PRs before they open.\n\nThe reviewer is a fixed pipeline, not a single agent. The review strategy is\ngenerated per PR: three lens agents propose review dimensions from the PR's\ncontent, one reviewer definition runs once per dimension, and later agents\nverify, challenge and combine the findings. Everything that can be computed\nis code: diff parsing, clustering, blast radius, evidence extraction,\nscoring, deduplication, line mapping and the review event.\n\nThe pipeline is **recall-first**. It posts every evidenced finding above the\nconfidence threshold, up to the comment cap. A precision gate exists as an\noperator dial and is off by default.\n\nAcceptance is measured, not asserted: see Verification.\n\n## Pipeline\n\nEach numbered phase is a stage of the `pr-review` workflow. Each agent call is\nan `agent` StepExecution; fan-out phases record one child per call.\n\n1. **Intake.** Code reads the PR metadata and diff statistics: type,\n complexity, languages, areas touched, risk signals. Depth comes from lines\n changed: under 100 is `quick` (at most 3 dimensions), under 500 is\n `standard` (6), otherwise `deep` (12). An operator may set depth\n explicitly.\n2. **Anatomy.** Code parses the diff into files and hunks, clusters related\n files, and computes the blast radius: unchanged files that import or are\n imported by changed files. One agent writes the PR narrative, risk\n surfaces, changes that do not fit the narrative, and gaps between the\n description and the diff.\n3. **Dimension selection.** Three lens agents run in parallel over the intake\n (including its machine-written score), anatomy and diff, each able to read\n the repository. A PR that intake scores as likely machine-written adds a\n hallucination-check dimension: claims, citations and behaviour the diff\n does not actually implement.\n - **behaviour:** where the old and new code diverge for any input; API\n contracts, concurrency and state, security, error handling, data flow;\n - **mechanics:** whether the code runs at the language and framework\n level; types, signatures and every caller, decorators and middleware,\n framework contracts, imports;\n - **fit:** patterns, complexity, abstraction, test adequacy, documentation\n that no longer matches, dependencies, migration completeness.\n\n Each lens returns review dimensions: a name, a reviewer prompt written for\n this PR, target files, context files and a priority. Code merges them,\n drops duplicates and keeps the highest-priority dimensions up to the\n depth's cap.\n4. **Review.** One reviewer definition runs once per dimension, in parallel,\n at most 8 at a time. Each reads its target and context files and may\n follow references. Each finding carries file, line range, severity\n (`critical`, `important`, `suggestion`, `nitpick`), title, body, an\n optional suggested fix, quoted evidence, confidence and tags. A reviewer\n that exhausts its turn cap or deadline before its terminal tool call\n records a distinct StepExecution status from a reviewer that completed\n and found nothing; synthesis and the posted review treat the two\n differently (the former is unreviewed, the latter is reviewed-clean).\n5. **Verification layer**, over the review findings:\n - code extracts an evidence package per finding: the code at the cited\n lines and snippets from its callers;\n - an evidence verifier checks high-priority findings against their\n packages;\n - an adversary confirms or challenges each finding, looks for issues every\n reviewer missed, and is more sceptical when intake scores the PR as\n likely machine-written;\n - a compound agent looks, within each cluster of confirmed findings, for\n defects that only exist in combination.\n6. **Coverage and consistency**, in parallel:\n - a coverage gate checks every cluster and high-exposure blast-radius file\n was reviewed, and runs gap reviewers for what was not, at most 2 rounds;\n - consistency verification lists every cross-location obligation the\n changed code creates (what must be true elsewhere for this line to be\n correct), then verifies each by reading both endLine truncated
|
||||
"body": "# PR review agent\n\n**Status:** Approved\n**Authority:** `docs/architecture/agent-system.md`; supersedes the reviewer\nexecutor and blocking rule in `docs/architecture/adversarial-review.md`, and\nthe designs in `docs/prds/operator-pr-review.md` and\n`docs/prds/review-calibration.md`.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (every agent\ncall is a StepExecution), `docs/prds/external-agent-harness.md` (any agent\nstep may run on a harness), `docs/prds/inline-review.md` (posting),\n`docs/prds/pr-review-remediation.md` (consumes the findings).\n\n## Decision\n\nArchie ships a pull request reviewer as its first end-to-end product. It\nreviews any pull request on a watched repository, a PR an operator names, and\narchie's own PRs before they open.\n\nThe reviewer is a fixed pipeline, not a single agent. The review strategy is\ngenerated per PR: three lens agents propose review dimensions from the PR's\ncontent, one reviewer definition runs once per dimension, and later agents\nverify, challenge and combine the findings. Everything that can be computed\nis code: diff parsing, clustering, blast radius, evidence extraction,\nscoring, deduplication, line mapping and the review event.\n\nThe pipeline is **recall-first**. It posts every evidenced finding above the\nconfidence threshold, up to the comment cap. A precision gate exists as an\noperator dial and is off by default.\n\nAcceptance is measured, not asserted: see Verification.\n\n## Pipeline\n\nEach numbered phase is a stage of the `pr-review` workflow. Each agent call is\nan `agent` StepExecution; fan-out phases record one child per call.\n\n1. **Intake.** Code reads the PR metadata and diff statistics: type,\n complexity, languages, areas touched, risk signals. Depth comes from lines\n changed: under 100 is `quick` (at most 3 dimensions), under 500 is\n `standard` (6), otherwise `deep` (12). An operator may set depth\n explicitly.\n2. **Anatomy.** Code parses the diff into files and hunks, clusters related\n files, and computes the blast radius: unchanged files that import or are\n imported by changed files. One agent writes the PR narrative, risk\n surfaces, changes that do not fit the narrative, and gaps between the\n description and the diff.\n3. **Dimension selection.** Three lens agents run in parallel over the intake\n (including its machine-written score), anatomy and diff, each able to read\n the repository. A PR that intake scores as likely machine-written adds a\n hallucination-check dimension: claims, citations and behaviour the diff\n does not actually implement.\n - **behaviour:** where the old and new code diverge for any input; API\n contracts, concurrency and state, security, error handling, data flow;\n - **mechanics:** whether the code runs at the language and framework\n level; types, signatures and every caller, decorators and middleware,\n framework contracts, imports;\n - **fit:** patterns, complexity, abstraction, test adequacy, documentation\n that no longer matches, dependencies, migration completeness.\n\n Each lens returns review dimensions: a name, a reviewer prompt written for\n this PR, target files, context files and a priority. Code merges them,\n drops duplicates and keeps the highest-priority dimensions up to the\n depth's cap.\n4. **Review.** One reviewer definition runs once per dimension, in parallel,\n at most 8 at a time. Each reads its target and context files and may\n follow references. Each finding carries file, line range, severity\n (`critical`, `important`, `suggestion`, `nitpick`), title, body, an\n optional suggested fix, quoted evidence, confidence and tags. A reviewer\n that exhausts its turn cap or deadline before its terminal tool call\n records a distinct StepExecution status from a reviewer that completed\n and found nothing; synthesis and the posted review treat the two\n differently (the former is unreviewed, the latter is reviewed-clean).\n5. **Verification layer**, over the review findings:\n - code extracts an evidence package per finding: the code at the cited\n lines and snippets from its callers;\n - an evidence verifier checks high-priority findings against their\n packages;\n - an adversary confirms or challenges each finding, looks for issues every\n reviewer missed, and is more sceptical when intake scores the PR as\n likely machine-written;\n - a compound agent looks, within each cluster of confirmed findings, for\n defects that only exist in combination.\n6. **Coverage and consistency**, in parallel:\n - a coverage gate checks every cluster and high-exposure blast-radius file\n was reviewed, and runs gap reviewers for what was not, at most 2 rounds;\n - consistency verification lists every cross-location obligation the\n changed code creates (what must be true elsewhere for this line to be\n correct), then verifies each by reading both endLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/pr-review-operator-response/",
|
||||
"title": "PR review: the operator's answer to the approval gate",
|
||||
"section": "PRDs",
|
||||
"body": "# PR review: the operator's answer to the approval gate\n\n**Status:** Draft\n**Authority:** `docs/prds/pr-review-agent.md` (\"Operator approval\") for the gate, and `docs/prds/state-store-contract.md` (rev. 2e) for every task field that crosses the State Store.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (a run cannot resume partway through its stage list).\n\n## Decision 1 — where the operator answers\n\nThe dashboard's `POST /api/tasks/{id}/action` (`internal/webui/api_tasks.go`, `handleTaskAction`) is the authoritative surface: the only one that resolves the actor from a credential and can carry a payload. Chat is a shortcut, not a second authority — `/approve` (`internal/gateway/gateway.go`, `Router.handleApprove`) answers the post decision with every offered finding selected, and gains no instruction or selection syntax.\n\n`taskstate.Action` gains `rereview`, and `taskstate.Actions` returns `approve`, `reject` and `rereview` for `WaitingHuman`. `approve` posts the review, `reject` discards it through the existing terminal decline, `rereview` reruns the review phases with instructions. The action body carries `instructions`, required by `rereview`, and `findings`, the finding keys `approve` posts; an absent selection means all of them.\n\nThe path is `handleTaskAction` → `applyOperatorTaskAction` → `Chat.Contract.ApplyOperatorTaskAction` → the daemon's `LocalChatAdapter` → `taskactions.Client` (NATS `archie.gateway.task-action`) → `taskactions.Service.Apply` → one guarded store write that records the decision and requeues. Chat's `/approve` path (`StoreTaskController.Approve` → `chatTaskControllerAdapter.ApproveChatTask`) must reach that same write through the same function, so the two surfaces cannot record different decisions for one operator intent.\n\nA requeue resumes the workflow the waiting state names, never a name the handler hardcodes. `taskactions.Service.apply` and `chatTaskControllerAdapter.ApproveChatTask` requeue every approval under `implement`, which is the feasibility handoff: an approval whose wait recorded `pr-review` would run the implement workflow. Feasibility must keep that handoff by naming `implement` on the task before it enters the wait, which is the invariant the handler needs — the workflow a task carries is the workflow it runs next. `Requeue` treats an empty workflow as \"keep the task's\".\n\n### The review the operator answers\n\nThe gate writes the review it is holding onto the task before it ends the run: one JSON column, `review_gate`, a sibling of `review_payload`, holding the scored findings, the head SHA, the pull request's identity and the workflow the wait resumes. The run's existing `Store.Update` persists it. The same column carries the answer back: the response path's guarded write fills in the outcome, the selection and the instructions, and clears the offer for a re-review.\n\nThe operator reads the offer through the existing task read — `taskView.Task` on `GET /api/tasks` and `GET /api/tasks/{id}/debug` — so no new read endpoint is needed.\n\nThis record is load-bearing because the run that computed the findings ends at the gate and the engine cannot resume it partway. Without it, \"post the selected findings\" cannot mean the findings the operator saw: the resumed run recomputes them with agent calls. An `approve` resume therefore posts the recorded review filtered by the selection, and the review the resumed phases recompute never reaches the output.\n\n## Decision 2 — the cap\n\nThe cap is a column: `rereview_rounds` on `tasks`, `bigint NOT NULL DEFAULT 0`, read by the response path and incremented in the same guarded write that requeues a re-review. A re-review at the cap is refused with the conflict sentinel the dashboard answers 409 with, and nothing is requeued. Two is a constant in the domain, not a config knob: the PRD fixes it.\n\n`Task.Attempt` cannot carry the count, because it counts every requeue. The two alternatives are worse.\n\n- The execution tree measures passes through the gate, not operator requests. A crash recovery (`RecoverStale`) or an operator retry re-runs the pipeline to the gate and would consume the cap; and a stage has no step read (`task.Store` carries none), so this route needs a new interface method anyway.\n- `Task.Notes` is the agent memory ledger, not scratch. `AgentStage.buildRequest` passes it into every generic agent request, `agentexec` renders it as a prompt section, and `persistAppendedNotes` appends to it. A counter parsed out of a field agents write to is the encoded-state defect this repository rejects, and instructions parked there would reach every unrelated workflow's agents.\n\n`RemediationRounds` is the precedent: one writer, one reader, one meaning, kept separate from `RetryCount` so unrelated retries cannot draw a budget down.\n\nThe write crosses the State Store. A goose migration adds the column; `internal/infrastructure/postgres/queriesLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/pr-review-remediation/",
|
||||
@@ -444,11 +456,17 @@
|
||||
"section": "PRDs",
|
||||
"body": "# Review calibration -- decision\n\n**Status:** Draft\n**Beads issue:** `archie-core-h019.5` (parent `archie-core-h019`)\n\n\u003e Superseded by `docs/prds/pr-review-agent.md`.\n\nDecides the half of calibration `docs/prds/adversarial-self-review.md` left\nopen: \"the adversarial-confirmation mechanism that keeps a finding from\nsurfacing on a single reviewer's say-so.\"\n\n## The problem, precisely\n\nThe shipped contract (`internal/domain/workflow/review_findings.go`, h019.2)\nalready makes only a **confirmed** error-level finding block\n(`Blocking() == Verdict == confirmed \u0026\u0026 Level == error`), and a zero-finding\nreview is already distinguishable from one that never ran (`ReviewStatus`).\nWhat is missing is the other half: a reviewer can report _nothing_\nand say only \"I found nothing\" in free text. The operator cannot tell \"I\nchecked the five things that matter and they are clean\" from \"I did not look.\"\nThat is the failure the epic names -- a findings surface that consumes attention\nwithout earning trust.\n\n## Decisions\n\n### 1. \"Checked and cleared\" becomes structured, not prose\n\n`ReviewReport` gains `Checked []ReviewCheck`:\n\n```go\ntype ReviewCheck struct {\n Property string // what was verified clean, e.g. \"nil-safety of Foo callers\"\n Evidence string // how it was verified, e.g. \"read all 4 callers; each nil-checks\"\n}\n```\n\n`Evidence` is required, for the same reason `ReviewFinding.FailureScenario` is:\na check without evidence is an assertion, and the whole point is to separate a\nverified property from an unexamined one. A zero-finding review is credible\nonly when it lists what it checked.\n\n### 2. Report checks through a typed tool, like findings\n\nThe reviewer calls a new capture tool `record_checked` (mirroring\n`record_finding`) once per property it verified clean. The same environmental\nargument as findings applies: the reviewer cannot end its turn having silently\nskipped the checklist. `ReviewReport.Summary` stays as the human overview.\n\n### 3. Style nits are out of contract\n\nA finding must describe wrong behaviour or a crash. Style, formatting, naming,\nand preference nits are explicitly out of contract -- they are the padding that\ntrains the operator to skim. Stated in the reviewer prompt and in the\n`ReviewFinding` type's own documentation. Not machine-enforced: deciding \"is\nthis a style nit\" from free text is unreliable, and a wrong rejection is worse\nthan a tolerated one. The category enum already has no style bucket; the rule\ncloses the gap that would otherwise let one hide under `other`.\n\n### 4. Surfacing\n\nThe PR-body findings section lists the cleared properties alongside findings,\nso \"ran and found nothing\" arrives with its evidence. (Park reasons keep\nrendering only blocking findings -- what a human needs to act on, not the full\nchecklist.)\n\n## What this does not decide\n\n- Adding a second, independent reviewer pass. The verdict distinction plus the\n recorded checks is the confirmation mechanism; a second model pass is a\n separate, heavier decision and nothing in `h019.5` requires it.\n- How many findings can survive before `MaxSteps` is exhausted (a budget\n question, already satisfied by the per-stage budget).\n\n## Packages this touches\n\n- `internal/domain/workflow` -- `ReviewCheck`, `ReviewReport.Checked`, validation.\n- `internal/app/agentworker` -- `record_checked` tool, prompt rules.\n- `internal/domain/workflow/review.go` -- render cleared properties on the PR body.\n"
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/review-settings-resource/",
|
||||
"title": "Review settings resource",
|
||||
"section": "PRDs",
|
||||
"body": "# Review settings resource\n\n**Status:** Draft\n**Beads issue:** `archie-core-0ry6`\n**Authority:** `docs/prds/runtime-control-plane.md` for the resource model,\n`docs/architecture/configuration.md` for ownership, and\n`docs/prds/pr-review-agent.md` for what the two dials do.\n\n## Decision\n\n`review.precision_gate` and `review.approve_before_post` are one control-plane\nresource, `review-settings`. The file's `[review]` section seeds it, the stored\ndocument outranks the file from then on, and the dashboard writes it through the\nsame settings machinery as every other resource.\n\nThe resource is deployment-wide, like every definition the State Store seeds\n(`storecontract.DefaultOrgID`): not per-repository, per-identity, or per-run.\n\n## Why its own kind\n\nThe two dials are pr-review policy — what a review may post, and whether a human\ndecides — not a limit on a task run. The document that carries task limits\nbelongs to that feature. `runtime-control-plane.md` splits settings by feature\nand refuses documents unrelated features share; `configuration.md` puts a typed\nruntime setting with the domain whose behaviour it describes. One document per\nfeature is the rule, not one document per settings page.\n\nA new kind also has no legacy documents. Its first value is the seed derived\nfrom the file, so \"the store outranks the file\" begins from an explicit copy of\nboth fields rather than from a document that predates them.\n\n## Document\n\nNew file `internal/app/controlplane/review_settings.go`:\n\n```go\nconst ReviewSettingsKind = \"review-settings\"\n\ntype reviewSettings struct {\n PrecisionGate bool `json:\"precision_gate\"`\n ApproveBeforePost bool `json:\"approve_before_post\"`\n}\n```\n\nIts `Definition` sets `ApplyMode: \"live\"`, `Document: reviewSettings{}`, and a\n`Seed` returning `reviewSettings{cfg.Review.PrecisionGate, cfg.Review.ApproveBeforePost}`,\nand a `Validate` wrapping `validateAs`: both fields are free booleans with no cross-field\nrule, so the validator is a named place to put one rather than an inline no-op\n(`validatePluginSettings` is the precedent). No `Normalize`, because the kind\nhas no second wire shape to converge.\n\n## Seeding\n\n`ImportConfig`/`seededValues` write the seed for a kind the store does not hold,\nso a deployment that never edited the dials runs the values in `[review]`. Once\na document is stored, the file key stops mattering, which is the rule for every\ndatabase-owned setting.\n\n## Apply\n\n`runtimeConfigFrom` in `internal/app/controlplane/runtime_config.go` layers the\nstored document over the file's `Review`, assigning `out.Review.PrecisionGate`\nand `out.Review.ApproveBeforePost` field by field. Field-by-field, not\n`out.Review = ...`: a field added to `config.Review` later keeps the file's value\nuntil the document carries it.\n\n`ReviewSettingsKind` joins `runtimeResourceKinds` in\n`internal/app/archied/control_plane.go`, so a new stored version re-runs the\nlayering and republishes through `config.Holder`. Live rather than\nrestart-required, because every consumer re-reads it: `Review` is on\n`reloadableFields` in `internal/app/archied/reload.go`, and the value reaches a\nrun through `Daemon.configFor` → `Config.ForTask` → `TaskConfig.ToConfig` →\n`TaskContext.Cfg`, which `stagePRPrecisionGate` and `stagePROperatorApproval`\nread. The next task run uses the new value; a run in flight keeps the value it\nstarted with.\n\n## Projection and page\n\n`BuildConfigView` gains a plain `Review` field carrying both values out of\n`cfg.Review`, the way `BudgetsView` carries the layered budgets, so the published\nconfig snapshot tells any reader what is in force. The Review page renders the\ntwo toggles from the control-plane store's draft for `review-settings` — the\nshape `SystemTasksPage.vue` uses for `workflow-execution-settings` — and shows\nthe projected value beside them; a Settings route plus a `PAGE_RESOURCES` entry\n(`ui/src/stores/control-plane.ts`) bind the page to the kind.\n\nNothing else re-advertises them. `configFieldDescriptors` and `ConfigView.Schema`\ngain no `review.*` row: the config catalog's generic renderer was deleted, so a\ndescriptor row adds a field nothing can edit, which is the shape `becb0b5e`\nremoved.\n\n## What the test proves\n\n`internal/app/controlplane/runtime_config_test.go` adds the round trip that\nbuilds the effective config the way boot does: seed the kind from a file config\nwith both dials set one way, store a document with both flipped, layer it\nthrough `runtimeConfigFrom`, and assert the effective `cfg.Review` equals the\nstored document. A second case covers the absent kind, where the file's values\nstay in effect. The assertion is on `cfg.Review` — the value `TaskContext.Cfg`\ncarries into the two stages — not on a struct the test set directly.\n`TestRuntimeResourceKindsApplyLive` gains `review-settings` as live. The\nvalidator needs no `validators_agree_test.goLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/runtime-control-plane/",
|
||||
"title": "Runtime control plane",
|
||||
"section": "PRDs",
|
||||
"body": "# Runtime control plane\n\n**Status:** Approved \n**Date:** 2026-09-20 \n**Tracking:** `archie-core-j28m`, `archie-core-1786637498420-327-bacadaae`\n\n## Outcome\n\nArchie can be managed from the Web UI or a messaging channel. Both use the same\nAPI.\n\n## Model\n\nEach feature owns its settings and commands. The database stores them. There is\nno global configuration document.\n\nThis includes:\n\n- workflows;\n- identities and bindings;\n- providers and models;\n- repository policies;\n- channels and schedules;\n- capture mappings and bindings;\n- tool settings;\n- prompts and personas;\n- plugin settings.\n\nPlugin code remains installed code. Only its settings are stored in the\ndatabase.\n\nEach feature defines its data, validation, permissions, editor fields, and how\nchanges take effect. Each feature also defines its allowed actions. For example,\nidentities can be created, suspended, and retired.\n\nRepository and identity settings remain separate. They are not stored as large\nrecords that unrelated features must share.\n\nThe State Store process hosts the control-plane API on its existing gRPC server.\nNo new process is added. Feature code validates changes. Storage code only\nwrites versions and audit records.\n\nEvery write records who made it, where it came from, a request ID, and the\nversion being edited. The API layer sets the actor; the request body cannot.\nThe server validates the change, then writes it with its audit record. It rejects\nedits to an old version.\n\n## API\n\nThe API has four operations:\n\n1. **Catalogue** tells clients what can be viewed or changed and how to edit it.\n2. **Query** reads data and versions without exposing secrets.\n3. **Command** asks the owning feature to make a change.\n4. **Watch** streams changes and errors to clients.\n\nComplex features can use custom screens. They still use the same commands.\n\nA dashboard session or channel sender must map to an Archie identity before it\ncan run a command. Messaging syntax maps to the same commands as the Web UI.\n\n`archie-ui` receives only an API client. It does not receive a database handle,\ndaemon pointer, global configuration object, or secret value.\n\nThe database stores the requested settings. Each process reports the version it\nis using or why it failed. The UI shows both.\n\nSome changes require a restart. The UI shows them as pending. When a live change\nreplaces a running component, Archie starts and checks the new one before\nswitching. If the check fails, the old one keeps running.\n\n## Bootstrap, migration, and recovery\n\nBootstrap contains only the database path, listen addresses, credentials, TLS,\nand encryption-key references. State Store and control-plane RPCs use the same\nendpoint.\n\nWhen upgraded, existing instances are migrated to the database. Existing\ndatabase values are not overwritten, with one exception: a resource whose\ndocument the build ships is refreshed when the seed itself wrote the stored\ncopy, because that copy is a previous release's document rather than an\noperator's. After migration, settings in TOML are ignored and cannot block\nState Store startup.\n\nThe Web UI remains available when `archied` or Gateway is down. It shows failed\nchanges and can correct or roll them back.\n\nOffline commands can back up, restore, validate, and roll back the database.\nRecovery does not require `archied` or the Web UI. The database stores secret\nreferences, not secret values.\n\n## Workflows\n\nUsers create and edit workflows in the Web UI. Workflows are stored as versioned\nYAML. Built-in workflows can be overridden and restored.\n\nA workflow is a list of steps. Each step has a known type and its own settings.\nPlugins can add step types. Each run keeps the workflow version it started with.\n\nYaegi, `.archie/stages/*.go`, and `.archie/gate.go` will be removed. Their useful\nbehaviour must first be available as normal workflow steps.\n\n## First implementation\n\nThe first implementation makes task limits editable: maximum model steps,\nmaximum runtime, and maximum consecutive gate failures.\n\nIt includes:\n\n- validated task-limit fields;\n- version history and audit records;\n- the four API operations;\n- authenticated Web UI and messaging access;\n- live updates and apply status;\n- a Pinia-backed editor in the Web UI;\n- a messaging command using the same API;\n- migration of existing `[budgets]` values;\n- an end-to-end restart test.\n\nRunning tasks keep the limits they started with. New tasks use the latest applied\nversion. This replaces the old settings overlay instead of extending it.\n\n## Not included in the first implementation\n\n- Moving bootstrap or secret values into normal settings.\n- A generic JSON editor.\n- Rebuilding every settings screen at once.\n- Changing the limits of a running task.\n"
|
||||
"body": "# Runtime control plane\n\n**Status:** Approved \n**Date:** 2026-09-20 \n**Tracking:** `archie-core-j28m`, `archie-core-1786637498420-327-bacadaae`\n\n## Outcome\n\nArchie can be managed from the Web UI or a messaging channel. Both use the same\nAPI.\n\n## Model\n\nEach feature owns its settings and commands. The database stores them. There is\nno global configuration document.\n\nThis includes:\n\n- workflows;\n- identities and bindings;\n- providers and models;\n- repository policies;\n- channels and schedules;\n- capture mappings and bindings;\n- tool settings;\n- prompts and personas;\n- plugin settings.\n\nPlugin code remains installed code. Only its settings are stored in the\ndatabase.\n\nEach feature defines its data, validation, permissions, editor fields, and how\nchanges take effect. Each feature also defines its allowed actions. For example,\nidentities can be created, suspended, and retired.\n\nRepository and identity settings remain separate. They are not stored as large\nrecords that unrelated features must share.\n\nThe State Store process hosts the control-plane API on its existing gRPC server.\nNo new process is added. Feature code validates changes. Storage code only\nwrites versions and audit records.\n\nEvery write records who made it, where it came from, a request ID, and the\nversion being edited. The API layer sets the actor; the request body cannot.\nThe server validates the change, then writes it with its audit record. It rejects\nedits to an old version.\n\nThe request ID is the write's idempotency key, and it is scoped per org and\nper kind, never global. The key is `(org_id, kind, request_id)` — the unique\nindex `idx_resource_history_org_kind_request` (migration `0021_resources_per_org.sql`)\n— so replaying a request ID is the same org's prior write of the same kind, and\na request ID reused across orgs or across kinds is a separate write. This is the\nsame scoping as the task idempotency key `org/identity/owner/repo/number`\n(`docs/prds/orgs-and-access.md`).\n\n## API\n\nThe API has four operations:\n\n1. **Catalogue** tells clients what can be viewed or changed and how to edit it.\n2. **Query** reads data and versions without exposing secrets.\n3. **Command** asks the owning feature to make a change.\n4. **Watch** streams changes and errors to clients.\n\nComplex features can use custom screens. They still use the same commands.\n\nA dashboard session or channel sender must map to an Archie identity before it\ncan run a command. Messaging syntax maps to the same commands as the Web UI.\n\n`archie-ui` receives only an API client. It does not receive a database handle,\ndaemon pointer, global configuration object, or secret value.\n\nThe database stores the requested settings. Each process reports the version it\nis using or why it failed. The UI shows both.\n\nSome changes require a restart. The UI shows them as pending. When a live change\nreplaces a running component, Archie starts and checks the new one before\nswitching. If the check fails, the old one keeps running.\n\n## Bootstrap, migration, and recovery\n\nBootstrap contains only the database path, listen addresses, credentials, TLS,\nand encryption-key references. State Store and control-plane RPCs use the same\nendpoint.\n\nWhen upgraded, existing instances are migrated to the database. Existing\ndatabase values are not overwritten, with one exception: a resource whose\ndocument the build ships is refreshed when the seed itself wrote the stored\ncopy, because that copy is a previous release's document rather than an\noperator's. After migration, settings in TOML are ignored and cannot block\nState Store startup.\n\nThe Web UI remains available when `archied` or Gateway is down. It shows failed\nchanges and can correct or roll them back.\n\nOffline commands can back up, restore, validate, and roll back the database.\nRecovery does not require `archied` or the Web UI. The database stores secret\nreferences, not secret values.\n\n## Workflows\n\nUsers create and edit workflows in the Web UI. Workflows are stored as versioned\nYAML. Built-in workflows can be overridden and restored.\n\nA workflow is a list of steps. Each step has a known type and its own settings.\nPlugins can add step types. Each run keeps the workflow version it started with.\n\nYaegi, `.archie/stages/*.go`, and `.archie/gate.go` will be removed. Their useful\nbehaviour must first be available as normal workflow steps.\n\n## First implementation\n\nThe first implementation makes task limits editable: maximum model steps,\nmaximum runtime, and maximum consecutive gate failures.\n\nIt includes:\n\n- validated task-limit fields;\n- version history and audit records;\n- the four API operations;\n- authenticated Web UI and messaging access;\n- live updates and apply status;\n- a Pinia-backed editor in the Web UI;\n- a messaging command using the same API;\n- migration of existing `[budgets]` values;\n- an end-to-end restart test.\n\nRuLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/self-update-transaction/",
|
||||
@@ -480,6 +498,12 @@
|
||||
"section": "PRDs",
|
||||
"body": "# Skill curator (reference implementation) -- decision\n\n**Status:** Approved\n**Date:** 2026-09-02\n**Beads issue:** `archie-core-1786637499206`\n**Parent epic:** `archie-core-1786637500725` (curator engine)\n\n## Decision\n\nThe skill curator's v1 scope is **mechanical validation and safe\nnormalisation only** -- it never rewrites skill prose, never deletes a\nskill, and never calls the model. Each pass:\n\n1. Lists every skill under one configured root.\n2. Reads each `SKILL.md` and attempts to parse its frontmatter\n (`internal/skill.Parse`).\n3. Records an `Action` for any structural problem: a parse failure, or a\n missing `name`/`description` in the frontmatter that parsed.\n4. Normalises trivial whitespace (trailing whitespace per line, exactly\n one trailing newline) and writes back **only** when normalisation\n actually changes the bytes, recording an `Action` for that write.\n\nNothing here needs judgement. \"Review, prune, and improve\" -- the epic's\nframing -- is real for v1 but deliberately narrow: review is structural\nvalidation, improve is whitespace normalisation, and prune is _reporting_\na problem, never removing a skill. `Manifest.Skills = true` still marks\nthis curator agentic (existing `agentic()` rule), so it receives model\naccess from the registrar -- unused but nothing about the contract\nor the runtime needs to change to let a later version reason about\ncontent quality using it.\n\n## Why this scope, not more\n\nAn autonomous process that rewrites or deletes a user's skill content\nis a real risk with no undo path (no draft/approval flow for\ncurator writes exists yet, unlike the workflow engine's PR-based review).\nThe epic's own stated purpose for this issue is proving the curator\ncontract holds for a second, structurally different consumer than the\nsession-memory curator (files-as-skills vs. observations-as-memory), not\nsolving \"what makes a skill good.\" Mechanical validation is real,\nuseful, safe to run unattended, and fully exercises\nList/Read/Write -- every method `curator.SkillStore` declares except\n`Delete`, which this curator's Pass never has a safe reason to call in\nv1 (kept implemented on the store adapter regardless, since the\ninterface requires it and a future consumer may).\n\n## Why `internal/skill` isn't `curator.SkillStore` directly\n\n`internal/skill.Discover` parses every `SKILL.md` under a root and\nreturns an error for the **whole** call if any one of them fails to\nparse -- the read side was built for \"load the catalogue to start the\ndaemon,\" where one bad skill failing the whole boot is arguably correct,\nnot for \"review each skill and report what's wrong with it\nindividually.\" A per-skill parse failure must become one `Action`, not\nabort the pass before any skill is reviewed. So\n`internal/infrastructure/skillcurator` gets its own thin store adapter\nthat lists raw directory names and reads/writes raw `SKILL.md` bytes\nper name; the skill curator itself calls `internal/skill.Parse` on each\none, isolated per skill.\n\n## Root scope\n\nOne root only, resolved the same way `loadWorkflows` already resolves\n`skillsBase` (`cfg.SkillsDir`, falling back to `cfg.WorkDir`) -- not\nevery root `skill.CatalogRoots`/`DefaultRoots` would read for a model\n(which layers a shared dir over the per-profile one). Write and Delete\nneed one unambiguous target; a curator that could write into a shared,\nmulti-tenant skills directory is a bigger decision than this issue\ncovers. The model still sees the full multi-root catalogue through the\nexisting read path -- this curator only maintains the local one.\n\n## Shape\n\n### `internal/infrastructure/skillcurator/store.go`\n\n```go\n// Store implements curator.SkillStore over one root directory's\n// .agents/skills/*/SKILL.md files, tolerating a per-skill parse failure\n// as the curator's problem to report rather than this store's problem\n// to abort on.\ntype Store struct { root string }\n\nfunc NewStore(root string) *Store\n\nfunc (s *Store) List(ctx context.Context) ([]curator.SkillRef, error)\nfunc (s *Store) Read(ctx context.Context, name string) (curator.Skill, error)\nfunc (s *Store) Write(ctx context.Context, sk curator.Skill) error\nfunc (s *Store) Delete(ctx context.Context, name string) error\n```\n\n`Read` best-effort parses frontmatter for `Skill.Description` (empty on\nparse failure -- not this method's job to report that; `Pass` does, via\nits own `skill.Parse` call on `Skill.Content`). `Write` rejects a name\nthat does not already exist as a directory (this store curates existing\nskills; creating a new one is out of scope). `Delete` removes the whole\nskill directory, matching \"delete a skill,\" not just `SKILL.md`.\n\n### `internal/infrastructure/skillcurator/skillcurator.go`\n\n```go\n// Curator implements domain/curator.CuratorEngine. See\n// docs/prds/skill-curator.md for what a pass does and why.\ntype Curator struct { /* unexported: registrar view, interval *Line truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/soul/",
|
||||
"title": "SOUL: an agent's user-authored personality",
|
||||
"section": "PRDs",
|
||||
"body": "# SOUL: an agent's user-authored personality\n\n**Status:** Draft\n**Authority:** `docs/architecture/identity.md`, “Three-layer identity model”;\n`docs/prds/orgs-and-access.md` for identity and org ownership.\n\n## Decision\n\nSOUL supplies an agent's name, register and warmth. It belongs to the agent's\nimmutable `IdentityID`, not its session, model, installation or container\nprofile. Rename preserves it; changing a profile does not change who acts.\nSOUL never changes attribution, credentials, grants or identity lifecycle.\n`config.AgentProfile` remains an execution environment, not another identity.\n\nUse the existing three layers: invariants, user-authored SOUL, and curator-derived\nrepresentation. Curators may propose SOUL changes but never apply them.\n\n## Files and ownership\n\nThe config directory is `${XDG_CONFIG_HOME:-~/.config}/archie`, or the directory\nof an explicitly selected config file. The default/root agent uses `SOUL.md`\ndirectly there; additional agents use `agents/\u003cidentity-id\u003e/SOUL.md`. The root is\nthe identity selected by application composition, never the initiating user or\na session's personality selection. No file creates or selects an identity.\n\nThe common case keeps a human-friendly filename; additional agents get separate\nasset directories, following `.agents/skills/\u003cname\u003e/SKILL.md` without confusing\nthese directories with `[containers.profiles.\u003cname\u003e]`. Reject display-name or\nslug directories: rename and same-name agents across orgs would require another\nname-resolution rule. Immutable IDs avoid both problems without an org directory.\n\nWhen a file-owned agent's file is absent, create the shipped default with\nexclusive creation. Never overwrite an existing file, including an empty one.\nA creation failure still permits the turn using the embedded shipped default.\nShip that document independently of `agent.ShippedPersonas`, so deleting personas\ncannot delete the fallback.\n\nResolve once at the start of every new generation:\n\n1. A stored SOUL entry for the acting identity is authoritative.\n2. Without an entry, read that identity's file again.\n3. An absent, empty, unreadable or invalid selected source uses the shipped default.\n\nThe first explicit dashboard/API save takes ownership permanently. Thereafter\nfile edits are ignored: emit one ownership notice and show the owning stored\nversion in the dashboard. Restoring the shipped default writes a stored value;\nit does not remove ownership. Refuse writes that remove an owned entry.\n\nThis ownership boundary is the design's one stateful rule. Files seed the\neditable value, not an automatic durable import. Reject seed-once import because\nit breaks next-message file editing; reject file-always-wins because it makes\ndashboard saves ineffective. Neither a restart nor an upgrade overwrites an\nowned value, following the precedence of `controlplane.AgentProfileKind` without\ncopying its startup import timing.\n\n## Prompt and failure contract\n\nKeep `archie.md.tpl`'s XML-escaped\n`\u003csoul purpose=\"identity_and_style\" trust=\"data\"\u003e` block before\n`\u003cinstruction_precedence\u003e`. SOUL is style reference data; the invariant sections,\ntool inventory and runtime metadata remain authoritative and uneditable.\nPreserve `TestBuildSystemPromptSoulCannotOverrideInvariants`.\n\nAccept UTF-8 plain Markdown with at most 8 KiB in its XML-escaped body. This bounds\npersonality to a small prompt contribution; count the complete rendered prompt\nin the turn's context budget and never truncate invariant sections. Bound file\nreads before decoding; reject oversize input rather than truncating instructions.\nFile and stored writes use the same validator; an invalid save leaves the stored\nversion unchanged. The shipped default must pass that validator.\n\nRead and creation errors never fail a turn. Log once per identity/source failure\nepisode, retry on the next turn, and allow a new warning after recovery. If the\nstore cannot establish whether an owned entry exists, use the shipped default,\nnot a potentially shadowed file. Do not log SOUL text. Each turn keeps its resolved\nsnapshot; edits affect the next generation without restart or SIGHUP.\n\n## Cross-layer delivery\n\n- `internal/domain/agent` owns SOUL content validation and the identity-keyed read\n contract; `internal/domain/identity` remains the actor/lifecycle authority.\n- The configuration infrastructure resolves the config directory and handles\n file discovery, exclusive default creation and bounded per-turn reads.\n Application composition resolves the acting `IdentityID` through the existing\n identity repository/bindings and supplies it to the gateway.\n- Register `SoulKind` (`agent-souls`) in the control plane with a live descriptor\n and an org-scoped document mapping identity IDs to `{text}`. Persist it through\n the State Store's existing resource/history contracts, not a new table,\nLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/state-store-contract/",
|
||||
"title": "State Store — contract boundary \u0026 transport (ratification)",
|
||||
@@ -534,11 +558,23 @@
|
||||
"section": "PRDs",
|
||||
"body": "# Webhook intake security -- decision\n\n**Status:** Approved\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-t2db.5`, blocks `archie-core-t2db.4`\n\n## Why this is decided ahead of the code it constrains\n\n`archie-core-t2db` (no-code playbooks) and `archie-core-7d5u` (typed event\nsources) both land a public intake surface whose eventual output is an agent\nwith tool access and a repo. Neither capture (`t2db.1`) nor the intake family\n(`7d5u.3`) has shipped yet, so this decision exists to be a contract the two\nepics build against, not a retrofit once the surface exists. A webhook URL\nthat can start an agent is remote code execution with extra steps if any of\nthe five points below is missing on day one.\n\n## 1. Per-source authentication\n\nReuse the per-route HMAC-SHA256 scheme already in\n`internal/channels/webhook/webhook.go` (`validateHMAC`), unchanged: a source\nregisters with a shared secret; the signature is checked against the raw\nbody. Do not invent a second scheme for intake.\n\n**Capture and triggering split at the signature, not at a separate flag.** An\nevent with a missing or invalid signature is still captured (visible in the\ninspector, marked unauthenticated) so an operator can diagnose a\nmisconfigured sender -- but an unauthenticated event can never be matched\nagainst a binding. Authentication is what promotes an event from \"something\narrived\" to \"something that can run a workflow.\"\n\n## 2. Human approval before a binding arms\n\nA binding has three states: `draft -\u003e pending_approval -\u003e armed`. Only\n`armed` bindings are evaluated against incoming events. Creating or editing a\nbinding's matcher, mapping, or target workflow always sets it to\n`pending_approval`, including edits to an already-armed binding -- an edit is\nnot silently re-armed, or the approval gate is decorative. Approval requires\nan authenticated dashboard session; there is no unattended approval path.\n\nModel: `internal/channels/telegram`'s dangerous-command approval flow\n(`dangerousAction` / `pendingApproval` in `dangerous.go` / `approval.go`) is\nthe existing precedent for \"a human decision gates code from running\" in this\ncodebase -- follow its shape rather than designing a new approval primitive.\n\n## 3. Rate and volume limits, per source\n\nA token-bucket limiter keyed by source ID, not global -- one noisy or hostile\nsender must not exhaust another source's capture budget. Precedent for\nper-key volume capping: `internal/pairing/store.go`'s `MaxPendingPerPlatform`.\n\nOn limit exceeded, the request is rejected (HTTP 429), not silently dropped.\nA silently dropped event is indistinguishable from a sender that never fired\n-- the same invisible-failure shape the update-verification work\n(`archie-core-522`) exists to prevent. The rejection itself is observable in\nthe inspector as a distinct outcome from \"not received.\"\n\n## 4. Blast radius bounded by code, not by prompt wording\n\nA playbook-triggered task runs through exactly the same enforcement as every\nother task: the `[[repos.gate]]` quality gate, `diff_cap_lines`, the\nsandboxed per-task container (`internal/container`), worktree isolation, and\n\"the model never runs git.\" No new relaxation is introduced for\nplaybook-triggered work, and none is needed -- the existing constraints are\nthe control.\n\nMapped payload fields are treated as ordinary user-controlled text inside an\nagent prompt, exactly like an inbound chat message never as trusted\ninstructions. \"The agent should ignore instructions in the payload\" is not a\ncontrol and must not appear as one anywhere in this feature -- per the\nenvironmental-enforcement principle (`docs/architecture/agent-system.md`,\n\"Existing invariants to preserve\"), the boundary is the gate and the sandbox,\nnot a sentence in the system prompt.\n\n## 5. Retention and redaction of captured payloads\n\n**Retention:** captured events (bound or not) are purged after 7 days,\nconfigurable per deployment. Unbound events exist to be mapped against; nothing\njustifies keeping them indefinitely once that window has passed, and an\nunbounded capture store is the same disk-exhaustion risk item 3 addresses at\nthe request-rate level.\n\n**Redaction:** before a captured payload is persisted, apply the same\nkey-name heuristic `internal/gateway/stream.go`'s `sensitiveParameterKey`\nalready uses for tool-call parameters (matches `token`, `secret`, `password`,\n`passwd`, `api_key`, `apikey`, `authorization`, `credential`, `cookie`,\n`private_key` against normalised JSON keys) and replace matching values with\nthe same `[redacted]` marker. This is best-effort, not a guarantee -- a\nsender that names a secret field something the heuristic misses will still\nhave it captured. Document that limitation where captured payloads are\ndisplayed, rather than implying the redaction is complete.\n\n**Visibility:** captured payloads, redacted or not, are visible only to an\nauthenticated dashboarLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/workflow-call-outputs/",
|
||||
"title": "Declared outputs of a workflow run",
|
||||
"section": "PRDs",
|
||||
"body": "# Declared outputs of a workflow run\n\n**Status:** Draft\n**Date:** 2026-09-30\n**Bead:** `archie-core-t2db.43`\n**Authority:** `docs/prds/event-automation.md` for what a declared output is,\n`docs/prds/workflow-calls.md` for the call and its grants,\n`docs/prds/state-store-contract.md` for the State Store contract the wire\nchange must satisfy.\n**Supersedes:** `docs/prds/workflow-calls.md`, \"Waiting on the callee\": the\ncallee's transition detail is the human summary, not the stand-in for\nstructured outputs.\n\nA workflow declares named, typed outputs. A run writes each one once. A\n`wait: true` caller receives them, and may publish one as its own.\n\n## Where outputs are declared\n\n`outputs:` on the workflow definition, beside `inputs:`, decoded into\n`task.WorkflowInterface.Outputs map[string]OutputSpec` where\n`OutputSpec{Type, Required}`:\n\n```yaml\nid: firewall-contain\noutputs:\n contained: { type: bool, required: true }\n summary: { type: string }\n```\n\nNames and types are the input grammar: the same identifier shape, the same\ntype set, and the same check in `WorkflowInterface.Validate`, which both\n`ParseDefinition` and `ParseWorkflowInterface` call.\n\n**Forced by:** `ParseDefinition` decodes with `KnownFields(true)`, so a key no\ndecoded struct declares is refused — the declaration must live on\n`WorkflowInterface` for an author to write it at all, and that is where\n`docs/development/workflows.md` puts every field a process without the engine\nmust read. The type set is the shared value vocabulary: `TypeAccepts` and\n`ValueType` already check a mapped parameter and an input with it. `required`\ndefaults to false, and the generated YAML for a builtin that declares no\noutputs is unchanged, because a definition's digest pins a run.\n\n**Rejected:** outputs per step, or per call step, because a workflow returns a\nresult as itself rather than as one of its stages. Also `task.Definition`\ngaining outputs: no reader needs them — a binding offers inputs, and a call's\noutputs assignment is checked against the callee's YAML in the same\ncollection — and a declared field with no reader parses and does nothing.\n\n## How a run writes one\n\nEach declared output is offered to the run's agent stages as a capture tool\nnamed after the output, whose arguments are `{\"value\": ...}`; the accepted\ncall's `value` member is the output's value. The engine holds the accepted\nvalues on the task context and writes them into its own task row in the\n`Store.Update` call the run already makes. A deterministic stage may set the\nsame values in Go, and the run's finish check validates them identically.\n\n**Forced by:** a run's only structured results are capture calls\n(`agentexec.CaptureTool`, `Result.Captures`), and the workflow interface\nalready declares when a stage needs them (`task.WorkflowNeeds.Captures`).\nDeriving the tools from the declaration means an output cannot be written\nunder a name nobody declared, and it keeps one route to structured agent\noutput rather than adding a second beside the capture path. The row write is\nthe run's existing write: the task row is where a run's own state lives, and\nthe container already holds `Update` on its own row.\n\nThe tool is offered with `MaxCalls: 1`, and a stage is not offered an output\nthe run has written, so a run writes each output once: a repeat within a stage\nis answered with the tool's rejection and the agent carries on, and a later\nstage cannot overwrite a value. A stage's own capture tool may not share a\nname with a declared output, because the value would otherwise be ambiguous.\n\n**Rejected:** a `workflow.output` step type, because assigning the run's\nresult from a step needs a grammar for \"the previous stage's result\" that does\nnot exist, and a step that reaches the agent would be the second path\n`docs/development/workflows.md` forbids. Also a write RPC of its own: the\ncontainer's `Update` already reaches its own row, and a second write verb for\none column is a second thing to keep in step.\n\n## How a caller receives one\n\nOn a successful `wait: true` call the caller's stage holds the callee's\noutputs as a `map[string]any`. A call step's `outputs:` setting publishes one\nas one of the caller's own outputs:\n\n```yaml\nsteps:\n - type: workflow.call\n workflow: firewall-contain\n wait: true\n inputs: { src_ip: inputs.src_ip }\n outputs: { contained: outputs.contained }\n```\n\nThe key names the callee's declared name; the value names one of the caller's\nown with the prefix the call's `inputs:` already uses. A published value joins\nthe caller's output set for its run, so a caller returns its callee's result to\nits own caller, and a run may publish a callee's output and write its own\nothers.\n\nLifetime is the attempt. The run starts with an empty output set, every later\nstage of that attempt sees the values, and the finish write replaces the row's\nset wholesale — so nothing an attemLine truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/workflow-calls/",
|
||||
"title": "Workflows calling workflows",
|
||||
"section": "PRDs",
|
||||
"body": "# Workflows calling workflows\n\n**Status:** Draft\n**Date:** 2026-09-26\n**Bead:** `archie-core-t2db.41`\n**Extends:** `docs/prds/event-automation.md` (workflow/binding/profile model), `docs/prds/state-store-contract.md` (two new RPCs), `docs/prds/execution-tree-state-machine.md` (call kind)\n\n## The call, end to end\n\n```\ncaller run (in agent container)\n workflow.call step ──StartCall──▶ State Store EnqueueCallTask(caller, workflow, inputs)\n │ callee task row: org/identity/owner/repo\n │ derived from the caller row server-side\n ▼\n callee task (queued, org of the caller)\n │ daemon claims, pins the callee's OWN\n │ definition → its OWN agent profile\n ▼\n callee run (its own container, own tree)\n wait: true ──poll──▶ State Store WorkflowCallStatus(caller, callee)\n wait: false ──continue\n```\n\nA `workflow.call` step starts the callee as a first-class task and, with\n`wait: true`, waits for its terminal state. The callee is never run \"inside\"\nthe caller: it gets its own task row, its own pinned definition, its own\nprofile, its own container, and its own outcome on its own run\n(`execution-tree-state-machine.md`).\n\n## Save-time checks (the collection is the authority)\n\nAll semantic checks run in `workflow.ValidateDefinitionCollection`, the gate\nevery save and load of the `workflow-definitions` resource passes through.\nA single-definition parse cannot see callees, so it validates shape only.\n\n- **Callee exists.** A call to a workflow outside the collection is refused.\n- **Inputs checked.** A call value is a literal or a reference\n (`inputs.\u003cname\u003e`) to the calling workflow's declared input. References must\n name a declared input; the resolved type must satisfy the callee's declared\n type; unknown callee inputs are refused; a required callee input must be\n assigned.\n- **Self-calls refused.** The call graph over the collection is walked from\n every workflow; a workflow that reaches itself, directly or through other\n workflows, is refused with the cycle path. (Cycles can otherwise deadlock:\n `wait: true` callers hold their containers.)\n- **Repository.** `workflow.call` needs no repository, so it is legal in a\n `repository: none` workflow: the callee inherits the caller's task record\n (owner/repo/identity/org), and the callee's own `repository` mode decides\n whether its run clones.\n- **Org enablement** (bead `archie-core-t2db.42`): the single\n instance-wide collection the current model serves is what \"enabled for\n the caller's org\" resolves against — an undefined callee parks. The\n callee's org is derived from the caller's row, so a callee is never\n visible outside its caller's org. The per-org enable/disable catalogue\n lands with `archie-core-t2db.42` and becomes the one source of this\n predicate.\n\n## Runtime\n\n**Depth limit.** `workflow.MaxCallDepth` (5). The calling stage refuses a call\nthat would exceed it; the store re-checks on insert (callee depth = caller\ndepth + 1), so no path can enqueue past the limit.\n\n**Starting the callee.** The caller runs inside a container whose task-scoped\nState Store grant authorizes only `Update`/`Transition`/`InsertEvent` on its\nown task — it can neither enqueue a task nor read another one. One new\ncontract RPC starts the callee:\n\n- `EnqueueCallTask(caller_task_id, workflow, inputs_json) → task`\n — the store reads the caller row, refuses a caller that is not `running`,\n refuses past the depth limit, and inserts the callee row deriving org,\n workspace, identity, owner, repo and a fresh synthetic issue number from\n the caller (the same org COALESCE `EnqueueIssue`/`InsertChatTask` use). The\n callee row carries `call_parent_task_id` and `call_depth`.\n\n**Waiting on the callee.** One new read RPC returns a call's callee:\n\n- `WorkflowCallStatus(caller_task_id, call_task_id) → (status, detail)` — the\n handler verifies `call_parent_task_id` matches, so a caller reads only its\n own callees, and returns the callee's status plus its latest transition\n detail (the callee's summary — the wait:true \"outputs\" until a structured\n outputs model lands).\n\nBoth RPCs are admitted to task-scoped grants on `caller_task_id == granted\ntask ID` (`authorizesTaskScopedCall` gains two request-shape cases). This is\nthe one sanctioned widening of a task grant: the callee is the caller's child,\nand starting it is the mechanism by which one event becomes several runs. The\nwidenings stay request-shape checks; the parent-child authorization is a\nhandler-side row check, so the interceptor never reads the database.\n\n**Schema.** `tasks` gains Line truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/workflow-step-vocabulary/",
|
||||
"title": "Workflow step vocabulary: what a step type is and how binaries agree on it",
|
||||
"section": "PRDs",
|
||||
"body": "# Workflow step vocabulary: what a step type is and how binaries agree on it\n\n**Status:** Draft\n**Date:** 2026-09-29\n**Beads:** `archie-core-m7oc` (agreement between separately deployed binaries), `archie-core-ze7y` (the deleted `.archie/stages/*.go` equivalent)\n**Extends:** `docs/architecture/plugins-and-extensions.md` (plugin engine rule), `docs/prds/event-automation.md` (the workflow model step types serve)\n\n**Authority:** `docs/architecture/plugins-and-extensions.md` — the workflow step vocabulary is a Workflow capability family, and its extension point follows the plugin engine rule.\n\n## Decision\n\nA workflow step type is a named Go factory: the word a YAML workflow writes in\na step's `type` field, plus the factory that builds its stage. The name is a\nstable identifier; the identity also declares what plugin family claims it,\nso a contribution that shadows a shipped stage is refused instead of\noverriding it.\n\nThe Workflow domain owns the typed family contract — `StepTypeProvider` for a\ncontribution and `Manager` as the owning registry. A provider contributes a\nwhole set of step types under its own stable-identifier name; the manager\nvalidates the entire contribution before applying any of it (named provider,\nwell-formed step-type names, non-nil factories, no step type claimed twice,\nno claim on a name another provider holds) and applies it atomically. The\ngeneric `plugin.Plugin` contract stays metadata-only.\n\nThe vocabulary is a compiled-in, closed set. `internal/infrastructure/workflowsteps`\nis the one place a step type is bundled into any binary that resolves one: the\nshipped workflow stages, the repository-hook replacements (`workflow.diff-rules`)\nand the repository-free event steps each arrive as one provider, and adding a\nstep type means adding a provider to that set. Interpreted plugin sources\ncannot contribute step types: the Yaegi plugin host satisfies only the\nmetadata contract, and no bridge to `StepFactory` exists.\n\nEvery binary that resolves a step type builds its manager from that same\nprovider set at its composition root, before its first resolution, and\ninjects the manager everywhere it resolves definitions. Agreement is per\nbuild: the State Store, the daemon's workflow-definitions client and the agent\nworker built from the same source register identical vocabularies by\nconstruction; binaries built from different sources can disagree, and only a\nmatching deploy fixes that.\n\nAn unknown step type is refused twice, and never degraded. The validator\n`ParseDefinition` refuses a definition naming an unregistered type at save\ntime, on the validating side the composition root built (the State Store's\nworkflow-definitions resource validates there before anything is stored). The\ncompiler refuses the same definition at run, so a binary handed a definition\nwritten against a vocabulary it does not have fails that run loudly instead of\nsilently skipping steps. Both checks live in the Workflow domain against the\ninjected manager, which is why no binary can opt out of the check by\nsubstituting its own registry.\n\n## Relationship between the questions\n\n`archie-core-ze7y` is the same question seen from the deleted\nrepository-authored stages: those `.archie/stages/*.go` files held arbitrary\nstages, and their gate half landed as `workflow.diff-rules` settings rather\nthan interpreted Go. Their stage half has no replacement and does not get one\nthat approximates it: no interpreted-Go stage type is built, and no command\nstep type is created by this decision to stand in for them. The refusal of\n`.archie/stages/*.go` stays until the maintainer's trust decision\n(`archie-core-ndm8`) says otherwise.\n\nIf that decision ever allows a command step type, it arrives as a provider in\nthe compiled-in set under the rules above — a new contributor of exactly one\nstep type at a named provider. It is not a plugin contribution mechanism, and\nit does not change any answer in this document.\n\n## Rejected alternatives\n\n- **A vocabulary version or hash on the wire.** The executing side already\n refuses an unknown step type at its first compile, which is the mismatch\n detector, and it fails closed. A wire field would only narrow the\n deploy-skew window that a matching deploy closes, at the cost of a new\n contract field.\n- **A Yaegi-plugin-to-`StepFactory` bridge.** It would run repository-supplied\n interpreted code inside the daemon's process with daemon privileges. The\n trust boundary invariant requires repository-authored code to run in the\n task container, not in an interpreter embeddable into any binary.\n- **A package-level process-global registrar.** Global mutable registry state\n is unreadable at the composition root, collides with itself when one\n process builds the vocabulary twice, and is what made the first attempt at\n this question worse instead of better.\n- **Degrading on an unknown step type.** A run that skipsLine truncated
|
||||
}
|
||||
],
|
||||
"nav": [
|
||||
@@ -821,10 +857,18 @@
|
||||
"title": "Playbook data-flow and condition syntax",
|
||||
"url": "/dev/prds/playbook-expression-syntax/"
|
||||
},
|
||||
{
|
||||
"title": "Plugin settings reconcile live",
|
||||
"url": "/dev/prds/plugin-settings-live/"
|
||||
},
|
||||
{
|
||||
"title": "PR review agent",
|
||||
"url": "/dev/prds/pr-review-agent/"
|
||||
},
|
||||
{
|
||||
"title": "PR review: the operator's answer to the approval gate",
|
||||
"url": "/dev/prds/pr-review-operator-response/"
|
||||
},
|
||||
{
|
||||
"title": "PR review comment remediation -- decision",
|
||||
"url": "/dev/prds/pr-review-remediation/"
|
||||
@@ -845,6 +889,10 @@
|
||||
"title": "Review calibration -- decision",
|
||||
"url": "/dev/prds/review-calibration/"
|
||||
},
|
||||
{
|
||||
"title": "Review settings resource",
|
||||
"url": "/dev/prds/review-settings-resource/"
|
||||
},
|
||||
{
|
||||
"title": "Runtime control plane",
|
||||
"url": "/dev/prds/runtime-control-plane/"
|
||||
@@ -869,6 +917,10 @@
|
||||
"title": "Skill curator (reference implementation) -- decision",
|
||||
"url": "/dev/prds/skill-curator/"
|
||||
},
|
||||
{
|
||||
"title": "SOUL: an agent's user-authored personality",
|
||||
"url": "/dev/prds/soul/"
|
||||
},
|
||||
{
|
||||
"title": "State Store — contract boundary \u0026 transport (ratification)",
|
||||
"url": "/dev/prds/state-store-contract/"
|
||||
@@ -905,9 +957,17 @@
|
||||
"title": "Webhook intake security -- decision",
|
||||
"url": "/dev/prds/webhook-intake-security/"
|
||||
},
|
||||
{
|
||||
"title": "Declared outputs of a workflow run",
|
||||
"url": "/dev/prds/workflow-call-outputs/"
|
||||
},
|
||||
{
|
||||
"title": "Workflows calling workflows",
|
||||
"url": "/dev/prds/workflow-calls/"
|
||||
},
|
||||
{
|
||||
"title": "Workflow step vocabulary: what a step type is and how binaries agree on it",
|
||||
"url": "/dev/prds/workflow-step-vocabulary/"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -31,6 +31,50 @@ should look.
|
||||
after visiting Settings and Logs: the UI process serves HTTP/1.1 and six
|
||||
persistent streams can starve every later browser request.
|
||||
|
||||
## Testing the dashboard
|
||||
|
||||
Authority: the runner itself is `ui/package.json`'s `test` script: `node --test
|
||||
--experimental-strip-types test/*.test.ts` over `ui/test/*.test.ts`. It runs
|
||||
pure TypeScript modules with `node`'s type stripping — no SFC loader, no DOM,
|
||||
no second dependency beyond what the app already carries. That is the standard,
|
||||
and a decision record for when it is deliberately not enough:
|
||||
|
||||
- **Test archie's logic in a module; leave SFC wiring alone.** Wire shapes,
|
||||
draft state, copy conditions, filters and payload building live in plain
|
||||
`.ts` files beside the components and are tested in `ui/test/`. Components
|
||||
stay thin, so there is normally nothing in an SFC that is archie logic.
|
||||
- **Installed primitives are upstream-tested, not re-tested here.**
|
||||
`reka-ui` (2.10.5 here) maintains colocated tests per component in its own
|
||||
repo (Vitest/jsdom + axe: Switch, TagsInput, NumberField, Select, the
|
||||
arrow-navigation composables). The npm package ships none of those files, so
|
||||
this coverage is inherited through the version bound, not the tarball —
|
||||
`grep -l test node_modules/reka-ui` finds nothing. Duplicating them as local
|
||||
mount tests would test a dependency.
|
||||
- **A `.vue` import cannot run under the module runner.** `node --test` fails
|
||||
such a module with `ERR_UNKNOWN_FILE_EXTENSION: .vue` and the runner has no
|
||||
built-in loader for SFCs. A file under `ui/test/` that imports a `.vue` is
|
||||
therefore a broken test, not a weak one.
|
||||
- **If a DOM mount harness is ever needed** (a page-level accessibility
|
||||
contract, cross-component interaction, focus behaviour): `happy-dom` +
|
||||
`@vue/test-utils` + `vitest` scoped to a separate DOM test glob, keeping the
|
||||
module runner for everything else. Measured on the ui tree (vitest 5.0.2,
|
||||
happy-dom 20.14.5, @vue/test-utils 2.5.1): they add 24 top-level
|
||||
`node_modules` entries and 27 MB on a 296 MB install, a working 2-test mount
|
||||
suite (role=radiogroup vs role=switch, aria-label, click activation against
|
||||
the real SFCs) runs in 1.2 s on top of the current 0.5 s module suite, and
|
||||
`vue-tsc` (7.4 s) dominates `task test:ui` either way. The single-runner
|
||||
alternative (absorbing the existing suite into vitest) was measured
|
||||
infeasible as-is: all 29 existing files import `node:test`, which vitest
|
||||
cannot collect ("No test suite found in file" for every one), so it would be
|
||||
a rewrite of the whole suite, and its runtime was not measured. Nothing in
|
||||
`ui/dist` or CI changes either way; happy-dom is pure JS, no browser binary.
|
||||
- **What the module standard does not verify:** whether a call site passes an
|
||||
`aria-label` or which `v-if` branch shows. Those live in SFC templates and
|
||||
are only visible with a mount. If a page's accessible-name wiring is the
|
||||
risk, say so in the page's task and reach for the harness decision above
|
||||
rather than quietly growing mount tests. Never suppress a lint rule or a
|
||||
test to keep SFC behaviour out of the gate.
|
||||
|
||||
## Adding an API route
|
||||
|
||||
1. The handler goes in the `internal/webui/api_<concern>.go` file for its
|
||||
|
||||
@@ -34,9 +34,13 @@ restating them.
|
||||
index, so stage regenerated contracts before running it.
|
||||
- Stage by path, or read `git status` before `git add -A`. Tests that start
|
||||
embedded NATS or write logs must use `t.TempDir()`; anything they leave in the
|
||||
tree gets committed. `task check` also adds linter-tool lines to `go.sum`
|
||||
(bead 1e37); restore `go.sum` before committing unless you changed a
|
||||
dependency.
|
||||
tree gets committed. `task check` writes no module file on a consistent tree,
|
||||
root or `tools/` (verified at HEAD with `GOFLAGS` unset and with `-mod=mod`):
|
||||
a stale `go.mod` or `go.sum` fails it, with `missing go.sum entry` or
|
||||
`updates to go.mod needed`, rather than being repaired. Only a global
|
||||
`GOFLAGS=-mod=mod` in `go env` re-enables that repair, which is why the
|
||||
`go -C tools` steps pass `-mod=readonly`; if `go.sum` goes dirty, run
|
||||
`go mod tidy` and commit the result on its own.
|
||||
- A change to `docs/prds/`, `docs/architecture/` **or `docs/development/`**
|
||||
carries the regenerated `docs/data/generated/dev-docs.json`: run
|
||||
`task docs:artifact` (not `task docs:check`, which checks contracts only).
|
||||
|
||||
@@ -40,6 +40,17 @@ vocabulary it registers, so both must see the same step types.
|
||||
- Examples in `examples/workflows/` are parsed against the registered
|
||||
vocabulary by `TestExampleWorkflowsParse`. Add one for a new capability.
|
||||
|
||||
## Enabling and disabling
|
||||
|
||||
The `workflow-enablement` control-plane resource lists, per org, the workflows
|
||||
that org has disabled; anything unlisted is enabled. Its readers are the
|
||||
daemon's binding dispatch (a disabled workflow's binding is skipped like an
|
||||
unarmed one), the dashboard's binding save (refused) and binding list
|
||||
(`workflow_disabled`), and work requests. A new reader of "may this org run
|
||||
this workflow" goes through `task.WorkflowEnablement.Enabled`, never a second
|
||||
list. The dashboard acts in the default org until access resolves a caller's
|
||||
own.
|
||||
|
||||
## Starting a workflow from chat
|
||||
|
||||
`task_spawn` accepts `workflow` and an `inputs` object. The gateway forwards
|
||||
|
||||
@@ -59,6 +59,7 @@
|
||||
"/news/archie/1.43.0/": "/news/1.43.0/",
|
||||
"/news/archie/1.44.0/": "/news/1.44.0/",
|
||||
"/news/archie/1.44.1/": "/news/1.44.1/",
|
||||
"/news/archie/1.45.0/": "/news/1.45.0/",
|
||||
"/news/archie/1.5.0/": "/news/1.5.0/",
|
||||
"/news/archie/1.6.0/": "/news/1.6.0/",
|
||||
"/news/archie/1.7.0/": "/news/1.7.0/",
|
||||
@@ -135,6 +136,7 @@
|
||||
"/news/archied/1.43.0/": "/news/1.43.0/",
|
||||
"/news/archied/1.44.0/": "/news/1.44.0/",
|
||||
"/news/archied/1.44.1/": "/news/1.44.1/",
|
||||
"/news/archied/1.45.0/": "/news/1.45.0/",
|
||||
"/news/archied/1.5.0/": "/news/1.5.0/",
|
||||
"/news/archied/1.6.0/": "/news/1.6.0/",
|
||||
"/news/archied/1.7.0/": "/news/1.7.0/",
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
[
|
||||
{
|
||||
"version": "1.45.0",
|
||||
"date": "2026-09-30",
|
||||
"body": "### archied\n\n- fix(eda): reject a result map missing a field or needing a conversion\n- feat(eda): per-org workflow enable/disable gating bindings\n- feat(telegram): carry inbound media through the chat turn\n- fix(prsource): use a CodeQL-recognised containment guard when extracting archives\n- feat(store): key control-plane resources by (org, kind)\n- fix(store): refuse resource keys the audit key cannot disambiguate\n- fix(daemon): resume the task branch before container acquisition\n- feat(webui): show attachment metadata in the session transcript\n- feat(webui): name a workflow's declared inputs on the work-request form\n- feat(controlplane): carry the caller org from the request principal\n- fix(webui): the workflow handlers act in the caller's org\n- feat(kitrun): render the OAuth credential file with sentinels\n- feat(store): record accepted package authority against the pin\n- fix(workflow): the own-PR flow no longer parks on the operator-approval gate\n- feat(gateway): a TTL-bounded dedup gate for redelivered platform messages\n- feat(egress): capture the OAuth scope set and carry it to the harness secret\n- feat(kitrun): render the OAuth scopes and omit an uncaptured primary key\n- feat(store): project installed package contributions into org resources\n- feat(gateway): coalesce rapid-fire text events into one turn\n- fix(config): expand plugin, module and skills dirs like every other operator path\n- fix(webui): drop the config schema's editability advert\n- feat(webui): events counts, YAML reading, log rows and dashboard filters\n\n### archie-agent\n\n- feat(eda): per-org workflow enable/disable gating bindings\n- feat(telegram): carry inbound media through the chat turn\n- fix(prsource): use a CodeQL-recognised containment guard when extracting archives\n- feat(store): key control-plane resources by (org, kind)\n- feat(controlplane): carry the caller org from the request principal\n- feat(kitrun): render the OAuth credential file with sentinels\n- feat(store): record accepted package authority against the pin\n- fix(workflow): the own-PR flow no longer parks on the operator-approval gate\n- feat(egress): capture the OAuth scope set and carry it to the harness secret\n- feat(kitrun): render the OAuth scopes and omit an uncaptured primary key\n- feat(store): project installed package contributions into org resources"
|
||||
},
|
||||
{
|
||||
"version": "1.44.1",
|
||||
"date": "2026-09-29",
|
||||
|
||||
@@ -118,7 +118,12 @@ typed values are checked at load; `dyn` values are checked by the module decode.
|
||||
|
||||
**D5 — where the result map becomes the `Result` struct.** Settled:
|
||||
`ModuleRegistry` stays schema-agnostic. `ModuleRegistry.DecodeResult`
|
||||
(`internal/domain/eda/module`) is the conversion site.
|
||||
(`internal/domain/eda/module`) is the conversion site. It accepts a value only
|
||||
when it already carries the field's declared type, and a map only when it
|
||||
carries every declared field. A conversion changes the value's representation --
|
||||
a number for a string field is the named case -- and an absent field reads as the
|
||||
zero value, so each is reported rather than becoming a quietly wrong value in a
|
||||
later expression.
|
||||
|
||||
## Verification
|
||||
|
||||
@@ -128,5 +133,8 @@ typed values are checked at load; `dyn` values are checked by the module decode.
|
||||
dynamically indexed `actions` read fails.
|
||||
- An arg KEY the kind's `Args` schema does not define fails the load, and so
|
||||
does `message: '123'` for the `log` kind. `message: event.title` loads.
|
||||
- A result map missing a declared field, and one whose value is not already the
|
||||
field's type, are both reported by the decode; a value that would only be
|
||||
convertible (a number for a string field) is not accepted.
|
||||
- The one-action `workflow` playbook shape, its dispatch, and its existing
|
||||
load-time checks are unchanged.
|
||||
@@ -4,9 +4,6 @@
|
||||
**Date:** 2026-09-03
|
||||
**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`
|
||||
**Beads issue:** `archie-core-t2db.14` (expression environment).
|
||||
Unbuilt decisions: `archie-core-t2db.25` (J1 action ids),
|
||||
`archie-core-t2db.26` (J2 args as CEL),
|
||||
`archie-core-t2db.27` (Result schemas declared to CEL).
|
||||
|
||||
**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression
|
||||
mechanism covers both an action's `when` condition and its `args` values.
|
||||
@@ -222,6 +219,14 @@ cost of CEL is measured and bounded.
|
||||
applied to every expression; the linter and daemon use the same value so they
|
||||
agree. Alternative considered: exact number is a constant to tune at first
|
||||
use.
|
||||
- **J6 -- `args` failure semantics.** An `args` value that cannot be evaluated
|
||||
aborts its playbook run: the action and every later action in that playbook do
|
||||
not run, because data has no false value to fall back on and an action run
|
||||
without its arguments is a silent one. Other playbooks matching the same event
|
||||
still run, and the task is unaffected: a playbook run never changes routing.
|
||||
The daemon log reports the failure, naming the task, the playbook and the
|
||||
action; no task event is written. Alternative considered: mirroring J3,
|
||||
rejected because a predicate has a meaningful false and data does not.
|
||||
|
||||
## Interaction with the execution-time gaps
|
||||
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
# Plugin settings reconcile live
|
||||
|
||||
**Status:** Draft
|
||||
**Tracking:** GitHub [#1154](https://github.com/samcharles93/archie-core/issues/1154), epic [#998](https://github.com/samcharles93/archie-core/issues/998)
|
||||
**Authority:** `docs/prds/runtime-control-plane.md` decides each feature owns how
|
||||
its changes take effect; `docs/architecture/plugins-and-extensions.md` owns the
|
||||
plugin engine rule; `docs/prds/control-plane-apply-status.md` owns where apply
|
||||
reports go.
|
||||
|
||||
## Outcome
|
||||
|
||||
An operator drops a plugin, module or secret-engine file into the configured
|
||||
directory while archied runs, and the daemon loads it without a restart. An
|
||||
edit to an installed file replaces what is running. Deleting an installed file
|
||||
never stops running code; apply status says so. A stored change to
|
||||
`plugin-settings` applies live for additions instead of requiring a restart.
|
||||
|
||||
## Triggers
|
||||
|
||||
`plugin-settings` joins the kinds archied watches (`PluginSettingsKind` moves
|
||||
to the live list in `internal/app/archied/control_plane.go`). After the daemon
|
||||
layers a new version it reconciles the three directories the document names.
|
||||
A second trigger needs no document change: a poller reconciles on an interval,
|
||||
so a file dropped into a directory loads on its own. The poll interval is the
|
||||
apply-status restamp constant; no filesystem-watch dependency is added. The
|
||||
poller reads the current document's directories from the running config each
|
||||
tick, so a changed `plugin_dir`, `module_dir` or `secret_engine_dir` retargets
|
||||
reconciliation without a restart.
|
||||
|
||||
## Rules
|
||||
|
||||
- Adding and changing load; removing never unloads. Yaegi interpreters are not
|
||||
reclaimed while a caller still exists and there is no unload path; the
|
||||
running set can only grow.
|
||||
- Each file is loaded through a fresh interpreter (the boot pattern). A changed
|
||||
file therefore loads cleanly, and replacement switches consumers to the new
|
||||
interpretation; files are content-hashed so only new and changed files load.
|
||||
- A file that fails to load is skipped and reported; already-running code keeps
|
||||
running. This is the degrade-and-skip convention `plugin.LoadDir` and
|
||||
`Registry.LoadDir` already apply at boot. Boot keeps its reject-at-load rule
|
||||
for the module directory; a broken file at runtime skips without aborting.
|
||||
- A directory that does not exist reconciles to an empty set.
|
||||
|
||||
## The three registries
|
||||
|
||||
- **Secret engines** (`internal/secret`). Reconciliation re-runs
|
||||
`Registry.LoadDir` against the live directory. `Register` replaces by name,
|
||||
so an edited engine takes effect for the next resolution. A deleted engine
|
||||
stays registered: existing references keep resolving through it until
|
||||
restart, and the record says so. Engines are stateless resolvers, so a swap
|
||||
needs no lifecycle step.
|
||||
- **Modules** (`internal/domain/eda/module`): reconciliation re-runs
|
||||
`ModuleRegistry.Register` for the kind's file. The invoker maps
|
||||
last-write-wins, and playbook dispatch resolves through the registry at
|
||||
invoke time, so the swap reaches a running playbook store unchanged. A file
|
||||
deleted for a kind leaves the old invoker in place, running; reported.
|
||||
- **Plugins** (`internal/plugin`): `Host` grows running-state operations beside
|
||||
`Register`, `Start` and `Stop`. One adds a single module while the host is
|
||||
running: it is inserted in dependency order, started alone, and on failure
|
||||
rolled back alone, leaving every previously started module running. The
|
||||
other replaces a module whose manifest id matches: stop it, start the new
|
||||
one, and restart the old one when the new start fails. Replacing a module
|
||||
that another depends on is refused, not swapped. These operations obey the
|
||||
plugin engine rule: the host still invokes start and stop itself, isolates
|
||||
the failure to the one module, and keeps a rollback. A deleted file makes no
|
||||
host call: the module runs on and is reported.
|
||||
|
||||
## `skills_dir`
|
||||
|
||||
`skills_dir` is not part of this change. The daemon refuses a candidate
|
||||
`plugin-settings` document whose `skills_dir` differs from the value it is
|
||||
running, reporting that the field applies on restart, until a skill
|
||||
reconciliation exists to carry it. The refusal is the existing rejected-edit
|
||||
apply-status shape: the store keeps the version, the document is not applied,
|
||||
last-known-good keeps running.
|
||||
|
||||
## Apply status
|
||||
|
||||
One record: `archied` on `plugin-settings`. The reconciliation outcome is
|
||||
reported against the version the process is running. The record's error field
|
||||
carries the outstanding reconciliation problem: a deleted file still running, a
|
||||
file that failed to load, or the directory failing to read. A reconciliation
|
||||
with nothing outstanding clears the error. The version stays the version the
|
||||
process is running; an add that succeeded is live even while a removal is
|
||||
outstanding, and the error says so rather than showing the kind as
|
||||
restart-required. archie-gateway keeps boot-time loading; its record simply
|
||||
stays on the version it last applied until it restarts, which the per-process
|
||||
apply-status page already reads correctly.
|
||||
@@ -121,7 +121,11 @@ With `review.approve_before_post = true`, synthesis ends in
|
||||
`waiting_human`. The operator chooses which findings to post, rejects the
|
||||
review, or asks for a re-review with instructions. A re-review reruns phases
|
||||
3 to 8 with the instructions added to the lens and reviewer prompts, at most
|
||||
2 times. A review with no findings posts nothing and does not wait.
|
||||
2 times. A review with no findings posts nothing and does not wait. The gate
|
||||
belongs to the standalone `pr-review` workflow only; archie's own-PR stage list
|
||||
drops it, because a park there would end an implement run before its PR
|
||||
exists (`docs/prds/pr-review-operator-response.md`,
|
||||
"The gate is not reachable from archie's own PRs").
|
||||
|
||||
## Isolation
|
||||
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
# PR review: the operator's answer to the approval gate
|
||||
|
||||
**Status:** Draft
|
||||
**Authority:** `docs/prds/pr-review-agent.md` ("Operator approval") for the gate, and `docs/prds/state-store-contract.md` (rev. 2e) for every task field that crosses the State Store.
|
||||
**Compounds with:** `docs/prds/execution-tree-state-machine.md` (a run cannot resume partway through its stage list).
|
||||
|
||||
## Decision 1 — where the operator answers
|
||||
|
||||
The dashboard's `POST /api/tasks/{id}/action` (`internal/webui/api_tasks.go`, `handleTaskAction`) is the authoritative surface: the only one that resolves the actor from a credential and can carry a payload. Chat is a shortcut, not a second authority — `/approve` (`internal/gateway/gateway.go`, `Router.handleApprove`) answers the post decision with every offered finding selected, and gains no instruction or selection syntax.
|
||||
|
||||
`taskstate.Action` gains `rereview`, and `taskstate.Actions` returns `approve`, `reject` and `rereview` for `WaitingHuman`. `approve` posts the review, `reject` discards it through the existing terminal decline, `rereview` reruns the review phases with instructions. The action body carries `instructions`, required by `rereview`, and `findings`, the finding keys `approve` posts; an absent selection means all of them.
|
||||
|
||||
The path is `handleTaskAction` → `applyOperatorTaskAction` → `Chat.Contract.ApplyOperatorTaskAction` → the daemon's `LocalChatAdapter` → `taskactions.Client` (NATS `archie.gateway.task-action`) → `taskactions.Service.Apply` → one guarded store write that records the decision and requeues. Chat's `/approve` path (`StoreTaskController.Approve` → `chatTaskControllerAdapter.ApproveChatTask`) must reach that same write through the same function, so the two surfaces cannot record different decisions for one operator intent.
|
||||
|
||||
A requeue resumes the workflow the waiting state names, never a name the handler hardcodes. `taskactions.Service.apply` and `chatTaskControllerAdapter.ApproveChatTask` requeue every approval under `implement`, which is the feasibility handoff: an approval whose wait recorded `pr-review` would run the implement workflow. Feasibility must keep that handoff by naming `implement` on the task before it enters the wait, which is the invariant the handler needs — the workflow a task carries is the workflow it runs next. `Requeue` treats an empty workflow as "keep the task's".
|
||||
|
||||
### The review the operator answers
|
||||
|
||||
The gate writes the review it is holding onto the task before it ends the run: one JSON column, `review_gate`, a sibling of `review_payload`, holding the scored findings, the head SHA, the pull request's identity and the workflow the wait resumes. The run's existing `Store.Update` persists it. The same column carries the answer back: the response path's guarded write fills in the outcome, the selection and the instructions, and clears the offer for a re-review.
|
||||
|
||||
The operator reads the offer through the existing task read — `taskView.Task` on `GET /api/tasks` and `GET /api/tasks/{id}/debug` — so no new read endpoint is needed.
|
||||
|
||||
This record is load-bearing because the run that computed the findings ends at the gate and the engine cannot resume it partway. Without it, "post the selected findings" cannot mean the findings the operator saw: the resumed run recomputes them with agent calls. An `approve` resume therefore posts the recorded review filtered by the selection, and the review the resumed phases recompute never reaches the output.
|
||||
|
||||
## Decision 2 — the cap
|
||||
|
||||
The cap is a column: `rereview_rounds` on `tasks`, `bigint NOT NULL DEFAULT 0`, read by the response path and incremented in the same guarded write that requeues a re-review. A re-review at the cap is refused with the conflict sentinel the dashboard answers 409 with, and nothing is requeued. Two is a constant in the domain, not a config knob: the PRD fixes it.
|
||||
|
||||
`Task.Attempt` cannot carry the count, because it counts every requeue. The two alternatives are worse.
|
||||
|
||||
- The execution tree measures passes through the gate, not operator requests. A crash recovery (`RecoverStale`) or an operator retry re-runs the pipeline to the gate and would consume the cap; and a stage has no step read (`task.Store` carries none), so this route needs a new interface method anyway.
|
||||
- `Task.Notes` is the agent memory ledger, not scratch. `AgentStage.buildRequest` passes it into every generic agent request, `agentexec` renders it as a prompt section, and `persistAppendedNotes` appends to it. A counter parsed out of a field agents write to is the encoded-state defect this repository rejects, and instructions parked there would reach every unrelated workflow's agents.
|
||||
|
||||
`RemediationRounds` is the precedent: one writer, one reader, one meaning, kept separate from `RetryCount` so unrelated retries cannot draw a budget down.
|
||||
|
||||
The write crosses the State Store. A goose migration adds the column; `internal/infrastructure/postgres/queries/tasks.sql` and its sqlc output carry it; `internal/infrastructure/postgres/store.go` maps it; `proto/state/v1/state.proto` carries it as a new `Task` field with its regenerated contracts and `staterpc/values.go` mapping both ways; `internal/domain/workflow/task/task.go` declares it; `internal/infrastructure/taskactions/store.go` adds it to the `taskactions.Task` projection the response path reads. The write itself is a guarded requeue method beside `RetryTask`, which increments a counter, and `BeginRemediation`, which records a payload in the same transaction: one transition-table guard, one increment and one requeue in one write. It must join every column-listing query, which `docs/development/state-store.md` names.
|
||||
|
||||
Tests that prove it: the guarded write increments for a re-review and for no other requeue; a re-review at the cap is refused and leaves the task waiting; and the field survives the wire, which `TestTaskProtoCarriesEveryDomainField` asserts for every field of the domain task.
|
||||
|
||||
## Decision 3 — how the instructions reach the stages
|
||||
|
||||
The recorded instructions enter the pipeline at phase 3 and phase 4 only, which is what the PRD asks for: a labelled operator-instructions block appended to `runLens`'s mission in `stagePRLenses` and to `runReviewer`'s mission.
|
||||
|
||||
They cannot ride `AgentStage.buildRequest`. Every pr-review stage builds its `agentexec.Request` in `runPRReviewAgentRecorded`, which sets no `Request.Notes`, so nothing the generic path threads reaches these calls. The pipeline restores the instructions from the task record at the same point it restores the review, and the two mission builders read them there.
|
||||
|
||||
## The gate is not reachable from archie's own PRs
|
||||
|
||||
`StagePRReviewAndOpenPR` splices `prReviewDecisionStages()`, the list `PRReview()` uses, so the gate is reachable from the implement workflow too. It must not be. The change is one entry moving: `stagePROperatorApproval()` leaves the shared list for `PRReview()`'s own stage list, immediately before the merge gate, and the own-PR splice keeps taking the shared list unchanged. This is a prerequisite for the response channel and a change of its own.
|
||||
|
||||
Three reasons, the second deciding it alone.
|
||||
|
||||
- The gate's question is which findings to post, and on archie's own PRs there is no pull request to post to at the point the splice reaches the gate. That trigger's human decision point is its park on an unchallenged blocking finding.
|
||||
- Approving a wait inside the implement workflow re-enters that workflow at its first stage against a worktree that already carries the change. A builder that then reports no changes sets `BuildNoChanges`, `StageCommitPush` calls `closeNoChangesIssue`, and the issue is closed with no pull request ever opened. Nothing the gate decides is worth a route to silently closing an issue.
|
||||
- Short-circuiting instead would put a conditional no-op in every stage whose work a wait had completed, which changes the resume semantics of the whole pipeline rather than this gate.
|
||||
|
||||
## What a full restart gives, and what it costs
|
||||
|
||||
A requeue re-runs the workflow's stage list from its first stage; the engine has no partway resume. That satisfies the PRD's "reruns phases 3 to 8" for free, and it is acceptable for the standalone pipeline precisely because that pipeline has no stage that writes and then closes. The costs:
|
||||
|
||||
- Phases 1 and 2 recompute. Both are code except two agent calls — the machine-written score and the anatomy narrative — so those re-run, and a changed score can change whether the hallucination-check dimension is added.
|
||||
- The pull request's snapshot is checked out again.
|
||||
- The run's cost and wall-clock budgets are per attempt, so a re-review starts with a fresh allowance, which the cap bounds.
|
||||
|
||||
## Verification
|
||||
|
||||
- A gate wait on the standalone `pr-review` workflow answered `approve` posts exactly the findings the operator selected, and no finding the resumed phases produced.
|
||||
- A re-review at the cap is refused, the task stays waiting, and the count survives a store round trip.
|
||||
- A re-review's instructions reach the lens and reviewer missions; a first run's do not.
|
||||
- The gate is unreachable from the implement workflow's stage list.
|
||||
@@ -0,0 +1,123 @@
|
||||
# Review settings resource
|
||||
|
||||
**Status:** Draft
|
||||
**Beads issue:** `archie-core-0ry6`
|
||||
**Authority:** `docs/prds/runtime-control-plane.md` for the resource model,
|
||||
`docs/architecture/configuration.md` for ownership, and
|
||||
`docs/prds/pr-review-agent.md` for what the two dials do.
|
||||
|
||||
## Decision
|
||||
|
||||
`review.precision_gate` and `review.approve_before_post` are one control-plane
|
||||
resource, `review-settings`. The file's `[review]` section seeds it, the stored
|
||||
document outranks the file from then on, and the dashboard writes it through the
|
||||
same settings machinery as every other resource.
|
||||
|
||||
The resource is deployment-wide, like every definition the State Store seeds
|
||||
(`storecontract.DefaultOrgID`): not per-repository, per-identity, or per-run.
|
||||
|
||||
## Why its own kind
|
||||
|
||||
The two dials are pr-review policy — what a review may post, and whether a human
|
||||
decides — not a limit on a task run. The document that carries task limits
|
||||
belongs to that feature. `runtime-control-plane.md` splits settings by feature
|
||||
and refuses documents unrelated features share; `configuration.md` puts a typed
|
||||
runtime setting with the domain whose behaviour it describes. One document per
|
||||
feature is the rule, not one document per settings page.
|
||||
|
||||
A new kind also has no legacy documents. Its first value is the seed derived
|
||||
from the file, so "the store outranks the file" begins from an explicit copy of
|
||||
both fields rather than from a document that predates them.
|
||||
|
||||
## Document
|
||||
|
||||
New file `internal/app/controlplane/review_settings.go`:
|
||||
|
||||
```go
|
||||
const ReviewSettingsKind = "review-settings"
|
||||
|
||||
type reviewSettings struct {
|
||||
PrecisionGate bool `json:"precision_gate"`
|
||||
ApproveBeforePost bool `json:"approve_before_post"`
|
||||
}
|
||||
```
|
||||
|
||||
Its `Definition` sets `ApplyMode: "live"`, `Document: reviewSettings{}`, and a
|
||||
`Seed` returning `reviewSettings{cfg.Review.PrecisionGate, cfg.Review.ApproveBeforePost}`,
|
||||
and a `Validate` wrapping `validateAs`: both fields are free booleans with no cross-field
|
||||
rule, so the validator is a named place to put one rather than an inline no-op
|
||||
(`validatePluginSettings` is the precedent). No `Normalize`, because the kind
|
||||
has no second wire shape to converge.
|
||||
|
||||
## Seeding
|
||||
|
||||
`ImportConfig`/`seededValues` write the seed for a kind the store does not hold,
|
||||
so a deployment that never edited the dials runs the values in `[review]`. Once
|
||||
a document is stored, the file key stops mattering, which is the rule for every
|
||||
database-owned setting.
|
||||
|
||||
## Apply
|
||||
|
||||
`runtimeConfigFrom` in `internal/app/controlplane/runtime_config.go` layers the
|
||||
stored document over the file's `Review`, assigning `out.Review.PrecisionGate`
|
||||
and `out.Review.ApproveBeforePost` field by field. Field-by-field, not
|
||||
`out.Review = ...`: a field added to `config.Review` later keeps the file's value
|
||||
until the document carries it.
|
||||
|
||||
`ReviewSettingsKind` joins `runtimeResourceKinds` in
|
||||
`internal/app/archied/control_plane.go`, so a new stored version re-runs the
|
||||
layering and republishes through `config.Holder`. Live rather than
|
||||
restart-required, because every consumer re-reads it: `Review` is on
|
||||
`reloadableFields` in `internal/app/archied/reload.go`, and the value reaches a
|
||||
run through `Daemon.configFor` → `Config.ForTask` → `TaskConfig.ToConfig` →
|
||||
`TaskContext.Cfg`, which `stagePRPrecisionGate` and `stagePROperatorApproval`
|
||||
read. The next task run uses the new value; a run in flight keeps the value it
|
||||
started with.
|
||||
|
||||
## Projection and page
|
||||
|
||||
`BuildConfigView` gains a plain `Review` field carrying both values out of
|
||||
`cfg.Review`, the way `BudgetsView` carries the layered budgets, so the published
|
||||
config snapshot tells any reader what is in force. The Review page renders the
|
||||
two toggles from the control-plane store's draft for `review-settings` — the
|
||||
shape `SystemTasksPage.vue` uses for `workflow-execution-settings` — and shows
|
||||
the projected value beside them; a Settings route plus a `PAGE_RESOURCES` entry
|
||||
(`ui/src/stores/control-plane.ts`) bind the page to the kind.
|
||||
|
||||
Nothing else re-advertises them. `configFieldDescriptors` and `ConfigView.Schema`
|
||||
gain no `review.*` row: the config catalog's generic renderer was deleted, so a
|
||||
descriptor row adds a field nothing can edit, which is the shape `becb0b5e`
|
||||
removed.
|
||||
|
||||
## What the test proves
|
||||
|
||||
`internal/app/controlplane/runtime_config_test.go` adds the round trip that
|
||||
builds the effective config the way boot does: seed the kind from a file config
|
||||
with both dials set one way, store a document with both flipped, layer it
|
||||
through `runtimeConfigFrom`, and assert the effective `cfg.Review` equals the
|
||||
stored document. A second case covers the absent kind, where the file's values
|
||||
stay in effect. The assertion is on `cfg.Review` — the value `TaskContext.Cfg`
|
||||
carries into the two stages — not on a struct the test set directly.
|
||||
`TestRuntimeResourceKindsApplyLive` gains `review-settings` as live. The
|
||||
validator needs no `validators_agree_test.go` case: it wraps `validateAs` with
|
||||
no rule, because no combination of the two booleans is invalid, and a case that
|
||||
cannot fail is a test the project then carries for nothing.
|
||||
|
||||
## Rejected: extending workflow-execution-settings
|
||||
|
||||
That resource is `workflow.ExecutionSettings`, "the limits captured when a
|
||||
workflow execution starts". Putting review policy in it makes the review feature
|
||||
a field of the limits document, gives a boolean edit a ride on a document whose
|
||||
candidate is validated and swapped as one unit with the budgets, and forces
|
||||
absence-aware pointer fields onto booleans so a document stored before the fields
|
||||
existed does not turn a file's `precision_gate = true` into `false`. It buys one
|
||||
less kind at the cost of the ownership rule the model is built on.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Per-repository, per-identity, or per-workflow review settings. A scope key is
|
||||
a later decision with its own resource shape.
|
||||
- Any other review knob. A field joins the document only with a consumer, the
|
||||
rule every config field carries.
|
||||
- The reviewer pipeline's behaviour. This document places the settings; it does
|
||||
not change what they do.
|
||||
@@ -45,6 +45,14 @@ version being edited. The API layer sets the actor; the request body cannot.
|
||||
The server validates the change, then writes it with its audit record. It rejects
|
||||
edits to an old version.
|
||||
|
||||
The request ID is the write's idempotency key, and it is scoped per org and
|
||||
per kind, never global. The key is `(org_id, kind, request_id)` — the unique
|
||||
index `idx_resource_history_org_kind_request` (migration `0021_resources_per_org.sql`)
|
||||
— so replaying a request ID is the same org's prior write of the same kind, and
|
||||
a request ID reused across orgs or across kinds is a separate write. This is the
|
||||
same scoping as the task idempotency key `org/identity/owner/repo/number`
|
||||
(`docs/prds/orgs-and-access.md`).
|
||||
|
||||
## API
|
||||
|
||||
The API has four operations:
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# SOUL: an agent's user-authored personality
|
||||
|
||||
**Status:** Draft
|
||||
**Authority:** `docs/architecture/identity.md`, “Three-layer identity model”;
|
||||
`docs/prds/orgs-and-access.md` for identity and org ownership.
|
||||
|
||||
## Decision
|
||||
|
||||
SOUL supplies an agent's name, register and warmth. It belongs to the agent's
|
||||
immutable `IdentityID`, not its session, model, installation or container
|
||||
profile. Rename preserves it; changing a profile does not change who acts.
|
||||
SOUL never changes attribution, credentials, grants or identity lifecycle.
|
||||
`config.AgentProfile` remains an execution environment, not another identity.
|
||||
|
||||
Use the existing three layers: invariants, user-authored SOUL, and curator-derived
|
||||
representation. Curators may propose SOUL changes but never apply them.
|
||||
|
||||
## Files and ownership
|
||||
|
||||
The config directory is `${XDG_CONFIG_HOME:-~/.config}/archie`, or the directory
|
||||
of an explicitly selected config file. The default/root agent uses `SOUL.md`
|
||||
directly there; additional agents use `agents/<identity-id>/SOUL.md`. The root is
|
||||
the identity selected by application composition, never the initiating user or
|
||||
a session's personality selection. No file creates or selects an identity.
|
||||
|
||||
The common case keeps a human-friendly filename; additional agents get separate
|
||||
asset directories, following `.agents/skills/<name>/SKILL.md` without confusing
|
||||
these directories with `[containers.profiles.<name>]`. Reject display-name or
|
||||
slug directories: rename and same-name agents across orgs would require another
|
||||
name-resolution rule. Immutable IDs avoid both problems without an org directory.
|
||||
|
||||
When a file-owned agent's file is absent, create the shipped default with
|
||||
exclusive creation. Never overwrite an existing file, including an empty one.
|
||||
A creation failure still permits the turn using the embedded shipped default.
|
||||
Ship that document independently of `agent.ShippedPersonas`, so deleting personas
|
||||
cannot delete the fallback.
|
||||
|
||||
Resolve once at the start of every new generation:
|
||||
|
||||
1. A stored SOUL entry for the acting identity is authoritative.
|
||||
2. Without an entry, read that identity's file again.
|
||||
3. An absent, empty, unreadable or invalid selected source uses the shipped default.
|
||||
|
||||
The first explicit dashboard/API save takes ownership permanently. Thereafter
|
||||
file edits are ignored: emit one ownership notice and show the owning stored
|
||||
version in the dashboard. Restoring the shipped default writes a stored value;
|
||||
it does not remove ownership. Refuse writes that remove an owned entry.
|
||||
|
||||
This ownership boundary is the design's one stateful rule. Files seed the
|
||||
editable value, not an automatic durable import. Reject seed-once import because
|
||||
it breaks next-message file editing; reject file-always-wins because it makes
|
||||
dashboard saves ineffective. Neither a restart nor an upgrade overwrites an
|
||||
owned value, following the precedence of `controlplane.AgentProfileKind` without
|
||||
copying its startup import timing.
|
||||
|
||||
## Prompt and failure contract
|
||||
|
||||
Keep `archie.md.tpl`'s XML-escaped
|
||||
`<soul purpose="identity_and_style" trust="data">` block before
|
||||
`<instruction_precedence>`. SOUL is style reference data; the invariant sections,
|
||||
tool inventory and runtime metadata remain authoritative and uneditable.
|
||||
Preserve `TestBuildSystemPromptSoulCannotOverrideInvariants`.
|
||||
|
||||
Accept UTF-8 plain Markdown with at most 8 KiB in its XML-escaped body. This bounds
|
||||
personality to a small prompt contribution; count the complete rendered prompt
|
||||
in the turn's context budget and never truncate invariant sections. Bound file
|
||||
reads before decoding; reject oversize input rather than truncating instructions.
|
||||
File and stored writes use the same validator; an invalid save leaves the stored
|
||||
version unchanged. The shipped default must pass that validator.
|
||||
|
||||
Read and creation errors never fail a turn. Log once per identity/source failure
|
||||
episode, retry on the next turn, and allow a new warning after recovery. If the
|
||||
store cannot establish whether an owned entry exists, use the shipped default,
|
||||
not a potentially shadowed file. Do not log SOUL text. Each turn keeps its resolved
|
||||
snapshot; edits affect the next generation without restart or SIGHUP.
|
||||
|
||||
## Cross-layer delivery
|
||||
|
||||
- `internal/domain/agent` owns SOUL content validation and the identity-keyed read
|
||||
contract; `internal/domain/identity` remains the actor/lifecycle authority.
|
||||
- The configuration infrastructure resolves the config directory and handles
|
||||
file discovery, exclusive default creation and bounded per-turn reads.
|
||||
Application composition resolves the acting `IdentityID` through the existing
|
||||
identity repository/bindings and supplies it to the gateway.
|
||||
- Register `SoulKind` (`agent-souls`) in the control plane with a live descriptor
|
||||
and an org-scoped document mapping identity IDs to `{text}`. Persist it through
|
||||
the State Store's existing resource/history contracts, not a new table,
|
||||
process or SOUL RPC. An empty map owns no identity; saves preserve other entries
|
||||
and use expected-version checks. Validate that each key is an agent identity
|
||||
in the resource's org; begin with the default org without inventing org setup.
|
||||
Every save records the authenticated editor's identity, source, request ID and
|
||||
timestamp; file installation records the system identity, not a fictitious user.
|
||||
- Replace `TurnRunnerConfig.Personas` / `PersonaPromptSource` with the per-identity
|
||||
source in `prepareTurn`; name the `SystemPromptConfig` input SOUL. Both normal
|
||||
and streaming turns use this assembly path. Do not add personality to the
|
||||
worker environment, `AgentProfile` or workflow-stage prompts.
|
||||
- On an agent identity's dashboard page, expose its effective SOUL, file/default
|
||||
or stored source, shadowed file path, owning version, and Markdown editing.
|
||||
Save to the control plane, never the host file; show validation/version
|
||||
conflicts and offer restoring shipped content or a history revision as a save.
|
||||
Read effective file/default content and ownership metadata through a gateway
|
||||
query using the turn resolver; the separate UI process never reads the
|
||||
gateway host's files. Apply identity-management access rules and the existing
|
||||
shared live-update hub.
|
||||
- Update `docs/development/agent.md` and `control-plane.md` with this reach and
|
||||
precedence, `frontend-ui.md` with the gateway query, and the config wiring
|
||||
guidance with a copyable file example.
|
||||
|
||||
## Persona removal gate
|
||||
|
||||
SOUL supersedes `PersonaRegistry` and the persona-reconciliation proposal;
|
||||
identity-per-action supersedes personas as an identity mechanism.
|
||||
`PersonaRegistry.GetActive` composes the baseline with one of five styles from
|
||||
`agent.ShippedPersonas`; SOUL replaces the baseline, not that style catalogue.
|
||||
Per-session switching disappears: remove `/personality`, Telegram personality
|
||||
callbacks, the dashboard persona page, and chat wire fields `personas`,
|
||||
`active_personas` and `personas_available` without reusing their field numbers.
|
||||
Do not merge session selections into SOUL or retain a separate style axis.
|
||||
Custom persona text must remain available to copy before deleting its stored
|
||||
resource; no migration guesses one agent's SOUL from several sessions.
|
||||
|
||||
Remove persona loading, selectors and commands only after every chat composition
|
||||
supplies the identity-aware SOUL source, the fallback renders without personas,
|
||||
and the dashboard can edit it. SOUL delivery does not wait for org creation or a
|
||||
new container-profile model.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Two agents use distinct files and stored entries; renaming one or sharing a
|
||||
container profile does not change or mix their SOULs.
|
||||
- Before ownership, a file edit changes the next normal or streaming prompt;
|
||||
creating defaults never clobbers an existing file.
|
||||
- After a save, conflicting file edits and restarts cannot override the stored
|
||||
value; the ignored file and owning version are visible. Updating one agent
|
||||
preserves the other; stale saves fail and history restores are audited saves.
|
||||
- Missing, empty, unreadable, malformed and oversize selected inputs render the
|
||||
shipped default. Repeated read failures log once, recover on a later turn,
|
||||
and never abort generation.
|
||||
- Hostile markup stays escaped before singular, unchanged invariant blocks;
|
||||
prompt budgeting includes SOUL and cannot drop the rules.
|
||||
- Dashboard edits reach the next generation without restart; persona removal
|
||||
leaves every channel with a nonempty identity slot and preserved custom text.
|
||||
@@ -0,0 +1,213 @@
|
||||
# Declared outputs of a workflow run
|
||||
|
||||
**Status:** Draft
|
||||
**Date:** 2026-09-30
|
||||
**Bead:** `archie-core-t2db.43`
|
||||
**Authority:** `docs/prds/event-automation.md` for what a declared output is,
|
||||
`docs/prds/workflow-calls.md` for the call and its grants,
|
||||
`docs/prds/state-store-contract.md` for the State Store contract the wire
|
||||
change must satisfy.
|
||||
**Supersedes:** `docs/prds/workflow-calls.md`, "Waiting on the callee": the
|
||||
callee's transition detail is the human summary, not the stand-in for
|
||||
structured outputs.
|
||||
|
||||
A workflow declares named, typed outputs. A run writes each one once. A
|
||||
`wait: true` caller receives them, and may publish one as its own.
|
||||
|
||||
## Where outputs are declared
|
||||
|
||||
`outputs:` on the workflow definition, beside `inputs:`, decoded into
|
||||
`task.WorkflowInterface.Outputs map[string]OutputSpec` where
|
||||
`OutputSpec{Type, Required}`:
|
||||
|
||||
```yaml
|
||||
id: firewall-contain
|
||||
outputs:
|
||||
contained: { type: bool, required: true }
|
||||
summary: { type: string }
|
||||
```
|
||||
|
||||
Names and types are the input grammar: the same identifier shape, the same
|
||||
type set, and the same check in `WorkflowInterface.Validate`, which both
|
||||
`ParseDefinition` and `ParseWorkflowInterface` call.
|
||||
|
||||
**Forced by:** `ParseDefinition` decodes with `KnownFields(true)`, so a key no
|
||||
decoded struct declares is refused — the declaration must live on
|
||||
`WorkflowInterface` for an author to write it at all, and that is where
|
||||
`docs/development/workflows.md` puts every field a process without the engine
|
||||
must read. The type set is the shared value vocabulary: `TypeAccepts` and
|
||||
`ValueType` already check a mapped parameter and an input with it. `required`
|
||||
defaults to false, and the generated YAML for a builtin that declares no
|
||||
outputs is unchanged, because a definition's digest pins a run.
|
||||
|
||||
**Rejected:** outputs per step, or per call step, because a workflow returns a
|
||||
result as itself rather than as one of its stages. Also `task.Definition`
|
||||
gaining outputs: no reader needs them — a binding offers inputs, and a call's
|
||||
outputs assignment is checked against the callee's YAML in the same
|
||||
collection — and a declared field with no reader parses and does nothing.
|
||||
|
||||
## How a run writes one
|
||||
|
||||
Each declared output is offered to the run's agent stages as a capture tool
|
||||
named after the output, whose arguments are `{"value": ...}`; the accepted
|
||||
call's `value` member is the output's value. The engine holds the accepted
|
||||
values on the task context and writes them into its own task row in the
|
||||
`Store.Update` call the run already makes. A deterministic stage may set the
|
||||
same values in Go, and the run's finish check validates them identically.
|
||||
|
||||
**Forced by:** a run's only structured results are capture calls
|
||||
(`agentexec.CaptureTool`, `Result.Captures`), and the workflow interface
|
||||
already declares when a stage needs them (`task.WorkflowNeeds.Captures`).
|
||||
Deriving the tools from the declaration means an output cannot be written
|
||||
under a name nobody declared, and it keeps one route to structured agent
|
||||
output rather than adding a second beside the capture path. The row write is
|
||||
the run's existing write: the task row is where a run's own state lives, and
|
||||
the container already holds `Update` on its own row.
|
||||
|
||||
The tool is offered with `MaxCalls: 1`, and a stage is not offered an output
|
||||
the run has written, so a run writes each output once: a repeat within a stage
|
||||
is answered with the tool's rejection and the agent carries on, and a later
|
||||
stage cannot overwrite a value. A stage's own capture tool may not share a
|
||||
name with a declared output, because the value would otherwise be ambiguous.
|
||||
|
||||
**Rejected:** a `workflow.output` step type, because assigning the run's
|
||||
result from a step needs a grammar for "the previous stage's result" that does
|
||||
not exist, and a step that reaches the agent would be the second path
|
||||
`docs/development/workflows.md` forbids. Also a write RPC of its own: the
|
||||
container's `Update` already reaches its own row, and a second write verb for
|
||||
one column is a second thing to keep in step.
|
||||
|
||||
## How a caller receives one
|
||||
|
||||
On a successful `wait: true` call the caller's stage holds the callee's
|
||||
outputs as a `map[string]any`. A call step's `outputs:` setting publishes one
|
||||
as one of the caller's own outputs:
|
||||
|
||||
```yaml
|
||||
steps:
|
||||
- type: workflow.call
|
||||
workflow: firewall-contain
|
||||
wait: true
|
||||
inputs: { src_ip: inputs.src_ip }
|
||||
outputs: { contained: outputs.contained }
|
||||
```
|
||||
|
||||
The key names the callee's declared name; the value names one of the caller's
|
||||
own with the prefix the call's `inputs:` already uses. A published value joins
|
||||
the caller's output set for its run, so a caller returns its callee's result to
|
||||
its own caller, and a run may publish a callee's output and write its own
|
||||
others.
|
||||
|
||||
Lifetime is the attempt. The run starts with an empty output set, every later
|
||||
stage of that attempt sees the values, and the finish write replaces the row's
|
||||
set wholesale — so nothing an attempt wrote survives into the next, and a run
|
||||
that writes nothing stores nothing. Values reach a caller only from a
|
||||
successful terminal state: the caller assigns them after `callSucceeded`, so a
|
||||
parked or failed callee delivers nothing whatever its row holds. A call step
|
||||
with `wait: false` and an `outputs:` block is refused when the definition is
|
||||
saved, because nothing has finished when the caller moves on.
|
||||
|
||||
**Forced by:** caller and callee are separate runs in separate containers, so
|
||||
the caller's own row is the only durable place to hold a received value. The
|
||||
prefix rule is not a new grammar: in `inputs.<name>` the prefixed side is the
|
||||
caller's own value and the map key is the callee's, and `outputs.<name>` reads
|
||||
the same way in the other direction.
|
||||
|
||||
**Rejected:** an implicit same-name copy from callee to caller, because the
|
||||
caller's contract would then depend on a callee it does not name, and swapping
|
||||
the callee would silently change what the caller returns. Also letting a later
|
||||
stage read a callee's output in its mission text: that needs a stage-result
|
||||
reference namespace this model does not define.
|
||||
|
||||
## Failure rules
|
||||
|
||||
- **Undeclared value.** The agent path cannot produce one — the tool exists
|
||||
only for a declared output. A Go stage that sets an undeclared key parks the
|
||||
run before its outcome transition, and a stage capture tool sharing a
|
||||
declared output's name is refused when the run starts.
|
||||
- **Declared but never written.** With `required: true` the run parks naming
|
||||
the output and reaches no successful terminal state, so no caller observes a
|
||||
success missing a promised value. Without it the key is absent from the
|
||||
object rather than null, and a written `null` counts as not written, as
|
||||
`CheckInputs` treats a null input. A published output the callee never wrote
|
||||
leaves the caller's output absent, which the caller's own `required` then
|
||||
judges.
|
||||
- **Save time.** `WorkflowInterface.Validate` refuses an unknown type and a
|
||||
malformed name. A call step's `outputs:` assignment is checked by a
|
||||
`checkCallOutputs` beside `checkCallInputs`: the key must be declared by the
|
||||
callee, the reference must name an output the caller declares, and the
|
||||
callee's declared type must satisfy the caller's.
|
||||
- **Run time.** A captured value is re-validated against the declared type
|
||||
when the run reads it back, for the reason `readCaptures` re-checks every
|
||||
record: the captures file is writable by the harness user. A published
|
||||
callee value is re-checked against the caller's declared type before it
|
||||
enters the caller's set; a mismatch parks the caller naming the output.
|
||||
|
||||
**Rejected:** typing at save time only, because a captured value has no
|
||||
save-time form; and at run time only, because a definition error would surface
|
||||
as a failed run instead of a refused save.
|
||||
|
||||
## Storage and wire
|
||||
|
||||
- `tasks.outputs`, text, `NOT NULL DEFAULT ''`, written by the run's own row
|
||||
write — the shape `tasks.inputs` already has, with `EncodeOutputs` and
|
||||
`DecodeOutputs` beside `EncodeInputs`/`DecodeInputs`.
|
||||
- The `Task` message carries it as `outputs_json`, because the container's
|
||||
`Update` sends the whole row and a column the message cannot carry is a
|
||||
column that write drops.
|
||||
- `WorkflowCallStatusResponse` carries it as `outputs_json` beside `status`
|
||||
and `detail`, read from the callee's row by the handler that already
|
||||
verifies `call_parent_task_id`; `detail` keeps carrying the transition
|
||||
detail. The Go facade stays the two methods of `task.Caller`; no interface
|
||||
method is added.
|
||||
|
||||
**Forced by the grant:** a caller authenticates as its own task and may reach
|
||||
`WorkflowCallStatus` on its own task ID, `Update`/`Transition`/`InsertEvent`
|
||||
on its own row, and the step-execution pair on its own execution. It can read
|
||||
neither the callee's row nor the callee's steps. `WorkflowCallStatus` is
|
||||
therefore the only channel that exists, and it is already the sanctioned
|
||||
cross-run read — this model fills the placeholder it carries. The values live
|
||||
on the callee's row rather than on the caller's call step because the callee
|
||||
writes them and the caller only receives them.
|
||||
|
||||
The write order is part of the contract: a run persists its outputs before its
|
||||
outcome transition, so the status a caller sees when its wait ends already
|
||||
carries them. A set whose encoded form passes the bound the store applies to a
|
||||
structured payload (`clip`, as `ReviewPayload` is stored) is refused rather
|
||||
than clipped, because clipped JSON does not parse.
|
||||
|
||||
No sentinel moves: a missing required output is an engine park reason, and a
|
||||
caller asking about a task that is not its callee still gets `ErrCallNotYours`,
|
||||
so nothing new crosses `mapError`/`unmapError` and no canonical message string
|
||||
changes.
|
||||
|
||||
**Rejected:**
|
||||
- A new read RPC (`WorkflowCallOutputs`) — a second poll of a row the caller
|
||||
already polls, whose answer could disagree with the status beside it, and a
|
||||
second grant-widening row for the same parent-child check.
|
||||
- The step-execution record — `ListSteps` is granted by `read` on a run, so a
|
||||
caller cannot read the callee's steps at all, and making them readable would
|
||||
hand it the callee's whole step history, which the parent-child check exists
|
||||
to prevent. A step's `detail` is text clipped like every other detail.
|
||||
- The caller's call step as the store — the value originates in another
|
||||
process and must outlive the container that produced it.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- A run started by a binding: it has no caller, and its outputs live on its
|
||||
own row only.
|
||||
- A stage reference to a previous stage's captured value.
|
||||
- Assigning outputs in a caller cancelled while it waits.
|
||||
|
||||
## Verification
|
||||
|
||||
- A definition whose output type is not in the vocabulary, and a call
|
||||
publishing an output neither side declares, are refused when saved.
|
||||
- A `wait: true` caller receives a value its callee wrote, returns it as its
|
||||
own output, and the caller's row carries it.
|
||||
- A run whose required output is never written parks, and its caller's stage
|
||||
fails naming the output.
|
||||
- A call step publishing an output over `wait: false` is refused when saved.
|
||||
- A retried attempt that writes no outputs leaves an empty set on the row.
|
||||
- An example under `examples/workflows/` carries an `outputs:` block and a
|
||||
call that publishes one, which `TestExampleWorkflowsParse` covers.
|
||||
@@ -0,0 +1,85 @@
|
||||
# Workflow step vocabulary: what a step type is and how binaries agree on it
|
||||
|
||||
**Status:** Draft
|
||||
**Date:** 2026-09-29
|
||||
**Beads:** `archie-core-m7oc` (agreement between separately deployed binaries), `archie-core-ze7y` (the deleted `.archie/stages/*.go` equivalent)
|
||||
**Extends:** `docs/architecture/plugins-and-extensions.md` (plugin engine rule), `docs/prds/event-automation.md` (the workflow model step types serve)
|
||||
|
||||
**Authority:** `docs/architecture/plugins-and-extensions.md` — the workflow step vocabulary is a Workflow capability family, and its extension point follows the plugin engine rule.
|
||||
|
||||
## Decision
|
||||
|
||||
A workflow step type is a named Go factory: the word a YAML workflow writes in
|
||||
a step's `type` field, plus the factory that builds its stage. The name is a
|
||||
stable identifier; the identity also declares what plugin family claims it,
|
||||
so a contribution that shadows a shipped stage is refused instead of
|
||||
overriding it.
|
||||
|
||||
The Workflow domain owns the typed family contract — `StepTypeProvider` for a
|
||||
contribution and `Manager` as the owning registry. A provider contributes a
|
||||
whole set of step types under its own stable-identifier name; the manager
|
||||
validates the entire contribution before applying any of it (named provider,
|
||||
well-formed step-type names, non-nil factories, no step type claimed twice,
|
||||
no claim on a name another provider holds) and applies it atomically. The
|
||||
generic `plugin.Plugin` contract stays metadata-only.
|
||||
|
||||
The vocabulary is a compiled-in, closed set. `internal/infrastructure/workflowsteps`
|
||||
is the one place a step type is bundled into any binary that resolves one: the
|
||||
shipped workflow stages, the repository-hook replacements (`workflow.diff-rules`)
|
||||
and the repository-free event steps each arrive as one provider, and adding a
|
||||
step type means adding a provider to that set. Interpreted plugin sources
|
||||
cannot contribute step types: the Yaegi plugin host satisfies only the
|
||||
metadata contract, and no bridge to `StepFactory` exists.
|
||||
|
||||
Every binary that resolves a step type builds its manager from that same
|
||||
provider set at its composition root, before its first resolution, and
|
||||
injects the manager everywhere it resolves definitions. Agreement is per
|
||||
build: the State Store, the daemon's workflow-definitions client and the agent
|
||||
worker built from the same source register identical vocabularies by
|
||||
construction; binaries built from different sources can disagree, and only a
|
||||
matching deploy fixes that.
|
||||
|
||||
An unknown step type is refused twice, and never degraded. The validator
|
||||
`ParseDefinition` refuses a definition naming an unregistered type at save
|
||||
time, on the validating side the composition root built (the State Store's
|
||||
workflow-definitions resource validates there before anything is stored). The
|
||||
compiler refuses the same definition at run, so a binary handed a definition
|
||||
written against a vocabulary it does not have fails that run loudly instead of
|
||||
silently skipping steps. Both checks live in the Workflow domain against the
|
||||
injected manager, which is why no binary can opt out of the check by
|
||||
substituting its own registry.
|
||||
|
||||
## Relationship between the questions
|
||||
|
||||
`archie-core-ze7y` is the same question seen from the deleted
|
||||
repository-authored stages: those `.archie/stages/*.go` files held arbitrary
|
||||
stages, and their gate half landed as `workflow.diff-rules` settings rather
|
||||
than interpreted Go. Their stage half has no replacement and does not get one
|
||||
that approximates it: no interpreted-Go stage type is built, and no command
|
||||
step type is created by this decision to stand in for them. The refusal of
|
||||
`.archie/stages/*.go` stays until the maintainer's trust decision
|
||||
(`archie-core-ndm8`) says otherwise.
|
||||
|
||||
If that decision ever allows a command step type, it arrives as a provider in
|
||||
the compiled-in set under the rules above — a new contributor of exactly one
|
||||
step type at a named provider. It is not a plugin contribution mechanism, and
|
||||
it does not change any answer in this document.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
- **A vocabulary version or hash on the wire.** The executing side already
|
||||
refuses an unknown step type at its first compile, which is the mismatch
|
||||
detector, and it fails closed. A wire field would only narrow the
|
||||
deploy-skew window that a matching deploy closes, at the cost of a new
|
||||
contract field.
|
||||
- **A Yaegi-plugin-to-`StepFactory` bridge.** It would run repository-supplied
|
||||
interpreted code inside the daemon's process with daemon privileges. The
|
||||
trust boundary invariant requires repository-authored code to run in the
|
||||
task container, not in an interpreter embeddable into any binary.
|
||||
- **A package-level process-global registrar.** Global mutable registry state
|
||||
is unreadable at the composition root, collides with itself when one
|
||||
process builds the vocabulary twice, and is what made the first attempt at
|
||||
this question worse instead of better.
|
||||
- **Degrading on an unknown step type.** A run that skips steps it cannot
|
||||
compile would finish green while silently dropping the behaviour the
|
||||
definition asked for; refusal at save and at run is the only honest option.
|
||||
@@ -1408,6 +1408,7 @@ func (b *boot) buildDaemon() {
|
||||
KindWorkflows: b.kindWorkflows,
|
||||
LabelWorkflows: b.labelWorkflows,
|
||||
WorkflowDefinitions: b.workflowDefinitions,
|
||||
WorkflowEnablement: b.controlPlane,
|
||||
Playbooks: b.playbooks,
|
||||
}
|
||||
if identities, ok := b.stateStore.(identity.Repository); ok {
|
||||
|
||||
@@ -100,6 +100,80 @@ func (m *preparedChatTurnModel) ToolSummaries() []gateway.ToolSummary {
|
||||
return append([]gateway.ToolSummary(nil), m.toolInfo...)
|
||||
}
|
||||
|
||||
// buildTurnMessages maps the gateway's compressed history onto ai-sdk chat
|
||||
// messages. The turn's inbound media is not part of stored history, so it
|
||||
// is merged here into the final user message as content parts: the model
|
||||
// sees the caption text plus each attachment's bytes (or URL), exactly for
|
||||
// the turn it arrived in.
|
||||
func buildTurnMessages(request gateway.TurnModelRequest) []chat.Message {
|
||||
messages := make([]chat.Message, len(request.Messages))
|
||||
for i, message := range request.Messages {
|
||||
role := chat.RoleUser
|
||||
switch message.Role {
|
||||
case "assistant":
|
||||
role = chat.RoleAssistant
|
||||
case "system":
|
||||
role = chat.RoleSystem
|
||||
}
|
||||
messages[i] = chat.Message{Role: role, Content: message.Content}
|
||||
}
|
||||
parts := mediaParts(request.Media)
|
||||
if len(parts) == 0 {
|
||||
return messages
|
||||
}
|
||||
last := len(messages) - 1
|
||||
// The media belongs to the inbound message, which is always the final
|
||||
// user entry; if the view ever ends differently, content stays intact
|
||||
// over silently attaching parts to an unrelated message.
|
||||
if last < 0 || messages[last].Role != chat.RoleUser {
|
||||
return messages
|
||||
}
|
||||
userMessages := messages[last]
|
||||
userMessages.Parts = append(chat.Parts{chat.TextPart{Text: userMessages.Content}}, parts...)
|
||||
userMessages.Content = ""
|
||||
messages[last] = userMessages
|
||||
return messages
|
||||
}
|
||||
|
||||
// mediaParts converts inbound media attachments into ai-sdk content
|
||||
// parts: images get ImagePart, everything else FilePart. An attachment
|
||||
// carrying neither bytes nor a URL names nothing consumable and is
|
||||
// dropped -- the turn's text still reaches the model.
|
||||
func mediaParts(media []gateway.MediaAttachment) []chat.Part {
|
||||
if len(media) == 0 {
|
||||
return nil
|
||||
}
|
||||
parts := make([]chat.Part, 0, len(media))
|
||||
for _, att := range media {
|
||||
if att.Type == "image" {
|
||||
switch {
|
||||
case len(att.Data) > 0 && att.MIMEType != "":
|
||||
parts = append(parts, chat.ImagePart{Data: att.Data, MediaType: att.MIMEType})
|
||||
case att.URL != "":
|
||||
parts = append(parts, chat.ImagePart{URL: att.URL})
|
||||
}
|
||||
continue
|
||||
}
|
||||
part := chat.FilePart{Name: att.FileName, MediaType: att.MIMEType}
|
||||
if len(att.Data) > 0 {
|
||||
part.Data = att.Data
|
||||
} else {
|
||||
part.URL = att.URL
|
||||
}
|
||||
if len(part.Data) == 0 && part.URL == "" {
|
||||
continue
|
||||
}
|
||||
if part.MediaType == "" {
|
||||
part.MediaType = "application/octet-stream"
|
||||
}
|
||||
parts = append(parts, part)
|
||||
}
|
||||
if len(parts) == 0 {
|
||||
return nil
|
||||
}
|
||||
return parts
|
||||
}
|
||||
|
||||
func (m *preparedChatTurnModel) ToolSchemaTokens() int {
|
||||
return m.toolTokens
|
||||
}
|
||||
@@ -110,17 +184,7 @@ func (m *preparedChatTurnModel) Generate(
|
||||
stream gateway.TurnStream,
|
||||
) (string, error) {
|
||||
options := m.options
|
||||
options.Messages = make([]chat.Message, len(request.Messages))
|
||||
for i, message := range request.Messages {
|
||||
role := chat.RoleUser
|
||||
switch message.Role {
|
||||
case "assistant":
|
||||
role = chat.RoleAssistant
|
||||
case "system":
|
||||
role = chat.RoleSystem
|
||||
}
|
||||
options.Messages[i] = chat.Message{Role: role, Content: message.Content}
|
||||
}
|
||||
options.Messages = buildTurnMessages(request)
|
||||
// This is one provider response's output allowance, not a turn-
|
||||
// continuation budget. Tool loops remain free to continue.
|
||||
options.MaxTokens = request.MaxOutputTokens
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
package archied
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/samcharles93/ai-sdk/chat"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/gateway"
|
||||
)
|
||||
|
||||
// TestBuildTurnMessagesAttachesMediaPartsToTheFinalUserMessage: inbound
|
||||
// media only reaches the model when the turn's request is turned into
|
||||
// message parts; the attachment belongs on the message the user just sent.
|
||||
func TestBuildTurnMessagesAttachesMediaPartsToTheFinalUserMessage(t *testing.T) {
|
||||
request := gateway.TurnModelRequest{
|
||||
Messages: []gateway.CompressedMessage{
|
||||
{Role: "system", Content: "sys"},
|
||||
{Role: "user", Content: "what is this?"},
|
||||
},
|
||||
Media: []gateway.MediaAttachment{
|
||||
{Type: "image", MIMEType: "image/png", Data: []byte("png-bytes")},
|
||||
{Type: "image", URL: "https://example.invalid/pic.jpg"},
|
||||
{Type: "document", MIMEType: "application/pdf", FileName: "report.pdf", Data: []byte("pdf-bytes")},
|
||||
},
|
||||
}
|
||||
|
||||
messages := buildTurnMessages(request)
|
||||
if len(messages) != 2 {
|
||||
t.Fatalf("messages = %d, want 2", len(messages))
|
||||
}
|
||||
if messages[0].Content != "sys" || len(messages[0].Parts) != 0 {
|
||||
t.Errorf("system message = %#v, want plain content and no parts", messages[0])
|
||||
}
|
||||
if messages[1].Content != "" {
|
||||
t.Errorf("user content = %q, want it moved into a text part", messages[1].Content)
|
||||
}
|
||||
parts := messages[1].Parts
|
||||
if len(parts) != 4 {
|
||||
t.Fatalf("user parts = %d, want text plus three media parts: %#v", len(parts), parts)
|
||||
}
|
||||
if text, ok := parts[0].(chat.TextPart); !ok || text.Text != "what is this?" {
|
||||
t.Errorf("part 0 = %#v, want the caption text", parts[0])
|
||||
}
|
||||
image, ok := parts[1].(chat.ImagePart)
|
||||
if !ok || string(image.Data) != "png-bytes" || image.MediaType != "image/png" {
|
||||
t.Errorf("part 1 = %#v, want an image part carrying the png bytes", parts[1])
|
||||
}
|
||||
linked, ok := parts[2].(chat.ImagePart)
|
||||
if !ok || linked.URL != "https://example.invalid/pic.jpg" {
|
||||
t.Errorf("part 2 = %#v, want an image part pointing at the URL", parts[2])
|
||||
}
|
||||
file, ok := parts[3].(chat.FilePart)
|
||||
if !ok {
|
||||
t.Fatalf("part 3 = %#v, want a file part", parts[3])
|
||||
}
|
||||
if string(file.Data) != "pdf-bytes" || file.MediaType != "application/pdf" || file.Name != "report.pdf" {
|
||||
t.Errorf("file part = %#v, want the pdf bytes with type and name", file)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBuildTurnMessagesSkipsUndeliverableMedia: an attachment carrying
|
||||
// neither bytes nor a URL names nothing the model can consume, so it must
|
||||
// be dropped rather than sent as an empty part.
|
||||
func TestBuildTurnMessagesSkipsUndeliverableMedia(t *testing.T) {
|
||||
request := gateway.TurnModelRequest{
|
||||
Messages: []gateway.CompressedMessage{
|
||||
{Role: "user", Content: "look"},
|
||||
},
|
||||
Media: []gateway.MediaAttachment{
|
||||
{Type: "video"},
|
||||
{Type: "image", MIMEType: "image/png", Data: []byte("png")},
|
||||
},
|
||||
}
|
||||
messages := buildTurnMessages(request)
|
||||
fileSeen := false
|
||||
for _, p := range messages[0].Parts {
|
||||
if _, ok := p.(chat.FilePart); ok {
|
||||
fileSeen = true
|
||||
}
|
||||
}
|
||||
if fileSeen {
|
||||
t.Errorf("parts = %#v, want the empty attachment dropped", messages[0].Parts)
|
||||
}
|
||||
if len(messages[0].Parts) != 2 {
|
||||
t.Fatalf("user parts = %d, want text plus the one deliverable image", len(messages[0].Parts))
|
||||
}
|
||||
if text, ok := messages[0].Parts[0].(chat.TextPart); !ok || text.Text != "look" {
|
||||
t.Errorf("part 0 = %#v, want the text part", messages[0].Parts[0])
|
||||
}
|
||||
}
|
||||
|
||||
// TestBuildTurnMessagesWithoutMediaKeepsPlainContent is the regression
|
||||
// guard for the overwhelmingly common case: text-only turns stay exactly as
|
||||
// they were, with Content and no parts.
|
||||
func TestBuildTurnMessagesWithoutMediaKeepsPlainContent(t *testing.T) {
|
||||
request := gateway.TurnModelRequest{
|
||||
Messages: []gateway.CompressedMessage{
|
||||
{Role: "system", Content: "sys"},
|
||||
{Role: "user", Content: "plain"},
|
||||
{Role: "assistant", Content: "reply"},
|
||||
},
|
||||
}
|
||||
messages := buildTurnMessages(request)
|
||||
for i, want := range []struct {
|
||||
role chat.Role
|
||||
content string
|
||||
}{
|
||||
{chat.RoleSystem, "sys"},
|
||||
{chat.RoleUser, "plain"},
|
||||
{chat.RoleAssistant, "reply"},
|
||||
} {
|
||||
if messages[i].Content != want.content || messages[i].Parts != nil {
|
||||
t.Errorf("message %d = %#v, want content %q and no parts", i, messages[i], want.content)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -47,7 +47,7 @@ func (s *outageStore) down(stall time.Duration) {
|
||||
s.failing, s.stall = true, stall
|
||||
}
|
||||
|
||||
func (s *outageStore) Resource(ctx context.Context, kind string) (storecontract.Resource, error) {
|
||||
func (s *outageStore) Resource(ctx context.Context, orgID, kind string) (storecontract.Resource, error) {
|
||||
s.mu.Lock()
|
||||
failing, stall := s.failing, s.stall
|
||||
s.mu.Unlock()
|
||||
@@ -55,7 +55,7 @@ func (s *outageStore) Resource(ctx context.Context, kind string) (storecontract.
|
||||
time.Sleep(stall)
|
||||
return storecontract.Resource{}, errors.New("state store unreachable")
|
||||
}
|
||||
return s.TaskDB.Resource(ctx, kind)
|
||||
return s.TaskDB.Resource(ctx, orgID, kind)
|
||||
}
|
||||
|
||||
// TestWatchRetryGrowsWhileTheStateStoreIsDown is the outage this reconnect
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
pb "github.com/samcharles93/archie-core/internal/contracts/gateway/v1"
|
||||
"github.com/samcharles93/archie-core/internal/domain/messaging"
|
||||
"github.com/samcharles93/archie-core/internal/domain/taskactions"
|
||||
"github.com/samcharles93/archie-core/internal/sdnotify"
|
||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||
)
|
||||
|
||||
@@ -116,7 +117,7 @@ func TestGatewayServingAnnouncesReady(t *testing.T) {
|
||||
serveCh := make(chan error, 1)
|
||||
go func() { serveCh <- b.serveGatewayListener(ctx, ln, true, stubGatewayContract{}, nil) }()
|
||||
|
||||
if !listener.waitForState(t, 5*time.Second, readyState) {
|
||||
if !listener.waitForState(t, 5*time.Second, sdnotify.ReadyState) {
|
||||
t.Fatal("no READY=1 datagram within 5s of the Gateway serving")
|
||||
}
|
||||
|
||||
|
||||
@@ -81,7 +81,7 @@ func (b *boot) setupKitLauncher(ctx context.Context) {
|
||||
b.kitLauncher = &kitrun.Launcher{
|
||||
Pool: pool, Fetch: fetcher, Proxy: proxy, Networks: networks,
|
||||
AgentBinary: agentBinary, CAFile: egress.CACertPath(caDir),
|
||||
Config: b.cfgHolder, Secrets: b.secrets, Grants: grants,
|
||||
Config: b.cfgHolder, Secrets: b.secrets, Grants: grants, OAuth: oauthStore,
|
||||
}
|
||||
log.Info("kit harness runs enabled", "proxy", ln.Addr().String())
|
||||
}
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
package archied
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"context"
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"net"
|
||||
"testing"
|
||||
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/credentials/insecure"
|
||||
"google.golang.org/grpc/test/bufconn"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/app/controlplane"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
|
||||
)
|
||||
|
||||
// contributedLayer ships one workflow definition file's portable YAML.
|
||||
func contributedLayer(t *testing.T, path, yaml string) []byte {
|
||||
t.Helper()
|
||||
var raw bytes.Buffer
|
||||
writer := tar.NewWriter(&raw)
|
||||
if err := writer.WriteHeader(&tar.Header{Typeflag: tar.TypeReg, Name: path, Size: int64(len(yaml))}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := writer.Write([]byte(yaml)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return raw.Bytes()
|
||||
}
|
||||
|
||||
func packageDescriptor() storepkg.Descriptor {
|
||||
return storepkg.Descriptor{
|
||||
APIVersion: storepkg.APIVersion, DisplayName: "Review pack", Version: "1",
|
||||
Contributes: storepkg.Contributions{Workflows: []string{"workflows/review.yaml"}},
|
||||
Files: []storepkg.File{{Path: "workflows/review.yaml", Mode: 0o644}},
|
||||
}
|
||||
}
|
||||
|
||||
// TestPackageContributionsLiveRoundTrip drives the State Store's own
|
||||
// composition — the projections stateStoreDeps wires, the install and remove
|
||||
// RPCs the dashboard calls, and the workflow-definitions resource the daemon
|
||||
// reads per dispatch — over one real gRPC connection: installing a package
|
||||
// makes its workflow a usable org resource, and removing it stops being
|
||||
// usable, both without restarting anything.
|
||||
func TestPackageContributionsLiveRoundTrip(t *testing.T) {
|
||||
db := pgstore.Open(t)
|
||||
projections, err := newPackageProjections(db.Pool)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
const yaml = "id: review\nsteps:\n - type: bootstrap.apply\n"
|
||||
service := storepkg.Service{
|
||||
Registry: packageLayerRegistry{descriptor: packageDescriptor(), layer: contributedLayer(t, "workflows/review.yaml", yaml)},
|
||||
Store: postgres.NewInstalledPackages(db.Pool),
|
||||
Projections: projections,
|
||||
}
|
||||
listener := bufconn.Listen(1 << 20)
|
||||
server := grpc.NewServer()
|
||||
staterpc.RegisterServer(server, staterpc.Deps{Packages: service})
|
||||
go func() { _ = server.Serve(listener) }()
|
||||
t.Cleanup(func() { server.Stop(); _ = listener.Close() })
|
||||
conn, err := grpc.NewClient("passthrough:///state",
|
||||
grpc.WithTransportCredentials(insecure.NewCredentials()),
|
||||
grpc.WithContextDialer(func(ctx context.Context, _ string) (net.Conn, error) { return listener.DialContext(ctx) }))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = conn.Close() })
|
||||
client := staterpc.NewClient(conn)
|
||||
const digest = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
||||
|
||||
if _, err := client.InstallPackage(t.Context(), "org-a", "review", "localhost:5000/review", digest); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// The contribution is usable: the org resource the daemon's
|
||||
// WorkflowDefinitions client reads carries it now, no restart involved.
|
||||
resource, err := postgres.NewResources(db.Pool).Resource(t.Context(), "org-a", controlplane.WorkflowDefinitionsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
collection, err := workflow.DecodeDefinitionCollection(resource.Value, workflow.BuiltinStepRegistry())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := collection.DefinitionByID("review"); !ok {
|
||||
t.Fatal("installed package's workflow is not an org workflow after install")
|
||||
}
|
||||
if err := client.RemoveInstalled(t.Context(), "org-a", "review"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resource, err = postgres.NewResources(db.Pool).Resource(t.Context(), "org-a", controlplane.WorkflowDefinitionsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
collection, err = workflow.DecodeDefinitionCollection(resource.Value, workflow.BuiltinStepRegistry())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := collection.DefinitionByID("review"); ok {
|
||||
t.Fatal("removed package's workflow still dispatches: the org resource kept it")
|
||||
}
|
||||
if entries, err := postgres.NewPackageContributions(db.Pool).Entries(t.Context(), "org-a", "review"); err != nil || len(entries) != 0 {
|
||||
t.Fatalf("contributions ledger after removal = %#v, %v", entries, err)
|
||||
}
|
||||
}
|
||||
|
||||
type packageLayerRegistry struct {
|
||||
descriptor storepkg.Descriptor
|
||||
layer []byte
|
||||
}
|
||||
|
||||
func (r packageLayerRegistry) Fetch(context.Context, string, string) (storepkg.Descriptor, []byte, error) {
|
||||
return r.descriptor, r.layer, nil
|
||||
}
|
||||
|
||||
// TestStateStoreDepsWiresPackageProjections pins the wiring the live round
|
||||
// trip drives: stateStoreDeps builds projections on the State Store's pool and
|
||||
// carries them into the packages service it composes.
|
||||
func TestStateStoreDepsWiresPackageProjections(t *testing.T) {
|
||||
file := "state_store.go"
|
||||
fset := token.NewFileSet()
|
||||
parsed, err := parser.ParseFile(fset, file, nil, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var calls []string
|
||||
ast.Inspect(parsed, func(node ast.Node) bool {
|
||||
if call, ok := node.(*ast.CallExpr); ok {
|
||||
if fn, ok := call.Fun.(*ast.Ident); ok {
|
||||
calls = append(calls, fn.Name)
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
found := false
|
||||
for _, call := range calls {
|
||||
if call == "newPackageProjections" {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatalf("stateStoreDeps (%s body) wires no package projections", file)
|
||||
}
|
||||
}
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/app/controlplane"
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgtest"
|
||||
"github.com/samcharles93/archie-core/internal/secret"
|
||||
@@ -51,7 +52,7 @@ func TestStateStoreBootSeedsProviderSourceReference(t *testing.T) {
|
||||
go func() { errCh <- RunStateStore(ctx, StateStoreOptions{Config: path, Listen: "127.0.0.1:0"}) }()
|
||||
deadline := time.After(10 * time.Second)
|
||||
for {
|
||||
resource, err := store.Resource(t.Context(), controlplane.ProviderSettingsKind)
|
||||
resource, err := store.Resource(t.Context(), storecontract.DefaultOrgID, controlplane.ProviderSettingsKind)
|
||||
if err == nil {
|
||||
if strings.Contains(string(resource.Value), "ARCHIE_PROVIDER_") || !strings.Contains(string(resource.Value), ref.Key) {
|
||||
t.Fatalf("stored provider settings = %s, want source reference and no derived env name", resource.Value)
|
||||
|
||||
@@ -1,22 +1,18 @@
|
||||
// sd_notify.go implements systemd's sd_notify(3) protocol for archied: one
|
||||
// READY=1 when the run loop starts serving, and a WATCHDOG=1 heartbeat while
|
||||
// the run loop is demonstrably still making progress.
|
||||
// sd_notify.go wires the daemon into internal/sdnotify's implementation of
|
||||
// systemd's sd_notify(3) protocol: one READY=1 when the run loop starts
|
||||
// serving, and a WATCHDOG=1 heartbeat while the run loop is demonstrably still
|
||||
// making progress.
|
||||
//
|
||||
// The protocol is three environment variables and one datagram per state, so
|
||||
// this file speaks it directly instead of taking a module dependency: the whole
|
||||
// client is net.DialUnix plus a Write. Nothing here is fatal -- a process with
|
||||
// no NOTIFY_SOCKET (a plain terminal, Docker, a user unit without WatchdogSec)
|
||||
// sends nothing and serves exactly as before, and a socket that cannot be
|
||||
// written to is logged and ignored.
|
||||
// The protocol client and the heartbeat sampler live in internal/sdnotify
|
||||
// because archie-messaging, a separate process that cannot import this
|
||||
// package, is the next process that must be able to announce itself
|
||||
// (archie-core-g5g8). What stays here is what is the daemon's own: which
|
||||
// marker counts as progress (daemon.LastPollAt) and how long the loop may go
|
||||
// without beginning a poll pass.
|
||||
//
|
||||
// The heartbeat is deliberately not a bare ticker. A ticker fires whether or
|
||||
// not the loop it stands for is alive, so it would keep systemd satisfied while
|
||||
// the event loop was wedged -- the exact failure the watchdog exists to catch.
|
||||
// The sampler instead reads the daemon's own progress marker
|
||||
// (daemon.LastPollAt, stamped by the poll loop at the start of every pass) and
|
||||
// withholds the heartbeat once that marker is older than the loop's own poll
|
||||
// cadence. While passes keep beginning, beats flow at half of WatchdogSec; a
|
||||
// loop blocked inside a pass stops being reported as alive.
|
||||
// Nothing here is fatal -- a process with no NOTIFY_SOCKET (a plain terminal,
|
||||
// Docker, a user unit without WatchdogSec) sends nothing and serves exactly as
|
||||
// before, and a socket that cannot be written to is logged and ignored.
|
||||
//
|
||||
// Consuming the states is the unit's business: READY=1 needs Type=notify, and
|
||||
// the heartbeat needs WatchdogSec.
|
||||
@@ -25,75 +21,12 @@ package archied
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"net"
|
||||
"os"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/sdnotify"
|
||||
)
|
||||
|
||||
const (
|
||||
// notifySocketEnv names the socket a manager passes to a Type=notify unit.
|
||||
notifySocketEnv = "NOTIFY_SOCKET"
|
||||
// watchdogUsecEnv carries WatchdogSec in microseconds.
|
||||
watchdogUsecEnv = "WATCHDOG_USEC"
|
||||
// watchdogPIDEnv names the process whose beats the watchdog counts.
|
||||
watchdogPIDEnv = "WATCHDOG_PID"
|
||||
|
||||
// readyState and watchdogState are the two sd_notify states archied sends.
|
||||
readyState = "READY=1"
|
||||
watchdogState = "WATCHDOG=1"
|
||||
|
||||
// watchdogHeartbeatDivisor is sd_notify(3)'s recommendation: beat at half
|
||||
// of WatchdogSec, so one missed beat is tolerated and two are not.
|
||||
watchdogHeartbeatDivisor = 2
|
||||
)
|
||||
|
||||
// notifier writes sd_notify states to the socket named in the environment. A
|
||||
// notifier with no address -- the normal case away from systemd -- sends
|
||||
// nothing and says nothing.
|
||||
type notifier struct {
|
||||
addr string
|
||||
log *slog.Logger
|
||||
}
|
||||
|
||||
// newNotifier resolves the notify socket. The address is read from the
|
||||
// environment systemd sets for the process, so systemd replaces the previous
|
||||
// unit's socket with the new one across a restart.
|
||||
func newNotifier(getenv func(string) string, log *slog.Logger) notifier {
|
||||
return notifier{addr: getenv(notifySocketEnv), log: log}
|
||||
}
|
||||
|
||||
// send delivers one state line. A failure is logged, never returned: a
|
||||
// notification is an observation about the daemon, and no observation is worth
|
||||
// refusing to serve over.
|
||||
func (n notifier) send(state string) {
|
||||
if n.addr == "" {
|
||||
return
|
||||
}
|
||||
if err := sendNotify(n.addr, state); err != nil {
|
||||
n.log.Warn("systemd notify failed", "state", state, "socket", n.addr, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
// sendNotify writes one state datagram to a unixgram socket. sd_notify(3) is
|
||||
// one datagram per state, so each send dials afresh: a manager restart recreates
|
||||
// its socket, and a connection cached across one would fail every send after it.
|
||||
// A leading '@' names the Linux abstract namespace, which user managers use;
|
||||
// net.UnixAddr takes that syntax verbatim.
|
||||
func sendNotify(addr, state string) error {
|
||||
conn, err := net.DialUnix("unixgram", nil, &net.UnixAddr{Name: addr, Net: "unixgram"})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { _ = conn.Close() }()
|
||||
if _, err := conn.Write([]byte(state)); err != nil {
|
||||
return fmt.Errorf("write %q: %w", state, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// announceReady tells systemd the daemon has finished starting. It is called at
|
||||
// the moment a process declares itself serving: the daemon's run loop (next to
|
||||
// healthSurface.markServing), and the serve steps of archie-gateway and
|
||||
@@ -101,42 +34,35 @@ func sendNotify(addr, state string) error {
|
||||
// own and would otherwise hang a Type=notify unit until TimeoutStartSec while
|
||||
// perfectly healthy.
|
||||
func (b *boot) announceReady() {
|
||||
notifyReady(b.log)
|
||||
}
|
||||
|
||||
// notifyReady is announceReady without the boot: the Gateway's serving tail
|
||||
// runs on a boot it owns, but a free helper keeps the announcement one call
|
||||
// rather than a second notion of "started" on each surface.
|
||||
func notifyReady(log *slog.Logger) {
|
||||
newNotifier(os.Getenv, log).send(readyState)
|
||||
sdnotify.Ready(b.log)
|
||||
}
|
||||
|
||||
// startWatchdog arms the heartbeat sampler for the daemon's run loop. It does
|
||||
// nothing unless a manager named a notify socket and asked for a watchdog
|
||||
// (WATCHDOG_USEC), so every deployment without systemd is unaffected.
|
||||
func (b *boot) startWatchdog(ctx context.Context) {
|
||||
n := newNotifier(os.Getenv, b.log)
|
||||
if n.addr == "" {
|
||||
n := sdnotify.New(os.Getenv, b.log)
|
||||
if !n.Enabled() {
|
||||
return
|
||||
}
|
||||
interval, ok := watchdogHeartbeat(os.Getenv, os.Getpid(), b.log)
|
||||
interval, ok := sdnotify.Heartbeat(os.Getenv, os.Getpid(), b.log)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
|
||||
sampler := loopWatchdog{
|
||||
interval: interval,
|
||||
cadence: b.loopCadence,
|
||||
progress: b.d.LastPollAt,
|
||||
beat: func() { n.send(watchdogState) },
|
||||
log: b.log,
|
||||
sampler := sdnotify.Watchdog{
|
||||
Interval: interval,
|
||||
Cadence: b.loopCadence,
|
||||
Progress: b.d.LastPollAt,
|
||||
Beat: func() { n.Send(sdnotify.WatchdogState) },
|
||||
Log: b.log,
|
||||
}
|
||||
b.log.Info("systemd watchdog armed",
|
||||
"heartbeat", interval.String(),
|
||||
"loop_cadence", b.loopCadence().String())
|
||||
|
||||
go func() {
|
||||
if err := sampler.run(ctx); err != nil && !errors.Is(err, context.Canceled) {
|
||||
if err := sampler.Run(ctx); err != nil && !errors.Is(err, context.Canceled) {
|
||||
b.log.Error("systemd watchdog stopped", "err", err)
|
||||
}
|
||||
}()
|
||||
@@ -161,86 +87,3 @@ func (b *boot) loopCadence() time.Duration {
|
||||
}
|
||||
return cadence
|
||||
}
|
||||
|
||||
// watchdogHeartbeat reads systemd's watchdog contract and returns the heartbeat
|
||||
// cadence, which is half of WatchdogSec. ok is false when systemd asked for no
|
||||
// watchdog, and also when it asked for one from a different process:
|
||||
// WATCHDOG_PID names the process whose beats count, so a unit watching another
|
||||
// process must not have this one beating on its behalf.
|
||||
func watchdogHeartbeat(getenv func(string) string, pid int, log *slog.Logger) (time.Duration, bool) {
|
||||
raw := getenv(watchdogUsecEnv)
|
||||
if raw == "" {
|
||||
return 0, false
|
||||
}
|
||||
if rawPID := getenv(watchdogPIDEnv); rawPID != "" {
|
||||
wantPID, err := strconv.Atoi(rawPID)
|
||||
if err != nil || wantPID != pid {
|
||||
log.Warn("systemd watchdog disabled: WATCHDOG_PID is not this process",
|
||||
"watchdog_pid", rawPID, "pid", pid, "err", err)
|
||||
return 0, false
|
||||
}
|
||||
}
|
||||
watchdog, err := time.ParseDuration(raw + "us")
|
||||
if err != nil || watchdog <= 0 {
|
||||
log.Warn("systemd watchdog disabled: WATCHDOG_USEC unreadable", "value", raw, "err", err)
|
||||
return 0, false
|
||||
}
|
||||
return watchdog / watchdogHeartbeatDivisor, true
|
||||
}
|
||||
|
||||
// loopWatchdog samples the run loop's progress marker and emits the watchdog
|
||||
// heartbeat only while that marker is fresh. Sampling is periodic; the beat is
|
||||
// not, because it is licensed by the marker rather than by the clock.
|
||||
type loopWatchdog struct {
|
||||
// interval is the sampling cadence: half of systemd's WatchdogSec.
|
||||
interval time.Duration
|
||||
// cadence reports how long the loop may go between poll passes.
|
||||
cadence func() time.Duration
|
||||
// progress reports the start of the loop's most recent poll pass, or the
|
||||
// zero time when no pass has begun yet.
|
||||
progress func() time.Time
|
||||
// beat emits one WATCHDOG=1.
|
||||
beat func()
|
||||
log *slog.Logger
|
||||
}
|
||||
|
||||
// run samples until ctx ends. It returns ctx.Err() on shutdown, which is the
|
||||
// only way out: this sampler never gives up on its own, because a sampler that
|
||||
// stopped would leave systemd watching a process nothing reports for.
|
||||
func (w loopWatchdog) run(ctx context.Context) error {
|
||||
armedAt := time.Now()
|
||||
stalled := false
|
||||
ticker := time.NewTicker(w.interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-ticker.C:
|
||||
}
|
||||
|
||||
last := w.progress()
|
||||
if last.IsZero() {
|
||||
// No pass has begun at all. Measure staleness from arming, so a
|
||||
// loop that never starts polling is caught like one that stopped.
|
||||
last = armedAt
|
||||
}
|
||||
cadence := w.cadence()
|
||||
switch age := time.Since(last); {
|
||||
case age > cadence:
|
||||
if !stalled {
|
||||
w.log.Warn("systemd watchdog: run loop has not begun a poll pass within its interval; withholding the heartbeat until it does",
|
||||
"stale_for", age.String(), "loop_cadence", cadence.String())
|
||||
stalled = true
|
||||
}
|
||||
default:
|
||||
if stalled {
|
||||
w.log.Info("systemd watchdog: run loop resumed; heartbeats resumed",
|
||||
"last_pass_age", age.String())
|
||||
stalled = false
|
||||
}
|
||||
w.beat()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -18,8 +18,16 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/daemon"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgtest"
|
||||
"github.com/samcharles93/archie-core/internal/sdnotify"
|
||||
)
|
||||
|
||||
// The protocol-level properties of the notify client -- the '@' address form,
|
||||
// the disabled and unusable socket contract, and the half-of-WatchdogSec
|
||||
// heartbeat -- are pinned in internal/sdnotify, where the client now lives.
|
||||
// These tests pin what is the daemon's: that its three serving entry points
|
||||
// announce, that the sampler samples daemon.LastPollAt, and that the whole
|
||||
// path stays silent and harmless without a NOTIFY_SOCKET.
|
||||
|
||||
// notifyRecorder captures every record at every level. The other recording
|
||||
// handler in this package keeps warnings only; these tests need "this path
|
||||
// logged nothing at all" as positive evidence that a disabled notifier stayed
|
||||
@@ -49,6 +57,7 @@ func (h *notifyRecorder) all() []slog.Record {
|
||||
return append([]slog.Record(nil), h.records...)
|
||||
}
|
||||
|
||||
// at returns the captured records at one level.
|
||||
func (h *notifyRecorder) at(level slog.Level) []slog.Record {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
@@ -83,7 +92,7 @@ func listenNotify(t *testing.T, addr string) *notifyListener {
|
||||
t.Fatalf("listen for notify datagrams at %q: %v", addr, err)
|
||||
}
|
||||
t.Cleanup(func() { _ = conn.Close() })
|
||||
t.Setenv(notifySocketEnv, addr)
|
||||
t.Setenv(sdnotify.NotifySocketEnv, addr)
|
||||
return ¬ifyListener{conn: conn}
|
||||
}
|
||||
|
||||
@@ -174,93 +183,12 @@ func pollPasses(ctx context.Context, d *daemon.Daemon, on *atomic.Bool, every ti
|
||||
}
|
||||
}
|
||||
|
||||
// TestNotifyReachesAbstractSocket pins the '@' address form. systemd is not
|
||||
// restricted to filesystem paths, and a client that stripped the prefix would
|
||||
// connect to a file that does not exist instead of the abstract namespace.
|
||||
func TestNotifyReachesAbstractSocket(t *testing.T) {
|
||||
listener := listenNotify(t, fmt.Sprintf("@archie-notify-test-%d", os.Getpid()))
|
||||
|
||||
(&boot{log: slog.New(slog.DiscardHandler)}).announceReady()
|
||||
|
||||
if states := listener.states(t, 300*time.Millisecond); !slices.Contains(states, readyState) {
|
||||
t.Errorf("states received = %v, want %q among them", states, readyState)
|
||||
}
|
||||
}
|
||||
|
||||
// TestNotifierDisabledIsSilentAndUnusableSocketWarns pins the two ends of the
|
||||
// delivery contract: no socket means nothing is attempted and nothing is said,
|
||||
// while a socket that cannot be written to is reported once and then ignored.
|
||||
// Neither may ever be fatal.
|
||||
func TestNotifierDisabledIsSilentAndUnusableSocketWarns(t *testing.T) {
|
||||
rec := ¬ifyRecorder{}
|
||||
log := slog.New(rec)
|
||||
|
||||
newNotifier(func(string) string { return "" }, log).send(readyState)
|
||||
if got := rec.all(); len(got) != 0 {
|
||||
t.Errorf("a notifier with no socket logged %d records, want none: %v", len(got), got)
|
||||
}
|
||||
|
||||
missing := filepath.Join(t.TempDir(), "absent", "n.sock")
|
||||
newNotifier(func(string) string { return missing }, log).send(readyState)
|
||||
warns := rec.at(slog.LevelWarn)
|
||||
if len(warns) != 1 {
|
||||
t.Fatalf("sends to an unusable socket logged %d warnings, want exactly 1", len(warns))
|
||||
}
|
||||
if errs := rec.at(slog.LevelError); len(errs) != 0 {
|
||||
t.Errorf("sends to an unusable socket logged %d errors, want none", len(errs))
|
||||
}
|
||||
}
|
||||
|
||||
// TestWatchdogHeartbeatInterval pins the interval convention: systemd passes
|
||||
// WatchdogSec as WATCHDOG_USEC and sd_notify(3) heartbeats at half of it, and
|
||||
// WATCHDOG_PID names the process whose beats count.
|
||||
func TestWatchdogHeartbeatInterval(t *testing.T) {
|
||||
const self = 42
|
||||
tests := []struct {
|
||||
name string
|
||||
usec string
|
||||
pidEnv string
|
||||
want time.Duration
|
||||
wantOK bool
|
||||
}{
|
||||
{name: "no watchdog requested", wantOK: false},
|
||||
{name: "half of a 30s watchdog", usec: "30000000", want: 15 * time.Second, wantOK: true},
|
||||
{name: "sub-second watchdog", usec: "20000", want: 10 * time.Millisecond, wantOK: true},
|
||||
{name: "unparseable", usec: "soon", wantOK: false},
|
||||
{name: "zero", usec: "0", wantOK: false},
|
||||
{name: "negative", usec: "-1", wantOK: false},
|
||||
{name: "watchdog pid is this process", usec: "20000", pidEnv: "42", want: 10 * time.Millisecond, wantOK: true},
|
||||
{name: "watchdog pid is another process", usec: "20000", pidEnv: "7", wantOK: false},
|
||||
{name: "watchdog pid is unreadable", usec: "20000", pidEnv: "seven", wantOK: false},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
getenv := func(key string) string {
|
||||
switch key {
|
||||
case watchdogUsecEnv:
|
||||
return tt.usec
|
||||
case watchdogPIDEnv:
|
||||
return tt.pidEnv
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
got, ok := watchdogHeartbeat(getenv, self, slog.New(slog.DiscardHandler))
|
||||
if ok != tt.wantOK || got != tt.want {
|
||||
t.Errorf("watchdogHeartbeat(usec=%q, pid=%q) = %v, %v; want %v, %v",
|
||||
tt.usec, tt.pidEnv, got, ok, tt.want, tt.wantOK)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestNotifyReadyAndWatchdogHeartbeat is the run-loop path end to end: a real
|
||||
// NOTIFY_SOCKET, the daemon's own boot path announcing READY, and the run
|
||||
// loop's own progress marker producing WATCHDOG heartbeats.
|
||||
func TestNotifyReadyAndWatchdogHeartbeat(t *testing.T) {
|
||||
listener := newNotifyListener(t)
|
||||
t.Setenv(watchdogUsecEnv, "20000") // 20ms watchdog: heartbeat every 10ms
|
||||
t.Setenv(sdnotify.WatchdogUsecEnv, "20000") // 20ms watchdog: heartbeat every 10ms
|
||||
|
||||
b, d := newNotifyTestBoot(t, 50*time.Millisecond)
|
||||
ctx, cancel := context.WithCancel(t.Context())
|
||||
@@ -274,11 +202,11 @@ func TestNotifyReadyAndWatchdogHeartbeat(t *testing.T) {
|
||||
b.startWatchdog(ctx)
|
||||
|
||||
states := listener.states(t, 300*time.Millisecond)
|
||||
if !slices.Contains(states, readyState) {
|
||||
t.Errorf("states received = %v, want %q among them", states, readyState)
|
||||
if !slices.Contains(states, sdnotify.ReadyState) {
|
||||
t.Errorf("states received = %v, want %q among them", states, sdnotify.ReadyState)
|
||||
}
|
||||
if countState(states, watchdogState) == 0 {
|
||||
t.Errorf("states received = %v, want at least one %q among them", states, watchdogState)
|
||||
if countState(states, sdnotify.WatchdogState) == 0 {
|
||||
t.Errorf("states received = %v, want at least one %q among them", states, sdnotify.WatchdogState)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -288,7 +216,7 @@ func TestNotifyReadyAndWatchdogHeartbeat(t *testing.T) {
|
||||
// kept alive by an independent ticker.
|
||||
func TestWatchdogWithholdsHeartbeatWhenRunLoopStalls(t *testing.T) {
|
||||
listener := newNotifyListener(t)
|
||||
t.Setenv(watchdogUsecEnv, "20000") // 20ms watchdog: heartbeat every 10ms
|
||||
t.Setenv(sdnotify.WatchdogUsecEnv, "20000") // 20ms watchdog: heartbeat every 10ms
|
||||
|
||||
const cadence = 50 * time.Millisecond
|
||||
b, d := newNotifyTestBoot(t, cadence)
|
||||
@@ -300,7 +228,7 @@ func TestWatchdogWithholdsHeartbeatWhenRunLoopStalls(t *testing.T) {
|
||||
go pollPasses(ctx, d, &polling, 5*time.Millisecond)
|
||||
|
||||
b.startWatchdog(ctx)
|
||||
if beats := countState(listener.states(t, 200*time.Millisecond), watchdogState); beats == 0 {
|
||||
if beats := countState(listener.states(t, 200*time.Millisecond), sdnotify.WatchdogState); beats == 0 {
|
||||
t.Fatal("no WATCHDOG=1 within 200ms while the run loop was polling")
|
||||
}
|
||||
|
||||
@@ -323,12 +251,12 @@ func TestWatchdogWithholdsHeartbeatWhenRunLoopStalls(t *testing.T) {
|
||||
// terminal, and a systemd unit with no WatchdogSec all take this path, and none
|
||||
// of them may see a datagram, a warning, or a failed start.
|
||||
func TestNotifyUnsetIsNoOp(t *testing.T) {
|
||||
t.Setenv(watchdogUsecEnv, "20000")
|
||||
t.Setenv(sdnotify.WatchdogUsecEnv, "20000")
|
||||
// t.Setenv cannot unset a variable: register the restoration first, then
|
||||
// remove it for real so "unset" means unset.
|
||||
t.Setenv(notifySocketEnv, "")
|
||||
if err := os.Unsetenv(notifySocketEnv); err != nil {
|
||||
t.Fatalf("unset %s: %v", notifySocketEnv, err)
|
||||
t.Setenv(sdnotify.NotifySocketEnv, "")
|
||||
if err := os.Unsetenv(sdnotify.NotifySocketEnv); err != nil {
|
||||
t.Fatalf("unset %s: %v", sdnotify.NotifySocketEnv, err)
|
||||
}
|
||||
|
||||
rec := ¬ifyRecorder{}
|
||||
@@ -349,8 +277,8 @@ func TestNotifyUnsetIsNoOp(t *testing.T) {
|
||||
// would be a regression on every deployment that has no systemd at all.
|
||||
func TestNotifySocketFailureIsNotFatal(t *testing.T) {
|
||||
missing := filepath.Join(t.TempDir(), "absent", "n.sock")
|
||||
t.Setenv(notifySocketEnv, missing)
|
||||
t.Setenv(watchdogUsecEnv, "20000")
|
||||
t.Setenv(sdnotify.NotifySocketEnv, missing)
|
||||
t.Setenv(sdnotify.WatchdogUsecEnv, "20000")
|
||||
|
||||
rec := ¬ifyRecorder{}
|
||||
b, d := newNotifyTestBoot(t, 50*time.Millisecond)
|
||||
@@ -405,7 +333,7 @@ func TestStateStoreBootAnnouncesReady(t *testing.T) {
|
||||
errCh := make(chan error, 1)
|
||||
go func() { errCh <- RunStateStore(ctx, StateStoreOptions{Config: path, Listen: "127.0.0.1:0"}) }()
|
||||
|
||||
if !listener.waitForState(t, 15*time.Second, readyState) {
|
||||
if !listener.waitForState(t, 15*time.Second, sdnotify.ReadyState) {
|
||||
select {
|
||||
case bootErr := <-errCh:
|
||||
t.Fatalf("RunStateStore exited before announcing ready: %v", bootErr)
|
||||
|
||||
@@ -17,6 +17,7 @@ import (
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
"google.golang.org/grpc"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/app/controlplane"
|
||||
@@ -289,10 +290,20 @@ func (b *boot) openStateStore(ctx context.Context) error {
|
||||
func (b *boot) stateStoreDeps(grants *staterpc.TaskGrants) staterpc.Deps {
|
||||
deps := staterpc.Deps{Tasks: b.st, Log: b.log, Grants: grants}
|
||||
if b.pg != nil {
|
||||
deps.Packages = storepkg.Service{
|
||||
packages := storepkg.Service{
|
||||
Registry: registry.LocalRegistry{},
|
||||
Store: postgres.NewInstalledPackages(b.pg),
|
||||
}
|
||||
// A projection failure degrades the org resources a package would have
|
||||
// contributed to, the way a group above degrades: the State Store boots
|
||||
// and reports, rather than refusing to serve the task tables.
|
||||
projections, err := newPackageProjections(b.pg)
|
||||
if err != nil {
|
||||
b.log.Error("build package projections", "err", err)
|
||||
} else {
|
||||
packages.Projections = projections
|
||||
}
|
||||
deps.Packages = packages
|
||||
}
|
||||
if identities, ok := b.st.(identity.Repository); ok {
|
||||
deps.Identities = identities
|
||||
@@ -494,3 +505,22 @@ func stateStoreServerOpts(listen, token string, grants *staterpc.TaskGrants) (op
|
||||
grpc.ChainStreamInterceptor(grants.StreamInterceptor(token)),
|
||||
}, false, nil
|
||||
}
|
||||
|
||||
// newPackageProjections builds the family projectors an installed package's
|
||||
// contributions apply through: workflows become org workflow definitions, the
|
||||
// one resource kind the runtime reads live today. The step vocabulary
|
||||
// (stepVocabulary) is the same one this process's control plane validates
|
||||
// dashboard edits with, so a definition the projector admits is compilable
|
||||
// where it dispatches (openStateStoreControlPlane for the pairing).
|
||||
func newPackageProjections(pool *pgxpool.Pool) (map[string]storepkg.FamilyProjector, error) {
|
||||
steps, err := stepVocabulary()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("register workflow step vocabulary: %w", err)
|
||||
}
|
||||
workflows, err := controlplane.NewWorkflowPackageProjector(
|
||||
postgres.NewResources(pool), postgres.NewPackageContributions(pool), steps)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build workflow package projector: %w", err)
|
||||
}
|
||||
return map[string]storepkg.FamilyProjector{storepkg.FamilyWorkflows: workflows}, nil
|
||||
}
|
||||
@@ -30,6 +30,7 @@ import (
|
||||
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
|
||||
)
|
||||
|
||||
// The audit identity every offline rollback records. A rollback is one more
|
||||
@@ -135,7 +136,7 @@ func replayRevision(ctx context.Context, resources controlplane.ResourceStore, o
|
||||
if !server.Owns(options.Kind) {
|
||||
return "", fmt.Errorf("unknown resource kind %q", options.Kind)
|
||||
}
|
||||
current, err := resources.Resource(ctx, options.Kind)
|
||||
current, err := resources.Resource(ctx, storecontract.DefaultOrgID, options.Kind)
|
||||
if errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
return "", fmt.Errorf("%s has no stored value to roll back", options.Kind)
|
||||
}
|
||||
@@ -146,7 +147,7 @@ func replayRevision(ctx context.Context, resources controlplane.ResourceStore, o
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
replaced, err := server.Command(ctx, &pb.CommandRequest{
|
||||
replaced, err := server.Command(ctx, controlplanerpc.CommandRequest(ctx, &pb.CommandRequest{
|
||||
Kind: options.Kind,
|
||||
Command: "replace",
|
||||
ValueJson: revision.Value,
|
||||
@@ -154,7 +155,7 @@ func replayRevision(ctx context.Context, resources controlplane.ResourceStore, o
|
||||
Actor: OfflineRollbackActor,
|
||||
Source: OfflineRollbackSource,
|
||||
RequestId: fmt.Sprintf("recovery-rollback-%s-%d", options.Kind, time.Now().UTC().UnixNano()),
|
||||
})
|
||||
}))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("replay %s at revision %d: %w", options.Kind, revision.Version, err)
|
||||
}
|
||||
@@ -165,7 +166,7 @@ func replayRevision(ctx context.Context, resources controlplane.ResourceStore, o
|
||||
// revisionToReplay picks the value to put back: the revision the operator
|
||||
// named, or the newest one older than the value the resource carries now.
|
||||
func revisionToReplay(ctx context.Context, st controlplane.ResourceStore, kind string, current, requested int64) (storecontract.Resource, error) {
|
||||
history, err := st.ResourceHistory(ctx, kind, 0)
|
||||
history, err := st.ResourceHistory(ctx, storecontract.DefaultOrgID, kind, 0)
|
||||
if err != nil {
|
||||
return storecontract.Resource{}, fmt.Errorf("read %s history: %w", kind, err)
|
||||
}
|
||||
|
||||
@@ -54,11 +54,12 @@ func putResource(t *testing.T, pool *pgxpool.Pool, kind, value, label string) {
|
||||
t.Helper()
|
||||
resources := postgres.NewResources(pool)
|
||||
expected := int64(0)
|
||||
if current, err := resources.Resource(t.Context(), kind); err == nil {
|
||||
if current, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, kind); err == nil {
|
||||
expected = current.Version
|
||||
}
|
||||
if _, err := resources.PutResource(t.Context(), storecontract.ResourceWrite{
|
||||
Kind: kind, Value: []byte(value), Actor: label, Source: "test",
|
||||
OrgID: storecontract.DefaultOrgID,
|
||||
Kind: kind, Value: []byte(value), Actor: label, Source: "test",
|
||||
RequestID: fmt.Sprintf("%s-%d", label, expected+1), ExpectedVersion: expected,
|
||||
}); err != nil {
|
||||
t.Fatalf("put %s: %v", kind, err)
|
||||
@@ -125,7 +126,7 @@ func TestPostgresRecoveryBackupAndRestore(t *testing.T) {
|
||||
if !strings.Contains(summary, "after the snapshot") {
|
||||
t.Errorf("restore must say writes after the snapshot are lost; summary = %q", summary)
|
||||
}
|
||||
current, err := postgres.NewResources(pool).Resource(t.Context(), controlplane.ModelRoleAssignmentsKind)
|
||||
current, err := postgres.NewResources(pool).Resource(t.Context(), storecontract.DefaultOrgID, controlplane.ModelRoleAssignmentsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -188,7 +189,7 @@ func TestPostgresRecoveryRollback(t *testing.T) {
|
||||
if _, err := RunStateStoreRecovery(t.Context(), options); err != nil {
|
||||
t.Fatalf("rollback: %v", err)
|
||||
}
|
||||
current, err := postgres.NewResources(pool).Resource(t.Context(), kind)
|
||||
current, err := postgres.NewResources(pool).Resource(t.Context(), storecontract.DefaultOrgID, kind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -17,6 +17,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/domain/health"
|
||||
"github.com/samcharles93/archie-core/internal/domain/messaging"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/sdnotify"
|
||||
"github.com/samcharles93/archie-core/internal/secret"
|
||||
)
|
||||
|
||||
@@ -199,6 +200,18 @@ func (s *Service) Start(ctx context.Context) error {
|
||||
})
|
||||
}
|
||||
|
||||
// Every configured channel has been handed its lifetime and the service
|
||||
// blocks here serving them. systemd's READY=1 asserts the same fact this
|
||||
// line has just established, so the announcement goes out here rather than
|
||||
// from a second notion of "started" (the convention archie-core-1174 set
|
||||
// for the daemon, the Gateway and the State Store; archie-core-g5g8 is this
|
||||
// process joining them). Not earlier: a channel whose configuration is
|
||||
// invalid fails composition, above, and READY must not precede it. Not
|
||||
// later: a unit whose TimeoutStartSec expires first would restart a healthy
|
||||
// process. Nothing is consumed until the unit says Type=notify, so this is
|
||||
// inert on today's install.
|
||||
sdnotify.Ready(s.log)
|
||||
|
||||
<-ctx.Done()
|
||||
|
||||
for _, ch := range s.channels {
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
package archiemessaging
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"net"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/sdnotify"
|
||||
)
|
||||
|
||||
// notifyListener is a stand-in for systemd's NOTIFY_SOCKET: a unixgram socket
|
||||
// that collects the state lines the service sends. It is bound to the real
|
||||
// NOTIFY_SOCKET for the duration of the test.
|
||||
type notifyListener struct {
|
||||
conn *net.UnixConn
|
||||
}
|
||||
|
||||
// listenNotify binds a notify socket in the test's temp dir and points
|
||||
// NOTIFY_SOCKET at it. A unix socket path is capped near 108 bytes, so the
|
||||
// short file name is what keeps the path inside t.TempDir() under that.
|
||||
func listenNotify(t *testing.T) *notifyListener {
|
||||
t.Helper()
|
||||
addr := filepath.Join(t.TempDir(), "n.sock")
|
||||
conn, err := net.ListenUnixgram("unixgram", &net.UnixAddr{Name: addr, Net: "unixgram"})
|
||||
if err != nil {
|
||||
t.Fatalf("listen for notify datagrams at %q: %v", addr, err)
|
||||
}
|
||||
t.Cleanup(func() { _ = conn.Close() })
|
||||
t.Setenv(sdnotify.NotifySocketEnv, addr)
|
||||
return ¬ifyListener{conn: conn}
|
||||
}
|
||||
|
||||
// waitForState reads datagrams until one carries want or the window closes.
|
||||
func (l *notifyListener) waitForState(t *testing.T, window time.Duration, want string) bool {
|
||||
t.Helper()
|
||||
deadline := time.Now().Add(window)
|
||||
buf := make([]byte, 1024)
|
||||
for time.Now().Before(deadline) {
|
||||
if err := l.conn.SetReadDeadline(time.Now().Add(50 * time.Millisecond)); err != nil {
|
||||
t.Fatalf("set notify read deadline: %v", err)
|
||||
}
|
||||
n, _, err := l.conn.ReadFromUnix(buf)
|
||||
if err != nil {
|
||||
var netErr net.Error
|
||||
if errors.As(err, &netErr) && netErr.Timeout() {
|
||||
continue
|
||||
}
|
||||
t.Fatalf("read notify datagram: %v", err)
|
||||
}
|
||||
if string(buf[:n]) == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// TestServiceAnnouncesReadyWhenServing: archie-messaging run as a Type=notify
|
||||
// unit must send READY=1 at the moment it declares itself serving, or systemd
|
||||
// kills a healthy process once TimeoutStartSec expires. It could not before:
|
||||
// the client lived in internal/app/archied, which this process cannot import
|
||||
// (archie-core-g5g8). The announcement is judged against the same evidence the
|
||||
// other serve paths use -- the process itself says it is serving -- and not
|
||||
// against a second notion of started invented here.
|
||||
func TestServiceAnnouncesReadyWhenServing(t *testing.T) {
|
||||
listener := listenNotify(t)
|
||||
|
||||
srv, err := compose(t.Context(), deps{
|
||||
Config: ResolvedConfig{Options: Options{ShutdownTimeout: 2 * time.Second}},
|
||||
Log: slog.New(slog.DiscardHandler),
|
||||
Chat: &dummyChatContract{},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("compose: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithCancel(t.Context())
|
||||
started := make(chan error, 1)
|
||||
go func() { started <- srv.Start(ctx) }()
|
||||
t.Cleanup(func() {
|
||||
cancel()
|
||||
srv.Stop()
|
||||
select {
|
||||
case err := <-started:
|
||||
if err != nil {
|
||||
t.Errorf("srv.Start returned %v, want a clean stop", err)
|
||||
}
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Error("srv.Start did not return after Stop, so a channel leaked its goroutine")
|
||||
}
|
||||
})
|
||||
|
||||
if !listener.waitForState(t, 5*time.Second, sdnotify.ReadyState) {
|
||||
t.Fatal("no READY=1 datagram within 5s of the Messaging Service serving")
|
||||
}
|
||||
}
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
|
||||
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
|
||||
"github.com/samcharles93/archie-core/internal/domain/messaging"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
|
||||
)
|
||||
|
||||
type messagingControlPlane struct{ client pb.ControlPlaneServiceClient }
|
||||
@@ -46,7 +47,7 @@ func (c messagingControlPlane) Catalog(ctx context.Context) ([]messaging.Setting
|
||||
}
|
||||
|
||||
func (c messagingControlPlane) Query(ctx context.Context, kind string) (messaging.SettingResource, error) {
|
||||
response, err := c.client.Query(ctx, &pb.QueryRequest{Kind: kind})
|
||||
response, err := c.client.Query(ctx, controlplanerpc.QueryRequest(ctx, kind))
|
||||
if err != nil {
|
||||
return messaging.SettingResource{}, settingsRPCError(err)
|
||||
}
|
||||
@@ -62,7 +63,7 @@ func (c messagingControlPlane) Command(ctx context.Context, command messaging.Se
|
||||
if err != nil {
|
||||
return messaging.SettingResource{}, errors.Join(messaging.ErrSettingsValidation, err)
|
||||
}
|
||||
response, err := c.client.Command(ctx, &pb.CommandRequest{Kind: command.Kind, Command: "replace", ValueJson: value, ExpectedVersion: command.ExpectedVersion, RequestId: command.RequestID, Actor: command.Actor, Source: command.Source})
|
||||
response, err := c.client.Command(ctx, controlplanerpc.CommandRequest(ctx, &pb.CommandRequest{Kind: command.Kind, Command: "replace", ValueJson: value, ExpectedVersion: command.ExpectedVersion, RequestId: command.RequestID, Actor: command.Actor, Source: command.Source}))
|
||||
if err != nil {
|
||||
return messaging.SettingResource{}, settingsRPCError(err)
|
||||
}
|
||||
|
||||
@@ -87,6 +87,11 @@ func TestRoutesWithoutAContractDegradeExplicitly(t *testing.T) {
|
||||
name: "workflow registry", method: http.MethodGet, path: "/api/workflows",
|
||||
want: http.StatusOK, emptyJSON: `"definitions":null`,
|
||||
},
|
||||
{
|
||||
name: "workflow enabled toggle", method: http.MethodPut, path: "/api/workflows/implement/enabled",
|
||||
body: `{"enabled":false}`,
|
||||
want: http.StatusServiceUnavailable,
|
||||
},
|
||||
{
|
||||
name: "dashboard work request", method: http.MethodPost, path: "/api/work-requests",
|
||||
body: `{"repository":"acme/widget","workflow":"implement","title":"do it"}`,
|
||||
|
||||
@@ -70,7 +70,7 @@ func (c *Client) Catalog(ctx context.Context) ([]*pb.ResourceDescriptor, error)
|
||||
}
|
||||
|
||||
func (c *WorkflowDefinitionsClient) WorkflowDefinitions(ctx context.Context) (workflow.WorkflowDefinitionCollection, int64, error) {
|
||||
response, err := c.rpc.Query(ctx, &pb.QueryRequest{Kind: WorkflowDefinitionsKind})
|
||||
response, err := c.rpc.Query(ctx, controlplanerpc.QueryRequest(ctx, WorkflowDefinitionsKind))
|
||||
if err != nil {
|
||||
return workflow.WorkflowDefinitionCollection{}, 0, controlplanerpc.ClientError(err)
|
||||
}
|
||||
@@ -90,7 +90,7 @@ func (c *WorkflowDefinitionsClient) ReplaceWorkflowDefinitions(ctx context.Conte
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
response, err := c.rpc.Command(ctx, &pb.CommandRequest{Kind: WorkflowDefinitionsKind, Command: "replace", ValueJson: value, ExpectedVersion: expectedVersion, Actor: actor, Source: source, RequestId: requestID})
|
||||
response, err := c.rpc.Command(ctx, controlplanerpc.CommandRequest(ctx, &pb.CommandRequest{Kind: WorkflowDefinitionsKind, Command: "replace", ValueJson: value, ExpectedVersion: expectedVersion, Actor: actor, Source: source, RequestId: requestID}))
|
||||
if err != nil {
|
||||
return 0, controlplanerpc.ClientError(err)
|
||||
}
|
||||
@@ -98,7 +98,7 @@ func (c *WorkflowDefinitionsClient) ReplaceWorkflowDefinitions(ctx context.Conte
|
||||
}
|
||||
|
||||
func (c *Client) WorkflowExecutionSettings(ctx context.Context) (workflow.ExecutionSettings, int64, error) {
|
||||
response, err := c.rpc.Query(ctx, &pb.QueryRequest{Kind: WorkflowExecutionSettingsKind})
|
||||
response, err := c.rpc.Query(ctx, controlplanerpc.QueryRequest(ctx, WorkflowExecutionSettingsKind))
|
||||
if err != nil {
|
||||
return workflow.ExecutionSettings{}, 0, controlplanerpc.ClientError(err)
|
||||
}
|
||||
@@ -114,7 +114,7 @@ func (c *Client) ReplaceWorkflowExecutionSettings(ctx context.Context, settings
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
response, err := c.rpc.Command(ctx, &pb.CommandRequest{Kind: WorkflowExecutionSettingsKind, Command: "replace", ValueJson: value, ExpectedVersion: expectedVersion, Actor: actor, Source: source, RequestId: requestID})
|
||||
response, err := c.rpc.Command(ctx, controlplanerpc.CommandRequest(ctx, &pb.CommandRequest{Kind: WorkflowExecutionSettingsKind, Command: "replace", ValueJson: value, ExpectedVersion: expectedVersion, Actor: actor, Source: source, RequestId: requestID}))
|
||||
if err != nil {
|
||||
return 0, controlplanerpc.ClientError(err)
|
||||
}
|
||||
@@ -122,7 +122,7 @@ func (c *Client) ReplaceWorkflowExecutionSettings(ctx context.Context, settings
|
||||
}
|
||||
|
||||
func (c *Client) WatchWorkflowExecutionSettings(ctx context.Context, afterVersion int64) (<-chan AppliedSettings, error) {
|
||||
stream, err := c.rpc.Watch(ctx, &pb.WatchRequest{Kind: WorkflowExecutionSettingsKind, AfterVersion: afterVersion})
|
||||
stream, err := c.rpc.Watch(ctx, controlplanerpc.WatchRequest(ctx, WorkflowExecutionSettingsKind, afterVersion))
|
||||
if err != nil {
|
||||
return nil, controlplanerpc.ClientError(err)
|
||||
}
|
||||
@@ -152,7 +152,7 @@ type AppliedResource struct {
|
||||
// refused to decode -- the store answered with that version, and resuming
|
||||
// before it would re-offer it on every reconnect.
|
||||
func (c *Client) WatchResource(ctx context.Context, kind string, afterVersion int64) (<-chan AppliedResource, error) {
|
||||
stream, err := c.rpc.Watch(ctx, &pb.WatchRequest{Kind: kind, AfterVersion: afterVersion})
|
||||
stream, err := c.rpc.Watch(ctx, controlplanerpc.WatchRequest(ctx, kind, afterVersion))
|
||||
if err != nil {
|
||||
return nil, controlplanerpc.ClientError(err)
|
||||
}
|
||||
@@ -168,7 +168,7 @@ type AppliedSettings struct {
|
||||
}
|
||||
|
||||
func (c *Client) Personas(ctx context.Context) (agent.PersonaCollection, int64, error) {
|
||||
response, err := c.rpc.Query(ctx, &pb.QueryRequest{Kind: PersonasKind})
|
||||
response, err := c.rpc.Query(ctx, controlplanerpc.QueryRequest(ctx, PersonasKind))
|
||||
if err != nil {
|
||||
return agent.PersonaCollection{}, 0, controlplanerpc.ClientError(err)
|
||||
}
|
||||
@@ -183,7 +183,7 @@ type AppliedPersonas struct {
|
||||
}
|
||||
|
||||
func (c *Client) WatchPersonas(ctx context.Context, afterVersion int64) (<-chan AppliedPersonas, error) {
|
||||
stream, err := c.rpc.Watch(ctx, &pb.WatchRequest{Kind: PersonasKind, AfterVersion: afterVersion})
|
||||
stream, err := c.rpc.Watch(ctx, controlplanerpc.WatchRequest(ctx, PersonasKind, afterVersion))
|
||||
if err != nil {
|
||||
return nil, controlplanerpc.ClientError(err)
|
||||
}
|
||||
@@ -241,7 +241,7 @@ func sendUpdate[T any](ctx context.Context, out chan<- T, update T) bool {
|
||||
}
|
||||
|
||||
func (c *Client) Schedules(ctx context.Context) ([]scheduling.JobSpec, int64, error) {
|
||||
response, err := c.rpc.Query(ctx, &pb.QueryRequest{Kind: SchedulesKind})
|
||||
response, err := c.rpc.Query(ctx, controlplanerpc.QueryRequest(ctx, SchedulesKind))
|
||||
if err != nil {
|
||||
return nil, 0, controlplanerpc.ClientError(err)
|
||||
}
|
||||
@@ -257,7 +257,7 @@ func (c *Client) ReplaceSchedules(ctx context.Context, jobs []scheduling.JobSpec
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
response, err := c.rpc.Command(ctx, &pb.CommandRequest{Kind: SchedulesKind, Command: "replace", ValueJson: value, ExpectedVersion: expectedVersion, Actor: actor, Source: source, RequestId: requestID})
|
||||
response, err := c.rpc.Command(ctx, controlplanerpc.CommandRequest(ctx, &pb.CommandRequest{Kind: SchedulesKind, Command: "replace", ValueJson: value, ExpectedVersion: expectedVersion, Actor: actor, Source: source, RequestId: requestID}))
|
||||
if err != nil {
|
||||
return 0, controlplanerpc.ClientError(err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
package controlplane
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"testing"
|
||||
|
||||
"google.golang.org/grpc"
|
||||
|
||||
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
|
||||
"github.com/samcharles93/archie-core/internal/domain/org"
|
||||
)
|
||||
|
||||
// orgRecordingClient is the rpc end that records the org of every request it
|
||||
// is handed, so a test can read back what the client put on the wire.
|
||||
type orgRecordingClient struct {
|
||||
queryOrg string
|
||||
commandOrg string
|
||||
watchOrg string
|
||||
}
|
||||
|
||||
func (*orgRecordingClient) Catalog(context.Context, *pb.CatalogRequest, ...grpc.CallOption) (*pb.CatalogResponse, error) {
|
||||
panic("Catalog is not part of the org-stamping path")
|
||||
}
|
||||
|
||||
func (c *orgRecordingClient) Query(_ context.Context, in *pb.QueryRequest, _ ...grpc.CallOption) (*pb.QueryResponse, error) {
|
||||
c.queryOrg = in.GetOrgId()
|
||||
return &pb.QueryResponse{Resource: &pb.Resource{Kind: in.GetKind()}}, nil
|
||||
}
|
||||
|
||||
func (*orgRecordingClient) History(context.Context, *pb.HistoryRequest, ...grpc.CallOption) (*pb.HistoryResponse, error) {
|
||||
panic("History is not part of the org-stamping path")
|
||||
}
|
||||
|
||||
func (*orgRecordingClient) Audit(context.Context, *pb.AuditRequest, ...grpc.CallOption) (*pb.AuditResponse, error) {
|
||||
return &pb.AuditResponse{}, nil
|
||||
}
|
||||
|
||||
func (c *orgRecordingClient) Command(_ context.Context, in *pb.CommandRequest, _ ...grpc.CallOption) (*pb.CommandResponse, error) {
|
||||
c.commandOrg = in.GetOrgId()
|
||||
return &pb.CommandResponse{Resource: &pb.Resource{Version: 1}}, nil
|
||||
}
|
||||
|
||||
func (c *orgRecordingClient) Watch(_ context.Context, in *pb.WatchRequest, _ ...grpc.CallOption) (grpc.ServerStreamingClient[pb.WatchResponse], error) {
|
||||
c.watchOrg = in.GetOrgId()
|
||||
return eofWatchStream{}, nil
|
||||
}
|
||||
|
||||
// eofWatchStream is the client end of a stream that ends before it carries
|
||||
// anything: the watch under test only builds the request.
|
||||
type eofWatchStream struct{ grpc.ClientStream }
|
||||
|
||||
func (eofWatchStream) Recv() (*pb.WatchResponse, error) { return nil, io.EOF }
|
||||
|
||||
func TestClientStampsCallerOrgOnEveryRequestShape(t *testing.T) {
|
||||
// The table holds the org, not a context: a stored context.Context field is
|
||||
// what containedctx rejects, and each case only needs to say what the caller
|
||||
// put on its context.
|
||||
tests := []struct {
|
||||
name string
|
||||
orgID string
|
||||
want string
|
||||
}{
|
||||
{name: "no org on the context is the default org", want: string(org.DefaultOrgID)},
|
||||
{name: "org on the context travels on the wire", orgID: "acme", want: "acme"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
if tt.orgID != "" {
|
||||
ctx = org.WithOrg(ctx, org.OrgID(tt.orgID))
|
||||
}
|
||||
rpc := &orgRecordingClient{}
|
||||
client := NewRPCClient(rpc)
|
||||
|
||||
if _, _, err := client.Query(ctx, ChannelSettingsKind, func([]byte) error { return nil }); err != nil {
|
||||
t.Fatalf("Query: %v", err)
|
||||
}
|
||||
if _, err := client.ReplaceSchedules(ctx, nil, 0, "actor", "source", "request"); err != nil {
|
||||
t.Fatalf("ReplaceSchedules: %v", err)
|
||||
}
|
||||
if _, err := client.WatchResource(ctx, WorkflowExecutionSettingsKind, 0); err != nil {
|
||||
t.Fatalf("WatchResource: %v", err)
|
||||
}
|
||||
|
||||
if rpc.queryOrg != tt.want || rpc.commandOrg != tt.want || rpc.watchOrg != tt.want {
|
||||
t.Fatalf("wire orgs = query %q command %q watch %q, want %q each",
|
||||
rpc.queryOrg, rpc.commandOrg, rpc.watchOrg, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -31,8 +31,8 @@ var (
|
||||
// root asserts the State Store against it, so this is the single definition of
|
||||
// what a control-plane backing store must offer.
|
||||
type ResourceStore interface {
|
||||
Resource(context.Context, string) (storecontract.Resource, error)
|
||||
ResourceHistory(context.Context, string, int) ([]storecontract.Resource, error)
|
||||
Resource(ctx context.Context, orgID, kind string) (storecontract.Resource, error)
|
||||
ResourceHistory(ctx context.Context, orgID, kind string, limit int) ([]storecontract.Resource, error)
|
||||
Audit(ctx context.Context, table string, keys []string, limit int) ([]storecontract.AuditEntry, error)
|
||||
PutResource(context.Context, storecontract.ResourceWrite) (storecontract.Resource, error)
|
||||
}
|
||||
@@ -78,7 +78,7 @@ func (s *Server) Query(ctx context.Context, request *pb.QueryRequest) (*pb.Query
|
||||
if _, ok := s.definitions[request.GetKind()]; !ok {
|
||||
return nil, status.Error(codes.NotFound, "resource not found")
|
||||
}
|
||||
resource, err := s.store.Resource(ctx, request.Kind)
|
||||
resource, err := s.store.Resource(ctx, requestOrg(request.GetOrgId()), request.Kind)
|
||||
if err != nil {
|
||||
return nil, mapError(err)
|
||||
}
|
||||
@@ -96,7 +96,7 @@ func (s *Server) History(ctx context.Context, request *pb.HistoryRequest) (*pb.H
|
||||
if limit <= 0 || limit > defaultHistoryLimit {
|
||||
limit = defaultHistoryLimit
|
||||
}
|
||||
revisions, err := s.store.ResourceHistory(ctx, request.Kind, limit)
|
||||
revisions, err := s.store.ResourceHistory(ctx, requestOrg(request.GetOrgId()), request.Kind, limit)
|
||||
if err != nil {
|
||||
return nil, mapError(err)
|
||||
}
|
||||
@@ -149,7 +149,7 @@ func (s *Server) Command(ctx context.Context, request *pb.CommandRequest) (*pb.C
|
||||
if err != nil {
|
||||
return nil, mapError(err)
|
||||
}
|
||||
resource, err := s.store.PutResource(ctx, storecontract.ResourceWrite{Kind: request.Kind, Value: value, Actor: request.Actor, Source: request.Source, RequestID: request.RequestId, ExpectedVersion: request.ExpectedVersion, At: time.Now().UTC()})
|
||||
resource, err := s.store.PutResource(ctx, storecontract.ResourceWrite{OrgID: requestOrg(request.GetOrgId()), Kind: request.Kind, Value: value, Actor: request.Actor, Source: request.Source, RequestID: request.RequestId, ExpectedVersion: request.ExpectedVersion, At: time.Now().UTC()})
|
||||
if err != nil {
|
||||
return nil, mapError(err)
|
||||
}
|
||||
@@ -164,7 +164,7 @@ func (s *Server) Watch(request *pb.WatchRequest, stream pb.ControlPlaneService_W
|
||||
ticker := time.NewTicker(250 * time.Millisecond)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
resource, err := s.store.Resource(stream.Context(), request.Kind)
|
||||
resource, err := s.store.Resource(stream.Context(), requestOrg(request.GetOrgId()), request.Kind)
|
||||
if err == nil && resource.Version > version {
|
||||
if err := stream.Send(&pb.WatchResponse{Resource: resourceProto(resource)}); err != nil {
|
||||
return err
|
||||
@@ -182,7 +182,16 @@ func (s *Server) Watch(request *pb.WatchRequest, stream pb.ControlPlaneService_W
|
||||
}
|
||||
|
||||
func resourceProto(resource storecontract.Resource) *pb.Resource {
|
||||
return &pb.Resource{Kind: resource.Kind, Version: resource.Version, ValueJson: resource.Value, UpdatedAt: timestamp(resource.At)}
|
||||
return &pb.Resource{OrgId: resource.OrgID, Kind: resource.Kind, Version: resource.Version, ValueJson: resource.Value, UpdatedAt: timestamp(resource.At)}
|
||||
}
|
||||
|
||||
// requestOrg is the org a request acts in. Until the caller's org is derived
|
||||
// server-side (access), a request that names none acts in the default org.
|
||||
func requestOrg(orgID string) string {
|
||||
if orgID == "" {
|
||||
return storecontract.DefaultOrgID
|
||||
}
|
||||
return orgID
|
||||
}
|
||||
|
||||
func timestamp(value time.Time) *timestamppb.Timestamp {
|
||||
|
||||
@@ -79,7 +79,7 @@ func TestImportConfigSeedsWorkflowDefinitionsWithoutOverwritingOverride(t *testi
|
||||
if _, _, err := server.ImportConfig(t.Context(), config.Config{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resource, err := resources.Resource(t.Context(), WorkflowDefinitionsKind)
|
||||
resource, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -95,13 +95,13 @@ func TestImportConfigSeedsWorkflowDefinitionsWithoutOverwritingOverride(t *testi
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := resources.PutResource(t.Context(), storecontract.ResourceWrite{Kind: WorkflowDefinitionsKind, Value: value, ExpectedVersion: resource.Version, Actor: "test", Source: "test", RequestID: "override"}); err != nil {
|
||||
if _, err := resources.PutResource(t.Context(), storecontract.ResourceWrite{OrgID: storecontract.DefaultOrgID, Kind: WorkflowDefinitionsKind, Value: value, ExpectedVersion: resource.Version, Actor: "test", Source: "test", RequestID: "override"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, _, err := server.ImportConfig(t.Context(), config.Config{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := resources.Resource(t.Context(), WorkflowDefinitionsKind)
|
||||
got, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -119,7 +119,7 @@ func TestImportConfigSeedsPersonasWithoutOverwritingEdits(t *testing.T) {
|
||||
if _, _, err := server.ImportConfig(t.Context(), config.Config{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resource, err := resources.Resource(t.Context(), PersonasKind)
|
||||
resource, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, PersonasKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -132,13 +132,13 @@ func TestImportConfigSeedsPersonasWithoutOverwritingEdits(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := resources.PutResource(t.Context(), storecontract.ResourceWrite{Kind: PersonasKind, Value: value, ExpectedVersion: resource.Version, Actor: "test", Source: "test", RequestID: "persona-edit"}); err != nil {
|
||||
if _, err := resources.PutResource(t.Context(), storecontract.ResourceWrite{OrgID: storecontract.DefaultOrgID, Kind: PersonasKind, Value: value, ExpectedVersion: resource.Version, Actor: "test", Source: "test", RequestID: "persona-edit"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, _, err := server.ImportConfig(t.Context(), config.Config{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := resources.Resource(t.Context(), PersonasKind)
|
||||
got, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, PersonasKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -161,7 +161,7 @@ func TestImportWorkflowExecutionSettingsDoesNotOverwriteExistingValue(t *testing
|
||||
if _, err := server.ImportWorkflowExecutionSettings(t.Context(), workflow.ExecutionSettings{MaxModelToolSteps: 99, MaxRuntime: time.Hour, MaxConsecutiveGateFailures: 9}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resource, err := resources.Resource(t.Context(), WorkflowExecutionSettingsKind)
|
||||
resource, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowExecutionSettingsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -186,7 +186,7 @@ func TestRegistryRoutesAndValidatesDefinitions(t *testing.T) {
|
||||
if _, _, err := server.ImportConfig(t.Context(), config.Config{Containers: config.ContainerConfig{Image: "archie-agent:test", PullPolicy: "missing"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := resources.Resource(t.Context(), ContainerRuntimePoliciesKind); err != nil {
|
||||
if _, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, ContainerRuntimePoliciesKind); err != nil {
|
||||
t.Fatalf("container runtime policies were not seeded: %v", err)
|
||||
}
|
||||
|
||||
@@ -220,7 +220,7 @@ func TestProviderSeedKeepsReferencesAndNeverResolvedSecrets(t *testing.T) {
|
||||
if _, _, err := server.ImportConfig(context.Background(), cfg); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resource, err := resources.Resource(t.Context(), ProviderSettingsKind)
|
||||
resource, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, ProviderSettingsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -308,12 +308,38 @@ func TestImportConfigSkipsASeedItCannotValidate(t *testing.T) {
|
||||
if len(skipped) != 1 || skipped[0].Kind != RepositoryPoliciesKind || skipped[0].Err == nil {
|
||||
t.Fatalf("skipped = %+v, want the repository policies and the reason", skipped)
|
||||
}
|
||||
if _, err := resources.Resource(t.Context(), RepositoryPoliciesKind); !errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
if _, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, RepositoryPoliciesKind); !errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
t.Errorf("repository policies Resource = %v, want ErrResourceNotFound: an invalid seed must not be stored", err)
|
||||
}
|
||||
// Kinds that are fine are still seeded: one skipped kind does not abandon
|
||||
// the rest of the migration.
|
||||
if _, err := resources.Resource(t.Context(), WorkflowDefinitionsKind); err != nil {
|
||||
if _, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind); err != nil {
|
||||
t.Errorf("workflow definitions: %v, want them seeded alongside the skipped kind", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A request names the org whose copy of a kind it acts on; one naming none acts
|
||||
// in the default org, so a write for one org is invisible to the others.
|
||||
func TestControlPlaneRequestsActInTheirOrg(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
resources := pgstore.Open(t)
|
||||
defer resources.Close()
|
||||
server := testServer(t, resources)
|
||||
written, err := server.Command(t.Context(), &pb.CommandRequest{
|
||||
Kind: WorkflowEnablementKind, Command: "replace", ValueJson: []byte(`{"orgs":{}}`),
|
||||
Actor: "test", Source: "test", RequestId: "acme-write", OrgId: "acme",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if written.Resource.OrgId != "acme" || written.Resource.Version != 1 {
|
||||
t.Fatalf("written = %+v, want acme at version 1", written.Resource)
|
||||
}
|
||||
if _, err := server.Query(t.Context(), &pb.QueryRequest{Kind: WorkflowEnablementKind, OrgId: "acme"}); err != nil {
|
||||
t.Fatalf("acme Query: %v", err)
|
||||
}
|
||||
if _, err := server.Query(t.Context(), &pb.QueryRequest{Kind: WorkflowEnablementKind}); status.Code(err) != codes.NotFound {
|
||||
t.Fatalf("default-org Query = %v, want NotFound: acme's write leaked into the default org", err)
|
||||
}
|
||||
}
|
||||
@@ -285,7 +285,7 @@ func TestWatchServesOnlyVersionsAfterTheRequestedOne(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := resources.PutResource(ctx, storecontract.ResourceWrite{Kind: WorkflowExecutionSettingsKind, Value: value, ExpectedVersion: version, Actor: "test", Source: "test", RequestID: "watch-next"}); err != nil {
|
||||
if _, err := resources.PutResource(ctx, storecontract.ResourceWrite{OrgID: storecontract.DefaultOrgID, Kind: WorkflowExecutionSettingsKind, Value: value, ExpectedVersion: version, Actor: "test", Source: "test", RequestID: "watch-next"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
select {
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
// Schemas are not declared here: each Definition names its document type, and
|
||||
// Definition.Descriptor derives the JSON Schema from it (internal/app/controlplane/schema.go).
|
||||
func builtinDefinitions(steps workflow.StepRegistry) []Definition {
|
||||
definitions := []Definition{workflowDefinition(), workflowDefinitionsDefinition(steps), personaDefinition(), scheduleDefinition(), agentProfileDefinition(), credentialBindingsDefinition()}
|
||||
definitions := []Definition{workflowDefinition(), workflowDefinitionsDefinition(steps), workflowEnablementDefinition(), personaDefinition(), scheduleDefinition(), agentProfileDefinition(), credentialBindingsDefinition()}
|
||||
definitions = append(definitions, modelDefinitions()...)
|
||||
definitions = append(definitions, operationalDefinitions()...)
|
||||
return definitions
|
||||
|
||||
@@ -45,7 +45,7 @@ func TestImportConfigRefusesAResolvedProviderSnapshot(t *testing.T) {
|
||||
if !reported {
|
||||
t.Errorf("skipped = %+v, want provider-settings reported", skipped)
|
||||
}
|
||||
if _, err := resources.Resource(t.Context(), ProviderSettingsKind); err == nil {
|
||||
if _, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, ProviderSettingsKind); err == nil {
|
||||
t.Fatal("provider-settings was stored from a resolved snapshot")
|
||||
} else if !strings.Contains(err.Error(), "not found") {
|
||||
t.Fatalf("Resource = %v, want %v", err, storecontract.ErrResourceNotFound)
|
||||
|
||||
@@ -53,12 +53,33 @@ func operationalDefinitions() []Definition {
|
||||
}
|
||||
}
|
||||
|
||||
// validatePluginSettings accepts every value: the four directories are free-form
|
||||
// operator paths with no cross-field rule and no rule the configuration package
|
||||
// applies either (configuration.Validate has no say over them), so there is
|
||||
// nothing yet for this validator to enforce. It stays a real function rather than
|
||||
// an inline no-op so the place to add a rule is obvious when one exists -- e.g.
|
||||
// "a path that is absolute" or "a skills dir under the plugin dir".
|
||||
// validatePluginSettings accepts every value, deliberately. The four directories
|
||||
// are free-form operator paths, and that is the decision recorded for
|
||||
// archie-core #1143 rather than a validator nobody has written yet.
|
||||
//
|
||||
// Two candidate rules were considered and are refused:
|
||||
//
|
||||
// - That the directory must exist. Every consumer reads a missing directory as
|
||||
// an empty one by design: plugin.LoadDir and secret.Registry.LoadDir return
|
||||
// no entries and no error on os.IsNotExist, loadModules skips a kind whose
|
||||
// file is absent, and playbook.Load returns an empty store. The no-error half
|
||||
// is pinned by internal/secret's TestLoadDirNonexistent. An existence rule
|
||||
// would turn each of those deliberate no-ops into a refused startup.
|
||||
// - That the path must be absolute, or that skills_dir must sit under
|
||||
// plugin_dir. No directory setting in this repository is validated for shape
|
||||
// (work_dir, state_dir, chat.workspace, the routing and playbook paths and an
|
||||
// MCP server's work_dir are all free-form), so a rule for these four alone
|
||||
// would be arbitrary. It would also fail closed in the wrong place: a stored
|
||||
// document is layered over the file and the boot path validates the result,
|
||||
// so a rule added here refuses to start a daemon whose only problem is a
|
||||
// value the store already holds.
|
||||
//
|
||||
// A "~/..." value is not the missing rule either: it is a path-expansion
|
||||
// concern, handled where the file is read (configuration.applyGeneralDefaults),
|
||||
// not a reason to refuse the document.
|
||||
//
|
||||
// It stays a real function rather than an inline no-op so that a rule, if one is
|
||||
// ever settled, has one obvious home.
|
||||
func validatePluginSettings(input []byte) error {
|
||||
return validateAs(input, func(pluginSettings) error { return nil })
|
||||
}
|
||||
|
||||
@@ -156,7 +156,7 @@ func (r *seedRefusal) Unwrap() error { return r.err }
|
||||
// value this build derived is not one the resource's validator accepts, and
|
||||
// nothing was written.
|
||||
func (s *Server) seedKind(ctx context.Context, definition Definition, cfg config.Config) (int64, error) {
|
||||
stored, storedErr := s.store.Resource(ctx, definition.Kind)
|
||||
stored, storedErr := s.store.Resource(ctx, storecontract.DefaultOrgID, definition.Kind)
|
||||
absent := errors.Is(storedErr, storecontract.ErrResourceNotFound)
|
||||
if storedErr != nil && !absent {
|
||||
return 0, storedErr
|
||||
@@ -188,7 +188,7 @@ func (s *Server) seedKind(ctx context.Context, definition Definition, cfg config
|
||||
}
|
||||
}
|
||||
write := storecontract.ResourceWrite{
|
||||
Kind: definition.Kind, Value: value,
|
||||
OrgID: storecontract.DefaultOrgID, Kind: definition.Kind, Value: value,
|
||||
Actor: seedActor, Source: seedSource,
|
||||
RequestID: seedRequestID(definition.Kind, value),
|
||||
ExpectedVersion: expectedVersion(stored, absent), At: time.Now().UTC(),
|
||||
@@ -226,7 +226,7 @@ func (s *Server) shippedValueIsStale(ctx context.Context, definition Definition)
|
||||
// seed. It reads the ledger rather than the current row because the ledger is
|
||||
// what records who wrote a value, and the row does not.
|
||||
func (s *Server) seedWroteNewest(ctx context.Context, kind string) (bool, error) {
|
||||
history, err := s.store.ResourceHistory(ctx, kind, 1)
|
||||
history, err := s.store.ResourceHistory(ctx, storecontract.DefaultOrgID, kind, 1)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("read %s history: %w", kind, err)
|
||||
}
|
||||
@@ -305,7 +305,7 @@ func (s *Server) ValidateStored(ctx context.Context) (int, error) {
|
||||
checked := 0
|
||||
var failures []error
|
||||
for _, definition := range s.ordered {
|
||||
resource, err := s.store.Resource(ctx, definition.Kind)
|
||||
resource, err := s.store.Resource(ctx, storecontract.DefaultOrgID, definition.Kind)
|
||||
if errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -46,13 +46,14 @@ func staleWorkflowDefinitions(t *testing.T) []byte {
|
||||
// -- so a test does not restate a key or an identity it would then be pinning.
|
||||
func writeSeeded(t *testing.T, resources *pgstore.TaskDB, kind string, value []byte) storecontract.Resource {
|
||||
t.Helper()
|
||||
if current, err := resources.Resource(t.Context(), kind); err == nil {
|
||||
if current, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, kind); err == nil {
|
||||
t.Fatalf("%s already holds version %d; a seeded fixture writes the first revision", kind, current.Version)
|
||||
} else if !errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
t.Fatalf("read %s: %v", kind, err)
|
||||
}
|
||||
seeded, err := resources.PutResource(t.Context(), storecontract.ResourceWrite{
|
||||
Kind: kind, Value: value, ExpectedVersion: 0,
|
||||
OrgID: storecontract.DefaultOrgID,
|
||||
Kind: kind, Value: value, ExpectedVersion: 0,
|
||||
Actor: seedActor, Source: seedSource, RequestID: seedRequestID(kind, value),
|
||||
})
|
||||
if err != nil {
|
||||
@@ -66,7 +67,8 @@ func writeSeeded(t *testing.T, resources *pgstore.TaskDB, kind string, value []b
|
||||
func replaceAsOperator(t *testing.T, resources *pgstore.TaskDB, kind string, value []byte, expected int64) storecontract.Resource {
|
||||
t.Helper()
|
||||
replaced, err := resources.PutResource(t.Context(), storecontract.ResourceWrite{
|
||||
Kind: kind, Value: value, ExpectedVersion: expected,
|
||||
OrgID: storecontract.DefaultOrgID,
|
||||
Kind: kind, Value: value, ExpectedVersion: expected,
|
||||
Actor: "operator", Source: "archie-ui", RequestID: "operator-replace-" + kind,
|
||||
})
|
||||
if err != nil {
|
||||
@@ -96,7 +98,7 @@ func TestImportConfigRefreshesAStaleShippedValue(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("ImportConfig: %v", err)
|
||||
}
|
||||
refreshed, err := resources.Resource(t.Context(), WorkflowDefinitionsKind)
|
||||
refreshed, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -115,7 +117,7 @@ func TestImportConfigRefreshesAStaleShippedValue(t *testing.T) {
|
||||
}
|
||||
// A refresh is a write like any other: what it replaced stays a revision, so
|
||||
// the value the previous release shipped is still readable.
|
||||
history, err := resources.ResourceHistory(t.Context(), WorkflowDefinitionsKind, 0)
|
||||
history, err := resources.ResourceHistory(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -138,7 +140,7 @@ func TestImportConfigLeavesASeededValueAloneOnceItMatches(t *testing.T) {
|
||||
if _, _, err := server.ImportConfig(t.Context(), config.Config{}); err != nil {
|
||||
t.Fatalf("ImportConfig %d: %v", attempt, err)
|
||||
}
|
||||
stored, err := resources.Resource(t.Context(), WorkflowDefinitionsKind)
|
||||
stored, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -220,7 +222,7 @@ func TestImportConfigKeepsAStoredValueThatIsNotTheSeeds(t *testing.T) {
|
||||
if _, _, err := testServer(t, resources).ImportConfig(t.Context(), resolved); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
stored, err := resources.Resource(t.Context(), RepositoryPoliciesKind)
|
||||
stored, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, RepositoryPoliciesKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -243,7 +245,7 @@ func TestImportConfigKeepsAStoredValueThatIsNotTheSeeds(t *testing.T) {
|
||||
defer resources.Close()
|
||||
server := testServer(t, resources)
|
||||
want := tt.settle(t, resources)
|
||||
before, err := resources.Resource(t.Context(), tt.kind)
|
||||
before, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, tt.kind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -253,7 +255,7 @@ func TestImportConfigKeepsAStoredValueThatIsNotTheSeeds(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("ImportConfig %d: %v", attempt, err)
|
||||
}
|
||||
after, err := resources.Resource(t.Context(), tt.kind)
|
||||
after, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, tt.kind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -284,11 +286,11 @@ func TestImportConfigRecreatesAKindAnOperatorRemoved(t *testing.T) {
|
||||
if _, _, err := server.ImportConfig(t.Context(), config.Config{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
seeded, err := resources.Resource(t.Context(), WorkflowDefinitionsKind)
|
||||
seeded, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
history, err := resources.ResourceHistory(t.Context(), WorkflowDefinitionsKind, 0)
|
||||
history, err := resources.ResourceHistory(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -299,14 +301,14 @@ func TestImportConfigRecreatesAKindAnOperatorRemoved(t *testing.T) {
|
||||
if _, err := resources.Pool.Exec(t.Context(), `DELETE FROM resources WHERE kind = $1`, WorkflowDefinitionsKind); err != nil {
|
||||
t.Fatalf("remove the resource: %v", err)
|
||||
}
|
||||
if _, err := resources.Resource(t.Context(), WorkflowDefinitionsKind); !errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
if _, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind); !errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
t.Fatalf("resource after the removal = %v, want ErrResourceNotFound", err)
|
||||
}
|
||||
|
||||
if _, _, err := server.ImportConfig(t.Context(), config.Config{}); err != nil {
|
||||
t.Fatalf("ImportConfig after the removal: %v", err)
|
||||
}
|
||||
restored, err := resources.Resource(t.Context(), WorkflowDefinitionsKind)
|
||||
restored, err := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind)
|
||||
if err != nil {
|
||||
t.Fatalf("the removed %s was not re-seeded: %v", WorkflowDefinitionsKind, err)
|
||||
}
|
||||
|
||||
@@ -71,7 +71,7 @@ type storeReader struct {
|
||||
}
|
||||
|
||||
func (r storeReader) Query(ctx context.Context, kind string, decode func([]byte) error) (int64, bool, error) {
|
||||
resource, err := r.resources.Resource(ctx, kind)
|
||||
resource, err := r.resources.Resource(ctx, storecontract.DefaultOrgID, kind)
|
||||
if errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
seed, ok := r.seeds[kind]
|
||||
if !ok {
|
||||
|
||||
@@ -34,6 +34,7 @@ import (
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/app/agentworker"
|
||||
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
|
||||
@@ -267,7 +268,7 @@ func admitThroughStateStore(t *testing.T, steps *workflow.Manager, stepType stri
|
||||
t.Fatalf("build control plane server: %v", err)
|
||||
}
|
||||
|
||||
if resource, resourceErr := resources.Resource(t.Context(), WorkflowDefinitionsKind); resourceErr == nil {
|
||||
if resource, resourceErr := resources.Resource(t.Context(), storecontract.DefaultOrgID, WorkflowDefinitionsKind); resourceErr == nil {
|
||||
before = resource.Version
|
||||
}
|
||||
value, err := json.Marshal(probeCollection(stepType))
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
package controlplane
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/org"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
|
||||
)
|
||||
|
||||
const WorkflowEnablementKind = "workflow-enablement"
|
||||
|
||||
func workflowEnablementDefinition() Definition {
|
||||
return Definition{
|
||||
Kind: WorkflowEnablementKind,
|
||||
Title: "Workflow enablement",
|
||||
Document: task.WorkflowEnablement{},
|
||||
ApplyMode: "live",
|
||||
Seed: func(config.Config) any { return task.WorkflowEnablement{Orgs: map[org.OrgID]task.OrgWorkflows{}} },
|
||||
Validate: func(input []byte) error {
|
||||
return validateAs(input, func(enablement task.WorkflowEnablement) error { return enablement.Validate() })
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// WorkflowEnablement returns which workflows each org has disabled.
|
||||
func (c *Client) WorkflowEnablement(ctx context.Context) (task.WorkflowEnablement, error) {
|
||||
response, err := c.rpc.Query(ctx, controlplanerpc.QueryRequest(ctx, WorkflowEnablementKind))
|
||||
if err != nil {
|
||||
return task.WorkflowEnablement{}, controlplanerpc.ClientError(err)
|
||||
}
|
||||
var enablement task.WorkflowEnablement
|
||||
err = json.Unmarshal(response.Resource.ValueJson, &enablement)
|
||||
return enablement, err
|
||||
}
|
||||
@@ -0,0 +1,178 @@
|
||||
package controlplane
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
)
|
||||
|
||||
// projectionActor is the audit identity every package projection carries: the
|
||||
// write is made by the installed package, never by a request body.
|
||||
const projectionActor = "archie-state-store"
|
||||
|
||||
// projectWriteAttempts bounds one apply or withdrawal: each attempt re-reads
|
||||
// the org's document and re-merges, so a document the operator edited during
|
||||
// the projection is folded in, and only a persistently moving document
|
||||
// refuses the projection.
|
||||
const projectWriteAttempts = 3
|
||||
|
||||
// WorkflowProjector carries an installed package's workflow contributions
|
||||
// into the org's workflow-definitions resource, the same document an operator
|
||||
// edits on the dashboard (docs/prds/store.md, "Packages"). The readers the
|
||||
// daemon holds read that resource per dispatch, so a projected workflow is
|
||||
// usable at the next dispatch and a withdrawn one stops dispatching, without
|
||||
// restarting processes.
|
||||
//
|
||||
// Apply merges: a contributed id that belongs to an operator or to another
|
||||
// package is refused (storepkg.ErrContributionCollision), never silently
|
||||
// replaced.
|
||||
type WorkflowProjector struct {
|
||||
store ResourceStore
|
||||
ledger storepkg.ProjectionLedger
|
||||
steps workflow.StepRegistry
|
||||
}
|
||||
|
||||
var _ storepkg.FamilyProjector = (*WorkflowProjector)(nil)
|
||||
|
||||
// NewWorkflowPackageProjector builds the projector for storepkg.FamilyWorkflows
|
||||
// on the workflow step vocabulary the composition root registered: the
|
||||
// vocabulary the projected definitions are compiled and validated against,
|
||||
// the same one the control plane validates dashboard edits with.
|
||||
func NewWorkflowPackageProjector(resources ResourceStore, ledger storepkg.ProjectionLedger, manager *workflow.Manager) (*WorkflowProjector, error) {
|
||||
steps, err := stepRegistry(manager)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resources == nil || ledger == nil {
|
||||
return nil, errors.New("control plane: workflow projection needs a resource store and a contributions ledger")
|
||||
}
|
||||
return &WorkflowProjector{store: resources, ledger: ledger, steps: steps}, nil
|
||||
}
|
||||
|
||||
func (p *WorkflowProjector) Apply(ctx context.Context, orgID, name, digest string, contents map[string][]byte) error {
|
||||
entries := make([]workflow.WorkflowDefinitionEntry, 0, len(contents))
|
||||
recorded := make([]storepkg.ProjectionEntry, 0, len(contents))
|
||||
for contentName, content := range contents {
|
||||
id, err := workflow.DefinitionID(string(content))
|
||||
if err != nil {
|
||||
return fmt.Errorf("contributed workflow %q: %w", contentName, err)
|
||||
}
|
||||
entries = append(entries, workflow.WorkflowDefinitionEntry{ID: id, YAML: string(content)})
|
||||
recorded = append(recorded, storepkg.ProjectionEntry{Family: storepkg.FamilyWorkflows, EntryID: id})
|
||||
}
|
||||
// One apply, one request: a retried PutResource reuses the same request ID
|
||||
// and answers once, while a later install of the same package writes a
|
||||
// revision of its own.
|
||||
source := "package:" + name + "@" + digest
|
||||
if err := p.write(ctx, orgID, source, "package-project:"+digest, func(collection workflow.WorkflowDefinitionCollection) (workflow.WorkflowDefinitionCollection, error) {
|
||||
return mergeContribution(collection, entries, name)
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := p.ledger.Record(ctx, orgID, name, recorded); err != nil {
|
||||
return fmt.Errorf("record contributions of %q: %w", name, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *WorkflowProjector) Withdraw(ctx context.Context, orgID, name, digest string) error {
|
||||
entries, err := p.ledger.Entries(ctx, orgID, name)
|
||||
if err != nil {
|
||||
return fmt.Errorf("entries of package %q: %w", name, err)
|
||||
}
|
||||
withdrawn := make(map[string]struct{}, len(entries))
|
||||
for _, entry := range entries {
|
||||
if entry.Family == storepkg.FamilyWorkflows {
|
||||
withdrawn[entry.EntryID] = struct{}{}
|
||||
}
|
||||
}
|
||||
if len(withdrawn) == 0 {
|
||||
return nil
|
||||
}
|
||||
if err := p.write(ctx, orgID, "package-withdraw:"+name+"@"+digest, "package-withdraw:"+digest,
|
||||
func(collection workflow.WorkflowDefinitionCollection) (workflow.WorkflowDefinitionCollection, error) {
|
||||
merged := make([]workflow.WorkflowDefinitionEntry, 0, len(collection.Definitions))
|
||||
for _, item := range collection.Definitions {
|
||||
if _, gone := withdrawn[item.ID]; gone {
|
||||
continue
|
||||
}
|
||||
merged = append(merged, item)
|
||||
}
|
||||
return workflow.WorkflowDefinitionCollection{Definitions: merged}, nil
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
return p.ledger.Forget(ctx, orgID, name, storepkg.FamilyWorkflows)
|
||||
}
|
||||
|
||||
// write merges an org's stored workflow-definitions document and puts it
|
||||
// back, bounded by projectWriteAttempts: each attempt re-reads and re-merges,
|
||||
// so a document the operator edited during the projection is folded in, and
|
||||
// the write carries the ledger's replay identity of one episode.
|
||||
func (p *WorkflowProjector) write(ctx context.Context, orgID, source, requestPrefix string, merge func(workflow.WorkflowDefinitionCollection) (workflow.WorkflowDefinitionCollection, error)) error {
|
||||
requestID := requestPrefix + ":" + strconv.FormatInt(time.Now().UTC().UnixNano(), 36)
|
||||
for attempt := 0; ; attempt++ {
|
||||
collection, version := p.collection(ctx, orgID)
|
||||
merged, err := merge(collection)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
value, err := encodeWorkflowDefinitions(merged, p.steps)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := p.store.PutResource(ctx, storecontract.ResourceWrite{
|
||||
OrgID: orgID, Kind: WorkflowDefinitionsKind, Value: value, ExpectedVersion: version,
|
||||
Actor: projectionActor, Source: source, RequestID: requestID,
|
||||
}); errors.Is(err, storecontract.ErrResourceVersionConflict) && attempt < projectWriteAttempts-1 {
|
||||
continue
|
||||
} else if err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// collection reads an org's workflow-definitions document. An org that never
|
||||
// received one starts from the shipped definitions, the state a first write
|
||||
// seeds, so projection leaves the org with one answer either way.
|
||||
func (p *WorkflowProjector) collection(ctx context.Context, orgID string) (workflow.WorkflowDefinitionCollection, int64) {
|
||||
resource, err := p.store.Resource(ctx, orgID, WorkflowDefinitionsKind)
|
||||
if errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
return workflow.ShippedDefinitions(), 0
|
||||
}
|
||||
if err != nil {
|
||||
return workflow.WorkflowDefinitionCollection{}, 0
|
||||
}
|
||||
collection, err := workflow.DecodeDefinitionCollection(resource.Value, p.steps)
|
||||
if err != nil {
|
||||
return workflow.WorkflowDefinitionCollection{}, 0
|
||||
}
|
||||
return collection, resource.Version
|
||||
}
|
||||
|
||||
// mergeContribution merges a package's contributed entries into an org's
|
||||
// collection. An operator's entry with the same workflow id is a refusal, not
|
||||
// a silent replacement; the package's own entries reach here only after its
|
||||
// ledger rows were cleared, so colliding is always foreign work.
|
||||
func mergeContribution(base workflow.WorkflowDefinitionCollection, contributed []workflow.WorkflowDefinitionEntry, name string) (workflow.WorkflowDefinitionCollection, error) {
|
||||
replacing := make(map[string]struct{}, len(contributed))
|
||||
for _, entry := range contributed {
|
||||
replacing[entry.ID] = struct{}{}
|
||||
}
|
||||
merged := make([]workflow.WorkflowDefinitionEntry, 0, len(base.Definitions)+len(contributed))
|
||||
for _, entry := range base.Definitions {
|
||||
if _, incoming := replacing[entry.ID]; incoming {
|
||||
return workflow.WorkflowDefinitionCollection{}, fmt.Errorf("package %q: %w: workflow %q", name, storepkg.ErrContributionCollision, entry.ID)
|
||||
}
|
||||
merged = append(merged, entry)
|
||||
}
|
||||
merged = append(merged, contributed...)
|
||||
return workflow.WorkflowDefinitionCollection{Definitions: merged}, nil
|
||||
}
|
||||
@@ -0,0 +1,179 @@
|
||||
package controlplane
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
)
|
||||
|
||||
const testDigest = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
||||
|
||||
// testPackageProjector builds the workflow package projector against a real
|
||||
// migrated database and the production step vocabulary. The projector's
|
||||
// ledger rows describe installed packages, so a fixture installs one.
|
||||
func testPackageProjector(ctx context.Context, t *testing.T, orgID, pkgName, digest string) (*WorkflowProjector, ResourceStore, storepkg.ProjectionLedger) {
|
||||
t.Helper()
|
||||
db := pgstore.Open(t)
|
||||
installed := storepkg.Installed{
|
||||
OrgID: orgID, Name: pkgName, Reference: "localhost:5000/" + pkgName,
|
||||
Digest: digest, Descriptor: testPackageDescriptor(), Layer: []byte("layer"), UpdatePolicy: "manual",
|
||||
}
|
||||
if err := postgres.NewInstalledPackages(db.Pool).Install(ctx, installed); err != nil {
|
||||
t.Fatalf("install package record: %v", err)
|
||||
}
|
||||
resources := postgres.NewResources(db.Pool)
|
||||
ledger := postgres.NewPackageContributions(db.Pool)
|
||||
projector, err := NewWorkflowPackageProjector(resources, ledger, testSteps(t))
|
||||
if err != nil {
|
||||
t.Fatalf("build workflow projector: %v", err)
|
||||
}
|
||||
return projector, resources, ledger
|
||||
}
|
||||
|
||||
// testPackageDescriptor is the valid descriptor backing the ledger fixture.
|
||||
func testPackageDescriptor() storepkg.Descriptor {
|
||||
return storepkg.Descriptor{
|
||||
APIVersion: storepkg.APIVersion, DisplayName: "Review pack", Version: "1",
|
||||
Contributes: storepkg.Contributions{Workflows: []string{"workflows/review.yaml"}},
|
||||
}
|
||||
}
|
||||
|
||||
// contributedWorkflow builds a contributed workflow the shipped vocabulary
|
||||
// compiles, with a fresh id no shipped or operator workflow claims.
|
||||
func contributedWorkflow(name string) workflow.WorkflowDefinitionEntry {
|
||||
return workflow.WorkflowDefinitionEntry{ID: name, YAML: "id: " + name + "\nsteps:\n - type: bootstrap.apply\n"}
|
||||
}
|
||||
|
||||
// storedCollection returns an org's workflow-definitions document the way its
|
||||
// readers see it: shipped definitions stand in for an absent row.
|
||||
func storedCollection(ctx context.Context, t *testing.T, resources ResourceStore, orgID string) (workflow.WorkflowDefinitionCollection, int64) {
|
||||
t.Helper()
|
||||
resource, err := resources.Resource(ctx, orgID, WorkflowDefinitionsKind)
|
||||
if errors.Is(err, storecontract.ErrResourceNotFound) {
|
||||
return workflow.ShippedDefinitions(), 0
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
collection, err := workflow.DecodeDefinitionCollection(resource.Value, workflow.BuiltinStepRegistry())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return collection, resource.Version
|
||||
}
|
||||
|
||||
// operatorEdit installs a workflow the operator owns, exactly a dashboard
|
||||
// replacement does.
|
||||
func operatorEdit(ctx context.Context, t *testing.T, resources ResourceStore, orgID string, entry workflow.WorkflowDefinitionEntry) {
|
||||
t.Helper()
|
||||
collection, version := storedCollection(ctx, t, resources, orgID)
|
||||
collection.Definitions = append(collection.Definitions, entry)
|
||||
value, err := encodeWorkflowDefinitions(collection, testSteps(t).Registry())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := resources.PutResource(ctx, storecontract.ResourceWrite{
|
||||
OrgID: orgID, Kind: WorkflowDefinitionsKind, Value: value, ExpectedVersion: version,
|
||||
Actor: "operator", Source: "dashboard", RequestID: "operator-edit",
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func resourceVersionOf(ctx context.Context, t *testing.T, resources ResourceStore, orgID string) int64 {
|
||||
t.Helper()
|
||||
_, version := storedCollection(ctx, t, resources, orgID)
|
||||
return version
|
||||
}
|
||||
|
||||
func TestWorkflowProjectorApplyAndWithdraw(t *testing.T) {
|
||||
t.Parallel()
|
||||
ctx := context.Background()
|
||||
const pkgName = "review"
|
||||
const contributedID = "review"
|
||||
contributedYAML := contributedWorkflow(contributedID).YAML
|
||||
|
||||
// applied builds a projector that has projected one contributed workflow.
|
||||
applied := func(t *testing.T) (*WorkflowProjector, ResourceStore, storepkg.ProjectionLedger) {
|
||||
t.Helper()
|
||||
projector, resources, ledger := testPackageProjector(ctx, t, "org-a", pkgName, testDigest)
|
||||
if err := projector.Apply(ctx, "org-a", pkgName, testDigest, map[string][]byte{
|
||||
"workflows/review.yaml": []byte(contributedYAML),
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return projector, resources, ledger
|
||||
}
|
||||
|
||||
t.Run("apply makes the contributed workflow an org workflow without a restart", func(t *testing.T) {
|
||||
_, resources, ledger := applied(t)
|
||||
collection, _ := storedCollection(ctx, t, resources, "org-a")
|
||||
if _, ok := collection.DefinitionByID(contributedID); !ok {
|
||||
t.Fatal("org workflow-definitions does not carry the contributed workflow")
|
||||
}
|
||||
entries, err := ledger.Entries(ctx, "org-a", pkgName)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(entries) != 1 || entries[0] != (storepkg.ProjectionEntry{Family: storepkg.FamilyWorkflows, EntryID: contributedID}) {
|
||||
t.Fatalf("ledger = %#v", entries)
|
||||
}
|
||||
})
|
||||
t.Run("withdraw takes exactly the package's entries back out", func(t *testing.T) {
|
||||
projector, resources, _ := testPackageProjector(ctx, t, "org-a", pkgName, testDigest)
|
||||
other := contributedWorkflow("review-operator")
|
||||
if err := projector.Apply(ctx, "org-a", pkgName, testDigest, map[string][]byte{
|
||||
"workflows/review.yaml": []byte(contributedYAML),
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
operatorEdit(ctx, t, resources, "org-a", other)
|
||||
if err := projector.Withdraw(ctx, "org-a", pkgName, testDigest); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
collection, _ := storedCollection(ctx, t, resources, "org-a")
|
||||
if _, ok := collection.DefinitionByID(contributedID); ok {
|
||||
t.Fatal("the package's contributed workflow survives withdrawal")
|
||||
}
|
||||
if _, ok := collection.DefinitionByID(other.ID); !ok {
|
||||
t.Fatal("operator workflow was withdrawn by a package remove")
|
||||
}
|
||||
})
|
||||
t.Run("a contributed id colliding with an operator workflow is refused", func(t *testing.T) {
|
||||
projector, resources, _ := testPackageProjector(ctx, t, "org-a", pkgName, testDigest)
|
||||
operatorEdit(ctx, t, resources, "org-a", contributedWorkflow(contributedID))
|
||||
before := resourceVersionOf(ctx, t, resources, "org-a")
|
||||
if err := projector.Apply(ctx, "org-a", pkgName, testDigest, map[string][]byte{
|
||||
"workflows/review.yaml": []byte(contributedYAML),
|
||||
}); !errors.Is(err, storepkg.ErrContributionCollision) {
|
||||
t.Fatalf("colliding contribution error = %v", err)
|
||||
}
|
||||
if after := resourceVersionOf(ctx, t, resources, "org-a"); after != before {
|
||||
t.Fatal("refused contribution wrote the resource anyway")
|
||||
}
|
||||
})
|
||||
t.Run("an invalid contributed workflow is refused without a write", func(t *testing.T) {
|
||||
projector, resources, _ := testPackageProjector(ctx, t, "org-a", pkgName, testDigest)
|
||||
before := resourceVersionOf(ctx, t, resources, "org-a")
|
||||
if err := projector.Apply(ctx, "org-a", pkgName, testDigest, map[string][]byte{
|
||||
"workflows/broken.yaml": []byte("id: broken\nsteps:\n - type: not-a-step\n"),
|
||||
}); err == nil {
|
||||
t.Fatal("invalid workflow definition projected")
|
||||
}
|
||||
if after := resourceVersionOf(ctx, t, resources, "org-a"); after != before {
|
||||
t.Fatal("refused projection wrote the resource anyway")
|
||||
}
|
||||
})
|
||||
t.Run("withdraw of a package that never applied is a no-op", func(t *testing.T) {
|
||||
projector, _, _ := testPackageProjector(ctx, t, "org-x", pkgName, testDigest)
|
||||
if err := projector.Withdraw(ctx, "org-x", pkgName, testDigest); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -33,7 +33,7 @@ type executionSettingsDocument struct {
|
||||
// holds is left alone, and a kind it does not hold is seeded under the one seed
|
||||
// identity (see seedActor and seedRequestID).
|
||||
func (s *Server) ImportWorkflowExecutionSettings(ctx context.Context, settings workflow.ExecutionSettings) (int64, error) {
|
||||
resource, err := s.store.Resource(ctx, WorkflowExecutionSettingsKind)
|
||||
resource, err := s.store.Resource(ctx, storecontract.DefaultOrgID, WorkflowExecutionSettingsKind)
|
||||
if err == nil {
|
||||
return resource.Version, nil
|
||||
}
|
||||
@@ -48,7 +48,7 @@ func (s *Server) ImportWorkflowExecutionSettings(ctx context.Context, settings w
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
resource, err = s.store.PutResource(ctx, storecontract.ResourceWrite{Kind: WorkflowExecutionSettingsKind, Value: value, Actor: seedActor, Source: seedSource, RequestID: seedRequestID(WorkflowExecutionSettingsKind, value), ExpectedVersion: 0, At: time.Now().UTC()})
|
||||
resource, err = s.store.PutResource(ctx, storecontract.ResourceWrite{OrgID: storecontract.DefaultOrgID, Kind: WorkflowExecutionSettingsKind, Value: value, Actor: seedActor, Source: seedSource, RequestID: seedRequestID(WorkflowExecutionSettingsKind, value), ExpectedVersion: 0, At: time.Now().UTC()})
|
||||
return resource.Version, err
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
package telegram
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
|
||||
"github.com/go-telegram/bot"
|
||||
"github.com/go-telegram/bot/models"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/messaging"
|
||||
)
|
||||
|
||||
// inboundMedia pairs an inbound attachment with the textual note the
|
||||
// transcript stores in its place: stored history is text-only, so the
|
||||
// note is what later turns read once the attachment's bytes are gone.
|
||||
type inboundMedia struct {
|
||||
attachment messaging.MediaAttachment
|
||||
// note is a bracketed, self-describing placeholder such as "[photo]"
|
||||
// or "[document: report.pdf]".
|
||||
note string
|
||||
}
|
||||
|
||||
// extractInboundMedia maps a Telegram message onto the attachment it
|
||||
// carries. A Telegram message carries at most one file (a photo is the
|
||||
// set of its resolutions and collapses into one image), so the boolean is
|
||||
// false exactly for messages with nothing to download.
|
||||
func extractInboundMedia(msg *models.Message) (inboundMedia, bool) {
|
||||
switch {
|
||||
case len(msg.Photo) > 0:
|
||||
att := messaging.MediaAttachment{Type: "image", MIMEType: "image/jpeg"}
|
||||
for _, size := range msg.Photo {
|
||||
if largestPhoto(size, att) {
|
||||
att.FileID = size.FileID
|
||||
att.Width, att.Height = &size.Width, &size.Height
|
||||
if size.FileSize != 0 {
|
||||
att.FileSize = new(int64(size.FileSize))
|
||||
}
|
||||
}
|
||||
}
|
||||
return inboundMedia{att, "[photo]"}, true
|
||||
|
||||
case msg.Document != nil:
|
||||
return inboundMedia{messaging.MediaAttachment{
|
||||
Type: "document",
|
||||
FileID: msg.Document.FileID,
|
||||
MIMEType: msg.Document.MimeType,
|
||||
FileName: msg.Document.FileName,
|
||||
FileSize: nilIfZero(msg.Document.FileSize),
|
||||
}, documentNote(msg.Document.FileName)}, true
|
||||
|
||||
case msg.Video != nil:
|
||||
return inboundMedia{videoAttachment(msg.Video.FileID, msg.Video.MimeType, nilIfZero(msg.Video.FileSize), &msg.Video.Width, &msg.Video.Height, &msg.Video.Duration), "[video]"}, true
|
||||
|
||||
case msg.VideoNote != nil:
|
||||
size := int64(msg.VideoNote.FileSize)
|
||||
return inboundMedia{videoAttachment(msg.VideoNote.FileID, "video/mp4", nilIfZero(size), nil, nil, &msg.VideoNote.Duration), "[video message]"}, true
|
||||
|
||||
case msg.Audio != nil:
|
||||
return inboundMedia{messaging.MediaAttachment{
|
||||
Type: "audio",
|
||||
FileID: msg.Audio.FileID,
|
||||
MIMEType: msg.Audio.MimeType,
|
||||
FileName: msg.Audio.FileName,
|
||||
FileSize: nilIfZero(msg.Audio.FileSize),
|
||||
Duration: &msg.Audio.Duration,
|
||||
}, "[audio]"}, true
|
||||
|
||||
case msg.Voice != nil:
|
||||
return inboundMedia{messaging.MediaAttachment{
|
||||
Type: "audio",
|
||||
FileID: msg.Voice.FileID,
|
||||
MIMEType: msg.Voice.MimeType,
|
||||
FileSize: nilIfZero(msg.Voice.FileSize),
|
||||
Duration: &msg.Voice.Duration,
|
||||
}, "[voice message]"}, true
|
||||
}
|
||||
return inboundMedia{}, false
|
||||
}
|
||||
|
||||
// largestPhoto reports whether candidate is larger than the best size
|
||||
// recorded so far in att's width/height pair.
|
||||
func largestPhoto(candidate models.PhotoSize, att messaging.MediaAttachment) bool {
|
||||
if att.Width == nil {
|
||||
return true
|
||||
}
|
||||
candidateArea, bestArea := candidate.Width*candidate.Height, *att.Width**att.Height
|
||||
return candidateArea > bestArea
|
||||
}
|
||||
|
||||
func videoAttachment(fileID, mimeType string, fileSize *int64, width, height, duration *int) messaging.MediaAttachment {
|
||||
return messaging.MediaAttachment{
|
||||
Type: "video", FileID: fileID, MIMEType: mimeType,
|
||||
FileSize: fileSize, Width: width, Height: height, Duration: duration,
|
||||
}
|
||||
}
|
||||
|
||||
func documentNote(name string) string {
|
||||
if name == "" {
|
||||
return "[document]"
|
||||
}
|
||||
return fmt.Sprintf("[document: %s]", name)
|
||||
}
|
||||
|
||||
func nilIfZero(v int64) *int64 {
|
||||
if v == 0 {
|
||||
return nil
|
||||
}
|
||||
return &v
|
||||
}
|
||||
|
||||
// maxInboundDownloadBytes is the Bot API download ceiling: bots cannot
|
||||
// fetch files larger than 20 MB through getFile, so a larger attachment
|
||||
// must be refused with a clear notice rather than met with a Bot API 400
|
||||
// nobody reads.
|
||||
var maxInboundDownloadBytes int64 = 20 * 1024 * 1024
|
||||
|
||||
var errInboundMediaTooLarge = errors.New("file exceeds the bot download limit")
|
||||
|
||||
// downloadMedia turns a Telegram file id into the file's bytes: getFile
|
||||
// resolves the transfer path, then the file is fetched from the Bot API
|
||||
// server. Called on the turn lane, which is already running the rest of
|
||||
// the download-and-chat work for this message.
|
||||
func (g *Gateway) downloadMedia(ctx context.Context, b *bot.Bot, att messaging.MediaAttachment) ([]byte, error) {
|
||||
if att.FileID == "" {
|
||||
return nil, fmt.Errorf("media attachment carries no file id")
|
||||
}
|
||||
if att.FileSize != nil && *att.FileSize > maxInboundDownloadBytes {
|
||||
return nil, fmt.Errorf("%w: %d bytes", errInboundMediaTooLarge, *att.FileSize)
|
||||
}
|
||||
file, err := b.GetFile(ctx, &bot.GetFileParams{FileID: att.FileID})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("resolve telegram file: %w", err)
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, b.FileDownloadLink(file), nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("download telegram file: %w", err)
|
||||
}
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("download telegram file: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("download telegram file: status %d", resp.StatusCode)
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(resp.Body, maxInboundDownloadBytes+1))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("download telegram file: %w", err)
|
||||
}
|
||||
if int64(len(data)) > maxInboundDownloadBytes {
|
||||
return nil, errInboundMediaTooLarge
|
||||
}
|
||||
return data, nil
|
||||
}
|
||||
|
||||
// turnMessageText renders the text an inbound message's turn is built
|
||||
// from: plain text as received, or the attachment note plus caption when
|
||||
// the message carries a file. The note is what later turns read from
|
||||
// history, so a captioned photo whose caption alone would read as the
|
||||
// whole message ("what is this?") still names its attachment.
|
||||
func turnMessageText(msg *models.Message, media inboundMedia, hasMedia bool) string {
|
||||
if !hasMedia {
|
||||
return msg.Text
|
||||
}
|
||||
if msg.Caption == "" {
|
||||
return media.note
|
||||
}
|
||||
return media.note + "\n" + msg.Caption
|
||||
}
|
||||
|
||||
// unsupportedMediaNotice returns the refusal for content kinds Archie
|
||||
// cannot read, and "" for update content that is not a message to answer
|
||||
// at all -- a group member change or pin must stay silent, while a
|
||||
// sticker the sender meant Archie to react to must not disappear into the
|
||||
// void.
|
||||
func unsupportedMediaNotice(msg *models.Message) string {
|
||||
switch {
|
||||
case msg.Sticker != nil:
|
||||
return "I can't read stickers — send the image or file directly and I'll take a look."
|
||||
case msg.Location != nil:
|
||||
return "I can't use locations yet."
|
||||
case msg.Contact != nil:
|
||||
return "I can't use contact cards yet."
|
||||
case msg.Poll != nil:
|
||||
return "I can't use polls yet."
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// mediaReplyLabel names an attachment the turn could not serve, in the
|
||||
// sentence the sender reads.
|
||||
func mediaReplyLabel(att messaging.MediaAttachment) string {
|
||||
switch {
|
||||
case att.Type == "image":
|
||||
return "I received your photo "
|
||||
case att.Type == "document" && att.FileName != "":
|
||||
return fmt.Sprintf("I received %s ", att.FileName)
|
||||
case att.Type == "document":
|
||||
return "I received your file "
|
||||
case att.Type == "video":
|
||||
return "I received your video "
|
||||
default:
|
||||
return "I received your audio attachment "
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,383 @@
|
||||
package telegram
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/go-telegram/bot"
|
||||
"github.com/go-telegram/bot/models"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/messaging"
|
||||
)
|
||||
|
||||
// recordedAPI captures every Bot API request so a test can assert a reply
|
||||
// was sent (or, equally, that none was).
|
||||
type recordedAPI struct {
|
||||
mu sync.Mutex
|
||||
calls []string // "path body" lines, one per request
|
||||
}
|
||||
|
||||
func newRecordedAPI(t *testing.T, fileRoute func(w http.ResponseWriter, r *http.Request) bool) (*bot.Bot, *recordedAPI) {
|
||||
t.Helper()
|
||||
rec := &recordedAPI{}
|
||||
api := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if fileRoute != nil && fileRoute(w, r) {
|
||||
return
|
||||
}
|
||||
body := readBody(r)
|
||||
rec.mu.Lock()
|
||||
rec.calls = append(rec.calls, r.URL.Path+" "+body)
|
||||
rec.mu.Unlock()
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if strings.HasSuffix(r.URL.Path, "getFile") {
|
||||
_, _ = w.Write([]byte(`{"ok":true,"result":{"file_path":"photos/pic.jpg"}}`))
|
||||
return
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"ok":true,"result":true}`))
|
||||
}))
|
||||
t.Cleanup(api.Close)
|
||||
|
||||
b, err := bot.New("1:test", bot.WithServerURL(api.URL), bot.WithSkipGetMe())
|
||||
if err != nil {
|
||||
t.Fatalf("new test bot: %v", err)
|
||||
}
|
||||
return b, rec
|
||||
}
|
||||
|
||||
func readBody(r *http.Request) string {
|
||||
_ = r.ParseMultipartForm(1 << 20)
|
||||
if r.PostForm != nil {
|
||||
return r.PostForm.Encode()
|
||||
}
|
||||
return r.URL.RawQuery
|
||||
}
|
||||
|
||||
func (rec *recordedAPI) contains(fragment string) bool {
|
||||
rec.mu.Lock()
|
||||
defer rec.mu.Unlock()
|
||||
for _, call := range rec.calls {
|
||||
if strings.Contains(strings.ToLower(call), strings.ToLower(fragment)) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (rec *recordedAPI) callCount() int {
|
||||
rec.mu.Lock()
|
||||
defer rec.mu.Unlock()
|
||||
return len(rec.calls)
|
||||
}
|
||||
|
||||
func testMediaGateway(b *bot.Bot) *Gateway {
|
||||
g := New("1:test", []int64{42}, slog.New(slog.DiscardHandler))
|
||||
g.bot = b
|
||||
g.turns = messaging.NewTurns(g.log)
|
||||
return g
|
||||
}
|
||||
|
||||
// TestExtractInboundMediaMapsTelegramKinds walks every supported message
|
||||
// kind. A message's media decides the file the bot must download, so a
|
||||
// mapping error sends the wrong file or sends nothing at all.
|
||||
func TestExtractInboundMediaMapsTelegramKinds(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
msg *models.Message
|
||||
wantType string
|
||||
wantFileID string
|
||||
wantMIME string
|
||||
wantName string
|
||||
wantNote string
|
||||
wantNoMedia bool
|
||||
}{
|
||||
{
|
||||
name: "photo picks the largest size",
|
||||
msg: &models.Message{
|
||||
Photo: []models.PhotoSize{
|
||||
{FileID: "small", Width: 90, Height: 80},
|
||||
{FileID: "big", Width: 1280, Height: 960, FileSize: 150000},
|
||||
},
|
||||
},
|
||||
wantType: "image",
|
||||
wantFileID: "big",
|
||||
wantMIME: "image/jpeg",
|
||||
wantNote: "[photo]",
|
||||
},
|
||||
{
|
||||
name: "document",
|
||||
msg: &models.Message{
|
||||
Document: &models.Document{
|
||||
FileID: "doc1", FileName: "report.pdf", MimeType: "application/pdf", FileSize: 2048,
|
||||
},
|
||||
},
|
||||
wantType: "document",
|
||||
wantFileID: "doc1",
|
||||
wantMIME: "application/pdf",
|
||||
wantName: "report.pdf",
|
||||
wantNote: "[document: report.pdf]",
|
||||
},
|
||||
{
|
||||
name: "video",
|
||||
msg: &models.Message{
|
||||
Video: &models.Video{FileID: "vid1", Duration: 12, MimeType: "video/mp4"},
|
||||
},
|
||||
wantType: "video",
|
||||
wantFileID: "vid1",
|
||||
wantMIME: "video/mp4",
|
||||
wantNote: "[video]",
|
||||
},
|
||||
{
|
||||
name: "video note",
|
||||
msg: &models.Message{
|
||||
VideoNote: &models.VideoNote{FileID: "vnote1", Duration: 30},
|
||||
},
|
||||
wantType: "video",
|
||||
wantFileID: "vnote1",
|
||||
wantMIME: "video/mp4",
|
||||
wantNote: "[video message]",
|
||||
},
|
||||
{
|
||||
name: "audio",
|
||||
msg: &models.Message{
|
||||
Audio: &models.Audio{FileID: "aud1", FileName: "song.mp3", MimeType: "audio/mpeg", Duration: 200},
|
||||
},
|
||||
wantType: "audio",
|
||||
wantFileID: "aud1",
|
||||
wantMIME: "audio/mpeg",
|
||||
wantName: "song.mp3",
|
||||
wantNote: "[audio]",
|
||||
},
|
||||
{
|
||||
name: "voice",
|
||||
msg: &models.Message{
|
||||
Voice: &models.Voice{FileID: "voice1", MimeType: "audio/ogg", Duration: 8},
|
||||
},
|
||||
wantType: "audio",
|
||||
wantFileID: "voice1",
|
||||
wantMIME: "audio/ogg",
|
||||
wantNote: "[voice message]",
|
||||
},
|
||||
{
|
||||
name: "plain text message carries no media",
|
||||
msg: &models.Message{Text: "hello"},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got, ok := extractInboundMedia(tc.msg)
|
||||
if tc.wantType == "" {
|
||||
if ok {
|
||||
t.Fatalf("extractInboundMedia(%s) = ok, want none", tc.name)
|
||||
}
|
||||
return
|
||||
}
|
||||
if !ok {
|
||||
t.Fatalf("extractInboundMedia(%s) missed, want %s", tc.name, tc.wantType)
|
||||
}
|
||||
att := got.attachment
|
||||
if att.Type != tc.wantType {
|
||||
t.Errorf("Type = %q, want %q", att.Type, tc.wantType)
|
||||
}
|
||||
if att.FileID != tc.wantFileID {
|
||||
t.Errorf("FileID = %q, want %q", att.FileID, tc.wantFileID)
|
||||
}
|
||||
if att.MIMEType != tc.wantMIME {
|
||||
t.Errorf("MIMEType = %q, want %q", att.MIMEType, tc.wantMIME)
|
||||
}
|
||||
if att.FileName != tc.wantName {
|
||||
t.Errorf("FileName = %q, want %q", att.FileName, tc.wantName)
|
||||
}
|
||||
if got.note != tc.wantNote {
|
||||
t.Errorf("note = %q, want %q", got.note, tc.wantNote)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestTurnMessageTextKeepsCaptionWithPlaceholder: history is text-only, so
|
||||
// the stored turn text must name the attachment and keep any caption.
|
||||
func TestTurnMessageTextKeepsCaptionWithPlaceholder(t *testing.T) {
|
||||
media, ok := extractInboundMedia(&models.Message{
|
||||
Photo: []models.PhotoSize{{FileID: "a", Width: 10, Height: 10}},
|
||||
Caption: "what is this?",
|
||||
})
|
||||
if !ok {
|
||||
t.Fatal("media missed")
|
||||
}
|
||||
got := turnMessageText(&models.Message{Text: "", Caption: "what is this?"}, media, ok)
|
||||
want := "[photo]\nwhat is this?"
|
||||
if got != want {
|
||||
t.Errorf("turnMessageText = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestDefaultHandlerSubmitsPhotoTurnWithDownloadedMedia drives the one
|
||||
// behaviour at the heart of the gap: a photo update reaches the chat turn
|
||||
// with the file's bytes attached, not as a dropped message.
|
||||
func TestDefaultHandlerSubmitsPhotoTurnWithDownloadedMedia(t *testing.T) {
|
||||
photoBytes := []byte("jpeg-bytes")
|
||||
var fileServed bool
|
||||
b, rec := newRecordedAPI(t, func(w http.ResponseWriter, r *http.Request) bool {
|
||||
if strings.HasPrefix(r.URL.Path, "/file/bot1:test/") {
|
||||
w.Header().Set("Content-Type", "image/jpeg")
|
||||
_, _ = w.Write(photoBytes)
|
||||
fileServed = true
|
||||
return true
|
||||
}
|
||||
return false
|
||||
})
|
||||
|
||||
inboundCh := make(chan messaging.Inbound, 1)
|
||||
client := &fakeChatContract{
|
||||
streamFunc: func(_ context.Context, in messaging.Inbound) (<-chan messaging.ChatEvent, error) {
|
||||
inboundCh <- in
|
||||
ch := make(chan messaging.ChatEvent, 2)
|
||||
ch <- messaging.ChatEvent{Kind: "started", SessionID: "s1"}
|
||||
ch <- messaging.ChatEvent{Kind: "done", Text: "done", SessionID: "s1"}
|
||||
close(ch)
|
||||
return ch, nil
|
||||
},
|
||||
}
|
||||
g := testMediaGateway(b)
|
||||
|
||||
handler := g.defaultHandler(client)
|
||||
handler(context.Background(), b, &models.Update{Message: &models.Message{
|
||||
ID: 5,
|
||||
Date: 1,
|
||||
From: &models.User{ID: 42, Username: "sam"},
|
||||
Chat: models.Chat{ID: 7, Type: "private"},
|
||||
Photo: []models.PhotoSize{{FileID: "big", Width: 1280, Height: 960, FileSize: 150000}},
|
||||
Caption: "look at this",
|
||||
}})
|
||||
|
||||
select {
|
||||
case in := <-inboundCh:
|
||||
if len(in.Media) != 1 {
|
||||
t.Fatalf("inbound media = %#v, want one attachment", in.Media)
|
||||
}
|
||||
att := in.Media[0]
|
||||
if att.Type != "image" || att.MIMEType != "image/jpeg" || att.FileID != "big" {
|
||||
t.Errorf("attachment = %#v, want image with the photo's file id", att)
|
||||
}
|
||||
if string(att.Data) != string(photoBytes) {
|
||||
t.Errorf("Data = %q, want the downloaded photo bytes", att.Data)
|
||||
}
|
||||
want := "[photo]\nlook at this"
|
||||
if in.Message.Text != want {
|
||||
t.Errorf("Text = %q, want %q", in.Message.Text, want)
|
||||
}
|
||||
if in.Message.SourceID != "5" {
|
||||
t.Errorf("SourceID = %q, want the Telegram message id", in.Message.SourceID)
|
||||
}
|
||||
if !fileServed {
|
||||
t.Error("download never fetched the file bytes")
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatalf("photo update never reached the chat turn; api calls were:\n%s",
|
||||
strings.Join(rec.calls, "\n"))
|
||||
}
|
||||
}
|
||||
|
||||
// TestDefaultHandlerRepliesToUnsupportedMedia keeps the gap from returning
|
||||
// silently in the other direction: content types Archie cannot read get a
|
||||
// clear notice rather than the void.
|
||||
func TestDefaultHandlerRepliesToUnsupportedMedia(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
setup func(*models.Message)
|
||||
reply string
|
||||
}{
|
||||
{"sticker", func(m *models.Message) { m.Sticker = &models.Sticker{FileID: "s"} }, "sticker"},
|
||||
{"location", func(m *models.Message) { m.Location = &models.Location{} }, "location"},
|
||||
{"contact", func(m *models.Message) { m.Contact = &models.Contact{} }, "contact"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
b, rec := newRecordedAPI(t, nil)
|
||||
g := testMediaGateway(b)
|
||||
msg := &models.Message{
|
||||
ID: 6,
|
||||
Date: 1,
|
||||
From: &models.User{ID: 42, Username: "sam"},
|
||||
Chat: models.Chat{ID: 7, Type: "private"},
|
||||
}
|
||||
tc.setup(msg)
|
||||
g.defaultHandler(&fakeChatContract{})(context.Background(), b, &models.Update{Message: msg})
|
||||
if !rec.contains(tc.reply) {
|
||||
t.Errorf("no reply mentioning %q was sent; calls:\n%s", tc.reply, strings.Join(rec.calls, "\n"))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestDefaultHandlerSilentForNonContentMessages guards the other update
|
||||
// kinds a group can carry: no text, no media, no notice -- and so no reply.
|
||||
func TestDefaultHandlerSilentForNonContentMessages(t *testing.T) {
|
||||
b, rec := newRecordedAPI(t, nil)
|
||||
g := testMediaGateway(b)
|
||||
g.defaultHandler(&fakeChatContract{})(context.Background(), b, &models.Update{Message: &models.Message{
|
||||
ID: 7,
|
||||
Date: 1,
|
||||
From: &models.User{ID: 42},
|
||||
Chat: models.Chat{ID: 7, Type: "private"},
|
||||
}})
|
||||
if n := rec.callCount(); n != 0 {
|
||||
t.Errorf("%d API calls for a message with no text, media or notice, want 0", n)
|
||||
}
|
||||
}
|
||||
|
||||
// TestDownloadMediaCoversSuccessAndLimits exercises the download the turn
|
||||
// relies on: the Bot API getFile lookup, the file fetch, and the size gate.
|
||||
func TestDownloadMediaCoversSuccessAndLimits(t *testing.T) {
|
||||
fileBytes := []byte("file-bytes")
|
||||
b, rec := newRecordedAPI(t, func(w http.ResponseWriter, r *http.Request) bool {
|
||||
if strings.HasPrefix(r.URL.Path, "/file/bot1:test/") {
|
||||
_, _ = w.Write(fileBytes)
|
||||
return true
|
||||
}
|
||||
return false
|
||||
})
|
||||
g := testMediaGateway(b)
|
||||
|
||||
t.Run("success", func(t *testing.T) {
|
||||
data, err := g.downloadMedia(context.Background(), b,
|
||||
messaging.MediaAttachment{Type: "image", FileID: "f1", MIMEType: "image/jpeg"})
|
||||
if err != nil {
|
||||
t.Fatalf("downloadMedia() error = %v", err)
|
||||
}
|
||||
if string(data) != string(fileBytes) {
|
||||
t.Errorf("downloaded %q, want %q", data, fileBytes)
|
||||
}
|
||||
if !rec.contains("getfile") {
|
||||
t.Error("getFile never resolved the file path")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("oversized declared size is refused before download", func(t *testing.T) {
|
||||
old := maxInboundDownloadBytes
|
||||
maxInboundDownloadBytes = 4
|
||||
t.Cleanup(func() { maxInboundDownloadBytes = old })
|
||||
size := int64(10)
|
||||
if _, err := g.downloadMedia(context.Background(), b,
|
||||
messaging.MediaAttachment{Type: "document", FileID: "f2", FileSize: &size}); err == nil {
|
||||
t.Error("oversized declared size accepted")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("oversized response body is refused", func(t *testing.T) {
|
||||
old := maxInboundDownloadBytes
|
||||
maxInboundDownloadBytes = 4
|
||||
t.Cleanup(func() { maxInboundDownloadBytes = old })
|
||||
if _, err := g.downloadMedia(context.Background(), b,
|
||||
messaging.MediaAttachment{Type: "document", FileID: "f3"}); err == nil {
|
||||
t.Error("response larger than the limit accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -519,7 +519,7 @@ func (g *Gateway) startHandler() bot.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// ── default handler (non-command text → router) ─────────────
|
||||
// ── default handler (commands and non-command messages → router) ─────
|
||||
|
||||
func (g *Gateway) defaultHandler(client messaging.ChatContract) bot.HandlerFunc {
|
||||
return func(ctx context.Context, b *bot.Bot, update *models.Update) {
|
||||
@@ -529,7 +529,20 @@ func (g *Gateway) defaultHandler(client messaging.ChatContract) bot.HandlerFunc
|
||||
}
|
||||
|
||||
msg, ok := g.authorizedMessage(ctx, b, update)
|
||||
if !ok || msg.Text == "" {
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
// Media messages arrive before the text gates: a photo has no text,
|
||||
// so checking it first is what keeps attachments from being dropped.
|
||||
if _, isMedia := extractInboundMedia(msg); isMedia {
|
||||
g.submitTurn(ctx, b, msg, client)
|
||||
return
|
||||
}
|
||||
if notice := unsupportedMediaNotice(msg); notice != "" {
|
||||
g.sendMessage(ctx, b, msg.Chat.ID, msg.MessageThreadID, notice)
|
||||
return
|
||||
}
|
||||
if msg.Text == "" {
|
||||
return
|
||||
}
|
||||
if isModelSelectorRequest(msg.Text) {
|
||||
@@ -584,27 +597,52 @@ func (g *Gateway) handleCallback(ctx context.Context, b *bot.Bot, update *models
|
||||
// running here would block delivery of every later update -- including the
|
||||
// /stop meant to cancel it, which would sit unread until the turn it was
|
||||
// aimed at had already finished.
|
||||
// turnInbound renders a Telegram message as the channel-neutral inbound a
|
||||
// chat turn is built from, with media attachments attached. ok is false
|
||||
// for messages with neither text nor media, which are not turns at all.
|
||||
func turnInbound(msg *models.Message) (messaging.Inbound, bool) {
|
||||
media, hasMedia := extractInboundMedia(msg)
|
||||
if !hasMedia && msg.Text == "" {
|
||||
return messaging.Inbound{}, false
|
||||
}
|
||||
gm := messaging.Inbound{
|
||||
Platform: "telegram",
|
||||
Message: messaging.Message{
|
||||
// Telegram's message ID makes persistence idempotent: the store
|
||||
// derives a canonical ID from it, so a redelivered update is a
|
||||
// no-op rather than appending a duplicate or overwriting the
|
||||
// stored record, which must stay immutable. Date is the sender's
|
||||
// clock reading and is what history should be ordered by.
|
||||
SourceID: fmt.Sprintf("%d", msg.ID),
|
||||
ConversationID: conversationID(msg),
|
||||
Sender: msg.From.Username,
|
||||
SenderID: fmt.Sprintf("%d", msg.From.ID),
|
||||
Role: messaging.RoleUser,
|
||||
Text: turnMessageText(msg, media, hasMedia),
|
||||
At: time.Unix(int64(msg.Date), 0).UTC(),
|
||||
},
|
||||
}
|
||||
if hasMedia {
|
||||
gm.Media = []messaging.MediaAttachment{media.attachment}
|
||||
}
|
||||
return gm, true
|
||||
}
|
||||
|
||||
func (g *Gateway) submitTurn(ctx context.Context, b *bot.Bot, msg *models.Message, client messaging.ChatContract) {
|
||||
gm := messaging.Inbound{Platform: "telegram", Message: messaging.Message{
|
||||
// Telegram's message ID makes persistence idempotent: the store
|
||||
// derives a canonical ID from it, so a redelivered update is a
|
||||
// no-op rather than appending a duplicate or overwriting the
|
||||
// stored record, which must stay immutable. Date is the sender's
|
||||
// clock reading and is what history should be ordered by.
|
||||
SourceID: fmt.Sprintf("%d", msg.ID),
|
||||
ConversationID: conversationID(msg),
|
||||
Sender: msg.From.Username,
|
||||
SenderID: fmt.Sprintf("%d", msg.From.ID),
|
||||
Role: messaging.RoleUser,
|
||||
Text: msg.Text,
|
||||
At: time.Unix(int64(msg.Date), 0).UTC(),
|
||||
}}
|
||||
gm, isTurn := turnInbound(msg)
|
||||
if !isTurn {
|
||||
return
|
||||
}
|
||||
|
||||
// The lane key must be the session, so that /stop -- which resolves
|
||||
// the same key -- reaches the turn the sender is actually watching.
|
||||
session := conversationID(msg).String()
|
||||
chatID, threadID := msg.Chat.ID, msg.MessageThreadID
|
||||
g.turns.Submit(ctx, session, func(turnCtx context.Context) {
|
||||
if !g.fetchTurnMedia(turnCtx, b, &gm, chatID, threadID) {
|
||||
return
|
||||
}
|
||||
|
||||
// If the turn invokes a tool that requires human approval,
|
||||
// the dispatch layer blocks on this approver. Nil is fine
|
||||
// — most turns need no gating.
|
||||
@@ -629,53 +667,92 @@ func (g *Gateway) submitTurn(ctx context.Context, b *bot.Bot, msg *models.Messag
|
||||
return
|
||||
}
|
||||
|
||||
var reply string
|
||||
var streamErr error
|
||||
for ev := range events {
|
||||
switch ev.Kind {
|
||||
case "delta":
|
||||
live.Delta(ev.Text)
|
||||
case "tool":
|
||||
live.ToolCall(ev.Tool)
|
||||
case "media":
|
||||
live.Media(turnCtx, ev.Media)
|
||||
case "done":
|
||||
reply = ev.Text
|
||||
case "error":
|
||||
streamErr = errors.New(ev.Text)
|
||||
}
|
||||
}
|
||||
stopTyping()
|
||||
|
||||
// A cancelled turn is a /stop, not a fault. The stop handler has
|
||||
// already acknowledged it, and turnCtx is dead, so there is
|
||||
// nothing useful left to send from here -- but whatever was
|
||||
// already streamed stays, minus the cursor that would otherwise
|
||||
// claim the answer is still being written. abandon strips turnCtx's
|
||||
// own cancellation before it edits, so the cursor-drop is not itself
|
||||
// aborted by the /stop that triggered it.
|
||||
//
|
||||
// errors.Is(err, context.Canceled) reliably separates a /stop from a
|
||||
// fault; either way turnCtx is dead by the time we get here. A
|
||||
// genuine fault is marked as failed rather than left as a clean
|
||||
// partial: /stop is an acknowledged interruption, a provider error
|
||||
// is not, and an unmarked partial reads as a finished answer either
|
||||
// way.
|
||||
if streamErr != nil || turnCtx.Err() != nil {
|
||||
if errors.Is(turnCtx.Err(), context.Canceled) || (streamErr != nil && errors.Is(streamErr, context.Canceled)) {
|
||||
g.log.Info("chat turn stopped", "session", session)
|
||||
live.abandon(turnCtx)
|
||||
} else {
|
||||
g.log.Error("stream turn failed", "error", streamErr)
|
||||
live.abandonFailed(turnCtx)
|
||||
}
|
||||
reply, aborted := g.drainTurnEvents(turnCtx, session, live, stopTyping, events)
|
||||
if aborted {
|
||||
return
|
||||
}
|
||||
live.finalize(turnCtx, reply)
|
||||
})
|
||||
}
|
||||
|
||||
// ── helpers ──────────────────────────────────────────────────
|
||||
// drainTurnEvents consumes a streamed turn, rendering deltas, tool calls
|
||||
// and media as they arrive, and returns the final reply.
|
||||
//
|
||||
// A cancelled turn is a /stop, not a fault. The stop handler has already
|
||||
// acknowledged it, and turnCtx is dead, so there is
|
||||
// nothing useful left to send from here -- but whatever was
|
||||
// already streamed stays, minus the cursor that would otherwise
|
||||
// claim the answer is still being written. abandon strips turnCtx's
|
||||
// own cancellation before it edits, so the cursor-drop is not itself
|
||||
// aborted by the /stop that triggered it.
|
||||
//
|
||||
// errors.Is(err, context.Canceled) reliably separates a /stop from a
|
||||
// fault; either way turnCtx is dead by the time we get here. A
|
||||
// genuine fault is marked as failed rather than left as a clean
|
||||
// partial: /stop is an acknowledged interruption, a provider error
|
||||
// is not, and an unmarked partial reads as a finished answer either
|
||||
// way.
|
||||
//
|
||||
// aborted means the turn was abandoned (as a stop or a failure) and its
|
||||
// reply must not be finalized.
|
||||
func (g *Gateway) drainTurnEvents(
|
||||
turnCtx context.Context, session string,
|
||||
live *liveReply, stopTyping func(), events <-chan messaging.ChatEvent,
|
||||
) (string, bool) {
|
||||
var reply string
|
||||
var streamErr error
|
||||
for ev := range events {
|
||||
switch ev.Kind {
|
||||
case "delta":
|
||||
live.Delta(ev.Text)
|
||||
case "tool":
|
||||
live.ToolCall(ev.Tool)
|
||||
case "media":
|
||||
live.Media(turnCtx, ev.Media)
|
||||
case "done":
|
||||
reply = ev.Text
|
||||
case "error":
|
||||
streamErr = errors.New(ev.Text)
|
||||
}
|
||||
}
|
||||
stopTyping()
|
||||
|
||||
if streamErr != nil || turnCtx.Err() != nil {
|
||||
if errors.Is(turnCtx.Err(), context.Canceled) || (streamErr != nil && errors.Is(streamErr, context.Canceled)) {
|
||||
g.log.Info("chat turn stopped", "session", session)
|
||||
live.abandon(turnCtx)
|
||||
} else {
|
||||
g.log.Error("stream turn failed", "error", streamErr)
|
||||
live.abandonFailed(turnCtx)
|
||||
}
|
||||
return "", true
|
||||
}
|
||||
return reply, false
|
||||
}
|
||||
|
||||
// fetchTurnMedia downloads the turn's inbound attachment onto the lane,
|
||||
// where waiting for the Telegram API belongs: the update worker stays free
|
||||
// to serve other messages and a /stop while this download progresses. A
|
||||
// failed download is reported as a failed attachment without starting the
|
||||
// turn, so the answer does not pretend the media was read -- a later
|
||||
// turn's history only knows the note, not the picture, and the model must
|
||||
// not reason from what it never saw.
|
||||
func (g *Gateway) fetchTurnMedia(ctx context.Context, b *bot.Bot, gm *messaging.Inbound, chatID int64, threadID int) bool {
|
||||
if len(gm.Media) == 0 || gm.Media[0].Data != nil {
|
||||
return true
|
||||
}
|
||||
data, err := g.downloadMedia(ctx, b, gm.Media[0])
|
||||
if err != nil {
|
||||
g.log.Warn("inbound media download failed", "media_type", gm.Media[0].Type, "error", err)
|
||||
g.sendMessage(ctx, b, chatID, threadID,
|
||||
mediaReplyLabel(gm.Media[0])+"but I couldn't download it — try sending it again.")
|
||||
return false
|
||||
}
|
||||
gm.Media[0].Data = data
|
||||
return true
|
||||
}
|
||||
|
||||
// ── helpers ──────────────────────────────────────────────────────
|
||||
|
||||
// conversationID addresses the chat a Telegram message arrived in: the
|
||||
// chat ID, and the topic thread within it for supergroups.
|
||||
|
||||
@@ -213,6 +213,7 @@ type Resource struct {
|
||||
Version int64 `protobuf:"varint,2,opt,name=version,proto3" json:"version,omitempty"`
|
||||
ValueJson []byte `protobuf:"bytes,3,opt,name=value_json,json=valueJson,proto3" json:"value_json,omitempty"`
|
||||
UpdatedAt *timestamppb.Timestamp `protobuf:"bytes,4,opt,name=updated_at,json=updatedAt,proto3" json:"updated_at,omitempty"`
|
||||
OrgId string `protobuf:"bytes,5,opt,name=org_id,json=orgId,proto3" json:"org_id,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
@@ -275,9 +276,19 @@ func (x *Resource) GetUpdatedAt() *timestamppb.Timestamp {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *Resource) GetOrgId() string {
|
||||
if x != nil {
|
||||
return x.OrgId
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// Every request that names a resource acts in one org's copy of it. An empty
|
||||
// org_id is the default org.
|
||||
type QueryRequest struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Kind string `protobuf:"bytes,1,opt,name=kind,proto3" json:"kind,omitempty"`
|
||||
OrgId string `protobuf:"bytes,2,opt,name=org_id,json=orgId,proto3" json:"org_id,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
@@ -319,6 +330,13 @@ func (x *QueryRequest) GetKind() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *QueryRequest) GetOrgId() string {
|
||||
if x != nil {
|
||||
return x.OrgId
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
type QueryResponse struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Resource *Resource `protobuf:"bytes,1,opt,name=resource,proto3" json:"resource,omitempty"`
|
||||
@@ -454,7 +472,8 @@ type HistoryRequest struct {
|
||||
Kind string `protobuf:"bytes,1,opt,name=kind,proto3" json:"kind,omitempty"`
|
||||
// Limit caps the revisions returned, newest first. Zero means the server's
|
||||
// own default.
|
||||
Limit int32 `protobuf:"varint,2,opt,name=limit,proto3" json:"limit,omitempty"`
|
||||
Limit int32 `protobuf:"varint,2,opt,name=limit,proto3" json:"limit,omitempty"`
|
||||
OrgId string `protobuf:"bytes,3,opt,name=org_id,json=orgId,proto3" json:"org_id,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
@@ -503,6 +522,13 @@ func (x *HistoryRequest) GetLimit() int32 {
|
||||
return 0
|
||||
}
|
||||
|
||||
func (x *HistoryRequest) GetOrgId() string {
|
||||
if x != nil {
|
||||
return x.OrgId
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
type HistoryResponse struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Revisions []*Revision `protobuf:"bytes,1,rep,name=revisions,proto3" json:"revisions,omitempty"`
|
||||
@@ -790,6 +816,7 @@ type CommandRequest struct {
|
||||
// from browser/request JSON.
|
||||
Actor string `protobuf:"bytes,6,opt,name=actor,proto3" json:"actor,omitempty"`
|
||||
Source string `protobuf:"bytes,7,opt,name=source,proto3" json:"source,omitempty"`
|
||||
OrgId string `protobuf:"bytes,8,opt,name=org_id,json=orgId,proto3" json:"org_id,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
@@ -873,6 +900,13 @@ func (x *CommandRequest) GetSource() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *CommandRequest) GetOrgId() string {
|
||||
if x != nil {
|
||||
return x.OrgId
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
type CommandResponse struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Resource *Resource `protobuf:"bytes,1,opt,name=resource,proto3" json:"resource,omitempty"`
|
||||
@@ -921,6 +955,7 @@ type WatchRequest struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Kind string `protobuf:"bytes,1,opt,name=kind,proto3" json:"kind,omitempty"`
|
||||
AfterVersion int64 `protobuf:"varint,2,opt,name=after_version,json=afterVersion,proto3" json:"after_version,omitempty"`
|
||||
OrgId string `protobuf:"bytes,3,opt,name=org_id,json=orgId,proto3" json:"org_id,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
@@ -969,6 +1004,13 @@ func (x *WatchRequest) GetAfterVersion() int64 {
|
||||
return 0
|
||||
}
|
||||
|
||||
func (x *WatchRequest) GetOrgId() string {
|
||||
if x != nil {
|
||||
return x.OrgId
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
type WatchResponse struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Resource *Resource `protobuf:"bytes,1,opt,name=resource,proto3" json:"resource,omitempty"`
|
||||
@@ -1039,16 +1081,18 @@ const file_controlplane_v1_controlplane_proto_rawDesc = "" +
|
||||
"\rdefaults_json\x18\b \x01(\tR\fdefaultsJson\"\x10\n" +
|
||||
"\x0eCatalogRequest\"T\n" +
|
||||
"\x0fCatalogResponse\x12A\n" +
|
||||
"\tresources\x18\x01 \x03(\v2#.controlplane.v1.ResourceDescriptorR\tresources\"\x92\x01\n" +
|
||||
"\tresources\x18\x01 \x03(\v2#.controlplane.v1.ResourceDescriptorR\tresources\"\xa9\x01\n" +
|
||||
"\bResource\x12\x12\n" +
|
||||
"\x04kind\x18\x01 \x01(\tR\x04kind\x12\x18\n" +
|
||||
"\aversion\x18\x02 \x01(\x03R\aversion\x12\x1d\n" +
|
||||
"\n" +
|
||||
"value_json\x18\x03 \x01(\fR\tvalueJson\x129\n" +
|
||||
"\n" +
|
||||
"updated_at\x18\x04 \x01(\v2\x1a.google.protobuf.TimestampR\tupdatedAt\"\"\n" +
|
||||
"updated_at\x18\x04 \x01(\v2\x1a.google.protobuf.TimestampR\tupdatedAt\x12\x15\n" +
|
||||
"\x06org_id\x18\x05 \x01(\tR\x05orgId\"9\n" +
|
||||
"\fQueryRequest\x12\x12\n" +
|
||||
"\x04kind\x18\x01 \x01(\tR\x04kind\"F\n" +
|
||||
"\x04kind\x18\x01 \x01(\tR\x04kind\x12\x15\n" +
|
||||
"\x06org_id\x18\x02 \x01(\tR\x05orgId\"F\n" +
|
||||
"\rQueryResponse\x125\n" +
|
||||
"\bresource\x18\x01 \x01(\v2\x19.controlplane.v1.ResourceR\bresource\"\xbc\x01\n" +
|
||||
"\bRevision\x12\x18\n" +
|
||||
@@ -1059,10 +1103,11 @@ const file_controlplane_v1_controlplane_proto_rawDesc = "" +
|
||||
"\x06source\x18\x04 \x01(\tR\x06source\x12\x1d\n" +
|
||||
"\n" +
|
||||
"request_id\x18\x05 \x01(\tR\trequestId\x12*\n" +
|
||||
"\x02at\x18\x06 \x01(\v2\x1a.google.protobuf.TimestampR\x02at\":\n" +
|
||||
"\x02at\x18\x06 \x01(\v2\x1a.google.protobuf.TimestampR\x02at\"Q\n" +
|
||||
"\x0eHistoryRequest\x12\x12\n" +
|
||||
"\x04kind\x18\x01 \x01(\tR\x04kind\x12\x14\n" +
|
||||
"\x05limit\x18\x02 \x01(\x05R\x05limit\"J\n" +
|
||||
"\x05limit\x18\x02 \x01(\x05R\x05limit\x12\x15\n" +
|
||||
"\x06org_id\x18\x03 \x01(\tR\x05orgId\"J\n" +
|
||||
"\x0fHistoryResponse\x127\n" +
|
||||
"\trevisions\x18\x01 \x03(\v2\x19.controlplane.v1.RevisionR\trevisions\"\xc6\x02\n" +
|
||||
"\n" +
|
||||
@@ -1087,7 +1132,7 @@ const file_controlplane_v1_controlplane_proto_rawDesc = "" +
|
||||
"recordKeys\x12\x14\n" +
|
||||
"\x05limit\x18\x03 \x01(\x05R\x05limit\"F\n" +
|
||||
"\rAuditResponse\x125\n" +
|
||||
"\aentries\x18\x01 \x03(\v2\x1b.controlplane.v1.AuditEntryR\aentries\"\xd5\x01\n" +
|
||||
"\aentries\x18\x01 \x03(\v2\x1b.controlplane.v1.AuditEntryR\aentries\"\xec\x01\n" +
|
||||
"\x0eCommandRequest\x12\x12\n" +
|
||||
"\x04kind\x18\x01 \x01(\tR\x04kind\x12\x18\n" +
|
||||
"\acommand\x18\x02 \x01(\tR\acommand\x12\x1d\n" +
|
||||
@@ -1097,12 +1142,14 @@ const file_controlplane_v1_controlplane_proto_rawDesc = "" +
|
||||
"\n" +
|
||||
"request_id\x18\x05 \x01(\tR\trequestId\x12\x14\n" +
|
||||
"\x05actor\x18\x06 \x01(\tR\x05actor\x12\x16\n" +
|
||||
"\x06source\x18\a \x01(\tR\x06source\"H\n" +
|
||||
"\x06source\x18\a \x01(\tR\x06source\x12\x15\n" +
|
||||
"\x06org_id\x18\b \x01(\tR\x05orgId\"H\n" +
|
||||
"\x0fCommandResponse\x125\n" +
|
||||
"\bresource\x18\x01 \x01(\v2\x19.controlplane.v1.ResourceR\bresource\"G\n" +
|
||||
"\bresource\x18\x01 \x01(\v2\x19.controlplane.v1.ResourceR\bresource\"^\n" +
|
||||
"\fWatchRequest\x12\x12\n" +
|
||||
"\x04kind\x18\x01 \x01(\tR\x04kind\x12#\n" +
|
||||
"\rafter_version\x18\x02 \x01(\x03R\fafterVersion\"\\\n" +
|
||||
"\rafter_version\x18\x02 \x01(\x03R\fafterVersion\x12\x15\n" +
|
||||
"\x06org_id\x18\x03 \x01(\tR\x05orgId\"\\\n" +
|
||||
"\rWatchResponse\x125\n" +
|
||||
"\bresource\x18\x01 \x01(\v2\x19.controlplane.v1.ResourceR\bresource\x12\x14\n" +
|
||||
"\x05error\x18\x02 \x01(\tR\x05error2\xd9\x03\n" +
|
||||
|
||||
@@ -161,7 +161,12 @@ type Message struct {
|
||||
// channel, so its own name is "web" even for a Telegram turn, which made
|
||||
// Source.Platform -- the first component of the session natural key -- a
|
||||
// constant. Transport-only, never persisted: the session record keeps it.
|
||||
Platform string `protobuf:"bytes,10,opt,name=platform,proto3" json:"platform,omitempty"`
|
||||
Platform string `protobuf:"bytes,10,opt,name=platform,proto3" json:"platform,omitempty"`
|
||||
// media carries attachment metadata for a message that arrived with a
|
||||
// file. The attachment's bytes never cross this wire: a stored Media
|
||||
// with tool_name empty is a sender attachment, and the tool_name on
|
||||
// tool-produced events stays a stream-only field.
|
||||
Media []*Media `protobuf:"bytes,11,rep,name=media,proto3" json:"media,omitempty"`
|
||||
unknownFields protoimpl.UnknownFields
|
||||
sizeCache protoimpl.SizeCache
|
||||
}
|
||||
@@ -266,6 +271,13 @@ func (x *Message) GetPlatform() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Message) GetMedia() []*Media {
|
||||
if x != nil {
|
||||
return x.Media
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type ToolCall struct {
|
||||
state protoimpl.MessageState `protogen:"open.v1"`
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
|
||||
@@ -2818,7 +2830,7 @@ const file_gateway_v1_chat_proto_rawDesc = "" +
|
||||
"\n" +
|
||||
"created_at\x18\t \x01(\v2\x1a.google.protobuf.TimestampR\tcreatedAt\x12@\n" +
|
||||
"\x0elast_active_at\x18\n" +
|
||||
" \x01(\v2\x1a.google.protobuf.TimestampR\flastActiveAt\"\xa4\x02\n" +
|
||||
" \x01(\v2\x1a.google.protobuf.TimestampR\flastActiveAt\"\xcd\x02\n" +
|
||||
"\aMessage\x12\x1d\n" +
|
||||
"\n" +
|
||||
"message_id\x18\x01 \x01(\tR\tmessageId\x12\x1b\n" +
|
||||
@@ -2833,7 +2845,8 @@ const file_gateway_v1_chat_proto_rawDesc = "" +
|
||||
"\n" +
|
||||
"budget_key\x18\t \x01(\tR\tbudgetKey\x12\x1a\n" +
|
||||
"\bplatform\x18\n" +
|
||||
" \x01(\tR\bplatform\"|\n" +
|
||||
" \x01(\tR\bplatform\x12'\n" +
|
||||
"\x05media\x18\v \x03(\v2\x11.gateway.v1.MediaR\x05media\"|\n" +
|
||||
"\bToolCall\x12\x0e\n" +
|
||||
"\x02id\x18\x01 \x01(\tR\x02id\x12\x12\n" +
|
||||
"\x04name\x18\x02 \x01(\tR\x04name\x12\x1e\n" +
|
||||
@@ -3127,74 +3140,75 @@ var file_gateway_v1_chat_proto_depIdxs = []int32{
|
||||
50, // 0: gateway.v1.Session.created_at:type_name -> google.protobuf.Timestamp
|
||||
50, // 1: gateway.v1.Session.last_active_at:type_name -> google.protobuf.Timestamp
|
||||
50, // 2: gateway.v1.Message.at:type_name -> google.protobuf.Timestamp
|
||||
2, // 3: gateway.v1.Turn.tool_calls:type_name -> gateway.v1.ToolCall
|
||||
50, // 4: gateway.v1.Turn.created_at:type_name -> google.protobuf.Timestamp
|
||||
50, // 5: gateway.v1.Turn.updated_at:type_name -> google.protobuf.Timestamp
|
||||
0, // 6: gateway.v1.SnapshotResponse.sessions:type_name -> gateway.v1.Session
|
||||
48, // 7: gateway.v1.SnapshotResponse.models_by_provider:type_name -> gateway.v1.SnapshotResponse.ModelsByProviderEntry
|
||||
49, // 8: gateway.v1.SnapshotResponse.active_personas:type_name -> gateway.v1.SnapshotResponse.ActivePersonasEntry
|
||||
0, // 9: gateway.v1.GetSessionResponse.session:type_name -> gateway.v1.Session
|
||||
1, // 10: gateway.v1.RecentMessagesResponse.messages:type_name -> gateway.v1.Message
|
||||
4, // 11: gateway.v1.RecentTurnsResponse.turns:type_name -> gateway.v1.Turn
|
||||
1, // 12: gateway.v1.RouteRequest.message:type_name -> gateway.v1.Message
|
||||
1, // 13: gateway.v1.StreamRequest.message:type_name -> gateway.v1.Message
|
||||
2, // 14: gateway.v1.StreamResponse.tool:type_name -> gateway.v1.ToolCall
|
||||
3, // 15: gateway.v1.StreamResponse.media:type_name -> gateway.v1.Media
|
||||
0, // 16: gateway.v1.SaveSessionRequest.session:type_name -> gateway.v1.Session
|
||||
0, // 17: gateway.v1.GetSessionsByChannelResponse.sessions:type_name -> gateway.v1.Session
|
||||
0, // 18: gateway.v1.ListSessionsResponse.sessions:type_name -> gateway.v1.Session
|
||||
1, // 19: gateway.v1.SaveMessageRequest.message:type_name -> gateway.v1.Message
|
||||
1, // 20: gateway.v1.SaveMessagesRequest.messages:type_name -> gateway.v1.Message
|
||||
1, // 21: gateway.v1.ReplaceMessagesRequest.messages:type_name -> gateway.v1.Message
|
||||
1, // 22: gateway.v1.SearchMessagesResponse.messages:type_name -> gateway.v1.Message
|
||||
5, // 23: gateway.v1.SnapshotResponse.ModelsByProviderEntry.value:type_name -> gateway.v1.StringList
|
||||
6, // 24: gateway.v1.ChatService.Snapshot:input_type -> gateway.v1.SnapshotRequest
|
||||
8, // 25: gateway.v1.ChatService.GetSession:input_type -> gateway.v1.GetSessionRequest
|
||||
10, // 26: gateway.v1.ChatService.RecentMessages:input_type -> gateway.v1.RecentMessagesRequest
|
||||
12, // 27: gateway.v1.ChatService.RecentTurns:input_type -> gateway.v1.RecentTurnsRequest
|
||||
14, // 28: gateway.v1.ChatService.Route:input_type -> gateway.v1.RouteRequest
|
||||
16, // 29: gateway.v1.ChatService.Stream:input_type -> gateway.v1.StreamRequest
|
||||
18, // 30: gateway.v1.ChatService.Cancel:input_type -> gateway.v1.CancelRequest
|
||||
20, // 31: gateway.v1.ChatService.SetPersona:input_type -> gateway.v1.SetPersonaRequest
|
||||
22, // 32: gateway.v1.ChatService.ApplyTaskAction:input_type -> gateway.v1.ApplyTaskActionRequest
|
||||
24, // 33: gateway.v1.ChatService.ApplyOperatorTaskAction:input_type -> gateway.v1.ApplyOperatorTaskActionRequest
|
||||
27, // 34: gateway.v1.ChatService.SaveSession:input_type -> gateway.v1.SaveSessionRequest
|
||||
28, // 35: gateway.v1.ChatService.GetSessionsByChannel:input_type -> gateway.v1.GetSessionsByChannelRequest
|
||||
36, // 36: gateway.v1.ChatService.DeleteSession:input_type -> gateway.v1.DeleteSessionRequest
|
||||
37, // 37: gateway.v1.ChatService.TouchSession:input_type -> gateway.v1.TouchSessionRequest
|
||||
38, // 38: gateway.v1.ChatService.ListSessions:input_type -> gateway.v1.ListSessionsRequest
|
||||
39, // 39: gateway.v1.ChatService.SaveMessage:input_type -> gateway.v1.SaveMessageRequest
|
||||
40, // 40: gateway.v1.ChatService.DeleteRecentMessages:input_type -> gateway.v1.DeleteRecentMessagesRequest
|
||||
42, // 41: gateway.v1.ChatService.MessageCount:input_type -> gateway.v1.MessageCountRequest
|
||||
44, // 42: gateway.v1.ChatService.SaveMessages:input_type -> gateway.v1.SaveMessagesRequest
|
||||
45, // 43: gateway.v1.ChatService.ReplaceMessages:input_type -> gateway.v1.ReplaceMessagesRequest
|
||||
46, // 44: gateway.v1.ChatService.SearchMessages:input_type -> gateway.v1.SearchMessagesRequest
|
||||
7, // 45: gateway.v1.ChatService.Snapshot:output_type -> gateway.v1.SnapshotResponse
|
||||
9, // 46: gateway.v1.ChatService.GetSession:output_type -> gateway.v1.GetSessionResponse
|
||||
11, // 47: gateway.v1.ChatService.RecentMessages:output_type -> gateway.v1.RecentMessagesResponse
|
||||
13, // 48: gateway.v1.ChatService.RecentTurns:output_type -> gateway.v1.RecentTurnsResponse
|
||||
15, // 49: gateway.v1.ChatService.Route:output_type -> gateway.v1.RouteResponse
|
||||
17, // 50: gateway.v1.ChatService.Stream:output_type -> gateway.v1.StreamResponse
|
||||
19, // 51: gateway.v1.ChatService.Cancel:output_type -> gateway.v1.CancelResponse
|
||||
21, // 52: gateway.v1.ChatService.SetPersona:output_type -> gateway.v1.SetPersonaResponse
|
||||
23, // 53: gateway.v1.ChatService.ApplyTaskAction:output_type -> gateway.v1.ApplyTaskActionResponse
|
||||
25, // 54: gateway.v1.ChatService.ApplyOperatorTaskAction:output_type -> gateway.v1.ApplyOperatorTaskActionResponse
|
||||
26, // 55: gateway.v1.ChatService.SaveSession:output_type -> gateway.v1.SaveSessionResponse
|
||||
29, // 56: gateway.v1.ChatService.GetSessionsByChannel:output_type -> gateway.v1.GetSessionsByChannelResponse
|
||||
31, // 57: gateway.v1.ChatService.DeleteSession:output_type -> gateway.v1.DeleteSessionResponse
|
||||
32, // 58: gateway.v1.ChatService.TouchSession:output_type -> gateway.v1.TouchSessionResponse
|
||||
30, // 59: gateway.v1.ChatService.ListSessions:output_type -> gateway.v1.ListSessionsResponse
|
||||
33, // 60: gateway.v1.ChatService.SaveMessage:output_type -> gateway.v1.SaveMessageResponse
|
||||
41, // 61: gateway.v1.ChatService.DeleteRecentMessages:output_type -> gateway.v1.DeleteRecentMessagesResponse
|
||||
43, // 62: gateway.v1.ChatService.MessageCount:output_type -> gateway.v1.MessageCountResponse
|
||||
34, // 63: gateway.v1.ChatService.SaveMessages:output_type -> gateway.v1.SaveMessagesResponse
|
||||
35, // 64: gateway.v1.ChatService.ReplaceMessages:output_type -> gateway.v1.ReplaceMessagesResponse
|
||||
47, // 65: gateway.v1.ChatService.SearchMessages:output_type -> gateway.v1.SearchMessagesResponse
|
||||
45, // [45:66] is the sub-list for method output_type
|
||||
24, // [24:45] is the sub-list for method input_type
|
||||
24, // [24:24] is the sub-list for extension type_name
|
||||
24, // [24:24] is the sub-list for extension extendee
|
||||
0, // [0:24] is the sub-list for field type_name
|
||||
3, // 3: gateway.v1.Message.media:type_name -> gateway.v1.Media
|
||||
2, // 4: gateway.v1.Turn.tool_calls:type_name -> gateway.v1.ToolCall
|
||||
50, // 5: gateway.v1.Turn.created_at:type_name -> google.protobuf.Timestamp
|
||||
50, // 6: gateway.v1.Turn.updated_at:type_name -> google.protobuf.Timestamp
|
||||
0, // 7: gateway.v1.SnapshotResponse.sessions:type_name -> gateway.v1.Session
|
||||
48, // 8: gateway.v1.SnapshotResponse.models_by_provider:type_name -> gateway.v1.SnapshotResponse.ModelsByProviderEntry
|
||||
49, // 9: gateway.v1.SnapshotResponse.active_personas:type_name -> gateway.v1.SnapshotResponse.ActivePersonasEntry
|
||||
0, // 10: gateway.v1.GetSessionResponse.session:type_name -> gateway.v1.Session
|
||||
1, // 11: gateway.v1.RecentMessagesResponse.messages:type_name -> gateway.v1.Message
|
||||
4, // 12: gateway.v1.RecentTurnsResponse.turns:type_name -> gateway.v1.Turn
|
||||
1, // 13: gateway.v1.RouteRequest.message:type_name -> gateway.v1.Message
|
||||
1, // 14: gateway.v1.StreamRequest.message:type_name -> gateway.v1.Message
|
||||
2, // 15: gateway.v1.StreamResponse.tool:type_name -> gateway.v1.ToolCall
|
||||
3, // 16: gateway.v1.StreamResponse.media:type_name -> gateway.v1.Media
|
||||
0, // 17: gateway.v1.SaveSessionRequest.session:type_name -> gateway.v1.Session
|
||||
0, // 18: gateway.v1.GetSessionsByChannelResponse.sessions:type_name -> gateway.v1.Session
|
||||
0, // 19: gateway.v1.ListSessionsResponse.sessions:type_name -> gateway.v1.Session
|
||||
1, // 20: gateway.v1.SaveMessageRequest.message:type_name -> gateway.v1.Message
|
||||
1, // 21: gateway.v1.SaveMessagesRequest.messages:type_name -> gateway.v1.Message
|
||||
1, // 22: gateway.v1.ReplaceMessagesRequest.messages:type_name -> gateway.v1.Message
|
||||
1, // 23: gateway.v1.SearchMessagesResponse.messages:type_name -> gateway.v1.Message
|
||||
5, // 24: gateway.v1.SnapshotResponse.ModelsByProviderEntry.value:type_name -> gateway.v1.StringList
|
||||
6, // 25: gateway.v1.ChatService.Snapshot:input_type -> gateway.v1.SnapshotRequest
|
||||
8, // 26: gateway.v1.ChatService.GetSession:input_type -> gateway.v1.GetSessionRequest
|
||||
10, // 27: gateway.v1.ChatService.RecentMessages:input_type -> gateway.v1.RecentMessagesRequest
|
||||
12, // 28: gateway.v1.ChatService.RecentTurns:input_type -> gateway.v1.RecentTurnsRequest
|
||||
14, // 29: gateway.v1.ChatService.Route:input_type -> gateway.v1.RouteRequest
|
||||
16, // 30: gateway.v1.ChatService.Stream:input_type -> gateway.v1.StreamRequest
|
||||
18, // 31: gateway.v1.ChatService.Cancel:input_type -> gateway.v1.CancelRequest
|
||||
20, // 32: gateway.v1.ChatService.SetPersona:input_type -> gateway.v1.SetPersonaRequest
|
||||
22, // 33: gateway.v1.ChatService.ApplyTaskAction:input_type -> gateway.v1.ApplyTaskActionRequest
|
||||
24, // 34: gateway.v1.ChatService.ApplyOperatorTaskAction:input_type -> gateway.v1.ApplyOperatorTaskActionRequest
|
||||
27, // 35: gateway.v1.ChatService.SaveSession:input_type -> gateway.v1.SaveSessionRequest
|
||||
28, // 36: gateway.v1.ChatService.GetSessionsByChannel:input_type -> gateway.v1.GetSessionsByChannelRequest
|
||||
36, // 37: gateway.v1.ChatService.DeleteSession:input_type -> gateway.v1.DeleteSessionRequest
|
||||
37, // 38: gateway.v1.ChatService.TouchSession:input_type -> gateway.v1.TouchSessionRequest
|
||||
38, // 39: gateway.v1.ChatService.ListSessions:input_type -> gateway.v1.ListSessionsRequest
|
||||
39, // 40: gateway.v1.ChatService.SaveMessage:input_type -> gateway.v1.SaveMessageRequest
|
||||
40, // 41: gateway.v1.ChatService.DeleteRecentMessages:input_type -> gateway.v1.DeleteRecentMessagesRequest
|
||||
42, // 42: gateway.v1.ChatService.MessageCount:input_type -> gateway.v1.MessageCountRequest
|
||||
44, // 43: gateway.v1.ChatService.SaveMessages:input_type -> gateway.v1.SaveMessagesRequest
|
||||
45, // 44: gateway.v1.ChatService.ReplaceMessages:input_type -> gateway.v1.ReplaceMessagesRequest
|
||||
46, // 45: gateway.v1.ChatService.SearchMessages:input_type -> gateway.v1.SearchMessagesRequest
|
||||
7, // 46: gateway.v1.ChatService.Snapshot:output_type -> gateway.v1.SnapshotResponse
|
||||
9, // 47: gateway.v1.ChatService.GetSession:output_type -> gateway.v1.GetSessionResponse
|
||||
11, // 48: gateway.v1.ChatService.RecentMessages:output_type -> gateway.v1.RecentMessagesResponse
|
||||
13, // 49: gateway.v1.ChatService.RecentTurns:output_type -> gateway.v1.RecentTurnsResponse
|
||||
15, // 50: gateway.v1.ChatService.Route:output_type -> gateway.v1.RouteResponse
|
||||
17, // 51: gateway.v1.ChatService.Stream:output_type -> gateway.v1.StreamResponse
|
||||
19, // 52: gateway.v1.ChatService.Cancel:output_type -> gateway.v1.CancelResponse
|
||||
21, // 53: gateway.v1.ChatService.SetPersona:output_type -> gateway.v1.SetPersonaResponse
|
||||
23, // 54: gateway.v1.ChatService.ApplyTaskAction:output_type -> gateway.v1.ApplyTaskActionResponse
|
||||
25, // 55: gateway.v1.ChatService.ApplyOperatorTaskAction:output_type -> gateway.v1.ApplyOperatorTaskActionResponse
|
||||
26, // 56: gateway.v1.ChatService.SaveSession:output_type -> gateway.v1.SaveSessionResponse
|
||||
29, // 57: gateway.v1.ChatService.GetSessionsByChannel:output_type -> gateway.v1.GetSessionsByChannelResponse
|
||||
31, // 58: gateway.v1.ChatService.DeleteSession:output_type -> gateway.v1.DeleteSessionResponse
|
||||
32, // 59: gateway.v1.ChatService.TouchSession:output_type -> gateway.v1.TouchSessionResponse
|
||||
30, // 60: gateway.v1.ChatService.ListSessions:output_type -> gateway.v1.ListSessionsResponse
|
||||
33, // 61: gateway.v1.ChatService.SaveMessage:output_type -> gateway.v1.SaveMessageResponse
|
||||
41, // 62: gateway.v1.ChatService.DeleteRecentMessages:output_type -> gateway.v1.DeleteRecentMessagesResponse
|
||||
43, // 63: gateway.v1.ChatService.MessageCount:output_type -> gateway.v1.MessageCountResponse
|
||||
34, // 64: gateway.v1.ChatService.SaveMessages:output_type -> gateway.v1.SaveMessagesResponse
|
||||
35, // 65: gateway.v1.ChatService.ReplaceMessages:output_type -> gateway.v1.ReplaceMessagesResponse
|
||||
47, // 66: gateway.v1.ChatService.SearchMessages:output_type -> gateway.v1.SearchMessagesResponse
|
||||
46, // [46:67] is the sub-list for method output_type
|
||||
25, // [25:46] is the sub-list for method input_type
|
||||
25, // [25:25] is the sub-list for extension type_name
|
||||
25, // [25:25] is the sub-list for extension extendee
|
||||
0, // [0:25] is the sub-list for field type_name
|
||||
}
|
||||
|
||||
func init() { file_gateway_v1_chat_proto_init() }
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -22,6 +22,7 @@ const (
|
||||
StateStoreService_InstallPackage_FullMethodName = "/state.v1.StateStoreService/InstallPackage"
|
||||
StateStoreService_GetInstalledPackage_FullMethodName = "/state.v1.StateStoreService/GetInstalledPackage"
|
||||
StateStoreService_ListInstalledPackages_FullMethodName = "/state.v1.StateStoreService/ListInstalledPackages"
|
||||
StateStoreService_AcceptPackageAuthority_FullMethodName = "/state.v1.StateStoreService/AcceptPackageAuthority"
|
||||
StateStoreService_RemoveInstalledPackage_FullMethodName = "/state.v1.StateStoreService/RemoveInstalledPackage"
|
||||
StateStoreService_RegisterTaskGrant_FullMethodName = "/state.v1.StateStoreService/RegisterTaskGrant"
|
||||
StateStoreService_RevokeTaskGrant_FullMethodName = "/state.v1.StateStoreService/RevokeTaskGrant"
|
||||
@@ -131,6 +132,10 @@ type StateStoreServiceClient interface {
|
||||
InstallPackage(ctx context.Context, in *InstallPackageRequest, opts ...grpc.CallOption) (*InstallPackageResponse, error)
|
||||
GetInstalledPackage(ctx context.Context, in *GetInstalledPackageRequest, opts ...grpc.CallOption) (*GetInstalledPackageResponse, error)
|
||||
ListInstalledPackages(ctx context.Context, in *ListInstalledPackagesRequest, opts ...grpc.CallOption) (*ListInstalledPackagesResponse, error)
|
||||
// AcceptPackageAuthority records the operator's accepted authority against
|
||||
// the pinned digest (the approve action; install is the create action).
|
||||
// Administrative until org-scoped principals land.
|
||||
AcceptPackageAuthority(ctx context.Context, in *AcceptPackageAuthorityRequest, opts ...grpc.CallOption) (*AcceptPackageAuthorityResponse, error)
|
||||
RemoveInstalledPackage(ctx context.Context, in *RemoveInstalledPackageRequest, opts ...grpc.CallOption) (*RemoveInstalledPackageResponse, error)
|
||||
// Administrative task credential lifecycle. Worker credentials cannot call these.
|
||||
RegisterTaskGrant(ctx context.Context, in *RegisterTaskGrantRequest, opts ...grpc.CallOption) (*RegisterTaskGrantResponse, error)
|
||||
@@ -335,6 +340,16 @@ func (c *stateStoreServiceClient) ListInstalledPackages(ctx context.Context, in
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) AcceptPackageAuthority(ctx context.Context, in *AcceptPackageAuthorityRequest, opts ...grpc.CallOption) (*AcceptPackageAuthorityResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(AcceptPackageAuthorityResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_AcceptPackageAuthority_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) RemoveInstalledPackage(ctx context.Context, in *RemoveInstalledPackageRequest, opts ...grpc.CallOption) (*RemoveInstalledPackageResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(RemoveInstalledPackageResponse)
|
||||
@@ -1288,6 +1303,10 @@ type StateStoreServiceServer interface {
|
||||
InstallPackage(context.Context, *InstallPackageRequest) (*InstallPackageResponse, error)
|
||||
GetInstalledPackage(context.Context, *GetInstalledPackageRequest) (*GetInstalledPackageResponse, error)
|
||||
ListInstalledPackages(context.Context, *ListInstalledPackagesRequest) (*ListInstalledPackagesResponse, error)
|
||||
// AcceptPackageAuthority records the operator's accepted authority against
|
||||
// the pinned digest (the approve action; install is the create action).
|
||||
// Administrative until org-scoped principals land.
|
||||
AcceptPackageAuthority(context.Context, *AcceptPackageAuthorityRequest) (*AcceptPackageAuthorityResponse, error)
|
||||
RemoveInstalledPackage(context.Context, *RemoveInstalledPackageRequest) (*RemoveInstalledPackageResponse, error)
|
||||
// Administrative task credential lifecycle. Worker credentials cannot call these.
|
||||
RegisterTaskGrant(context.Context, *RegisterTaskGrantRequest) (*RegisterTaskGrantResponse, error)
|
||||
@@ -1471,6 +1490,9 @@ func (UnimplementedStateStoreServiceServer) GetInstalledPackage(context.Context,
|
||||
func (UnimplementedStateStoreServiceServer) ListInstalledPackages(context.Context, *ListInstalledPackagesRequest) (*ListInstalledPackagesResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method ListInstalledPackages not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) AcceptPackageAuthority(context.Context, *AcceptPackageAuthorityRequest) (*AcceptPackageAuthorityResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method AcceptPackageAuthority not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) RemoveInstalledPackage(context.Context, *RemoveInstalledPackageRequest) (*RemoveInstalledPackageResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method RemoveInstalledPackage not implemented")
|
||||
}
|
||||
@@ -1819,6 +1841,24 @@ func _StateStoreService_ListInstalledPackages_Handler(srv interface{}, ctx conte
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_AcceptPackageAuthority_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(AcceptPackageAuthorityRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).AcceptPackageAuthority(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_AcceptPackageAuthority_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).AcceptPackageAuthority(ctx, req.(*AcceptPackageAuthorityRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_RemoveInstalledPackage_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(RemoveInstalledPackageRequest)
|
||||
if err := dec(in); err != nil {
|
||||
@@ -3455,6 +3495,10 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
|
||||
MethodName: "ListInstalledPackages",
|
||||
Handler: _StateStoreService_ListInstalledPackages_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "AcceptPackageAuthority",
|
||||
Handler: _StateStoreService_AcceptPackageAuthority_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "RemoveInstalledPackage",
|
||||
Handler: _StateStoreService_RemoveInstalledPackage_Handler,
|
||||
|
||||
@@ -29,6 +29,15 @@ func (d *Daemon) activeWorkflows(ctx context.Context) (workflow.WorkflowDefiniti
|
||||
return collection, err
|
||||
}
|
||||
|
||||
// workflowEnablement returns which workflows each org has disabled; with no
|
||||
// source wired every workflow is enabled.
|
||||
func (d *Daemon) workflowEnablement(ctx context.Context) (workflowtask.WorkflowEnablement, error) {
|
||||
if d.WorkflowEnablement == nil {
|
||||
return workflowtask.WorkflowEnablement{}, nil
|
||||
}
|
||||
return d.WorkflowEnablement.WorkflowEnablement(ctx)
|
||||
}
|
||||
|
||||
// resolveBindingTarget checks a matched binding against the workflow it
|
||||
// targets and resolves the task's repository and inputs. A non-empty reason
|
||||
// means the binding does not dispatch and the reason is recorded on it; ok
|
||||
|
||||
@@ -250,6 +250,12 @@ type Daemon struct {
|
||||
WorkflowDefinitions interface {
|
||||
WorkflowDefinitions(context.Context) (workflow.WorkflowDefinitionCollection, int64, error)
|
||||
}
|
||||
// WorkflowEnablement supplies which workflows each org has disabled. A
|
||||
// binding targeting a workflow its org disabled does not dispatch. Nil
|
||||
// means every workflow is enabled.
|
||||
WorkflowEnablement interface {
|
||||
WorkflowEnablement(context.Context) (workflowtask.WorkflowEnablement, error)
|
||||
}
|
||||
|
||||
// ToolRegistry is the central tool registry, wired by the composition
|
||||
// root. MCP-discovered tools and built-in tools are registered here
|
||||
@@ -667,16 +673,27 @@ func (d *Daemon) dispatchBindings(ctx context.Context) {
|
||||
d.Log.Warn("binding dispatch: workflow definitions unavailable", "error", err)
|
||||
return
|
||||
}
|
||||
enablement, err := d.workflowEnablement(ctx)
|
||||
if err != nil {
|
||||
d.Log.Warn("binding dispatch: workflow enablement unavailable", "error", err)
|
||||
return
|
||||
}
|
||||
for _, c := range captures {
|
||||
armed, err := d.BindingDispatcher.ArmedBindingsForSource(ctx, c.Source)
|
||||
if err != nil {
|
||||
d.Log.Warn("armed bindings lookup", "source", c.Source, "error", err)
|
||||
continue
|
||||
}
|
||||
for _, b := range armed {
|
||||
if !b.Matcher.Matches(c.Source, c.Dispatchable()) {
|
||||
continue
|
||||
}
|
||||
d.dispatchCapture(ctx, c, workflows, enablement)
|
||||
}
|
||||
}
|
||||
|
||||
// dispatchCapture offers one capture to every armed binding on its source. A
|
||||
// disabled workflow's binding is skipped like an unarmed one, so the capture
|
||||
// waits rather than recording a failure every cycle.
|
||||
func (d *Daemon) dispatchCapture(ctx context.Context, c storecontract.CapturedEvent, workflows workflow.WorkflowDefinitionCollection, enablement workflowtask.WorkflowEnablement) {
|
||||
armed, err := d.BindingDispatcher.ArmedBindingsForSource(ctx, c.Source)
|
||||
if err != nil {
|
||||
d.Log.Warn("armed bindings lookup", "source", c.Source, "error", err)
|
||||
return
|
||||
}
|
||||
for _, b := range armed {
|
||||
if b.Matcher.Matches(c.Source, c.Dispatchable()) && enablement.Enabled(b.OrgID, b.Workflow) {
|
||||
d.dispatchOneBinding(ctx, b, c, workflows)
|
||||
}
|
||||
}
|
||||
@@ -1402,6 +1419,16 @@ func (d *Daemon) prepareWorkspace(ctx context.Context, task *workflow.Task, tree
|
||||
}
|
||||
return dir, true
|
||||
}
|
||||
// A task that continues an already-open PR branch -- the remediate
|
||||
// workflow -- resumes that branch's remote tip instead of resetting onto
|
||||
// base. Preparing one the ordinary way discarded the commits the implement
|
||||
// run had already pushed, so the agent remediated a PR from a tree that no
|
||||
// longer contained it. This runs before Acquire, because the daemon holds
|
||||
// the forge credential that fetches the branch and the container must find
|
||||
// the PR's work already in place.
|
||||
if continuesOpenPRBranch(task) {
|
||||
return d.resumePRBranchWorkspace(ctx, task, trees)
|
||||
}
|
||||
// Every task gets an independent full clone. The former
|
||||
// PreparePersistent path shared objects with a per-repo bare cache;
|
||||
// go-git has no --dissociate, so a shared cache would stay a live
|
||||
@@ -1420,6 +1447,41 @@ func (d *Daemon) prepareWorkspace(ctx context.Context, task *workflow.Task, tree
|
||||
return dir, true
|
||||
}
|
||||
|
||||
// continuesOpenPRBranch reports whether the task must continue a branch archie
|
||||
// already pushed rather than start from base. Only the remediate workflow does:
|
||||
// it addresses review feedback on an open archie-owned PR and reuses that PR's
|
||||
// branch (docs/prds/pr-review-remediation.md decision 4), while every other
|
||||
// workflow starts fresh work. The workflow is read from the task row, where the
|
||||
// reaction consumer wrote it before queueing the run -- a decision about this
|
||||
// task that no later routing may overturn.
|
||||
func continuesOpenPRBranch(task *workflow.Task) bool {
|
||||
return task.Workflow == "remediate"
|
||||
}
|
||||
|
||||
// resumePRBranchWorkspace prepares a task's worktree on the branch its row
|
||||
// names, never on a branch recomputed from the title or labels: the PR was
|
||||
// opened from the persisted branch, and a retitled issue or relabelled task
|
||||
// would otherwise name a branch that does not exist. It fails closed -- there
|
||||
// is deliberately no fallback to preparing base, because that fallback is the
|
||||
// defect this route removes.
|
||||
func (d *Daemon) resumePRBranchWorkspace(ctx context.Context, task *workflow.Task, trees *worktree.Manager) (string, bool) {
|
||||
if task.Branch == "" {
|
||||
// A remediation task is queued against the branch its implement run
|
||||
// pushed, so an empty branch is a dispatch bug upstream and no retry
|
||||
// can fix it.
|
||||
d.Log.Error("remediate task has no persisted branch", "task", task.ID)
|
||||
d.parkRunningTask(ctx, task.ID, "worktree resume failed: remediate task has no branch to resume", taskstate.ParkNeedsHuman)
|
||||
return "", false
|
||||
}
|
||||
dir := trees.Dir(task.Owner, task.Repo, task.IssueNumber)
|
||||
if err := trees.Resume(ctx, dir, task.Branch); err != nil {
|
||||
d.Log.Error("worktree resume failed", "task", task.ID, "branch", task.Branch, "err", err)
|
||||
d.parkRunningTask(ctx, task.ID, "worktree resume failed: "+err.Error(), taskstate.ParkTransient)
|
||||
return "", false
|
||||
}
|
||||
return dir, true
|
||||
}
|
||||
|
||||
// cleanupTerminalTaskWorktree removes a task's worktree once its forge side is
|
||||
// settled, unless the worktree still holds work no commit captured.
|
||||
//
|
||||
|
||||
@@ -15,6 +15,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/domain/mapping"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
workflowtask "github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
@@ -422,3 +423,36 @@ func assertDispatchRecorded(t *testing.T, s *dispatchStores, bindingID string) {
|
||||
t.Fatalf("binding_dispatches row missing: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
type staticEnablement struct {
|
||||
value workflowtask.WorkflowEnablement
|
||||
}
|
||||
|
||||
func (e staticEnablement) WorkflowEnablement(context.Context) (workflowtask.WorkflowEnablement, error) {
|
||||
return e.value, nil
|
||||
}
|
||||
|
||||
// A binding whose org disabled its workflow does not dispatch, and the capture
|
||||
// waits: re-enabling the workflow dispatches it on the next cycle.
|
||||
func TestDispatchBindingsSkipsWorkflowItsOrgDisabled(t *testing.T) {
|
||||
s := openDispatchTestStore(t)
|
||||
mappingID := seedMapping(t, s, "sentry", mapping.Field{Name: "title", Path: "title", Type: mapping.TypeString, Required: true})
|
||||
bindingID, _ := seedArmedBinding(t, s, "sentry", mappingID)
|
||||
seedCapture(t, s, "sentry", true, `{"title":"hello"}`)
|
||||
stored, err := s.EdaStore.GetBinding(t.Context(), bindingID)
|
||||
if err != nil {
|
||||
t.Fatalf("GetBinding: %v", err)
|
||||
}
|
||||
|
||||
d := newDispatchDaemon(t, s)
|
||||
var enablement workflowtask.WorkflowEnablement
|
||||
d.WorkflowEnablement = staticEnablement{enablement.SetEnabled(stored.OrgID, stored.Workflow, false)}
|
||||
d.dispatchBindings(t.Context())
|
||||
if tasks, _ := s.Tasks(t.Context(), 10); len(tasks) != 0 {
|
||||
t.Fatalf("Tasks len = %d, want 0 while the workflow is disabled", len(tasks))
|
||||
}
|
||||
|
||||
d.WorkflowEnablement = staticEnablement{enablement.SetEnabled(stored.OrgID, stored.Workflow, true)}
|
||||
d.dispatchBindings(t.Context())
|
||||
assertDispatchRecorded(t, s, bindingID)
|
||||
}
|
||||
@@ -0,0 +1,185 @@
|
||||
package daemon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/go-git/go-git/v6"
|
||||
"github.com/go-git/go-git/v6/plumbing"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
"github.com/samcharles93/archie-core/internal/worktree"
|
||||
)
|
||||
|
||||
// pushedImplementRun models what a remediation task starts from: the implement
|
||||
// run prepared the clone, committed, and pushed its branch, so origin/<branch>
|
||||
// holds work base does not. title is the issue title the implement run saw,
|
||||
// which a later rename can move past the branch the PR actually lives on.
|
||||
func pushedImplementRun(t *testing.T, trees *worktree.Manager, title string) (dir, branch string, tip plumbing.Hash) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
dir, branch, err := trees.Prepare(ctx, "acme", "widget", "main", 7, title, "", "bug")
|
||||
if err != nil {
|
||||
t.Fatalf("Prepare() error = %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "widget.go"), []byte("package widget\n"), 0o600); err != nil {
|
||||
t.Fatalf("write widget.go: %v", err)
|
||||
}
|
||||
changed, err := trees.CommitAll(ctx, dir, "fix: the PR work")
|
||||
if err != nil || !changed {
|
||||
t.Fatalf("CommitAll() = (%v, %v), want (true, nil)", changed, err)
|
||||
}
|
||||
if err := trees.Push(ctx, dir, branch); err != nil {
|
||||
t.Fatalf("Push() error = %v", err)
|
||||
}
|
||||
return dir, branch, headOf(t, dir)
|
||||
}
|
||||
|
||||
// claimedTask returns the running row a dispatch leaves behind, carrying the
|
||||
// workflow and branch prepareWorkspace must route on. The row is read back from
|
||||
// the store, so the test drives prepareWorkspace from persisted state rather
|
||||
// than from a struct the test assembled.
|
||||
func claimedTask(t *testing.T, st *pgstore.TaskDB, title, workflowName, branch string) *workflow.Task {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
if _, err := st.EnqueueIssue(ctx, "acme", "widget", 7, title, "body", "bug", ""); err != nil {
|
||||
t.Fatalf("EnqueueIssue: %v", err)
|
||||
}
|
||||
task, err := st.ClaimNext(ctx)
|
||||
if err != nil || task == nil {
|
||||
t.Fatalf("ClaimNext = (%+v, %v)", task, err)
|
||||
}
|
||||
task.Workflow, task.Branch = workflowName, branch
|
||||
if err := st.Update(ctx, task); err != nil {
|
||||
t.Fatalf("persist task fields: %v", err)
|
||||
}
|
||||
|
||||
persisted, err := st.TaskByID(ctx, task.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("TaskByID: %v", err)
|
||||
}
|
||||
if persisted.Workflow != workflowName || persisted.Branch != branch {
|
||||
t.Fatalf("persisted row = (workflow %q, branch %q), want (%q, %q)",
|
||||
persisted.Workflow, persisted.Branch, workflowName, branch)
|
||||
}
|
||||
return persisted
|
||||
}
|
||||
|
||||
func headOf(t *testing.T, dir string) plumbing.Hash {
|
||||
t.Helper()
|
||||
r, err := git.PlainOpen(dir)
|
||||
if err != nil {
|
||||
t.Fatalf("PlainOpen(%s): %v", dir, err)
|
||||
}
|
||||
ref, err := r.Head()
|
||||
if err != nil {
|
||||
t.Fatalf("Head(%s): %v", dir, err)
|
||||
}
|
||||
return ref.Hash()
|
||||
}
|
||||
|
||||
// TestPrepareWorkspaceResumesThePersistedBranchForARemediation pins the defect
|
||||
// a remediation run used to hit: prepareWorkspace always called Manager.Prepare,
|
||||
// whose refresh resets the worktree onto origin/<base>, so a task continuing an
|
||||
// open PR branch lost that branch's commits before its container started and
|
||||
// the agent worked from base. It must instead resume the branch persisted on
|
||||
// the task row -- here a branch the task's current title would no longer name,
|
||||
// so a recomputed branch name cannot masquerade as the right answer.
|
||||
func TestPrepareWorkspaceResumesThePersistedBranchForARemediation(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
trees := newTestTrees(t, newLocalRemote(t, "acme", "widget"))
|
||||
|
||||
// The implement run pushed its PR work on the branch its then-title named.
|
||||
dir, branch, tip := pushedImplementRun(t, trees, "fix: original issue")
|
||||
|
||||
d, st, _ := testDaemon(t, 3, 0)
|
||||
// The issue was retitled since; the PR's branch is still the pushed one.
|
||||
task := claimedTask(t, st, "fix: retitled issue", "remediate", branch)
|
||||
|
||||
gotDir, ok := d.prepareWorkspace(ctx, task, trees, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
||||
if !ok {
|
||||
t.Fatal("prepareWorkspace() = false, want the remediation prepared on its PR branch")
|
||||
}
|
||||
if gotDir != dir {
|
||||
t.Errorf("prepareWorkspace() dir = %q, want the task's worktree %q", gotDir, dir)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(gotDir, "widget.go")); err != nil {
|
||||
t.Fatalf("widget.go missing after prepare: %v (the worktree was reset to base, discarding the PR branch's commits)", err)
|
||||
}
|
||||
if got := headOf(t, gotDir); got != tip {
|
||||
t.Errorf("worktree HEAD = %s, want the PR branch tip %s", got, tip)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPrepareWorkspaceFailsClosedWhenTheRemediationBranchIsGone pins the other
|
||||
// half of the same routing: a remediation whose branch is not on the remote
|
||||
// must park, not quietly fall back to preparing base -- the fallback is exactly
|
||||
// the state the fix removes, and a remediation run that cannot see its PR
|
||||
// branch has nothing correct to do.
|
||||
func TestPrepareWorkspaceFailsClosedWhenTheRemediationBranchIsGone(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
trees := newTestTrees(t, newLocalRemote(t, "acme", "widget"))
|
||||
|
||||
// The implement run's clone exists and holds its work, but the branch the
|
||||
// task row names was never pushed (a row pointing at the wrong branch is
|
||||
// the dispatch bug this guards).
|
||||
dir, _, tip := pushedImplementRun(t, trees, "fix: original issue")
|
||||
|
||||
d, st, _ := testDaemon(t, 3, 0)
|
||||
task := claimedTask(t, st, "fix: original issue", "remediate", "fix/7-never-pushed")
|
||||
|
||||
gotDir, ok := d.prepareWorkspace(ctx, task, trees, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
||||
if ok {
|
||||
t.Errorf("prepareWorkspace() = true (dir %q), want a fail-closed park: a missing PR branch must not be replaced by a base checkout", gotDir)
|
||||
}
|
||||
if gotDir != "" {
|
||||
t.Errorf("prepareWorkspace() dir = %q, want empty on failure", gotDir)
|
||||
}
|
||||
|
||||
persisted, err := st.TaskByID(ctx, task.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("TaskByID: %v", err)
|
||||
}
|
||||
if persisted.Status != workflow.StatusParked {
|
||||
t.Errorf("task status = %q, want parked", persisted.Status)
|
||||
}
|
||||
if !strings.Contains(persisted.ParkReason, "resume") {
|
||||
t.Errorf("park reason = %q, want it to name the failed resume", persisted.ParkReason)
|
||||
}
|
||||
if got := headOf(t, dir); got != tip {
|
||||
t.Errorf("worktree HEAD = %s, want it left on the PR branch tip %s", got, tip)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPrepareWorkspaceStartsBaseForANonRemediationTask keeps the resume route
|
||||
// from widening: a task that is not a remediation starts fresh on the branch
|
||||
// its current title names, even when its row carries a branch from an earlier
|
||||
// lifecycle (the row an operator requeue leaves behind).
|
||||
func TestPrepareWorkspaceStartsBaseForANonRemediationTask(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
trees := newTestTrees(t, newLocalRemote(t, "acme", "widget"))
|
||||
|
||||
_, pushedBranch, tip := pushedImplementRun(t, trees, "fix: original issue")
|
||||
|
||||
d, st, _ := testDaemon(t, 3, 0)
|
||||
task := claimedTask(t, st, "fix: retitled issue", "implement", pushedBranch)
|
||||
|
||||
gotDir, ok := d.prepareWorkspace(ctx, task, trees, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
||||
if !ok {
|
||||
t.Fatal("prepareWorkspace() = false, want a fresh preparation")
|
||||
}
|
||||
if got := headOf(t, gotDir); got == tip {
|
||||
t.Errorf("worktree HEAD = %s, want base rather than the persisted branch's tip: only a remediation resumes a branch", got)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(gotDir, "widget.go")); !os.IsNotExist(err) {
|
||||
t.Errorf("widget.go present after a non-remediation prepare (stat err = %v), want a fresh base checkout", err)
|
||||
}
|
||||
if task.Branch == pushedBranch {
|
||||
t.Errorf("task branch = %q, want the branch recomputed for fresh work rather than the persisted PR branch", task.Branch)
|
||||
}
|
||||
}
|
||||
@@ -8,6 +8,8 @@ import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/org"
|
||||
)
|
||||
|
||||
// Status is the binding lifecycle state. Mirrors telegram's
|
||||
@@ -65,11 +67,14 @@ type Binding struct {
|
||||
// Inputs assigns the workflow's declared inputs, each from a mapped
|
||||
// parameter or a constant; CheckWorkflow checks them when the binding is
|
||||
// saved.
|
||||
Inputs map[string]InputSource `json:"inputs,omitempty"`
|
||||
Version int `json:"version"`
|
||||
Status Status `json:"status"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Inputs map[string]InputSource `json:"inputs,omitempty"`
|
||||
// OrgID is the org that owns the binding; the store sets it and a
|
||||
// binding may target only a workflow that org has enabled.
|
||||
OrgID org.OrgID `json:"org_id"`
|
||||
Version int `json:"version"`
|
||||
Status Status `json:"status"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
// Validate checks a binding is well-formed before it is persisted: a
|
||||
|
||||
@@ -235,7 +235,8 @@ func (r *ModuleRegistry) Invoke(ctx context.Context, kind string, rawArgs map[st
|
||||
// CEL expression reads is the same Go struct the expression environment typed
|
||||
// (`actions.<id>.result.<field>`). The map is keyed by the lower-cased Go
|
||||
// field name, the same spelling expr registers as the CEL field name. An
|
||||
// unknown field or a wrong-typed value is a reported error.
|
||||
// unknown field, an absent declared field, or a value that is not already the
|
||||
// field's type is a reported error.
|
||||
func (r *ModuleRegistry) DecodeResult(kind string, raw map[string]any) (any, error) {
|
||||
k, ok := registry[kind]
|
||||
if !ok {
|
||||
@@ -246,7 +247,11 @@ func (r *ModuleRegistry) DecodeResult(kind string, raw map[string]any) (any, err
|
||||
|
||||
// decodeResultStruct marshals raw into a new value of the kind's Result
|
||||
// struct type, keyed by the lower-cased Go field name -- the same spelling
|
||||
// expr registers as the CEL field name.
|
||||
// expr registers as the CEL field name. Every declared field must be present
|
||||
// and every value must already carry that field's type: a missing field would
|
||||
// read as the zero value, and a conversion would change the value's
|
||||
// representation, so both are reported rather than becoming a quietly wrong
|
||||
// value in a later expression.
|
||||
func decodeResultStruct(kind string, t reflect.Type, raw map[string]any) (any, error) {
|
||||
if t == nil {
|
||||
return nil, fmt.Errorf("module %s: result schema is not set", kind)
|
||||
@@ -259,40 +264,55 @@ func decodeResultStruct(kind string, t reflect.Type, raw map[string]any) (any, e
|
||||
}
|
||||
|
||||
out := reflect.New(t).Elem()
|
||||
fields := make(map[string]reflect.Value, t.NumField())
|
||||
type declaredField struct {
|
||||
name string
|
||||
value reflect.Value
|
||||
}
|
||||
declared := make([]declaredField, 0, t.NumField())
|
||||
byName := make(map[string]reflect.Value, t.NumField())
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
f := t.Field(i)
|
||||
if !out.Field(i).CanSet() {
|
||||
continue
|
||||
}
|
||||
fields[strings.ToLower(f.Name)] = out.Field(i)
|
||||
name := strings.ToLower(t.Field(i).Name)
|
||||
declared = append(declared, declaredField{name: name, value: out.Field(i)})
|
||||
byName[name] = out.Field(i)
|
||||
}
|
||||
for key, val := range raw {
|
||||
fv, ok := fields[key]
|
||||
if !ok {
|
||||
|
||||
keys := make([]string, 0, len(raw))
|
||||
for key := range raw {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
for _, key := range keys {
|
||||
if _, ok := byName[key]; !ok {
|
||||
return nil, fmt.Errorf("module %s: unknown result field %q", kind, key)
|
||||
}
|
||||
if err := setResultField(kind, key, fv, val); err != nil {
|
||||
}
|
||||
for _, f := range declared {
|
||||
val, ok := raw[f.name]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("module %s: result field %q is missing, want %s", kind, f.name, f.value.Type())
|
||||
}
|
||||
if err := setResultField(kind, f.name, f.value, val); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return out.Interface(), nil
|
||||
}
|
||||
|
||||
// setResultField assigns val to fv, converting compatible Go types. A nil or
|
||||
// incompatible value is a reported error, never a silent zero-value fill.
|
||||
// setResultField assigns val to fv only when val's type is the field's type. A
|
||||
// conversion would change the value's representation -- a number silently
|
||||
// becoming a string is the named case -- so it is refused, as are a nil and a
|
||||
// wrong-typed value. Nothing here zero-fills.
|
||||
func setResultField(kind, field string, fv reflect.Value, val any) error {
|
||||
if val == nil {
|
||||
return fmt.Errorf("module %s: result.%s is nil, want %s", kind, field, fv.Type())
|
||||
}
|
||||
rv := reflect.ValueOf(val)
|
||||
if rv.Type().AssignableTo(fv.Type()) {
|
||||
fv.Set(rv)
|
||||
return nil
|
||||
if !rv.Type().AssignableTo(fv.Type()) {
|
||||
return fmt.Errorf("module %s: result.%s is %T, want %s", kind, field, val, fv.Type())
|
||||
}
|
||||
if rv.Type().ConvertibleTo(fv.Type()) {
|
||||
fv.Set(rv.Convert(fv.Type()))
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("module %s: result.%s is %T, want %s", kind, field, val, fv.Type())
|
||||
fv.Set(rv)
|
||||
return nil
|
||||
}
|
||||
@@ -169,9 +169,71 @@ func Run(a log.Args) log.Result {
|
||||
}
|
||||
}
|
||||
|
||||
// TestDecodeResultRejectsSchemaViolations pins the strict schema boundary:
|
||||
// every declared Result field must be present, and a value is accepted only
|
||||
// when its type is the field's type. An absent field would read as the zero
|
||||
// value, and a conversion changes the value's representation -- a number
|
||||
// silently becoming a string is the named case -- so both are failures rather
|
||||
// than a quietly wrong value in a later expression.
|
||||
func TestDecodeResultRejectsSchemaViolations(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
raw map[string]any
|
||||
wants []string
|
||||
}{
|
||||
{
|
||||
name: "absent field",
|
||||
raw: map[string]any{"written": true},
|
||||
wants: []string{"log", `result field "level"`, "missing"},
|
||||
},
|
||||
{
|
||||
name: "number where a string field is declared",
|
||||
raw: map[string]any{"written": true, "level": 42},
|
||||
wants: []string{"log", "result.level", "int", "want string"},
|
||||
},
|
||||
{
|
||||
name: "string where a bool field is declared",
|
||||
raw: map[string]any{"written": "true", "level": "info"},
|
||||
wants: []string{"log", "result.written", "want bool"},
|
||||
},
|
||||
}
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
r := New()
|
||||
got, err := r.DecodeResult("log", tc.raw)
|
||||
if err == nil {
|
||||
t.Fatalf("DecodeResult(%v) = %#v, want an error", tc.raw, got)
|
||||
}
|
||||
for _, want := range tc.wants {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("DecodeResult error = %q, want it to contain %q", err.Error(), want)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestDecodeResultAcceptsEveryFieldAtItsType is the other half of the strict
|
||||
// boundary: a complete map whose values already carry the schema's types
|
||||
// decodes unchanged.
|
||||
func TestDecodeResultAcceptsEveryFieldAtItsType(t *testing.T) {
|
||||
r := New()
|
||||
got, err := r.DecodeResult("log", map[string]any{"written": true, "level": "info"})
|
||||
if err != nil {
|
||||
t.Fatalf("DecodeResult: %v", err)
|
||||
}
|
||||
result, ok := got.(log.Result)
|
||||
if !ok {
|
||||
t.Fatalf("DecodeResult returned %T, want log.Result", got)
|
||||
}
|
||||
if want := (log.Result{Written: true, Level: "info"}); result != want {
|
||||
t.Fatalf("DecodeResult = %#v, want %#v", result, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeResultUnknownFieldIsError(t *testing.T) {
|
||||
r := New()
|
||||
_, err := r.DecodeResult("log", map[string]any{"written": true, "bogus": "x"})
|
||||
_, err := r.DecodeResult("log", map[string]any{"written": true, "level": "info", "bogus": "x"})
|
||||
if err == nil {
|
||||
t.Fatal("DecodeResult(unknown field) = nil, want error")
|
||||
}
|
||||
|
||||
@@ -735,8 +735,8 @@ func evalContext(input DispatchInput) expr.Context {
|
||||
//
|
||||
// Asymmetry with `when`: `when` is a predicate, so an evaluation error is
|
||||
// false (J3: skip + log); `args` is data, so an evaluation error has no
|
||||
// meaningful substitute and is returned to the caller to abort the dispatch.
|
||||
// Ratifying that rule against a real consumer is tracked by archie-core-1h05.
|
||||
// meaningful substitute and is returned to the caller to abort the playbook run
|
||||
// (J6, docs/prds/playbook-expression-syntax.md).
|
||||
func (s *Store) EvalArgs(a Action, input DispatchInput) (map[string]any, error) {
|
||||
if s == nil {
|
||||
return map[string]any{}, nil
|
||||
|
||||
@@ -124,6 +124,41 @@ func TestRunStopsThePlaybookOnAnInvokeError(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunAbortsThePlaybookOnAnArgsEvaluationFailure pins J6
|
||||
// (docs/prds/playbook-expression-syntax.md): an args value that cannot be
|
||||
// evaluated stops its playbook before the action runs, unlike a `when` that
|
||||
// errors, which skips the action and continues (J3). The error names the
|
||||
// playbook and the action whose args failed.
|
||||
func TestRunAbortsThePlaybookOnAnArgsEvaluationFailure(t *testing.T) {
|
||||
mods := &recordingModules{ModuleRegistry: module.New()}
|
||||
store := loadRunStore(t, mods, `
|
||||
trigger:
|
||||
kind: bug
|
||||
actions:
|
||||
- id: first
|
||||
position: module
|
||||
kind: log
|
||||
args: { message: 'event.absent' }
|
||||
- id: second
|
||||
position: module
|
||||
kind: log
|
||||
args: { message: '"second"' }
|
||||
`)
|
||||
|
||||
err := store.Run(t.Context(), &fakePlaybookLedger{}, slog.New(slog.DiscardHandler), bugInput())
|
||||
if err == nil {
|
||||
t.Fatal("Run(args evaluation failure) = nil, want the playbook aborted")
|
||||
}
|
||||
for _, want := range []string{"pb.yaml", "first", "evaluate args"} {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("Run error = %q, want it to contain %q", err.Error(), want)
|
||||
}
|
||||
}
|
||||
if len(mods.messages) != 0 {
|
||||
t.Fatalf("invoked messages = %q, want none: the playbook aborts before the first action runs", mods.messages)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunIgnoresNonMatchingAndWorkflowPlaybooks(t *testing.T) {
|
||||
mods := &recordingModules{ModuleRegistry: module.New()}
|
||||
store := loadRunStore(t, mods, twoLogActions)
|
||||
|
||||
@@ -21,7 +21,11 @@ type Secret struct {
|
||||
RefreshToken string
|
||||
TokenType string
|
||||
ExpiresAt time.Time
|
||||
UpdatedAt time.Time
|
||||
// Scopes are the scopes the provider granted, as the token response's
|
||||
// space-delimited `scope` field reported them (RFC 6749 section 5.1).
|
||||
// The set is kept across a refresh that does not repeat the field.
|
||||
Scopes []string
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
// Validate rejects a Secret with no identity or no access token: the setup
|
||||
|
||||
@@ -46,6 +46,15 @@ type Inbound struct {
|
||||
// record keeps the platform. Empty means the sender did not name its
|
||||
// channel, and the Gateway falls back to its own name.
|
||||
Platform string
|
||||
|
||||
// Media carries files the sender attached to this message (photos,
|
||||
// documents, voice, video) with their in-process bytes already
|
||||
// downloaded. Like Page and BudgetKey it is transport-only: it never
|
||||
// crosses a gRPC or persistence boundary, and the stored Message
|
||||
// records the attachment as a short textual note in its Text instead,
|
||||
// because download URLs expire and raw media bytes must not silently
|
||||
// inflate the transcript store. nil for a text-only message.
|
||||
Media []MediaAttachment
|
||||
}
|
||||
|
||||
// SpawnRequest is a chat-originated task creation request. Repo and
|
||||
|
||||
@@ -50,4 +50,10 @@ type MediaAttachment struct {
|
||||
// Raw carries platform-specific fields the common vocabulary does not
|
||||
// cover, so adapters never need a second attachment type.
|
||||
Raw map[string]any `json:"raw,omitempty"`
|
||||
|
||||
// Data carries the attachment bytes in-process. An inbound path sets
|
||||
// it after downloading the file; outbound senders never set it and
|
||||
// read Path or URL instead. It never crosses a wire or persistence
|
||||
// boundary -- it is a turn-scoped value, not a stored one.
|
||||
Data []byte `json:"data,omitempty"`
|
||||
}
|
||||
@@ -105,6 +105,16 @@ type Message struct {
|
||||
Role Role
|
||||
Text string
|
||||
|
||||
// Media carries attachment metadata for messages that arrived with a
|
||||
// file (photo, document, video, audio): the platform attachment id, its
|
||||
// name, type, and size, so the transcript records what was sent as
|
||||
// data rather than as prose. The attachment bytes and any embedded
|
||||
// Data field are deliberately NOT persisted -- they are turn-scoped --
|
||||
// so this is the metadata a later reader (search, transcript, a
|
||||
// re-look capability) can act on, including re-fetching through
|
||||
// FileID where the platform allows it.
|
||||
Media []MediaAttachment `json:"media,omitempty"`
|
||||
|
||||
// ToolCall and ToolResult are populated instead of Text for their
|
||||
// respective Roles; a Message carries exactly one of Text, ToolCall, or
|
||||
// ToolResult content.
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
package org
|
||||
|
||||
import "context"
|
||||
|
||||
// orgContextKey is the unexported key the acting org travels under. A named
|
||||
// struct type, not a string, so no other package can collide with it.
|
||||
type orgContextKey struct{}
|
||||
|
||||
// WithOrg attaches the org a request acts in. Every control-plane request
|
||||
// reads it back with OrgFromContext, so the org reaches the wire from one
|
||||
// place: the context the caller's principal put it on.
|
||||
func WithOrg(ctx context.Context, id OrgID) context.Context {
|
||||
return context.WithValue(ctx, orgContextKey{}, id)
|
||||
}
|
||||
|
||||
// OrgFromContext returns the org a request acts in, or DefaultOrgID when the
|
||||
// context carries none. DefaultOrgID is the single-operator install's org and
|
||||
// the org every record written before orgs existed belongs to.
|
||||
func OrgFromContext(ctx context.Context) OrgID {
|
||||
if id, ok := ctx.Value(orgContextKey{}).(OrgID); ok && id != "" {
|
||||
return id
|
||||
}
|
||||
return DefaultOrgID
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
package org
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// foreignContextKey stands in for another package's key: a value stored under
|
||||
// it must never be read back as the org.
|
||||
type foreignContextKey struct{}
|
||||
|
||||
func TestOrgFromContext(t *testing.T) {
|
||||
// build derives each case's context from a fresh background one. The table
|
||||
// holds no context.Context field on purpose: a stored context is what
|
||||
// containedctx rejects, and a builder keeps every case readable without one.
|
||||
tests := []struct {
|
||||
name string
|
||||
build func(context.Context) context.Context
|
||||
want OrgID
|
||||
}{
|
||||
{name: "unset is the default org", build: func(ctx context.Context) context.Context { return ctx }, want: DefaultOrgID},
|
||||
{name: "set org", build: func(ctx context.Context) context.Context { return WithOrg(ctx, "acme") }, want: "acme"},
|
||||
{
|
||||
name: "inner org overrides an outer one",
|
||||
build: func(ctx context.Context) context.Context { return WithOrg(WithOrg(ctx, "outer"), "inner") },
|
||||
want: "inner",
|
||||
},
|
||||
{
|
||||
name: "nil stored value is the default org",
|
||||
build: func(ctx context.Context) context.Context { return context.WithValue(ctx, orgContextKey{}, nil) },
|
||||
want: DefaultOrgID,
|
||||
},
|
||||
{
|
||||
name: "a value of a foreign type is the default org",
|
||||
build: func(ctx context.Context) context.Context { return context.WithValue(ctx, orgContextKey{}, "acme") },
|
||||
want: DefaultOrgID,
|
||||
},
|
||||
{
|
||||
name: "a value under a foreign key is the default org",
|
||||
build: func(ctx context.Context) context.Context {
|
||||
return context.WithValue(ctx, foreignContextKey{}, OrgID("acme"))
|
||||
},
|
||||
want: DefaultOrgID,
|
||||
},
|
||||
{name: "empty org is the default org", build: func(ctx context.Context) context.Context { return WithOrg(ctx, "") }, want: DefaultOrgID},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := OrgFromContext(tt.build(context.Background())); got != tt.want {
|
||||
t.Fatalf("OrgFromContext() = %q, want %q", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -521,11 +521,12 @@ var (
|
||||
ErrResourceVersionConflict = errors.New("resource version conflict")
|
||||
)
|
||||
|
||||
// Resource is one control-plane resource document: the stored value for a
|
||||
// kind, its revision, and the audit record written with the last write.
|
||||
// Resource is one control-plane resource document: the stored value for an
|
||||
// org's kind, its revision, and the audit record written with the last write.
|
||||
// Resource and ResourceWrite live here so the UI-side consumers can reference
|
||||
// them without linking the PostgreSQL implementation.
|
||||
type Resource struct {
|
||||
OrgID string
|
||||
Kind string
|
||||
Value []byte
|
||||
Version int64
|
||||
@@ -538,9 +539,23 @@ type Resource struct {
|
||||
}
|
||||
|
||||
// AuditTableResources is the sys_audit table name for control-plane resources,
|
||||
// keyed by resource kind.
|
||||
// keyed by ResourceAuditKey.
|
||||
const AuditTableResources = "resources"
|
||||
|
||||
// DefaultOrgID is the org every resource written before orgs existed belongs
|
||||
// to, and the org a request that names none acts in.
|
||||
const DefaultOrgID = "default"
|
||||
|
||||
// ResourceAuditKey is the sys_audit record key of one org's resource kind. The
|
||||
// default org keeps the bare kind, so audit rows written before resources were
|
||||
// per-org still resolve.
|
||||
func ResourceAuditKey(orgID, kind string) string {
|
||||
if orgID == DefaultOrgID {
|
||||
return kind
|
||||
}
|
||||
return orgID + "/" + kind
|
||||
}
|
||||
|
||||
// AuditEntry is one changed field of one record, as sys_audit holds it. Any
|
||||
// store table may write entries; Table and RecordKey locate the record.
|
||||
// OldValue and NewValue are JSON; an absent side is empty.
|
||||
@@ -562,6 +577,7 @@ type AuditEntry struct {
|
||||
// the optimistic-concurrency guard: a write whose expectation does not match
|
||||
// the stored revision is refused with ErrResourceVersionConflict.
|
||||
type ResourceWrite struct {
|
||||
OrgID string
|
||||
Kind string
|
||||
Value []byte
|
||||
Actor string
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
package storepkg
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Covers reports whether accepted authority is an upper bound of the
|
||||
// requested authority: every grant the request asks for is one the accepted
|
||||
// record grants, per field, never across fields. The acceptance record and
|
||||
// the update-widening check share this predicate (docs/prds/store.md,
|
||||
// "Authority": an update whose authority is equal or narrower applies).
|
||||
func (a Authority) Covers(b Authority) bool {
|
||||
return listCovers(a.CredentialServices, b.CredentialServices) &&
|
||||
listCovers(a.EgressHosts, b.EgressHosts) &&
|
||||
listCovers(a.ForgePermissions, b.ForgePermissions) &&
|
||||
listCovers(a.Triggers, b.Triggers) &&
|
||||
listCovers(a.Tools, b.Tools)
|
||||
}
|
||||
|
||||
func listCovers(have, want []string) bool {
|
||||
granted := make(map[string]struct{}, len(have))
|
||||
for _, grant := range have {
|
||||
granted[grant] = struct{}{}
|
||||
}
|
||||
for _, grant := range want {
|
||||
if _, ok := granted[grant]; !ok {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// Validate checks authority grant values: forge permissions use the
|
||||
// descriptor's vocabulary and every grant list rejects blank entries.
|
||||
func (a Authority) Validate() error {
|
||||
for _, permission := range a.ForgePermissions {
|
||||
if !validForgePermission(permission) {
|
||||
return fmt.Errorf("invalid forge permission %q", permission)
|
||||
}
|
||||
}
|
||||
fields := []struct {
|
||||
name string
|
||||
grants []string
|
||||
}{
|
||||
{name: "credential service", grants: a.CredentialServices},
|
||||
{name: "egress host", grants: a.EgressHosts},
|
||||
{name: "trigger", grants: a.Triggers},
|
||||
{name: "tool", grants: a.Tools},
|
||||
}
|
||||
for _, field := range fields {
|
||||
for i, grant := range field.grants {
|
||||
if strings.TrimSpace(grant) == "" {
|
||||
return fmt.Errorf("%s %d: name is required", field.name, i)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
package storepkg
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestAuthorityCovers(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
have Authority
|
||||
want Authority
|
||||
covers bool
|
||||
}{
|
||||
{
|
||||
name: "equal authority covers itself",
|
||||
have: Authority{ForgePermissions: []string{"read", "comment"}, Tools: []string{"shell"}},
|
||||
want: Authority{ForgePermissions: []string{"comment", "read"}, Tools: []string{"shell"}},
|
||||
covers: true,
|
||||
},
|
||||
{
|
||||
name: "wider accepted authority covers the narrower request",
|
||||
have: Authority{ForgePermissions: []string{"read", "comment", "review"}},
|
||||
want: Authority{ForgePermissions: []string{"read"}},
|
||||
covers: true,
|
||||
},
|
||||
{
|
||||
name: "an update asking for a forge permission the record lacks is wider",
|
||||
have: Authority{ForgePermissions: []string{"read"}},
|
||||
want: Authority{ForgePermissions: []string{"read", "push"}},
|
||||
covers: false,
|
||||
},
|
||||
{
|
||||
name: "grants are never interchangeable across fields",
|
||||
have: Authority{Tools: []string{"registry"}},
|
||||
want: Authority{CredentialServices: []string{"registry"}},
|
||||
covers: false,
|
||||
},
|
||||
{
|
||||
name: "coverage is checked along every field",
|
||||
have: Authority{
|
||||
CredentialServices: []string{"github"},
|
||||
EgressHosts: []string{"api.github.com"},
|
||||
ForgePermissions: []string{"read"},
|
||||
Triggers: []string{"repo/archie-core:pull_request"},
|
||||
Tools: []string{"shell"},
|
||||
},
|
||||
want: Authority{
|
||||
CredentialServices: []string{"github"},
|
||||
EgressHosts: []string{"internal.invalid"},
|
||||
},
|
||||
covers: false,
|
||||
},
|
||||
{
|
||||
name: "nothing accepted covers nothing requested",
|
||||
have: Authority{},
|
||||
want: Authority{},
|
||||
covers: true,
|
||||
},
|
||||
{
|
||||
name: "an empty accepted record covers any request at all",
|
||||
have: Authority{},
|
||||
want: Authority{Tools: []string{"shell"}},
|
||||
covers: false,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := tt.have.Covers(tt.want); got != tt.covers {
|
||||
t.Fatalf("Covers = %v, want %v", got, tt.covers)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAuthorityValidate(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
authority Authority
|
||||
wantErr string
|
||||
}{
|
||||
{name: "valid authority", authority: Authority{ForgePermissions: []string{"read"}, Tools: []string{"shell"}}},
|
||||
{name: "invalid forge permission", authority: Authority{ForgePermissions: []string{"admin"}}, wantErr: `invalid forge permission "admin"`},
|
||||
{name: "blank credential service", authority: Authority{CredentialServices: []string{" "}}, wantErr: "credential service 0: name is required"},
|
||||
{name: "blank egress host", authority: Authority{EgressHosts: []string{""}}, wantErr: "egress host 0: name is required"},
|
||||
{name: "blank trigger", authority: Authority{Triggers: []string{""}}, wantErr: "trigger 0: name is required"},
|
||||
{name: "blank tool", authority: Authority{Tools: []string{""}}, wantErr: "tool 0: name is required"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := tt.authority.Validate()
|
||||
if tt.wantErr == "" {
|
||||
if err != nil {
|
||||
t.Fatalf("Validate = %v, want nil", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err == nil || err.Error() != tt.wantErr {
|
||||
t.Fatalf("Validate = %v, want %q", err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
package storepkg
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// The descriptor families a package's Contributes names, and the family a
|
||||
// projector takes over.
|
||||
const (
|
||||
FamilyWorkflows = "workflows"
|
||||
FamilyPrompts = "prompts"
|
||||
FamilySkills = "skills"
|
||||
FamilyMCPServer = "mcpServers"
|
||||
FamilyDefaults = "defaults"
|
||||
)
|
||||
|
||||
// families is the family order projection follows, for a deterministic
|
||||
// install and withdrawal.
|
||||
var families = []string{FamilyWorkflows, FamilyPrompts, FamilySkills, FamilyMCPServer, FamilyDefaults}
|
||||
|
||||
// familyNames returns the names one descriptor family contributes.
|
||||
func (c Contributions) familyNames(family string) []string {
|
||||
switch family {
|
||||
case FamilyWorkflows:
|
||||
return c.Workflows
|
||||
case FamilyPrompts:
|
||||
return c.Prompts
|
||||
case FamilySkills:
|
||||
return c.Skills
|
||||
case FamilyMCPServer:
|
||||
return c.MCPServers
|
||||
case FamilyDefaults:
|
||||
return c.Defaults
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// FamilyProjector applies one descriptor family of an installed package into
|
||||
// the org resource that family contributes to, and withdraws its
|
||||
// contributions again on removal. Installing therefore makes a contribution
|
||||
// usable and removing stops it, both without restarting processes.
|
||||
type FamilyProjector interface {
|
||||
// Apply carries the contribution into the org's resources. contents maps
|
||||
// each contributed name to its layer content.
|
||||
Apply(ctx context.Context, orgID, name, digest string, contents map[string][]byte) error
|
||||
// Withdraw takes the contribution back out for an org's package.
|
||||
Withdraw(ctx context.Context, orgID, name, digest string) error
|
||||
}
|
||||
|
||||
// ProjectionEntry records one family's entry a package contributed to its org
|
||||
// resource, so removal can take exactly those entries back out.
|
||||
type ProjectionEntry struct {
|
||||
Family string
|
||||
EntryID string
|
||||
}
|
||||
|
||||
// ProjectionLedger keeps the entries an org's installed package projected.
|
||||
// Implementations live with the installed packages they describe.
|
||||
type ProjectionLedger interface {
|
||||
Record(context.Context, string, string, []ProjectionEntry) error
|
||||
Entries(ctx context.Context, orgID, name string) ([]ProjectionEntry, error)
|
||||
Forget(ctx context.Context, orgID, name, family string) error
|
||||
}
|
||||
|
||||
// resolveFamilyContents resolves every projected family of a package before
|
||||
// anything persists: a contribution the layer cannot back is an install that
|
||||
// can never become usable.
|
||||
func (s Service) resolveFamilyContents(descriptor Descriptor, layer []byte) (map[string]map[string][]byte, error) {
|
||||
contents := map[string]map[string][]byte{}
|
||||
for _, family := range families {
|
||||
if _, ok := s.Projections[family]; !ok {
|
||||
continue
|
||||
}
|
||||
entries, err := s.familyContents(descriptor, layer, family)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(entries) == 0 {
|
||||
continue
|
||||
}
|
||||
contents[family] = entries
|
||||
}
|
||||
return contents, nil
|
||||
}
|
||||
|
||||
// familyContents resolves one family's contributed names against the fetched
|
||||
// layer. A contributed name without layer content is an install that can
|
||||
// never make its contribution usable, so it refuses here.
|
||||
func (s Service) familyContents(descriptor Descriptor, layer []byte, family string) (map[string][]byte, error) {
|
||||
names := descriptor.Contributes.familyNames(family)
|
||||
if len(names) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
files, err := FilesFromLayer(layer)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
contents := make(map[string][]byte, len(names))
|
||||
for _, name := range names {
|
||||
content, ok := files[name]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("contributed %s %q is not a package file", family, name)
|
||||
}
|
||||
declared := false
|
||||
for _, file := range descriptor.Files {
|
||||
if file.Path == name {
|
||||
declared = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !declared {
|
||||
return nil, fmt.Errorf("contributed %s %q is not a declared package file", family, name)
|
||||
}
|
||||
contents[name] = content
|
||||
}
|
||||
return contents, nil
|
||||
}
|
||||
|
||||
// project carries a new installation's contributions into the org resources
|
||||
// their families project to. A family with no projector is persisted but not
|
||||
// projected.
|
||||
func (s Service) project(ctx context.Context, installed Installed, contents map[string]map[string][]byte) error {
|
||||
for _, family := range families {
|
||||
entries, ok := contents[family]
|
||||
if !ok || len(entries) == 0 {
|
||||
continue
|
||||
}
|
||||
projector, ok := s.Projections[family]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if err := projector.Apply(ctx, installed.OrgID, installed.Name, installed.Digest, entries); err != nil {
|
||||
return fmt.Errorf("project %s of package %q: %w", family, installed.Name, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// withdraw takes an installation's contributions back out of the org
|
||||
// resources they were projected into, before the package record goes: a
|
||||
// removal that fails its withdrawal keeps the package installed and its
|
||||
// contributions usable. Families without projectors withdraw nothing.
|
||||
func (s Service) withdraw(ctx context.Context, installed Installed) error {
|
||||
for _, family := range families {
|
||||
projector, ok := s.Projections[family]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if err := projector.Withdraw(ctx, installed.OrgID, installed.Name, installed.Digest); err != nil {
|
||||
return fmt.Errorf("withdraw %s of package %q: %w", family, installed.Name, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,208 @@
|
||||
package storepkg
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"compress/gzip"
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// contributedPackageDescriptor declares one contributed workflow file.
|
||||
const contributedPackage = `apiVersion: dev.archie.package.v1
|
||||
displayName: Review pack
|
||||
version: "1"
|
||||
contributes:
|
||||
workflows:
|
||||
- workflows/review.yaml
|
||||
files:
|
||||
- path: workflows/review.yaml
|
||||
mode: 0644
|
||||
`
|
||||
|
||||
func buildLayer(t *testing.T, files map[string][]byte, packed bool) []byte {
|
||||
t.Helper()
|
||||
var raw bytes.Buffer
|
||||
writer := tar.NewWriter(&raw)
|
||||
for _, name := range sortedFileNames(files) {
|
||||
if err := writer.WriteHeader(&tar.Header{
|
||||
Typeflag: tar.TypeReg, Name: name, Size: int64(len(files[name])), Mode: 0o644,
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := writer.Write(files[name]); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !packed {
|
||||
return raw.Bytes()
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
gz := gzip.NewWriter(&buf)
|
||||
if _, err := gz.Write(raw.Bytes()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := gz.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
func sortedFileNames(files map[string][]byte) []string {
|
||||
keys := make([]string, 0, len(files))
|
||||
for name := range files {
|
||||
keys = append(keys, name)
|
||||
}
|
||||
for i := range keys {
|
||||
for j := i + 1; j < len(keys); j++ {
|
||||
if keys[j] < keys[i] {
|
||||
keys[i], keys[j] = keys[j], keys[i]
|
||||
}
|
||||
}
|
||||
}
|
||||
return keys
|
||||
}
|
||||
|
||||
func TestFilesFromLayer(t *testing.T) {
|
||||
files := map[string][]byte{
|
||||
"workflows/review.yaml": []byte("id: review\nsteps:\n"),
|
||||
"workflows/other.yaml": []byte("id: other\nsteps:\n"),
|
||||
}
|
||||
for _, form := range []struct {
|
||||
name string
|
||||
layer []byte
|
||||
}{
|
||||
{name: "plain tar layer", layer: buildLayer(t, files, false)},
|
||||
{name: "gzipped tar layer", layer: buildLayer(t, files, true)},
|
||||
} {
|
||||
t.Run(form.name, func(t *testing.T) {
|
||||
extracted, err := FilesFromLayer(form.layer)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(extracted) != 2 || !bytes.Equal(extracted["workflows/review.yaml"], files["workflows/review.yaml"]) {
|
||||
t.Fatalf("files = %v", extracted)
|
||||
}
|
||||
})
|
||||
}
|
||||
t.Run("a non-tar layer is an error", func(t *testing.T) {
|
||||
if _, err := FilesFromLayer([]byte("not a tar")); err == nil {
|
||||
t.Fatal("non-tar layer extracted")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// fakeProjector records the contents one family receives, and the order the
|
||||
// service applies and withdraws them.
|
||||
type fakeProjector struct {
|
||||
contents map[string]map[string][]byte // org/name -> contributed name -> content
|
||||
applyCalled bool
|
||||
withdrawn []string
|
||||
applyErr error
|
||||
withdrawErr error
|
||||
}
|
||||
|
||||
func (p *fakeProjector) Apply(_ context.Context, orgID, name, _ string, contents map[string][]byte) error {
|
||||
p.applyCalled = true
|
||||
if p.applyErr != nil {
|
||||
return p.applyErr
|
||||
}
|
||||
p.contents[orgID+"/"+name] = contents
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *fakeProjector) Withdraw(_ context.Context, orgID, name, _ string) error {
|
||||
p.withdrawn = append(p.withdrawn, orgID+"/"+name)
|
||||
if p.withdrawErr != nil {
|
||||
return p.withdrawErr
|
||||
}
|
||||
delete(p.contents, orgID+"/"+name)
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestServiceInstallProjectsContributions(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
const digest = "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
|
||||
|
||||
descriptor, err := Decode(strings.NewReader(contributedPackage))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
workflowYAML := []byte("id: review\nsteps:\n")
|
||||
layer := buildLayer(t, map[string][]byte{"workflows/review.yaml": workflowYAML}, false)
|
||||
service := func(store Repository, projector *fakeProjector) Service {
|
||||
if projector == nil {
|
||||
return Service{Registry: &fakeLayeredRegistry{descriptor: descriptor, layer: layer}, Store: store}
|
||||
}
|
||||
return Service{Registry: &fakeLayeredRegistry{descriptor: descriptor, layer: layer}, Store: store, Projections: map[string]FamilyProjector{FamilyWorkflows: projector}}
|
||||
}
|
||||
|
||||
t.Run("a contributed workflow reaches the family projector", func(t *testing.T) {
|
||||
projector := &fakeProjector{contents: map[string]map[string][]byte{}}
|
||||
store := &fakeInstalledStore{records: map[string]Installed{}}
|
||||
if _, err := service(store, projector).Install(ctx, "org-a", "review", "localhost:5000/review", digest); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !projector.applyCalled {
|
||||
t.Fatal("projector never applied")
|
||||
}
|
||||
if got := projector.contents["org-a/review"]["workflows/review.yaml"]; !bytes.Equal(got, workflowYAML) {
|
||||
t.Fatalf("projected content = %q", got)
|
||||
}
|
||||
})
|
||||
t.Run("a failed projection is not a silent half install", func(t *testing.T) {
|
||||
projector := &fakeProjector{contents: map[string]map[string][]byte{}, applyErr: errors.New("no resource kind")}
|
||||
store := &fakeInstalledStore{records: map[string]Installed{}}
|
||||
if _, err := service(store, projector).Install(ctx, "org-a", "review", "localhost:5000/review", digest); err == nil {
|
||||
t.Fatal("install succeeded although the projection failed")
|
||||
}
|
||||
if _, ok := store.records["org-a/review"]; ok {
|
||||
t.Fatal("failed projection left an installed package without its contributions")
|
||||
}
|
||||
})
|
||||
t.Run("a contribution the layer does not carry refuses the install", func(t *testing.T) {
|
||||
empty := buildLayer(t, map[string][]byte{}, false)
|
||||
projector := &fakeProjector{contents: map[string]map[string][]byte{}}
|
||||
svc := Service{Registry: &fakeLayeredRegistry{descriptor: descriptor, layer: empty}, Store: &fakeInstalledStore{records: map[string]Installed{}}, Projections: map[string]FamilyProjector{FamilyWorkflows: projector}}
|
||||
if _, err := svc.Install(ctx, "org-a", "review", "localhost:5000/review", digest); err == nil {
|
||||
t.Fatal("contributed name without layer content installed")
|
||||
}
|
||||
})
|
||||
t.Run("removal withdraws before the record goes", func(t *testing.T) {
|
||||
projector := &fakeProjector{contents: map[string]map[string][]byte{}}
|
||||
store := &fakeInstalledStore{records: map[string]Installed{"org-a/review": {OrgID: "org-a", Name: "review", Descriptor: descriptor, Digest: digest, UpdatePolicy: "manual"}}}
|
||||
if err := service(store, projector).Remove(ctx, "org-a", "review"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := store.records["org-a/review"]; ok {
|
||||
t.Fatal("removed package record remains")
|
||||
}
|
||||
if len(projector.withdrawn) != 1 || projector.withdrawn[0] != "org-a/review" {
|
||||
t.Fatalf("withdrawn = %v", projector.withdrawn)
|
||||
}
|
||||
})
|
||||
t.Run("a failed withdraw keeps the whole installation", func(t *testing.T) {
|
||||
projector := &fakeProjector{contents: map[string]map[string][]byte{}, withdrawErr: errors.New("version conflict")}
|
||||
store := &fakeInstalledStore{records: map[string]Installed{"org-a/review": {OrgID: "org-a", Name: "review", Descriptor: descriptor, Digest: digest, UpdatePolicy: "manual"}}}
|
||||
if err := service(store, projector).Remove(ctx, "org-a", "review"); err == nil {
|
||||
t.Fatal("removal succeeded although the withdrawal failed")
|
||||
}
|
||||
if _, ok := store.records["org-a/review"]; !ok {
|
||||
t.Fatal("package removed but its withdrawable contributions were already gone")
|
||||
}
|
||||
})
|
||||
t.Run("a family without a projector installs unprojected", func(t *testing.T) {
|
||||
store := &fakeInstalledStore{records: map[string]Installed{}}
|
||||
if _, err := service(store, nil).Install(ctx, "org-a", "review", "localhost:5000/review", digest); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := store.records["org-a/review"]; !ok {
|
||||
t.Fatal("non-projected family refused to install")
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -11,6 +11,15 @@ var (
|
||||
ErrNotFound = errors.New("store package not found")
|
||||
ErrInstalled = errors.New("store package already installed")
|
||||
ErrRequired = errors.New("store package is required by another installed package")
|
||||
// ErrAuthorityNotDeclared: the operator asked to accept authority the
|
||||
// package itself does not declare, which is not an acceptance of that
|
||||
// package. Acceptance records the package's own declared grants against
|
||||
// the pinned digest (docs/prds/store.md, "Authority").
|
||||
ErrAuthorityNotDeclared = errors.New("store package does not declare this authority")
|
||||
// ErrContributionCollision: the package contributes an entry the org
|
||||
// resource already holds under an id the package never projected. The
|
||||
// contribution is refused rather than replacing the operator's entry.
|
||||
ErrContributionCollision = errors.New("package contribution replaces an entry it does not own")
|
||||
)
|
||||
|
||||
// Installed is one organisation's pinned Archie package.
|
||||
@@ -22,6 +31,10 @@ type Installed struct {
|
||||
Descriptor Descriptor
|
||||
Layer []byte
|
||||
UpdatePolicy string
|
||||
// AcceptedAuthority is the operator's acceptance, recorded against the
|
||||
// pinned digest. A nil Authority means the operator has not accepted;
|
||||
// until then the package's authority at use time is nothing.
|
||||
AcceptedAuthority *Authority
|
||||
}
|
||||
|
||||
// Repository owns installed package records and dependency-safe deletion.
|
||||
@@ -30,6 +43,9 @@ type Repository interface {
|
||||
Get(context.Context, string, string) (Installed, error)
|
||||
List(context.Context, string) ([]Installed, error)
|
||||
Remove(context.Context, string, string) error
|
||||
// Accept records the operator's accepted authority against the package's
|
||||
// currently pinned digest.
|
||||
Accept(context.Context, string, string, Authority) error
|
||||
}
|
||||
|
||||
// Registry reads a package by immutable OCI manifest digest.
|
||||
@@ -43,12 +59,18 @@ type Manager interface {
|
||||
GetInstalled(context.Context, string, string) (Installed, error)
|
||||
ListInstalled(context.Context, string) ([]Installed, error)
|
||||
RemoveInstalled(context.Context, string, string) error
|
||||
AcceptPackageAuthority(ctx context.Context, orgID, name string, accepted Authority) (Installed, error)
|
||||
}
|
||||
|
||||
// Service validates the registry content before persisting an installation.
|
||||
type Service struct {
|
||||
Registry Registry
|
||||
Store Repository
|
||||
// Projections carries, per descriptor family, the projector that applies
|
||||
// a package's contribution to the org resource that family projects to,
|
||||
// and withdraws it again on removal. A family with no entry is stored but
|
||||
// never projected: installing leaves it unused until a projector lands.
|
||||
Projections map[string]FamilyProjector
|
||||
}
|
||||
|
||||
var _ Manager = Service{}
|
||||
@@ -69,6 +91,30 @@ func (s Service) RemoveInstalled(ctx context.Context, orgID, name string) error
|
||||
return s.Remove(ctx, orgID, name)
|
||||
}
|
||||
|
||||
// AcceptPackageAuthority records the operator's acceptance against the pinned
|
||||
// digest. Only grants the package declares are acceptable, and the accepted
|
||||
// record may be narrower than the declaration: the record, not the descriptor,
|
||||
// is the authority the package is later checked against.
|
||||
func (s Service) AcceptPackageAuthority(ctx context.Context, orgID, name string, accepted Authority) (Installed, error) {
|
||||
if strings.TrimSpace(orgID) == "" || strings.TrimSpace(name) == "" {
|
||||
return Installed{}, errors.New("org and name are required")
|
||||
}
|
||||
if err := accepted.Validate(); err != nil {
|
||||
return Installed{}, fmt.Errorf("accepted authority: %w", err)
|
||||
}
|
||||
installed, err := s.Store.Get(ctx, orgID, name)
|
||||
if err != nil {
|
||||
return Installed{}, fmt.Errorf("installed package %q: %w", name, err)
|
||||
}
|
||||
if !installed.Descriptor.Authority.Covers(accepted) {
|
||||
return Installed{}, fmt.Errorf("package %q: %w", name, ErrAuthorityNotDeclared)
|
||||
}
|
||||
if err := s.Store.Accept(ctx, orgID, name, accepted); err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
return s.Store.Get(ctx, orgID, name)
|
||||
}
|
||||
|
||||
func (s Service) Install(ctx context.Context, orgID, name, reference, digest string) (Installed, error) {
|
||||
if strings.TrimSpace(orgID) == "" || strings.TrimSpace(name) == "" || strings.TrimSpace(reference) == "" {
|
||||
return Installed{}, errors.New("org, name and reference are required")
|
||||
@@ -96,9 +142,19 @@ func (s Service) Install(ctx context.Context, orgID, name, reference, digest str
|
||||
OrgID: orgID, Name: name, Reference: reference, Digest: digest,
|
||||
Descriptor: descriptor, Layer: layer, UpdatePolicy: "manual",
|
||||
}
|
||||
contents, err := s.resolveFamilyContents(descriptor, layer)
|
||||
if err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
if err := s.Store.Install(ctx, installed); err != nil {
|
||||
return Installed{}, err
|
||||
}
|
||||
if err := s.project(ctx, installed, contents); err != nil {
|
||||
// An install without its projected contributions is not usable. Take
|
||||
// whatever was projected back out, then the installation itself, so the
|
||||
// org is left with no package at all rather than a half-usable one.
|
||||
return Installed{}, errors.Join(err, s.withdraw(ctx, installed), s.Store.Remove(ctx, orgID, name))
|
||||
}
|
||||
return installed, nil
|
||||
}
|
||||
|
||||
@@ -106,5 +162,15 @@ func (s Service) Remove(ctx context.Context, orgID, name string) error {
|
||||
if strings.TrimSpace(orgID) == "" || strings.TrimSpace(name) == "" {
|
||||
return errors.New("org and name are required")
|
||||
}
|
||||
installed, err := s.Store.Get(ctx, orgID, name)
|
||||
if err != nil {
|
||||
return fmt.Errorf("installed package %q: %w", name, err)
|
||||
}
|
||||
// Withdraw before the record goes: the package's contributions must stop
|
||||
// being usable only once its installation stops being true. A withdrawal
|
||||
// that fails keeps the package installed and its record kept.
|
||||
if err := s.withdraw(ctx, installed); err != nil {
|
||||
return err
|
||||
}
|
||||
return s.Store.Remove(ctx, orgID, name)
|
||||
}
|
||||
@@ -57,6 +57,17 @@ func (s *fakeInstalledStore) Remove(_ context.Context, org, name string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *fakeInstalledStore) Accept(_ context.Context, org, name string, authority Authority) error {
|
||||
record, ok := s.records[org+"/"+name]
|
||||
if !ok {
|
||||
return ErrNotFound
|
||||
}
|
||||
accepted := authority
|
||||
record.AcceptedAuthority = &accepted
|
||||
s.records[org+"/"+name] = record
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestServiceInstall(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
valid, err := Decode(strings.NewReader(validDescriptor))
|
||||
@@ -101,6 +112,73 @@ func TestServiceInstall(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestServiceAcceptPackageAuthority(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
declared, err := Decode(strings.NewReader(validDescriptor))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
declared.Authority = Authority{ForgePermissions: []string{"read", "comment"}, Tools: []string{"shell"}}
|
||||
declared.Requires = nil
|
||||
install := func(t *testing.T) (Service, *fakeInstalledStore) {
|
||||
t.Helper()
|
||||
store := &fakeInstalledStore{records: map[string]Installed{}}
|
||||
svc := Service{Registry: &fakeRegistry{descriptor: declared}, Store: store}
|
||||
if _, err := svc.Install(ctx, "org-a", "review", "localhost:5000/review", "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return svc, store
|
||||
}
|
||||
t.Run("records the accepted authority against the pin", func(t *testing.T) {
|
||||
svc, store := install(t)
|
||||
accepted := Authority{ForgePermissions: []string{"read"}}
|
||||
record, err := svc.AcceptPackageAuthority(ctx, "org-a", "review", accepted)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if record.AcceptedAuthority == nil || !record.AcceptedAuthority.Covers(accepted) {
|
||||
t.Fatalf("accepted record = %#v", record.AcceptedAuthority)
|
||||
}
|
||||
stored, err := store.Get(ctx, "org-a", "review")
|
||||
if err != nil || stored.AcceptedAuthority == nil || !stored.AcceptedAuthority.Covers(accepted) {
|
||||
t.Fatalf("stored record = %#v, %v", stored.AcceptedAuthority, err)
|
||||
}
|
||||
// The acceptance record is the bound the package is later checked
|
||||
// against: the operator accepted a narrowed set, so the package's full
|
||||
// declaration now exceeds the accepted authority.
|
||||
if record.AcceptedAuthority.Covers(declared.Authority) {
|
||||
t.Fatal("accepted record covers authority the operator did not accept")
|
||||
}
|
||||
})
|
||||
t.Run("refuses grants the package does not declare", func(t *testing.T) {
|
||||
svc, store := install(t)
|
||||
_, err := svc.AcceptPackageAuthority(ctx, "org-a", "review", Authority{EgressHosts: []string{"internal.invalid"}})
|
||||
if !errors.Is(err, ErrAuthorityNotDeclared) {
|
||||
t.Fatalf("undeclared grant error = %v", err)
|
||||
}
|
||||
stored, err := store.Get(ctx, "org-a", "review")
|
||||
if err != nil || stored.AcceptedAuthority != nil {
|
||||
t.Fatalf("refused acceptance changed the record: %#v, %v", stored.AcceptedAuthority, err)
|
||||
}
|
||||
})
|
||||
t.Run("refuses an acceptance nothing installed", func(t *testing.T) {
|
||||
svc, _ := install(t)
|
||||
if _, err := svc.AcceptPackageAuthority(ctx, "org-a", "missing", Authority{}); !errors.Is(err, ErrNotFound) {
|
||||
t.Fatalf("unknown package error = %v", err)
|
||||
}
|
||||
})
|
||||
t.Run("rejects an invalid acceptance", func(t *testing.T) {
|
||||
svc, store := install(t)
|
||||
if _, err := svc.AcceptPackageAuthority(ctx, "org-a", "review", Authority{ForgePermissions: []string{"admin"}}); err == nil {
|
||||
t.Fatal("invalid forge permission accepted")
|
||||
}
|
||||
stored, err := store.Get(ctx, "org-a", "review")
|
||||
if err != nil || stored.AcceptedAuthority != nil {
|
||||
t.Fatalf("rejected acceptance changed the record: %#v, %v", stored.AcceptedAuthority, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestServiceInstallRequiresPinnedDependency(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
valid, err := Decode(strings.NewReader(validDescriptor))
|
||||
@@ -146,3 +224,16 @@ func TestServiceRemove(t *testing.T) {
|
||||
t.Fatalf("other org package removed: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// fakeLayeredRegistry is a registry returning a descriptor with its layer
|
||||
// bytes, the way a fetched package carries both.
|
||||
type fakeLayeredRegistry struct {
|
||||
descriptor Descriptor
|
||||
layer []byte
|
||||
}
|
||||
|
||||
func (r *fakeLayeredRegistry) Fetch(context.Context, string, string) (Descriptor, []byte, error) {
|
||||
return r.descriptor, r.layer, nil
|
||||
}
|
||||
|
||||
var _ Registry = (*fakeLayeredRegistry)(nil)
|
||||
@@ -0,0 +1,45 @@
|
||||
package storepkg
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"compress/gzip"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
)
|
||||
|
||||
// FilesFromLayer reads an installed package's layer into its files, keyed by
|
||||
// path. The fetch already validated the layer against Descriptor.Files and
|
||||
// held it within the package size limit; this recovers the content the
|
||||
// contributed names point at, plain or gzipped, whichever the package shipped.
|
||||
func FilesFromLayer(layer []byte) (map[string][]byte, error) {
|
||||
reader := io.Reader(bytes.NewReader(layer))
|
||||
if len(layer) >= 2 && layer[0] == 0x1f && layer[1] == 0x8b {
|
||||
gz, err := gzip.NewReader(reader)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read package layer: %w", err)
|
||||
}
|
||||
defer func() { _ = gz.Close() }()
|
||||
reader = gz
|
||||
}
|
||||
files := map[string][]byte{}
|
||||
tarReader := tar.NewReader(reader)
|
||||
for {
|
||||
header, err := tarReader.Next()
|
||||
if errors.Is(err, io.EOF) {
|
||||
return files, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read package layer: %w", err)
|
||||
}
|
||||
if header.Typeflag != tar.TypeReg {
|
||||
continue
|
||||
}
|
||||
content, err := io.ReadAll(tarReader)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read package file %q: %w", header.Name, err)
|
||||
}
|
||||
files[header.Name] = content
|
||||
}
|
||||
}
|
||||
@@ -363,6 +363,10 @@ func (f *fakeForge) VerifyPush(ctx context.Context, owner, repo string) error {
|
||||
// TestStageCommitPushClosesIssueWhenBuildNoChanges verifies that
|
||||
// StageCommitPush closes the issue with a comment when BuildNoChanges
|
||||
// is true, instead of trying to commit and failing.
|
||||
//
|
||||
// This is also the terminal half of the hazard archie-core-7nst describes: a
|
||||
// run re-entered against a worktree that already carries the change reaches
|
||||
// this path, and it must close the issue without ever opening a pull request.
|
||||
func TestStageCommitPushClosesIssueWhenBuildNoChanges(t *testing.T) {
|
||||
f := &fakeForge{}
|
||||
|
||||
@@ -389,6 +393,9 @@ func TestStageCommitPushClosesIssueWhenBuildNoChanges(t *testing.T) {
|
||||
} else {
|
||||
t.Log("CloseIssue called (no changes needed)")
|
||||
}
|
||||
if len(f.calls) != 0 {
|
||||
t.Errorf("forge calls = %v, want none: a no-changes run closes the issue and opens no pull request", f.calls)
|
||||
}
|
||||
if tc.Outcome.Status != StatusCompleted {
|
||||
t.Errorf("expected Outcome=Completed, got %s", tc.Outcome.Status)
|
||||
} else {
|
||||
|
||||
@@ -16,6 +16,8 @@ type PRReviewDecision struct {
|
||||
SkippedPhases []string
|
||||
}
|
||||
|
||||
// ReviewPullRequest refuses operator approval outright, so the gate is left
|
||||
// out of the stage list rather than reached and skipped.
|
||||
func ReviewPullRequest(ctx context.Context, tc *TaskContext) (PRReviewDecision, error) {
|
||||
if tc.Cfg.Review.ApproveBeforePost {
|
||||
return PRReviewDecision{}, fmt.Errorf("headless review cannot wait for operator approval")
|
||||
@@ -27,7 +29,7 @@ func ReviewPullRequest(ctx context.Context, tc *TaskContext) (PRReviewDecision,
|
||||
_ = os.RemoveAll(tc.prReview.snapshotDir)
|
||||
}
|
||||
}()
|
||||
for _, stage := range prReviewDecisionStages() {
|
||||
for _, stage := range prReviewDecisionStages(false) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return PRReviewDecision{}, err
|
||||
}
|
||||
|
||||
@@ -104,6 +104,12 @@ func renderBlockingFindingsDetail(findings []prreview.ScoredFinding) string {
|
||||
// the outcome the PR's existence already earned -- the same best-effort
|
||||
// convention the deleted StagePostReviewComments followed.
|
||||
//
|
||||
// The operator-approval gate is deliberately not spliced in here: it belongs
|
||||
// to the standalone pr-review workflow, where an operator's response
|
||||
// re-reviews a pull request that already exists to post to. This trigger
|
||||
// reviews a change with no PR and no place to post yet, so a wait here would
|
||||
// end the implement run before its PR was opened (archie-core-7nst).
|
||||
//
|
||||
// This must run every decision stage from inside one Stage.Run body rather
|
||||
// than as separate Stage entries in the workflow's list: the engine ends a
|
||||
// run the instant any stage sets tc.Outcome (workflow.go's Run loop), so a
|
||||
@@ -111,7 +117,7 @@ func renderBlockingFindingsDetail(findings []prreview.ScoredFinding) string {
|
||||
// spliced in as their own preceding stages -- OpenPR's own doc comment names
|
||||
// this exact pattern ("call this and then do so in the same stage").
|
||||
func StagePRReviewAndOpenPR(body func(*TaskContext) string) Stage {
|
||||
return stagePRReviewAndOpenPR(prReviewDecisionStages(), body)
|
||||
return stagePRReviewAndOpenPR(prReviewDecisionStages(false), body)
|
||||
}
|
||||
|
||||
// stagePRReviewAndOpenPR is StagePRReviewAndOpenPR's implementation, taking
|
||||
@@ -127,10 +133,8 @@ func stagePRReviewAndOpenPR(decisionStages []Stage, body func(*TaskContext) stri
|
||||
return fmt.Errorf("pr-review (%s): %w", stage.Name, err)
|
||||
}
|
||||
if tc.Outcome.Status != "" {
|
||||
// A decision stage ended the run itself (operator-approval's
|
||||
// waiting_human, or a decision stage's own park). Honor it
|
||||
// rather than proceed to open a PR the operator has not
|
||||
// cleared yet.
|
||||
// A decision stage ended the run itself. Honor it rather than
|
||||
// proceed to open a PR the run has already decided against.
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,8 +4,10 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/agentexec"
|
||||
"github.com/samcharles93/archie-core/internal/config"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/prreview"
|
||||
)
|
||||
@@ -168,6 +170,92 @@ func TestLocalPRSourceReadsFromTheTasksOwnWorktree(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// scriptedPRReviewPipeline answers every agent call the full decision
|
||||
// pipeline makes with the minimum its decoder accepts, ending with one
|
||||
// advisory (non-blocking) finding: enough for the pipeline to reach the point
|
||||
// of opening a PR, and nothing that parks on its own.
|
||||
func scriptedPRReviewPipeline() *concurrentAgentRunner {
|
||||
return &concurrentAgentRunner{byStage: func(stage string) (agentexec.Result, error) {
|
||||
switch {
|
||||
case stage == "intake-ai-score":
|
||||
return captureResult("score_ai_generated", map[string]any{"confidence": 0.1}), nil
|
||||
case strings.HasPrefix(stage, "lens-"):
|
||||
return captureResult("propose_dimensions", map[string]any{"dimensions": []map[string]any{{
|
||||
"name": "defect", "prompt": "review main.go", "target_files": []string{"main.go"}, "priority": 1.0,
|
||||
}}}), nil
|
||||
case strings.HasPrefix(stage, "reviewer-"):
|
||||
return captureResult("report_findings", map[string]any{"findings": []map[string]any{{
|
||||
"file": "main.go", "line_start": 1, "severity": "important", "title": "defect",
|
||||
"body": "broken", "evidence": "package main", "confidence": 0.9,
|
||||
}}}), nil
|
||||
case stage == "evidence-verifier":
|
||||
return captureResult("verify_findings", map[string]any{"verdicts": []map[string]any{{"index": 0, "supported": true}}}), nil
|
||||
case stage == "adversary":
|
||||
return captureResult("adversary_verdicts", map[string]any{"verdicts": []map[string]any{{"index": 0, "verdict": "confirmed"}}}), nil
|
||||
case stage == "consistency":
|
||||
return captureResult("report_findings", map[string]any{"findings": []any{}}), nil
|
||||
case stage == "merge-gate":
|
||||
return captureResult("classify_blocking", map[string]any{"blocking": false}), nil
|
||||
case stage == "polish":
|
||||
return captureResult("polish", map[string]any{"body": "tightened"}), nil
|
||||
default:
|
||||
return passedResult("phase complete"), nil
|
||||
}
|
||||
}}
|
||||
}
|
||||
|
||||
// TestStagePRReviewAndOpenPRDoesNotParkOnOperatorApproval is the regression
|
||||
// test for archie-core-7nst. The own-PR trigger splices the pr-review
|
||||
// decision stages into the implement workflow before it opens its PR, and
|
||||
// with review.approve_before_post set the operator-approval gate used to end
|
||||
// that run in waiting_human with no PR in existence. Approving such a park
|
||||
// re-runs the whole implement workflow against a worktree that already
|
||||
// carries the change; a builder that then finds nothing to do makes
|
||||
// StageCommitPush close the issue, so the run completes and the PR is never
|
||||
// opened. The gate belongs to the standalone pipeline only.
|
||||
func TestStagePRReviewAndOpenPRDoesNotParkOnOperatorApproval(t *testing.T) {
|
||||
tc, forge := ownPRTaskContext(t)
|
||||
tc.Task.Workflow = "implement"
|
||||
tc.Cfg.Review.ApproveBeforePost = true
|
||||
tc.Cfg.Models = map[string]string{"review": "provider/review", "classification": "provider/classification"}
|
||||
tc.Agent = scriptedPRReviewPipeline()
|
||||
|
||||
if err := StagePRReviewAndOpenPR(implementPRBody).Run(context.Background(), tc); err != nil {
|
||||
t.Fatalf("stage: %v", err)
|
||||
}
|
||||
if tc.Outcome.Status == StatusWaitingHuman {
|
||||
t.Fatalf("outcome = %q: the own-PR splice must not park on the operator-approval gate, there is no PR to post to yet", tc.Outcome.Status)
|
||||
}
|
||||
if tc.Outcome.Status != StatusPROpen {
|
||||
t.Fatalf("outcome = %q, want %q", tc.Outcome.Status, StatusPROpen)
|
||||
}
|
||||
if forge.prNumber == 0 {
|
||||
t.Fatal("no PR was opened")
|
||||
}
|
||||
}
|
||||
|
||||
// TestPRReviewWorkflowStillParksOnOperatorApproval is the other half of the
|
||||
// archie-core-7nst fix: the gate is dropped from the own-PR splice, not from
|
||||
// the pipeline. The standalone pr-review workflow still waits for the
|
||||
// operator before it posts and before its merge gate runs.
|
||||
func TestPRReviewWorkflowStillParksOnOperatorApproval(t *testing.T) {
|
||||
tc := baseTaskContext(t)
|
||||
tc.Cfg.Review.ApproveBeforePost = true
|
||||
tc.PRSource = &fakePRSource{diff: smallDiff, files: map[string]string{"main.go": "package main\n"}}
|
||||
tc.Agent = scriptedPRReviewPipeline()
|
||||
forge := &fakeForge{}
|
||||
tc.Forge = forge
|
||||
|
||||
Run(context.Background(), PRReview(), tc)
|
||||
|
||||
if tc.Outcome.Status != StatusWaitingHuman {
|
||||
t.Fatalf("outcome = %q, want %q: the standalone pipeline must still wait for the operator", tc.Outcome.Status, StatusWaitingHuman)
|
||||
}
|
||||
if len(forge.calls) != 0 || forge.reviewCalls != 0 {
|
||||
t.Errorf("forge calls = %v (reviews %d), want none: nothing posts before the operator responds", forge.calls, forge.reviewCalls)
|
||||
}
|
||||
}
|
||||
|
||||
// configRepoWithBase builds a minimal config.Repo whose BaseBranch()
|
||||
// resolves without a live git call.
|
||||
func configRepoWithBase(t *testing.T) config.Repo {
|
||||
|
||||
@@ -163,7 +163,7 @@ func PRReview() Workflow {
|
||||
"depth": {Type: "string"},
|
||||
},
|
||||
},
|
||||
Stages: append(prReviewDecisionStages(), stagePROutput()),
|
||||
Stages: append(prReviewDecisionStages(true), stagePROutput()),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -174,8 +174,17 @@ func PRReview() Workflow {
|
||||
// trigger): the decision of whether an unchallenged blocking finding parks
|
||||
// the task must happen before a PR exists to post to, while phase 9's
|
||||
// posting must happen after, once a PR number exists to anchor comments to.
|
||||
func prReviewDecisionStages() []Stage {
|
||||
return []Stage{
|
||||
//
|
||||
// withOperatorApproval splices the operator-approval gate in between
|
||||
// synthesis and the merge gate. Only the standalone pr-review workflow passes
|
||||
// true: there the operator's response re-reviews the pull request, and the
|
||||
// run is re-entered from the start anyway, which is safe because that
|
||||
// pipeline has no write-then-close stage. Archie's own PRs has no PR to give
|
||||
// the operator a posting decision about yet, and a park there would end the
|
||||
// implement run before it opened one (archie-core-7nst), so the splice drops
|
||||
// the gate instead.
|
||||
func prReviewDecisionStages(withOperatorApproval bool) []Stage {
|
||||
stages := []Stage{
|
||||
stagePRIntake(),
|
||||
stagePRAnatomy(),
|
||||
stagePRLenses(),
|
||||
@@ -184,9 +193,11 @@ func prReviewDecisionStages() []Stage {
|
||||
stagePRVerification(),
|
||||
stagePRCoverageConsistency(),
|
||||
stagePRSynthesis(),
|
||||
stagePROperatorApproval(),
|
||||
stagePRMergeGate(),
|
||||
}
|
||||
if withOperatorApproval {
|
||||
stages = append(stages, stagePROperatorApproval())
|
||||
}
|
||||
return append(stages, stagePRMergeGate())
|
||||
}
|
||||
|
||||
// stagePRIntake is pipeline phase 1: PR metadata and diff statistics, the
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
package task
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"maps"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/org"
|
||||
)
|
||||
|
||||
// WorkflowEnablement is the workflow-enablement resource value: which
|
||||
// workflows each org has turned off. A workflow is enabled for an org unless
|
||||
// that org lists it, so a newly shipped or added workflow is enabled
|
||||
// everywhere until an org disables it.
|
||||
type WorkflowEnablement struct {
|
||||
Orgs map[org.OrgID]OrgWorkflows `json:"orgs"`
|
||||
}
|
||||
|
||||
// OrgWorkflows is one org's workflow state.
|
||||
type OrgWorkflows struct {
|
||||
Disabled []string `json:"disabled"`
|
||||
}
|
||||
|
||||
// Enabled reports whether owner may run workflow.
|
||||
func (e WorkflowEnablement) Enabled(owner org.OrgID, workflow string) bool {
|
||||
return !slices.Contains(e.Orgs[owner].Disabled, workflow)
|
||||
}
|
||||
|
||||
// SetEnabled returns e with workflow enabled or disabled for owner.
|
||||
func (e WorkflowEnablement) SetEnabled(owner org.OrgID, workflow string, enabled bool) WorkflowEnablement {
|
||||
orgs := make(map[org.OrgID]OrgWorkflows, len(e.Orgs)+1)
|
||||
maps.Copy(orgs, e.Orgs)
|
||||
disabled := slices.DeleteFunc(slices.Clone(orgs[owner].Disabled), func(id string) bool { return id == workflow })
|
||||
if !enabled {
|
||||
disabled = append(disabled, workflow)
|
||||
}
|
||||
orgs[owner] = OrgWorkflows{Disabled: disabled}
|
||||
return WorkflowEnablement{Orgs: orgs}
|
||||
}
|
||||
|
||||
// Validate rejects blank org or workflow IDs and a workflow listed twice.
|
||||
func (e WorkflowEnablement) Validate() error {
|
||||
for owner, state := range e.Orgs {
|
||||
if strings.TrimSpace(string(owner)) == "" {
|
||||
return fmt.Errorf("workflow enablement: org ID is required")
|
||||
}
|
||||
seen := make(map[string]bool, len(state.Disabled))
|
||||
for _, workflow := range state.Disabled {
|
||||
if strings.TrimSpace(workflow) == "" {
|
||||
return fmt.Errorf("workflow enablement: org %q: workflow ID is required", owner)
|
||||
}
|
||||
if seen[workflow] {
|
||||
return fmt.Errorf("workflow enablement: org %q lists %q twice", owner, workflow)
|
||||
}
|
||||
seen[workflow] = true
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
package task
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/org"
|
||||
)
|
||||
|
||||
func TestWorkflowEnablementEnabled(t *testing.T) {
|
||||
enablement := WorkflowEnablement{Orgs: map[org.OrgID]OrgWorkflows{
|
||||
"acme": {Disabled: []string{"triage"}},
|
||||
}}
|
||||
cases := []struct {
|
||||
org org.OrgID
|
||||
workflow string
|
||||
want bool
|
||||
}{
|
||||
{"acme", "triage", false},
|
||||
{"acme", "implement", true},
|
||||
{"other", "triage", true},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := enablement.Enabled(c.org, c.workflow); got != c.want {
|
||||
t.Errorf("Enabled(%q, %q) = %v, want %v", c.org, c.workflow, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkflowEnablementSetEnabledRoundTrips(t *testing.T) {
|
||||
var enablement WorkflowEnablement
|
||||
enablement = enablement.SetEnabled("acme", "triage", false)
|
||||
enablement = enablement.SetEnabled("acme", "triage", false)
|
||||
if enablement.Enabled("acme", "triage") || len(enablement.Orgs["acme"].Disabled) != 1 {
|
||||
t.Fatalf("disable twice: %+v", enablement)
|
||||
}
|
||||
enablement = enablement.SetEnabled("acme", "triage", true)
|
||||
if !enablement.Enabled("acme", "triage") {
|
||||
t.Fatalf("re-enable: %+v", enablement)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkflowEnablementValidate(t *testing.T) {
|
||||
cases := map[string]struct {
|
||||
enablement WorkflowEnablement
|
||||
ok bool
|
||||
}{
|
||||
"empty": {WorkflowEnablement{}, true},
|
||||
"valid": {WorkflowEnablement{Orgs: map[org.OrgID]OrgWorkflows{"acme": {Disabled: []string{"triage"}}}}, true},
|
||||
"blank org": {WorkflowEnablement{Orgs: map[org.OrgID]OrgWorkflows{" ": {Disabled: []string{"triage"}}}}, false},
|
||||
"blank workflow": {WorkflowEnablement{Orgs: map[org.OrgID]OrgWorkflows{"acme": {Disabled: []string{""}}}}, false},
|
||||
"duplicate entry": {WorkflowEnablement{Orgs: map[org.OrgID]OrgWorkflows{"acme": {Disabled: []string{"triage", "triage"}}}}, false},
|
||||
}
|
||||
for name, c := range cases {
|
||||
if err := c.enablement.Validate(); (err == nil) != c.ok {
|
||||
t.Errorf("%s: Validate() = %v, want ok=%v", name, err, c.ok)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -141,17 +141,45 @@ func (a *LocalChatAdapter) Stream(ctx context.Context, in Inbound) (<-chan ChatE
|
||||
return nil, err
|
||||
}
|
||||
events := make(chan ChatEvent, 32)
|
||||
events <- ChatEvent{Kind: "started", SessionID: id}
|
||||
go a.stream(ctx, in, id, events)
|
||||
return events, nil
|
||||
}
|
||||
|
||||
func (a *LocalChatAdapter) stream(ctx context.Context, in Inbound, id string, events chan ChatEvent) {
|
||||
defer close(events)
|
||||
|
||||
started := a.prelude(ctx, in, id)
|
||||
if started.event != nil {
|
||||
events <- *started.event
|
||||
return
|
||||
}
|
||||
if !started.proceeds {
|
||||
// A covered fragment: the batch's turn runs on the call that
|
||||
// dispatched it. This caller contributes its text to that turn and
|
||||
// renders nothing of its own -- not even the started event, which
|
||||
// announces a turn that does not exist here.
|
||||
return
|
||||
}
|
||||
in = started.ready
|
||||
|
||||
// started announces the turn that runs; it comes after the gates and
|
||||
// the batch, and always before the first delta or the terminal event.
|
||||
events <- ChatEvent{Kind: "started", SessionID: id}
|
||||
|
||||
a.runStreamedTurn(ctx, in, id, events)
|
||||
}
|
||||
|
||||
// runStreamedTurn dispatches one turn whose payload the prelude produced and
|
||||
// pumps its events to the caller. The turn runs on its session lane -- the
|
||||
// lane key is the session, so /stop reaches the turn the sender is actually
|
||||
// watching -- while the pump stays on the caller's goroutine, so a
|
||||
// caller-cancelled stream stops the lane it is serving. The terminal event
|
||||
// lands after every mid-turn event that was already queued.
|
||||
func (a *LocalChatAdapter) runStreamedTurn(ctx context.Context, in Inbound, id string, events chan ChatEvent) {
|
||||
pending := make(chan ChatEvent, 32)
|
||||
stream := localChatStream{done: ctx.Done(), events: pending, sessionID: id}
|
||||
run := func(turnCtx context.Context) ChatEvent {
|
||||
reply, err := a.Router.RouteStream(turnCtx, in, stream)
|
||||
reply, err := a.Router.streamTurn(turnCtx, in, stream)
|
||||
if err != nil {
|
||||
return ChatEvent{Kind: "error", Text: err.Error(), SessionID: id}
|
||||
}
|
||||
@@ -167,6 +195,14 @@ func (a *LocalChatAdapter) stream(ctx context.Context, in Inbound, id string, ev
|
||||
} else {
|
||||
a.Turns.Submit(ctx, id, func(turnCtx context.Context) { result <- run(turnCtx) })
|
||||
}
|
||||
a.pumpTurnEvents(ctx, id, events, pending, result)
|
||||
}
|
||||
|
||||
// pumpTurnEvents streams one turn's events into its caller's channel: every
|
||||
// delta and tool event as it arrives, then the terminal event after them.
|
||||
// A caller whose context ends mid-stream has stopped the turn, and the lane
|
||||
// it occupies is stopped so the turn cannot keep running unobserved.
|
||||
func (a *LocalChatAdapter) pumpTurnEvents(ctx context.Context, id string, events chan ChatEvent, pending, result <-chan ChatEvent) {
|
||||
emit := func(event ChatEvent) {
|
||||
select {
|
||||
case events <- event:
|
||||
@@ -196,6 +232,52 @@ func (a *LocalChatAdapter) stream(ctx context.Context, in Inbound, id string, ev
|
||||
}
|
||||
}
|
||||
|
||||
// streamPrelude is what one streamed caller's turn is before any turn
|
||||
// exists: render a terminal event and stop, render nothing, or dispatch.
|
||||
type streamPrelude struct {
|
||||
// event is the terminal event the caller's stream renders when no turn
|
||||
// exists for it: a gate's prose or the batch's collection error. nil on
|
||||
// every path that proceeds.
|
||||
event *ChatEvent
|
||||
// ready is the payload the turn dispatches, and stays zero for every
|
||||
// caller whose stream renders no turn.
|
||||
ready Inbound
|
||||
// proceeds reports whether this caller dispatches the turn. A covered
|
||||
// fragment does not: it contributes only text to the dispatcher's turn,
|
||||
// so it renders nothing -- not even the started event, which announces
|
||||
// a turn that does not exist here.
|
||||
proceeds bool
|
||||
}
|
||||
|
||||
// prelude runs the per-message gates and the text batcher for one streamed
|
||||
// fragment, on this call's own goroutine and before the session lane, and
|
||||
// returns what this caller's stream does next.
|
||||
//
|
||||
// The gates run here rather than inside the session lane: a redelivered
|
||||
// fragment is a declined delivery to the gate it always reached and never
|
||||
// joins the batch it repeats; the rate budget is charged per fragment the
|
||||
// way it always was; and since the reply a gate returns renders today, the
|
||||
// prose is the done event of this very stream rather than a turn output.
|
||||
// Rapid-fire text fragments of one thought coalesce before the session lane
|
||||
// exists for the same reason: waiting inside a lane would make every
|
||||
// fragment queue behind the first fragment's batch window instead of
|
||||
// joining it.
|
||||
func (a *LocalChatAdapter) prelude(ctx context.Context, in Inbound, id string) streamPrelude {
|
||||
if a.Router.duplicateDelivery(in) {
|
||||
return streamPrelude{event: &ChatEvent{Kind: "done", Text: dedupReply, SessionID: id}}
|
||||
}
|
||||
if a.Router.checkRateLimit(in) {
|
||||
return streamPrelude{event: &ChatEvent{Kind: "done", Text: rateLimitReply, SessionID: id}}
|
||||
}
|
||||
ready, mine, err := a.Router.CollectTurn(ctx, in)
|
||||
if err != nil {
|
||||
return streamPrelude{event: &ChatEvent{Kind: "error", Text: err.Error(), SessionID: id}}
|
||||
}
|
||||
return streamPrelude{ready: ready, proceeds: mine}
|
||||
}
|
||||
|
||||
// localChatStream is the TurnStream piped into the router for one streamed
|
||||
// turn; its events move to the caller's channel below.
|
||||
type localChatStream struct {
|
||||
done <-chan struct{}
|
||||
events chan<- ChatEvent
|
||||
|
||||
@@ -5,6 +5,39 @@ import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// streamChat builds the local chat adapter the way the composition wires
|
||||
// it, pointed at the router under test; it is the production streaming
|
||||
// entry these streamed-turn tests drive.
|
||||
func streamChat(r *Router) *LocalChatAdapter {
|
||||
return &LocalChatAdapter{Router: r}
|
||||
}
|
||||
|
||||
// drainStreamed runs one streamed turn through the production chat adapter
|
||||
// and returns what its stream rendered: the deltas, then the terminal
|
||||
// event's kind and text.
|
||||
func drainStreamed(t *testing.T, chat *LocalChatAdapter, in Inbound) ([]string, string, string, error) {
|
||||
t.Helper()
|
||||
events, err := chat.Stream(context.Background(), in)
|
||||
if err != nil {
|
||||
return nil, "", "", err
|
||||
}
|
||||
rendered, drainErr := drained(events)
|
||||
if drainErr != nil {
|
||||
return nil, "", "", drainErr
|
||||
}
|
||||
var deltas []string
|
||||
kind, text := "", ""
|
||||
for _, event := range rendered {
|
||||
switch event.Kind {
|
||||
case "delta":
|
||||
deltas = append(deltas, event.Text)
|
||||
case "done", "error":
|
||||
kind, text = event.Kind, event.Text
|
||||
}
|
||||
}
|
||||
return deltas, kind, text, nil
|
||||
}
|
||||
|
||||
func TestLocalChatSnapshotAndStream(t *testing.T) {
|
||||
ctx := t.Context()
|
||||
sessions := NewSessionStoreMemory()
|
||||
|
||||
+79
-24
@@ -167,9 +167,9 @@ type Router struct {
|
||||
// reporting a value it does not have.
|
||||
Health HealthSource
|
||||
LLM LLMResponder // nil = LLM not wired yet
|
||||
// LLMStream is the optional streaming responder. When set, adapters
|
||||
// that can render partial output (see RouteStream) show the reply as
|
||||
// it generates; when nil, everything falls back to LLM.
|
||||
// LLMStream is the optional streaming responder. When set, the streamed
|
||||
// chat adapter (internal/gateway/chat_local.go) shows the reply as it
|
||||
// generates; when nil, everything falls back to LLM.
|
||||
LLMStream LLMStreamResponder
|
||||
// Identity is the archie identity this router belongs to (empty in
|
||||
// single-identity deployments). Propagated into SpawnRequest and
|
||||
@@ -206,7 +206,26 @@ type Router struct {
|
||||
// has a stable source but no person), else against Message.SenderID.
|
||||
// A message with neither is never limited: there is no key to charge
|
||||
// it against.
|
||||
Limiter *ratelimit.Limiter
|
||||
Limiter *ratelimit.Limiter
|
||||
// Dedup declines a redelivered platform message (same platform,
|
||||
// conversation and channel-native ID, delivered again inside the TTL
|
||||
// window from internal/gateway/message_dedup.go) before it re-enters
|
||||
// the turn pipeline. NewRouter wires the package default, so a
|
||||
// production router is always guarded; set a differently configured
|
||||
// gate to tune it, or nil to disable dedup entirely (test setups).
|
||||
// The window is deliberately not a config knob here: the durable
|
||||
// prior-reply replay in turn.go is the backstop for duplicates that
|
||||
// outlive it, and no composition choice between them needs exposing.
|
||||
Dedup *MessageDeduplicator
|
||||
// Batches coalesces rapid-fire plain-text fragments from one sender into
|
||||
// a single turn (internal/gateway/text_batch.go), so a chat client that
|
||||
// splits one thought across several messages costs one agent turn, not
|
||||
// one per fragment. It sits after the per-message gates (dedup, rate
|
||||
// limit) and before the responder: each fragment passes the gates as
|
||||
// the message it is, and the batch's single turn is the only dispatch.
|
||||
// NewRouter wires the package default, so a production router batches;
|
||||
// nil disables coalescing (test setups).
|
||||
Batches *TextBatchAggregator
|
||||
sessionTracker *sessionTracker
|
||||
gatewayName string
|
||||
// titlingMu guards titling, the set of sessions with a title proposal
|
||||
@@ -218,7 +237,11 @@ type Router struct {
|
||||
// NewRouter returns a Router. llm is optional -- when nil, non-command
|
||||
// messages get a "not configured" response.
|
||||
func NewRouter(store StatusReader, llm LLMResponder, gatewayName string) *Router {
|
||||
return &Router{Store: store, LLM: llm, gatewayName: gatewayName}
|
||||
return &Router{
|
||||
Store: store, LLM: llm, gatewayName: gatewayName,
|
||||
Dedup: NewMessageDeduplicator(0, 0, nil),
|
||||
Batches: NewTextBatchAggregator(0, 0, 0, nil, nil),
|
||||
}
|
||||
}
|
||||
|
||||
// InitSessions wires the session store and starts tracking sessions.
|
||||
@@ -277,16 +300,54 @@ func (r *Router) Route(ctx context.Context, in Inbound) (string, error) {
|
||||
// echoing the rate-limit prose as a successful delivery, or accepting an
|
||||
// event that was actually dropped (archie-core-1173).
|
||||
func (r *Router) RouteResult(ctx context.Context, in Inbound) (reply string, rateLimited bool, err error) {
|
||||
if r.duplicateDelivery(in) {
|
||||
return dedupReply, false, nil
|
||||
}
|
||||
if r.checkRateLimit(in) {
|
||||
return rateLimitReply, true, nil
|
||||
}
|
||||
reply, err = r.route(ctx, in)
|
||||
ready, mine, err := r.CollectTurn(ctx, in)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
if !mine {
|
||||
// A covered fragment: the batch's one turn answers the batch through
|
||||
// the call that dispatched it. This caller renders nothing of its
|
||||
// own; where the reply leaves as text (email is the only such
|
||||
// channel today), nothing is sent rather than the join sent twice.
|
||||
return "", false, nil
|
||||
}
|
||||
reply, err = r.route(ctx, ready)
|
||||
return reply, false, err
|
||||
}
|
||||
|
||||
// duplicateDelivery reports whether in is a repeat delivery inside the
|
||||
// dedup window, and records the delivery so later repeats see this one.
|
||||
// The check runs before the rate limit: a repeated delivery is the
|
||||
// channel's doing, not the sender's, so it must not spend the sender's
|
||||
// inbound budget. A nil Dedup (disabled) never blocks.
|
||||
func (r *Router) duplicateDelivery(in Inbound) bool {
|
||||
return r.Dedup != nil && !r.Dedup.Admit(in)
|
||||
}
|
||||
|
||||
// CollectTurn hands a gated inbound to the text batcher and returns the
|
||||
// payload this call is responsible for dispatching. Every gate has already
|
||||
// run by the time it is called -- a gate reply like the dedup prose is
|
||||
// computed for the message it is and never waits on a batch window, and
|
||||
// then the batch's single turn dispatches without running the gates again
|
||||
// (their answer is already recorded). A nil Batches passes through, as do
|
||||
// messages that are no fragment (internal/gateway/text_batch.go).
|
||||
func (r *Router) CollectTurn(ctx context.Context, in Inbound) (Inbound, bool, error) {
|
||||
if r.Batches == nil {
|
||||
return in, true, nil
|
||||
}
|
||||
return r.Batches.Collect(ctx, in)
|
||||
}
|
||||
|
||||
// route is Route's continuation once the caller has already cleared the
|
||||
// rate-limit check (or is deliberately skipping it, as RouteStream's
|
||||
// local-command fallthrough does to avoid charging one message twice).
|
||||
// rate-limit check (or is deliberately skipping it, as streamTurn's
|
||||
// local-command fallthrough and the streamed adapter's degraded-to-non-
|
||||
// streaming path do, to avoid charging one message twice).
|
||||
func (r *Router) route(ctx context.Context, in Inbound) (string, error) {
|
||||
text := strings.TrimSpace(in.Message.Text)
|
||||
cmd, _ := parseCmd(text, r.gatewayName)
|
||||
@@ -442,23 +503,17 @@ func (r *Router) dispatchSessionCommand(ctx context.Context, msg messaging.Messa
|
||||
return "", false, nil
|
||||
}
|
||||
|
||||
// RouteStream is Route for adapters that can render a reply progressively.
|
||||
// stream receives each new fragment of an LLM reply as it is generated, plus
|
||||
// each completed tool call, in the order the model produced them; the
|
||||
// complete reply is returned as usual.
|
||||
//
|
||||
// Only free-text messages stream. Gateway-local commands (/status, /model,
|
||||
// …) answer from local state in a single step, so they return through the
|
||||
// normal path with no delta callbacks. When no streaming responder is
|
||||
// configured, RouteStream is exactly Route -- callers get the whole reply
|
||||
// at the end and simply never see a delta, so an adapter can always call
|
||||
// RouteStream without checking first.
|
||||
func (r *Router) RouteStream(ctx context.Context, in Inbound, stream TurnStream) (string, error) {
|
||||
// streamTurn dispatches one streamed turn for an inbound the streamed chat
|
||||
// adapter has already prepared: the per-message gates (dedup, rate limit)
|
||||
// ran per fragment in the adapter's prelude, and the text batcher has
|
||||
// settled the turn's payload -- prelude's dispatcher carries the joined
|
||||
// payload for its whole batch here, so the gates answer no second time for
|
||||
// messages they already saw, and in particular the combined payload's
|
||||
// first-fragment source ID, which the dedup gate already recorded, does
|
||||
// not read back as a redelivery.
|
||||
func (r *Router) streamTurn(ctx context.Context, in Inbound, stream TurnStream) (string, error) {
|
||||
if r.LLMStream == nil || stream == nil {
|
||||
return r.Route(ctx, in)
|
||||
}
|
||||
if r.checkRateLimit(in) {
|
||||
return rateLimitReply, nil
|
||||
return r.route(ctx, in)
|
||||
}
|
||||
cmd, _ := parseCmd(strings.TrimSpace(in.Message.Text), r.gatewayName)
|
||||
if isLocalCommand(cmd) || strings.HasPrefix(cmd, "/") {
|
||||
|
||||
@@ -19,6 +19,9 @@ func senderInbound(senderID, text string) Inbound {
|
||||
func TestRouteRateLimitBlocksOverBudget(t *testing.T) {
|
||||
r := NewRouter(nil, fakeLLM, "test")
|
||||
r.Limiter = ratelimit.New(time.Minute, 1)
|
||||
// The batcher is off: nothing here is about coalescing, and its window
|
||||
// must not sit between a dispatch and this test's assertions.
|
||||
r.Batches = nil
|
||||
|
||||
reply, err := r.Route(context.Background(), senderInbound("u1", "hello"))
|
||||
if err != nil {
|
||||
@@ -40,6 +43,9 @@ func TestRouteRateLimitBlocksOverBudget(t *testing.T) {
|
||||
func TestRouteRateLimitIsPerSender(t *testing.T) {
|
||||
r := NewRouter(nil, fakeLLM, "test")
|
||||
r.Limiter = ratelimit.New(time.Minute, 1)
|
||||
// The batcher is off: these dispatches must be immediate, and each new
|
||||
// sender would otherwise wait out its own quiet window.
|
||||
r.Batches = nil
|
||||
|
||||
if _, err := r.Route(context.Background(), senderInbound("u1", "hi")); err != nil {
|
||||
t.Fatalf("Route: %v", err)
|
||||
@@ -55,6 +61,8 @@ func TestRouteRateLimitIsPerSender(t *testing.T) {
|
||||
|
||||
func TestRouteNoLimiterConfiguredNeverBlocks(t *testing.T) {
|
||||
r := NewRouter(nil, fakeLLM, "test")
|
||||
// The batcher is off: each allowed dispatch must be immediate here.
|
||||
r.Batches = nil
|
||||
|
||||
for i := range 5 {
|
||||
reply, err := r.Route(context.Background(), senderInbound("u1", "hello"))
|
||||
@@ -132,58 +140,61 @@ func TestRouteBudgetKeyLimitsWithoutSenderID(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestRouteStreamLocalCommandChargesExactlyOnce pins RouteStream's local
|
||||
// command fallthrough (into route, not Route) against double-charging the
|
||||
// budget: a status query must consume the same one unit whether it goes
|
||||
// through Route or RouteStream.
|
||||
func TestRouteStreamLocalCommandChargesExactlyOnce(t *testing.T) {
|
||||
// TestStreamLocalCommandChargesExactlyOnce pins the streamed adapter's local
|
||||
// command fallthrough (into route, not the streaming responder) against
|
||||
// double-charging the budget: a status query must consume the same one unit
|
||||
// whether it goes through Route or the adapter's Stream.
|
||||
func TestStreamLocalCommandChargesExactlyOnce(t *testing.T) {
|
||||
r := NewRouter(&fakeStore{counts: map[string]int{}}, nil, "test")
|
||||
r.Limiter = ratelimit.New(time.Minute, 2)
|
||||
// The batcher is off: the charge being pinned is per message, not per batch.
|
||||
r.Batches = nil
|
||||
r.LLMStream = func(context.Context, Inbound, TurnStream) (string, error) {
|
||||
return "streamed", nil
|
||||
}
|
||||
|
||||
if _, err := r.RouteStream(context.Background(), senderInbound("u1", "/status"), DeltaFunc(func(string) {})); err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
if _, _, _, err := drainStreamed(t, streamChat(r), senderInbound("u1", "/status")); err != nil {
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
|
||||
// Budget was 2; the local command must have consumed exactly one, so
|
||||
// exactly one more free-text message should still get through.
|
||||
reply, err := r.RouteStream(context.Background(), senderInbound("u1", "hello"), DeltaFunc(func(string) {}))
|
||||
_, kind, text, err := drainStreamed(t, streamChat(r), senderInbound("u1", "hello"))
|
||||
if err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
if reply != "streamed" {
|
||||
t.Fatalf("second call reply = %q, want it to have budget left", reply)
|
||||
if kind != "done" || text != "streamed" {
|
||||
t.Fatalf("second call terminal = (%s, %q), want it to have budget left", kind, text)
|
||||
}
|
||||
|
||||
reply, err = r.RouteStream(context.Background(), senderInbound("u1", "hello again"), DeltaFunc(func(string) {}))
|
||||
_, kind, text, err = drainStreamed(t, streamChat(r), senderInbound("u1", "hello again"))
|
||||
if err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
if reply != rateLimitReply {
|
||||
t.Fatalf("third call reply = %q, want the budget exhausted after 2 charged calls", reply)
|
||||
if kind != "done" || text != rateLimitReply {
|
||||
t.Fatalf("third call terminal = (%s, %q), want the budget exhausted after 2 charged calls", kind, text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRouteStreamRateLimitBlocksStreamedReply(t *testing.T) {
|
||||
func TestStreamRateLimitBlocksStreamedReply(t *testing.T) {
|
||||
r := NewRouter(nil, nil, "test")
|
||||
r.Limiter = ratelimit.New(time.Minute, 1)
|
||||
// The batcher is off: the blocked call must not wait on a batch window.
|
||||
r.Batches = nil
|
||||
streamCalls := 0
|
||||
r.LLMStream = func(context.Context, Inbound, TurnStream) (string, error) {
|
||||
streamCalls++
|
||||
return "streamed", nil
|
||||
}
|
||||
|
||||
if _, err := r.RouteStream(context.Background(), senderInbound("u1", "hello"), DeltaFunc(func(string) {})); err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
if _, _, _, err := drainStreamed(t, streamChat(r), senderInbound("u1", "hello")); err != nil {
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
reply, err := r.RouteStream(context.Background(), senderInbound("u1", "hello again"), DeltaFunc(func(string) {}))
|
||||
_, kind, text, err := drainStreamed(t, streamChat(r), senderInbound("u1", "hello again"))
|
||||
if err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
if reply != rateLimitReply {
|
||||
t.Fatalf("reply = %q, want the rate-limit reply", reply)
|
||||
if kind != "done" || text != rateLimitReply {
|
||||
t.Fatalf("terminal = (%s, %q), want the rate-limit reply", kind, text)
|
||||
}
|
||||
if streamCalls != 1 {
|
||||
t.Fatalf("LLMStream called %d times, want exactly 1 (the blocked call must not reach it)", streamCalls)
|
||||
|
||||
@@ -6,29 +6,28 @@ import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// RouteStream must degrade to Route when no streaming responder is set, so
|
||||
// adapters can always call it without first checking for support.
|
||||
func TestRouteStreamFallsBackWhenNoStreamResponder(t *testing.T) {
|
||||
// The streamed chat adapter must degrade to the blocking responder when no
|
||||
// streaming responder is set, so the chat contract's Stream can always be
|
||||
// called without first checking for support. The degrade lives in
|
||||
// streamTurn's contract now that every streamed entry is the adapter's.
|
||||
func TestStreamFallsBackWhenNoStreamResponder(t *testing.T) {
|
||||
r := NewRouter(nil, func(context.Context, Inbound) (string, error) {
|
||||
return "blocking reply", nil
|
||||
}, "telegram")
|
||||
|
||||
var deltas []string
|
||||
got, err := r.RouteStream(context.Background(), inbound("", "hello"), DeltaFunc(func(d string) {
|
||||
deltas = append(deltas, d)
|
||||
}))
|
||||
deltas, kind, text, err := drainStreamed(t, streamChat(r), inbound("", "hello"))
|
||||
if err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
if got != "blocking reply" {
|
||||
t.Errorf("reply = %q, want %q", got, "blocking reply")
|
||||
if kind != "done" || text != "blocking reply" {
|
||||
t.Errorf("terminal event = (%s, %q), want (done, \"blocking reply\")", kind, text)
|
||||
}
|
||||
if len(deltas) != 0 {
|
||||
t.Errorf("expected no deltas without a stream responder, got %v", deltas)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRouteStreamStreamsFreeText(t *testing.T) {
|
||||
func TestStreamStreamsFreeText(t *testing.T) {
|
||||
r := NewRouter(nil, nil, "telegram")
|
||||
r.LLMStream = func(_ context.Context, _ Inbound, stream TurnStream) (string, error) {
|
||||
for _, d := range []string{"a", "b", "c"} {
|
||||
@@ -37,21 +36,18 @@ func TestRouteStreamStreamsFreeText(t *testing.T) {
|
||||
return "abc", nil
|
||||
}
|
||||
|
||||
var sb strings.Builder
|
||||
got, err := r.RouteStream(context.Background(), inbound("", "hello"), DeltaFunc(func(d string) {
|
||||
sb.WriteString(d)
|
||||
}))
|
||||
deltas, kind, text, err := drainStreamed(t, streamChat(r), inbound("", "hello"))
|
||||
if err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
if got != "abc" || sb.String() != "abc" {
|
||||
t.Errorf("reply=%q deltas=%q, want both %q", got, sb.String(), "abc")
|
||||
if kind != "done" || text != "abc" || strings.Join(deltas, "") != "abc" {
|
||||
t.Errorf("terminal=(%s,%q) deltas=%q, want (done,\"abc\") rendering \"abc\"", kind, text, deltas)
|
||||
}
|
||||
}
|
||||
|
||||
// Local commands answer from local state in one step, so they must not be
|
||||
// sent to the streaming responder.
|
||||
func TestRouteStreamDoesNotStreamLocalCommands(t *testing.T) {
|
||||
func TestStreamDoesNotStreamLocalCommands(t *testing.T) {
|
||||
r := NewRouter(nil, nil, "telegram")
|
||||
streamed := false
|
||||
r.LLMStream = func(context.Context, Inbound, TurnStream) (string, error) {
|
||||
@@ -59,15 +55,15 @@ func TestRouteStreamDoesNotStreamLocalCommands(t *testing.T) {
|
||||
return "", nil
|
||||
}
|
||||
|
||||
if _, err := r.RouteStream(context.Background(), inbound("", "/status"), DeltaFunc(func(string) {})); err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
if _, _, _, err := drainStreamed(t, streamChat(r), inbound("", "/status")); err != nil {
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
if streamed {
|
||||
t.Error("/status must not be routed to the streaming responder")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRouteStreamRejectsUnknownCommandLocally(t *testing.T) {
|
||||
func TestStreamRejectsUnknownCommandLocally(t *testing.T) {
|
||||
r := NewRouter(nil, nil, "telegram")
|
||||
streamed := false
|
||||
r.LLMStream = func(context.Context, Inbound, TurnStream) (string, error) {
|
||||
@@ -75,14 +71,14 @@ func TestRouteStreamRejectsUnknownCommandLocally(t *testing.T) {
|
||||
return "ok", nil
|
||||
}
|
||||
|
||||
reply, err := r.RouteStream(context.Background(), inbound("", "/commands"), DeltaFunc(func(string) {}))
|
||||
_, kind, text, err := drainStreamed(t, streamChat(r), inbound("", "/commands"))
|
||||
if err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
if streamed {
|
||||
t.Error("unknown /commands must not reach the streaming responder")
|
||||
}
|
||||
if !strings.Contains(reply, "/help") {
|
||||
t.Fatalf("reply = %q, want /help guidance", reply)
|
||||
if kind != "done" || !strings.Contains(text, "/help") {
|
||||
t.Errorf("terminal = (%s, %q), want the local unknown-command reply with /help guidance", kind, text)
|
||||
}
|
||||
}
|
||||
@@ -349,15 +349,18 @@ func TestRouterRejectsUnknownCommand(t *testing.T) {
|
||||
return "fabricated streaming behavior", nil
|
||||
}
|
||||
|
||||
reply, err := r.RouteStream(context.Background(), inbound("", text), DeltaFunc(func(string) {}))
|
||||
_, kind, text, err := drainStreamed(t, streamChat(r), inbound("", text))
|
||||
if err != nil {
|
||||
t.Fatalf("RouteStream: %v", err)
|
||||
t.Fatalf("Stream: %v", err)
|
||||
}
|
||||
if llmCalls != 0 {
|
||||
t.Fatalf("unknown command invoked LLM %d times", llmCalls)
|
||||
}
|
||||
if !strings.Contains(reply, "/frobnicate") || !strings.Contains(reply, "/help") {
|
||||
t.Errorf("reply = %q, want unknown command and /help guidance", reply)
|
||||
if kind != "done" {
|
||||
t.Fatalf("terminal kind = %s, want a local done reply", kind)
|
||||
}
|
||||
if !strings.Contains(text, "/frobnicate") || !strings.Contains(text, "/help") {
|
||||
t.Errorf("reply = %q, want unknown command and /help guidance", text)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,518 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/messaging"
|
||||
)
|
||||
|
||||
// fragment builds one rapid-fire text message carrying the per-person
|
||||
// identity the batch key and the dedup gate read: the channel that brought
|
||||
// it, its conversation, a stable sender, and the channel-native message ID.
|
||||
// Two fragments from the same sender differ only in source ID and text.
|
||||
func fragment(n int, sender, text string) Inbound {
|
||||
return Inbound{
|
||||
Platform: "telegram",
|
||||
Message: messaging.Message{
|
||||
SourceID: fmt.Sprintf("tg-%d", n),
|
||||
ConversationID: messaging.ConversationID{ChannelID: "chat:1"},
|
||||
Sender: sender,
|
||||
SenderID: sender,
|
||||
Role: messaging.RoleUser,
|
||||
Text: text,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// streamOutcome records one streamed caller's rendered events, so the
|
||||
// dispatcher -- started, its deltas, its done -- can be told apart from a
|
||||
// covered fragment whose stream renders nothing at all.
|
||||
type streamOutcome struct {
|
||||
events []ChatEvent
|
||||
err error
|
||||
}
|
||||
|
||||
// drained reads a streamed turn's events until its channel closes. The
|
||||
// deadline is only a hang watchdog: a turn whose dispatch never lands must
|
||||
// fail the test with what it rendered so far, never stall the suite. The
|
||||
// timeout is returned, not raised with FailNow: draining runs on a helper
|
||||
// goroutine, and FailNow from one strands the outcome channel.
|
||||
func drained(events <-chan ChatEvent) ([]ChatEvent, error) {
|
||||
var got []ChatEvent
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
select {
|
||||
case event, ok := <-events:
|
||||
if !ok {
|
||||
return got, nil
|
||||
}
|
||||
got = append(got, event)
|
||||
case <-time.After(time.Second):
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
return got, fmt.Errorf("the streamed turn never closed its events; rendered so far %+v", got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// batchOf reads the open batch for a key, if one is open. Test-only: it is
|
||||
// how tests see a join land without racing a blocking Collect call.
|
||||
func batchOf(ag *TextBatchAggregator, key batchKey) *textBatch {
|
||||
ag.mu.Lock()
|
||||
defer ag.mu.Unlock()
|
||||
return ag.batches[key]
|
||||
}
|
||||
|
||||
// fragmentCount reports how many fragments key's open batch holds, or -1
|
||||
// when no batch is open. The read is taken under the aggregate's lock.
|
||||
func fragmentCount(ag *TextBatchAggregator, key batchKey) int {
|
||||
ag.mu.Lock()
|
||||
defer ag.mu.Unlock()
|
||||
if bat := ag.batches[key]; bat != nil {
|
||||
return len(bat.fragments)
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// waitForFragments polls until key's batch holds want fragments.
|
||||
func waitForFragments(t *testing.T, ag *TextBatchAggregator, key batchKey, want int) {
|
||||
t.Helper()
|
||||
deadline := time.Now().Add(2 * time.Second)
|
||||
for time.Now().Before(deadline) {
|
||||
if fragmentCount(ag, key) >= want {
|
||||
return
|
||||
}
|
||||
time.Sleep(time.Millisecond)
|
||||
}
|
||||
t.Fatalf("the batch never registered %d fragments for %v", want, key)
|
||||
}
|
||||
|
||||
// waitForSettled polls until key's batch has settled and left the map.
|
||||
func waitForSettled(t *testing.T, ag *TextBatchAggregator, key batchKey) {
|
||||
t.Helper()
|
||||
deadline := time.Now().Add(2 * time.Second)
|
||||
for time.Now().Before(deadline) {
|
||||
ag.mu.Lock()
|
||||
gone := ag.batches[key] == nil
|
||||
ag.mu.Unlock()
|
||||
if gone {
|
||||
return
|
||||
}
|
||||
time.Sleep(time.Millisecond)
|
||||
}
|
||||
t.Fatalf("the batch for %v never settled", key)
|
||||
}
|
||||
|
||||
// fakeBatchClock is the batcher's injectable clock: Now plus close timers
|
||||
// that fire when Advance moves past their deadline. Batch close times are
|
||||
// tested with this, never wall-clock sleeps; the same shape the curator
|
||||
// runtime's tests use.
|
||||
type fakeBatchClock struct {
|
||||
mu sync.Mutex
|
||||
now time.Time
|
||||
timers []*fakeBatchTimer
|
||||
}
|
||||
|
||||
type fakeBatchTimer struct {
|
||||
c chan time.Time
|
||||
at time.Time
|
||||
}
|
||||
|
||||
func newBatchClock(at time.Time) *fakeBatchClock {
|
||||
return &fakeBatchClock{now: at}
|
||||
}
|
||||
|
||||
func (fc *fakeBatchClock) Now() time.Time {
|
||||
fc.mu.Lock()
|
||||
defer fc.mu.Unlock()
|
||||
return fc.now
|
||||
}
|
||||
|
||||
func (fc *fakeBatchClock) After(d time.Duration) <-chan time.Time {
|
||||
fc.mu.Lock()
|
||||
defer fc.mu.Unlock()
|
||||
t := &fakeBatchTimer{c: make(chan time.Time, 1), at: fc.now.Add(d)}
|
||||
fc.timers = append(fc.timers, t)
|
||||
return t.c
|
||||
}
|
||||
|
||||
// Advance moves the clock forward and fires every timer whose deadline has
|
||||
// passed.
|
||||
func (fc *fakeBatchClock) Advance(d time.Duration) {
|
||||
fc.mu.Lock()
|
||||
fc.now = fc.now.Add(d)
|
||||
var fired []*fakeBatchTimer
|
||||
remains := fc.timers[:0]
|
||||
for _, t := range fc.timers {
|
||||
if !t.at.After(fc.now) {
|
||||
fired = append(fired, t)
|
||||
} else {
|
||||
remains = append(remains, t)
|
||||
}
|
||||
}
|
||||
fc.timers = remains
|
||||
fc.mu.Unlock()
|
||||
for _, t := range fired {
|
||||
select {
|
||||
case t.c <- t.at:
|
||||
default:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestRapidFireFragmentsTakeOneTurn pins the batcher's core contract:
|
||||
// fragments of one thought delivered back to back from one sender must
|
||||
// dispatch exactly one agent turn for the joined text, not one turn per
|
||||
// fragment. The fragments are registered one at a time -- the way a channel
|
||||
// delivers them -- and the batch closes when its quiet window passes.
|
||||
func TestRapidFireFragmentsTakeOneTurn(t *testing.T) {
|
||||
r := NewRouter(nil, nil, "telegram")
|
||||
clock := newBatchClock(time.Now())
|
||||
r.Batches = NewTextBatchAggregator(time.Hour, 0, 0, clock.Now, clock.After)
|
||||
|
||||
var mu sync.Mutex
|
||||
turns := 0
|
||||
var dispatched []string
|
||||
r.LLMStream = func(_ context.Context, in Inbound, stream TurnStream) (string, error) {
|
||||
mu.Lock()
|
||||
turns++
|
||||
dispatched = append(dispatched, in.Message.Text)
|
||||
mu.Unlock()
|
||||
stream.Delta("…")
|
||||
return "the answer", nil
|
||||
}
|
||||
|
||||
chat := &LocalChatAdapter{Router: r}
|
||||
key := batchKey{Platform: "telegram", Conversation: "chat:1", SenderID: "sam"}
|
||||
texts := []string{"first think", "about it", "then act"}
|
||||
outcomes := make(chan streamOutcome, len(texts))
|
||||
for i, text := range texts {
|
||||
go func(i int, text string) {
|
||||
events, err := chat.Stream(context.Background(), fragment(i, "sam", text))
|
||||
if err != nil {
|
||||
outcomes <- streamOutcome{err: err}
|
||||
return
|
||||
}
|
||||
rendered, drainErr := drained(events)
|
||||
outcomes <- streamOutcome{events: rendered, err: drainErr}
|
||||
}(i, text)
|
||||
waitForFragments(t, r.Batches, key, i+1)
|
||||
}
|
||||
clock.Advance(2 * time.Hour)
|
||||
|
||||
dispatchers, covered := 0, 0
|
||||
for range texts {
|
||||
outcome := <-outcomes
|
||||
if outcome.err != nil {
|
||||
t.Errorf("Stream: %v", outcome.err)
|
||||
continue
|
||||
}
|
||||
reply, sawStarted, sawDelta := "", false, false
|
||||
for _, event := range outcome.events {
|
||||
switch event.Kind {
|
||||
case "started":
|
||||
sawStarted = true
|
||||
case "delta":
|
||||
sawDelta = true
|
||||
case "done":
|
||||
reply = event.Text
|
||||
case "error":
|
||||
reply = "error: " + event.Text
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case reply == "the answer":
|
||||
dispatchers++
|
||||
if !sawStarted {
|
||||
t.Error("the turn that ran must open its stream with a started event")
|
||||
}
|
||||
if !sawDelta {
|
||||
t.Error("the fragment whose turn ran must see the reply stream through it, but no delta arrived")
|
||||
}
|
||||
case len(outcome.events) != 0:
|
||||
t.Errorf("a covered fragment's stream rendered %+v, want nothing at all", outcome.events)
|
||||
default:
|
||||
covered++
|
||||
}
|
||||
}
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if turns != 1 {
|
||||
t.Errorf("rapid-fire fragments from one sender started %d agent turns, want 1 (dispatched %q)", turns, dispatched)
|
||||
}
|
||||
if dispatchers != 1 {
|
||||
t.Errorf("%d of %d fragments rendered the turn's own reply, want exactly 1 dispatcher", dispatchers, len(texts))
|
||||
}
|
||||
if covered != len(texts)-1 {
|
||||
t.Errorf("%d fragments rendered nothing, want %d covered ones", covered, len(texts)-1)
|
||||
}
|
||||
if turns == 1 && dispatched[0] != strings.Join(texts, "\n") {
|
||||
t.Errorf("the turn dispatched %q, want the fragments joined in arrival order", dispatched[0])
|
||||
}
|
||||
}
|
||||
|
||||
// ---- component tests ---------------------------------------------------------
|
||||
|
||||
// TestTextBatchAggregatorJoinsThenReleasesOnQuietWindow walks the batcher
|
||||
// directly: fragments join in order, the batch stays open while fragments
|
||||
// keep arriving inside the window (each join re-arms the quiet deadline),
|
||||
// settling releases everyone, and the batch's dispatcher receives the
|
||||
// joined payload under the first fragment's identity. A fragment that
|
||||
// arrives after the batch settled starts a fresh batch.
|
||||
func TestTextBatchAggregatorJoinsThenReleasesOnQuietWindow(t *testing.T) {
|
||||
clock := newBatchClock(time.Date(2026, 9, 30, 12, 0, 0, 0, time.UTC))
|
||||
ag := NewTextBatchAggregator(time.Hour, 0, 0, clock.Now, clock.After)
|
||||
|
||||
key := batchKey{Platform: "telegram", Conversation: "chat:1", SenderID: "sam"}
|
||||
texts := []string{"first think", "about it", "then act"}
|
||||
results := make(chan collectResult, len(texts))
|
||||
for i, text := range texts {
|
||||
go func(i int, text string) {
|
||||
payload, mine, err := ag.Collect(context.Background(), fragment(i, "sam", text))
|
||||
results <- collectResult{payload: payload, mine: mine, err: err}
|
||||
}(i, text)
|
||||
waitForFragments(t, ag, key, i+1)
|
||||
}
|
||||
clock.Advance(2 * time.Hour)
|
||||
|
||||
got := 0
|
||||
for range len(texts) {
|
||||
out := <-results
|
||||
if out.err != nil {
|
||||
t.Errorf("Collect: %v", out.err)
|
||||
continue
|
||||
}
|
||||
got++
|
||||
if out.mine {
|
||||
first := fragment(0, "sam", "")
|
||||
if out.payload.Message.Text != strings.Join(texts, "\n") {
|
||||
t.Errorf("payload text = %q, want the fragments joined in arrival order", out.payload.Message.Text)
|
||||
}
|
||||
if out.payload.Message.SenderID != first.Message.SenderID ||
|
||||
out.payload.Message.SourceID != first.Message.SourceID ||
|
||||
out.payload.Message.ConversationID != first.Message.ConversationID {
|
||||
t.Errorf("payload identity = %v/%v, want the first fragment's", out.payload.Message.SenderID, out.payload.Message.SourceID)
|
||||
}
|
||||
} else if out.payload.Message.Text != "" {
|
||||
t.Error("a covered fragment received a payload, want none")
|
||||
}
|
||||
}
|
||||
if got != len(texts) {
|
||||
t.Errorf("%d of %d fragments were released, want all", got, len(texts))
|
||||
}
|
||||
|
||||
// A fragment that arrives after the batch settled starts a fresh batch
|
||||
// rather than joining a settled one, and its window is its own.
|
||||
next := fragment(10, "sam", "a later thought")
|
||||
fresh := make(chan collectResult, 1)
|
||||
go func() {
|
||||
payload, mine, err := ag.Collect(context.Background(), next)
|
||||
fresh <- collectResult{payload: payload, mine: mine, err: err}
|
||||
}()
|
||||
waitForFragments(t, ag, key, 1)
|
||||
clock.Advance(2 * time.Hour)
|
||||
out := <-fresh
|
||||
if out.err != nil || !out.mine || out.payload.Message.Text != "a later thought" {
|
||||
t.Errorf("a fresh batch's single fragment = %q, mine %v, err %v; want itself dispatched alone", out.payload.Message.Text, out.mine, out.err)
|
||||
}
|
||||
}
|
||||
|
||||
// collectResult pairs what one Collect call learned, for tests.
|
||||
type collectResult struct {
|
||||
payload Inbound
|
||||
mine bool
|
||||
err error
|
||||
}
|
||||
|
||||
// TestTextBatchAggregatorPassesNonFragmentsThrough pins what a batch never
|
||||
// holds: media, a command, a message no person sent, and one with no text
|
||||
// dispatch at once whatever the clock says, and carry their own payload.
|
||||
func TestTextBatchAggregatorPassesNonFragmentsThrough(t *testing.T) {
|
||||
clock := newBatchClock(time.Date(2026, 9, 30, 12, 0, 0, 0, time.UTC))
|
||||
ag := NewTextBatchAggregator(time.Hour, 0, 0, clock.Now, clock.After)
|
||||
key := batchKey{Platform: "telegram", Conversation: "chat:1", SenderID: "sam"}
|
||||
|
||||
media := fragment(1, "sam", "the attached log")
|
||||
media.Message.Media = []messaging.MediaAttachment{{Type: "document"}}
|
||||
command := fragment(2, "sam", "/status")
|
||||
stranger := fragment(3, "", "a route wrote this")
|
||||
empty := fragment(4, "sam", "")
|
||||
|
||||
for _, in := range []Inbound{media, command, stranger, empty} {
|
||||
payload, mine, err := ag.Collect(context.Background(), in)
|
||||
if err != nil {
|
||||
t.Fatalf("Collect(%q): %v", in.Message.Text, err)
|
||||
}
|
||||
if !mine || payload.Message.Text != in.Message.Text {
|
||||
t.Errorf("Collect(%q) = (payload %q, mine %v), want it dispatched as itself", in.Message.Text, payload.Message.Text, mine)
|
||||
}
|
||||
}
|
||||
if batchOf(ag, key) != nil {
|
||||
t.Error("non-fragment messages left a batch open behind them")
|
||||
}
|
||||
|
||||
// A lone plain-text fragment is its own batch: it dispatches once the
|
||||
// quiet window passes, with its payload unchanged.
|
||||
lone := make(chan collectResult, 1)
|
||||
go func() {
|
||||
payload, mine, err := ag.Collect(context.Background(), fragment(5, "lone-person", "just one message"))
|
||||
lone <- collectResult{payload: payload, mine: mine, err: err}
|
||||
}()
|
||||
waitForFragments(t, ag, batchKey{Platform: "telegram", Conversation: "chat:1", SenderID: "lone-person"}, 1)
|
||||
clock.Advance(2 * time.Hour)
|
||||
waitForSettled(t, ag, batchKey{Platform: "telegram", Conversation: "chat:1", SenderID: "lone-person"})
|
||||
out := <-lone
|
||||
if out.err != nil || !out.mine || out.payload.Message.Text != "just one message" {
|
||||
t.Errorf("a singleton = %q, mine %v, err %v; want itself dispatched once the window passed", out.payload.Message.Text, out.mine, out.err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTextBatchAggregatorClosesOnItsBounds pins the early close: a batch
|
||||
// that reaches its fragment bound settles at once, without waiting out the
|
||||
// window, and releases its fragments; a later fragment forms a fresh batch.
|
||||
func TestTextBatchAggregatorClosesOnItsBounds(t *testing.T) {
|
||||
clock := newBatchClock(time.Date(2026, 9, 30, 12, 0, 0, 0, time.UTC))
|
||||
ag := NewTextBatchAggregator(time.Hour, 2, 0, clock.Now, clock.After)
|
||||
key := batchKey{Platform: "telegram", Conversation: "chat:1", SenderID: "sam"}
|
||||
|
||||
results := make(chan collectResult, 2)
|
||||
for i := range 2 {
|
||||
go func(i int) {
|
||||
payload, mine, err := ag.Collect(context.Background(), fragment(i, "sam", fmt.Sprintf("fragment %d", i)))
|
||||
results <- collectResult{payload: payload, mine: mine, err: err}
|
||||
}(i)
|
||||
if i == 0 {
|
||||
waitForFragments(t, ag, key, 1)
|
||||
}
|
||||
}
|
||||
// Both settles happened without any clock movement: the bound, not the
|
||||
// window, closed the batch.
|
||||
waitForSettled(t, ag, key)
|
||||
|
||||
var payloads []string
|
||||
for range 2 {
|
||||
out := <-results
|
||||
if out.err != nil {
|
||||
t.Errorf("Collect: %v", out.err)
|
||||
continue
|
||||
}
|
||||
if out.mine {
|
||||
payloads = append(payloads, out.payload.Message.Text)
|
||||
}
|
||||
}
|
||||
if len(payloads) != 1 || payloads[0] != "fragment 0\nfragment 1" {
|
||||
t.Errorf("the bounded batch dispatched %q from %d dispatchers, want the two fragments joined by its single dispatcher", strings.Join(payloads, "|"), len(payloads))
|
||||
}
|
||||
|
||||
// A fragment arriving after the bound closed its batch waits on a fresh
|
||||
// batch window rather than being attached to nothing.
|
||||
after := make(chan collectResult, 1)
|
||||
go func() {
|
||||
payload, mine, err := ag.Collect(context.Background(), fragment(5, "sam", "after the bound"))
|
||||
after <- collectResult{payload: payload, mine: mine, err: err}
|
||||
}()
|
||||
waitForFragments(t, ag, key, 1)
|
||||
if bat := batchOf(ag, key); bat == nil || len(bat.fragments) != 1 {
|
||||
t.Error("the post-bound fragment did not start its own batch")
|
||||
}
|
||||
clock.Advance(2 * time.Hour)
|
||||
out := <-after
|
||||
if out.err != nil || !out.mine || out.payload.Message.Text != "after the bound" {
|
||||
t.Errorf("the post-bound fragment = %q, mine %v, err %v; want its own batch closed on the window", out.payload.Message.Text, out.mine, out.err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTextBatchAggregatorSeparatesSenders pins the batch key: two senders
|
||||
// in one conversation hold two open batches, and neither dispatches more
|
||||
// text than its sender wrote.
|
||||
func TestTextBatchAggregatorSeparatesSenders(t *testing.T) {
|
||||
clock := newBatchClock(time.Date(2026, 9, 30, 12, 0, 0, 0, time.UTC))
|
||||
ag := NewTextBatchAggregator(time.Hour, 0, 0, clock.Now, clock.After)
|
||||
keyOf := func(sender string) batchKey {
|
||||
return batchKey{Platform: "telegram", Conversation: "chat:1", SenderID: sender}
|
||||
}
|
||||
|
||||
results := make(chan collectResult, 3)
|
||||
go func() {
|
||||
payload, mine, err := ag.Collect(context.Background(), fragment(1, "sam", "did the build pass"))
|
||||
if mine {
|
||||
results <- collectResult{payload: payload, err: err}
|
||||
}
|
||||
}()
|
||||
waitForFragments(t, ag, keyOf("sam"), 1)
|
||||
go func() {
|
||||
payload, mine, err := ag.Collect(context.Background(), fragment(2, "kim", "same here, checking"))
|
||||
if mine {
|
||||
results <- collectResult{payload: payload, err: err}
|
||||
}
|
||||
}()
|
||||
waitForFragments(t, ag, keyOf("kim"), 1)
|
||||
go func() {
|
||||
payload, mine, err := ag.Collect(context.Background(), fragment(3, "sam", "and the tests"))
|
||||
if mine {
|
||||
results <- collectResult{payload: payload, err: err}
|
||||
}
|
||||
}()
|
||||
waitForFragments(t, ag, keyOf("sam"), 2)
|
||||
clock.Advance(2 * time.Hour)
|
||||
|
||||
var texts []string
|
||||
for range 2 {
|
||||
out := <-results
|
||||
if out.err != nil {
|
||||
t.Errorf("Collect: %v", out.err)
|
||||
}
|
||||
texts = append(texts, out.payload.Message.Text)
|
||||
}
|
||||
sam := strings.Join([]string{"did the build pass", "and the tests"}, "\n")
|
||||
for _, text := range texts {
|
||||
switch text {
|
||||
case sam, "same here, checking":
|
||||
default:
|
||||
t.Errorf("senders dispatched %q; want only %q and %q", text, sam, "same here, checking")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestTextBatchAggregatorHandsTheDispatchToAStillWaitingFragment pins the
|
||||
// handover: the caller that opens a batch may leave before the batch
|
||||
// settles, and the batch's turn must still run -- the first fragment still
|
||||
// waiting inherits the dispatch.
|
||||
func TestTextBatchAggregatorHandsTheDispatchToAStillWaitingFragment(t *testing.T) {
|
||||
clock := newBatchClock(time.Date(2026, 9, 30, 12, 0, 0, 0, time.UTC))
|
||||
ag := NewTextBatchAggregator(time.Hour, 0, 0, clock.Now, clock.After)
|
||||
key := batchKey{Platform: "telegram", Conversation: "chat:1", SenderID: "sam"}
|
||||
|
||||
leaderCtx, cancelLeader := context.WithCancel(context.Background())
|
||||
results := make(chan collectResult, 2)
|
||||
go func() {
|
||||
payload, mine, err := ag.Collect(leaderCtx, fragment(0, "sam", "half a thought"))
|
||||
results <- collectResult{payload: payload, mine: mine, err: err}
|
||||
}()
|
||||
waitForFragments(t, ag, key, 1)
|
||||
go func() {
|
||||
payload, mine, err := ag.Collect(context.Background(), fragment(1, "sam", "the other half"))
|
||||
results <- collectResult{payload: payload, mine: mine, err: err}
|
||||
}()
|
||||
waitForFragments(t, ag, key, 2)
|
||||
|
||||
cancelLeader()
|
||||
leader := <-results
|
||||
if leader.err == nil || leader.mine {
|
||||
t.Errorf("the cancelled dispatcher = mine %v, err %v; want its own error", leader.mine, leader.err)
|
||||
}
|
||||
|
||||
clock.Advance(2 * time.Hour)
|
||||
heir := <-results
|
||||
if heir.err != nil || !heir.mine {
|
||||
t.Fatalf("the remaining fragment = mine %v, err %v; want it to inherit the dispatch", heir.mine, heir.err)
|
||||
}
|
||||
if heir.payload.Message.Text != "half a thought\nthe other half" {
|
||||
t.Errorf("the inherited turn dispatched %q, want the whole batch joined", heir.payload.Message.Text)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The deduplicator's delivery window and memory bound. A redelivery
|
||||
// (webhook retry, Telegram long-poll re-fetch after a slow ack, forge
|
||||
// redelivery) reaches the gateway within seconds to a few minutes of the
|
||||
// first delivery, so five minutes clears the fast-path gate without ever
|
||||
// colliding with the durable prior-reply replay in turn.go, which remains
|
||||
// the backstop for a duplicate answered long before or long after the
|
||||
// window. Each entry is one key and a timestamp, so 1024 entries stay
|
||||
// comfortably under a megabyte.
|
||||
const (
|
||||
defaultDedupTTL = 5 * time.Minute
|
||||
defaultDedupCapacity = 1024
|
||||
)
|
||||
|
||||
// dedupKey identifies one platform message delivery: the channel that
|
||||
// carried it, the conversation it addressed, and its channel-native
|
||||
// message ID -- the same tuple a repeat delivery is keyed on. Two
|
||||
// components are not enough: Telegram message IDs are only unique per
|
||||
// chat, and different platforms reuse the same numeric ID ranges.
|
||||
type dedupKey struct {
|
||||
Platform string
|
||||
Conversation string
|
||||
SourceID string
|
||||
}
|
||||
|
||||
// MessageDeduplicator rejects a second delivery of the same platform
|
||||
// message within a TTL window, so at-least-once hand-off (webhook
|
||||
// retries, Telegram long-poll re-fetch, forge redelivery) does not
|
||||
// process one message twice -- a duplicate still inside the window never
|
||||
// reaches the turn pipeline at all.
|
||||
//
|
||||
// Memory is bounded by design, not by chance: at most capacity entries
|
||||
// are held. An insert brings the cache back under the bound by evicting
|
||||
// expired entries first (oldest recorded delivery first, exactly the
|
||||
// entries a TTL sweep would drop), then the oldest still-live delivery
|
||||
// once expired entries alone no longer make room. A rejected repeat
|
||||
// never extends the window: expiry is judged from the first delivery.
|
||||
//
|
||||
// One Router may be reached from several channel goroutines at once, so
|
||||
// every method holds a mutex; Admit's test-and-record is one atomic
|
||||
// step, which is what makes a concurrent duplicate see the race and
|
||||
// lose it.
|
||||
//
|
||||
// Admit runs before the turn, so a message whose first processing fails
|
||||
// stays inside the window until it expires; redeliveries of failed work
|
||||
// wait out the TTL, and turn.go's prior-reply path covers anything
|
||||
// that was actually completed in the meantime.
|
||||
type MessageDeduplicator struct {
|
||||
mu sync.Mutex
|
||||
// ttl is the delivery window; cap is the live-entry bound.
|
||||
ttl time.Duration
|
||||
cap int
|
||||
// now injects the clock, so expiry is testable without sleeping.
|
||||
now func() time.Time
|
||||
// entries maps each live key to when it was delivered; order is the
|
||||
// FIFO eviction queue, oldest delivery first. A live entry always has
|
||||
// at least one queue slot; a queue slot may outlive its entry after
|
||||
// an expiry re-arm and is dropped on the next sweep.
|
||||
entries map[dedupKey]time.Time
|
||||
order []dedupKey
|
||||
}
|
||||
|
||||
// NewMessageDeduplicator returns a gate with the given TTL window and
|
||||
// entry bound. Zero values select the package defaults; now injects the
|
||||
// clock, so tests judge expiry without sleeping. A nil now means
|
||||
// time.Now, which is the production shape.
|
||||
func NewMessageDeduplicator(ttl time.Duration, capacity int, now func() time.Time) *MessageDeduplicator {
|
||||
if ttl <= 0 {
|
||||
ttl = defaultDedupTTL
|
||||
}
|
||||
if capacity <= 0 {
|
||||
capacity = defaultDedupCapacity
|
||||
}
|
||||
if now == nil {
|
||||
now = time.Now
|
||||
}
|
||||
return &MessageDeduplicator{
|
||||
ttl: ttl,
|
||||
cap: capacity,
|
||||
now: now,
|
||||
entries: make(map[dedupKey]time.Time),
|
||||
}
|
||||
}
|
||||
|
||||
// Admit records this delivery and reports whether the message may be
|
||||
// processed: true for the first delivery inside the window (or the first
|
||||
// after an earlier one expired), false for a repeat that is already
|
||||
// inside the window. A message with no channel-native ID (its SourceID
|
||||
// is empty -- gateway-local commands, most dashboard turns) has no key to
|
||||
// deduplicate against and is always admitted.
|
||||
func (d *MessageDeduplicator) Admit(in Inbound) bool {
|
||||
if in.Message.SourceID == "" {
|
||||
return true
|
||||
}
|
||||
key := dedupKey{
|
||||
Platform: in.Platform,
|
||||
Conversation: in.Message.ConversationID.String(),
|
||||
SourceID: in.Message.SourceID,
|
||||
}
|
||||
|
||||
d.mu.Lock()
|
||||
defer d.mu.Unlock()
|
||||
at := d.now()
|
||||
if seen, live := d.entries[key]; live {
|
||||
if at.Sub(seen) < d.ttl {
|
||||
return false
|
||||
}
|
||||
// Expired: re-arm from the fresh delivery time by dropping the
|
||||
// stale queue slots for the key, then recording anew below.
|
||||
d.order = slices.DeleteFunc(d.order, func(k dedupKey) bool { return k == key })
|
||||
delete(d.entries, key)
|
||||
}
|
||||
d.evictLocked(at)
|
||||
d.entries[key] = at
|
||||
d.order = append(d.order, key)
|
||||
return true
|
||||
}
|
||||
|
||||
// evictLocked brings the cache under its bound ahead of one insert: it
|
||||
// sweeps every expired entry oldest-first, then evicts the oldest
|
||||
// still-live delivery. len(order) always covers every live entry, so the
|
||||
// live-entry loop below has a queue slot to evict from.
|
||||
func (d *MessageDeduplicator) evictLocked(at time.Time) {
|
||||
d.order = slices.DeleteFunc(d.order, func(k dedupKey) bool {
|
||||
seen, live := d.entries[k]
|
||||
if !live {
|
||||
return true
|
||||
}
|
||||
if at.Sub(seen) >= d.ttl {
|
||||
delete(d.entries, k)
|
||||
return true
|
||||
}
|
||||
return false
|
||||
})
|
||||
for len(d.entries) >= d.cap {
|
||||
key := d.order[0]
|
||||
d.order = d.order[1:]
|
||||
delete(d.entries, key)
|
||||
}
|
||||
}
|
||||
|
||||
// dedupReply is returned to a sender whose message arrived again inside
|
||||
// the dedup window. Like the rate-limit reply it is prose addressed to a
|
||||
// human reader of the chat; a webhook caller should treat any successful,
|
||||
// non-original reply the same way it treats rate-limit prose (see
|
||||
// archie-core-1173).
|
||||
const dedupReply = "I already received this message, so I'm ignoring this repeat delivery."
|
||||
Loaded 100 of 225 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user