merge(sams-cleanup): the PocketBase event-capture migration and its follow-ups

Moves event capture, mappings and bindings onto PocketBase-owned collections
in internal/infrastructure/edastore, and adds the read-only stateadmin
surface over the task database. Follow-ups included:

- the event feed cursor becomes a total order over a fixed-width timestamp
  with an opaque string cursor, so it no longer depends on rowid
  monotonicity;
- cron schedules get next-run arithmetic, so a cron job fires instead of
  returning ErrScheduleUnsupported;
- binding and mapping edits announce on the activity stream, so a dashboard
  edit is visible without a manual refresh;
- an unresolvable task identity fails closed rather than falling back to the
  root forge's credential;
- tool_call events project into the tool_calls collection, which had a
  schema and no writer.

Also: the branch is brought back to a passing task check, which it had not
been for the whole migration -- two pre-existing failures, an unrecorded
int64-to-string proto break and a stale generated docs artifact.
This commit is contained in:
sam committed 2026-09-23 03:33:49 +10:00
commit e231ef0786
152 files changed
+5479 -6880

No files matched your search

+1 -1
View File
@@ -158,7 +158,7 @@ Dolt remote is pushed explicitly (`bd dolt push`) and never assumed to be in syn
`.pi/control-plane-run/` holds a run's scaffold, lane plan, prompts, scripts and
gate output. It is gitignored and disposable. Treat it as run evidence only: a
fact worth keeping goes into this skill, `docs/`, or `ARCHITECTURE.md`, not into a
fact worth keeping goes into this skill or `docs/`, not into a
run directory the next clean deletes.
## Worked example
@@ -63,13 +63,13 @@ Do not simplify these facts into the target model:
| Workflow | `internal/domain/workflow` routes a `task.Task` into a `map[string]Workflow`, mutates a broad `TaskContext` through sequential stages. | `internal/domain/workflow/workflow.go` |
| Agent-stage boundary | Versioned `agentexec.Request`/`Result` carries one autonomous stage; validation correlates task, attempt, and stage. | `internal/agentexec/protocol.go` |
| Container handoff | `internal/taskrun` carries entire task plus `config.Repo` and `config.TaskConfig`; `internal/app/agentworker` owns routing and sequencing while its NATS adapter owns wire mechanics. | `internal/taskrun/taskrun.go`; `internal/app/agentworker/`; `internal/infrastructure/agenttransport/nats/` |
| Git ownership | Daemon-owned worktree operations clone, commit, push, diff, and clean up. | `internal/worktree/`; `internal/domain/workflow/steps.go`; `ARCHITECTURE.md` |
| Git ownership | Daemon-owned worktree operations clone, commit, push, diff, and clean up. | `internal/worktree/`; `internal/domain/workflow/steps.go`; `docs/architecture/agent-system.md` |
| Enforcement | Gates, read-only/protected paths, TDD inverted test gates, and diff caps are represented outside prompt prose. | `internal/gate/`; `internal/domain/workflow/{agent,tdd,steps}.go` |
| Configuration | `internal/config.Config`, `IdentityConfig`, `Repo`, and task snapshots mix input decoding with runtime concerns. Decoding itself lives in `internal/infrastructure/configuration`. | `internal/config/config.go`; `internal/infrastructure/configuration/` |
| Identity | Configured names and bot usernames act as process-local identity keys. There is no durable Identity domain. | `internal/daemon/daemon.go`; `internal/domain/storecontract/storecontract.go`; `internal/gateway/session.go` |
| Messaging | `internal/gateway` contains both the `Message` struct flow and richer `MessageEvent`; channel routing and task control are mixed with conversational behavior. | `internal/gateway/{gateway,messageevent,tasks}.go`; `internal/channels/` |
| Plugins | `plugin.Plugin` exposes only `Name` and `Version`; current loading interprets operator-installed Go files with Yaegi. | `internal/plugin/plugin.go` |
| Processes | Agent modes include in-process, subprocess, and NATS. Optional containers add a distinct boundary with known gaps. | `internal/config/config.go`; `internal/app/archied/`; `ARCHITECTURE.md` |
| Processes | Agent modes include in-process, subprocess, and NATS. Optional containers add a distinct boundary with known gaps. | `internal/config/config.go`; `internal/app/archied/`; `docs/architecture/organisation.md` |
### Interpret the current lifecycle honestly
@@ -98,13 +98,13 @@ not an open design.
| Invariant | Required action | Status and source |
| --- | --- | --- |
| The model never runs git | Keep clone, branch, commit, push, diff, and cleanup in deterministic workspace/worktree steps. | CURRENT: `ARCHITECTURE.md`; `internal/worktree/` |
| Deterministic constraints outrank prompts | Enforce gates, protected paths, read-only stages, test protection, and change caps outside model instructions. | CURRENT: `ARCHITECTURE.md`; `internal/gate/`; `internal/domain/workflow/` |
| Agent execution is a data boundary | Send bounded, versioned input; validate correlated output before applying it. | CURRENT foundation: `internal/agentexec/protocol.go`; `ARCHITECTURE.md` |
| The model never runs git | Keep clone, branch, commit, push, diff, and cleanup in deterministic workspace/worktree steps. | CURRENT: `docs/architecture/agent-system.md`; `internal/worktree/` |
| Deterministic constraints outrank prompts | Enforce gates, protected paths, read-only stages, test protection, and change caps outside model instructions. | CURRENT: `docs/architecture/agent-system.md`; `internal/gate/`; `internal/domain/workflow/` |
| Agent execution is a data boundary | Send bounded, versioned input; validate correlated output before applying it. | CURRENT foundation: `internal/agentexec/protocol.go`; `docs/architecture/agent-system.md` |
| Every workflow ends explicitly | Preserve terminal outcome or visible park. Preserve crash recovery of interrupted `running` work. | CURRENT: `internal/domain/workflow/workflow.go`; `internal/store/store.go` |
| Generic plugins stay metadata-only | Keep `plugin.Plugin` at `Name()`/`Version()`. Behavior on typed capability interface with owning `Registry`/`Manager`. | CURRENT-enforced rule: `internal/plugin/architecture_test.go` |
| Capability engines own lifecycle | Give resource-owning engines explicit start/health/stop semantics and failure isolation. | APPROVED rule: `ARCHITECTURE.md#plugin-engine-rule-strict` |
| Trust is explicit | Treat trusted operator-installed in-process code, repository code, out-of-process integrations, and container-isolated code as different trust classes. | APPROVED rule: `ARCHITECTURE.md#plugin-engine-rule-strict` |
| Capability engines own lifecycle | Give resource-owning engines explicit start/health/stop semantics and failure isolation. | APPROVED rule: `docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict` |
| Trust is explicit | Treat trusted operator-installed in-process code, repository code, out-of-process integrations, and container-isolated code as different trust classes. | APPROVED rule: `docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict` |
| Infrastructure stays replaceable | Keep forge, persistence, workspace, transport, container, and model details behind behavior-owned contracts. | APPROVED TARGET: `docs/architecture/dependencies-and-contracts.md` |
| Same-repository execution serializes by default | Preserve default unless explicit scheduling policy permits concurrency. | CURRENT: `internal/config/config.go`; `internal/daemon/daemon.go` |
+1 -1
View File
@@ -26,7 +26,7 @@ No root `README*` or `CONTRIBUTING*` file exists. Begin with:
```bash
sed -n '1,240p' CLAUDE.md
sed -n '1,240p' ARCHITECTURE.md
ls docs/architecture/
task --list
```
@@ -1,6 +1,6 @@
---
name: archie-docs-and-writing
description: Maintain Archie's documentation of record and write durable architecture decisions, migration plans, parity matrices, incident and dead-end records, feature ownership and deprecation records, operational runbooks, and documentation reviews. Use when deciding where an Archie fact belongs; reconciling code with ARCHITECTURE.md, CLAUDE.md, docs/prds, docs/archive, or generated contracts; changing tools/docsgen; documenting a feature so future maintainers can find its owners, consumers, invariants, superseded paths, deletion gates, evidence, and rollback; or checking documentation authority, drift, links, copied content, and source-artifact hygiene.
description: Maintain Archie's documentation of record and write durable architecture decisions, migration plans, parity matrices, incident and dead-end records, feature ownership and deprecation records, operational runbooks, and documentation reviews. Use when deciding where an Archie fact belongs; reconciling code with `CLAUDE.md`, `docs/architecture`, `docs/prds`, `docs/archive`, or generated contracts; changing `tools/docsgen`; documenting a feature so future maintainers can find its owners, consumers, invariants, superseded paths, deletion gates, evidence, and rollback; or checking documentation authority, drift, links, copied content, and source-artifact hygiene.
---
# Maintain Archie documentation
@@ -42,7 +42,6 @@ Verified on 2026-09-18:
|---|---|---|
| Live Go code, tests, `Taskfile.yml`, executable config parsing, composition under `internal/app/` (entered from `cmd/`) | `CURRENT` execution evidence | Trace producers and consumers; tests prove only asserted behavior |
| `CLAUDE.md` | Current contributor and safety protocol plus compact architecture orientation | Verify operational claims against code and `Taskfile.yml`; `AGENTS.md` is a symlink to it |
| `ARCHITECTURE.md` | Useful architecture history and partial current overview | Corroborate every inventory/status claim; its "Planned" list is stale for implemented skill support |
| `docs/prds/01-project-management.md` | Index and approved foundation for target architecture | Add or change target decisions in the focused document it names |
| `docs/architecture/*.md` | Focused target decisions, active review procedure, migration inventory | Read each file's status |
| `docs/architecture/migration-decisions.md` | `OPEN` migration inventory constrained by approved decisions | Close a question only after code-grounded review |
@@ -77,7 +76,7 @@ The repository root has no `README.md` or `CONTRIBUTING.md` as of 2026-09-18.
| Superseded design retained for context | `docs/archive/`, with historical status |
| Incident or rejected approach | `archie-failure-archaeology`'s chronology |
Do not duplicate a package map across `CLAUDE.md`, `ARCHITECTURE.md`, a PRD, and
Do not duplicate a package map across `CLAUDE.md`, a PRD, and
the site. Name the authoritative map and link to it.
## Write for future change
-436
View File
@@ -1,436 +0,0 @@
# archie-core Architecture
**archied** is a resident daemon that discovers forge work — GitHub and Gitea
issues assigned to its bot user, carrying its label, or either (`dispatch`
mode, per identity) — by polling (the default) or, single-identity
deployments only, reacting to forge webhook events the instant they arrive
(`Forge.Intake` = `webhook`/`both`, see `docs/prds/event-sources-and-
reactions.md`), works each matched issue in an isolated worktree through a
routed workflow, and opens pull requests for human review. A single daemon can
serve several identities across several forges, each with its own `bot_user`,
label, and repository set.
## Package Map
This lists the load-bearing packages, not all of them — `internal/` holds ~36.
The domain migration is in progress (`internal/domain/`,
`internal/infrastructure/`), so **the tree is authoritative over this table**.
See `docs/architecture/organisation.md` for the target structure and
`docs/architecture/migration-decisions.md` for the current position.
| Package | Role |
| ---------------------------- | ------------------------------------------------------------------------- |
| `cmd/archied/` | Binary entry: config→forge→store→runtime→workflows→daemon |
| `cmd/archie-agent/` | Binary entry: flags, environment, signals, and agentworker invocation |
| `internal/app/agentworker/` | Agent worker composition, lifecycle, routing, and workflow execution |
| `internal/infrastructure/agentboot/` | Worker boot-brief filesystem decoding |
| `internal/infrastructure/agentgit/` | Worker git safe-directory setup |
| `internal/infrastructure/agenttransport/nats/` | Worker core-NATS full-task handoff and RPC client construction |
| `internal/config/` | Config types only — loading lives in `infrastructure/configuration` |
| `internal/daemon/` | Resident loop: poll, enqueue, claim, route, process, reconcile |
| `internal/domain/workflow/` | Engine: stage lists, routing, shared steps, workflow definitions |
| `internal/domain/workflow/task/` | Task-execution vocabulary (`Task`, statuses, mid-run `Store`, `Definition`) importable without the engine |
| `internal/domain/messaging/` | Canonical messages plus the chat wire contracts (`ChatContract`, sessions, turns, streaming) shared by gateway, channels, webui, and archie-ui |
| `internal/channels/status/` | Operator-facing channel lifecycle ledger (`State`/`Descriptor`/`Status`/`Manager`); the plugin contract stays in `internal/channels` |
| `internal/app/archiemessaging/` | Messaging Service composition: Telegram/email/webhook channels, dialling the Gateway's `ChatContract` over gRPC. The daemon runs no channel and builds no `gateway.Router` |
| `internal/domain/workintake/`| `TaskEnvelope`, routing `Kind`, label vocabulary, task subjects |
| `internal/agentexec/` | Worker-local stage protocol and ai-sdk loop runner |
| `internal/forge/` | Forge interface: GitHub and Gitea implementations |
| `internal/forge/webhook/` | Optional forge webhook receiver: GitHub issue events → `workintake.TaskEnvelope`, same publish path as polling |
| `internal/webhookguard/` | Cross-cutting inbound-webhook mechanics: HMAC verification, per-source rate limiting |
| `internal/worktree/` | Git operations: clone, branch, commit, push, diff, cleanup |
| `internal/store/` | Task/capture/mapping/binding SQLite implementation; producer-owned contract interfaces live in `internal/domain/storecontract` (aliases kept here) |
| `internal/domain/storecontract/` | Producer-owned State Store read contracts consumed by webui, intake, and the gRPC adapters (state-store-contract rev. 2d) |
| `internal/events/` | In-process event bus: publish/subscribe for observability |
| `internal/infrastructure/` | Config loading, eventbus transport, model catalog |
| `internal/webui/` | Shared dashboard HTTP layer served by the archie-ui process |
| `internal/gateway/` (18.7k LOC) | Persistent-connection layer between archie and its users (Telegram, web UI, etc.); shared CommandRouter for dispatch |
| `internal/tools/` (20.9k LOC) | Central tool subsystem: built-in tool implementations, approval, execution contract |
| `internal/domain/memory/` | Memory engine contract: four typed scopes, CRUD + revisions, content scanner |
| `internal/infrastructure/memory/` | Builtin memory engine + its `builtin/` markdown store |
| `internal/secret/` | Secret reference resolution across named engines (env, vault, etc.) |
| `internal/plugin/` | Core plugin interface: Yaegi-interpreted `.go` plugins loaded from `~/.config/archie/plugins/` and registered with the daemon's extension registry |
| `internal/skill/` | Parses and discovers agentskills.io `SKILL.md` files |
| `ai-sdk/runtime` | External: provider-agnostic LLM runtime |
| `ai-sdk/agentloop` | External: gated agent loop with budgets and tool jails |
| `ai-sdk/core` | External: tool definitions, generate options |
## Task Lifecycle
```
queued → running(workflow:stage) → pr_open → merged | rejected
↓
waiting_human → (human approves) → queued(implement)
↓
parked
```
**Crash recovery:** tasks left in `running` are re-queued on startup. Parks are
never silent, but they no longer comment on the forge issue: the reason is
recorded on the task as `park_reason` (cleared on requeue) and surfaced by the
dashboard and `/api/tasks`. Retries are an explicit operator action capped by
`max_retries`, not a label a human removes.
**Adversarial review:** the implement workflow runs a fresh adversarial review
after the deterministic gates and before `pr_open` (see
`docs/architecture/adversarial-review.md`). A surviving blocking finding — or a
review that failed to run — parks the task with the findings as its
`park_reason`; otherwise the PR opens with a findings section on its body.
Opt-in per repo via `repo.review_enabled`.
**PR reconciliation:** the daemon polls open PRs, checks GitHub state
(`merged`/`closed`), transitions the task, and cleans up worktrees.
**Provenance by origin:** a task's origin determines which enqueue path
created it, though every origin converges on the same `tasks` table and the
same `Route`/`Run` lifecycle above once queued. Forge-sourced tasks — whether
discovered by polling or, single-identity only, delivered by the forge
webhook receiver (`internal/forge/webhook`) — enqueue via the same
`workintake.TaskEnvelope` and `PublishTask` path (see `docs/prds/event-sources-
and-reactions.md`'s decided contract); `TaskEnvelope.IdempotencyKey` is keyed
on `owner/repo/number`, not delivery source, so a webhook-delivered issue and
a later poll of the same issue dedup for free instead of double-enqueuing.
Playbook-binding-dispatched tasks enqueue
via `EnqueueChatTask`/`EnqueueBindingTask` (`internal/store`), the same direct
path chat-spawned tasks use, stamped with `binding_id`/`binding_version` for
provenance. See `docs/architecture/bindings.md` for the binding model and its
threat model in full.
## Workflow Engine
### Core types (`workflow.go`)
- `Workflow` = named list of `Stage`s
- `Stage` = `Name` + `Run(ctx, *TaskContext) error`
- `TaskContext` = mutable bag: Task, Repo, Config, Forge, Store, Agent runner,
worktree dir/branch, scratch fields
- `Outcome` = terminal decision (status + detail) -- ends the workflow
immediately
- `Registry` = `map[string]Workflow`
### Routing (`Route`)
Label-driven, no LLM involved. Pre-assigned workflow wins; then labels decide.
The built-in bindings are:
- `bug` → `tdd`
- `feature` → `feasibility`
- `bootstrap` → `bootstrap` (diagnostics, no LLM spend)
- Default → `implement`
Both the closed `bug`/`feature`/`bootstrap` set's target workflow and
arbitrary additional labels are YAML-overridable at daemon startup
(`internal/domain/workflow/routing.go`'s `LoadKindWorkflowsYAML`/
`LoadLabelWorkflowsYAML`, `WorkflowRoutingFile`/`WorkflowLabelsFile`/
`PlaybookDirs` config) -- see `docs/prds/eda-playbook-engine.md`. A label
already owned by the closed kind set, or a binding declared twice across
sources, is a reported load failure, not a silent override.
### Execution (`Run`)
Stages execute sequentially. After each stage:
- Task is persisted to SQLite
- Error → park the task (comment on issue, keep worktree)
- Outcome set → finish (transition task, update labels)
- Neither → continue to next stage
A workflow that ends without an outcome parks -- definition bugs must not vanish
silently.
## Workflows
### Bootstrap (diagnostics, no LLM)
```
prepare → apply(marker file) → commit-push → diff-cap → open-pr
```
Proves the plumbing: clone, push, PR. Label any test issue `bootstrap` to verify
end-to-end.
### Implement (default)
```
prepare → baseline-gate → plan(planner, read-only) → build(builder, gated) → commit-push → diff-cap → open-pr
```
- **baseline-gate:** runs the repo's gate at the base commit before anything is
touched
- **plan:** read-only agent explores codebase, produces implementation plan,
posts it as a comment
- **build:** full toolset agent implements against the plan behind the repo's
quality gate
- **diff-cap:** parks oversized changes (>400 lines default)
- **open-pr:** builder's summary becomes the PR body
### TDD (bug label)
```
prepare → baseline-gate → analyse(planner) → repro-tests(builder, inverted gate) → capture-proof → commit-repro → fix(builder, normal gate, test files protected) → commit-push → diff-cap → open-pr + repro-evidence
```
- **repro-tests:** the quality gate is **inverted** -- the test command must
FAIL. Proven failure is required.
- **capture-proof:** runs the test command deterministically, captures the
failing output
- **commit-repro:** commits the failing tests as the first commit
- **fix:** test files are **environmentally write-protected** (not
prompt-ruled). The agent fixes code, not tests.
- **open-pr + evidence:** posts both the PR and the captured failure proof
### Feasibility (feature label)
```
prepare → assess(planner, with decide tool) → [close_if_bad_fit | prd(planner)] → deliver → waiting_human
```
- **assess:** reads AGENT.md, ROADMAP.md, README. Calls a `decide` tool
(structured output, not parsed prose). Bad fit → close with reasons.
- **prd:** read-only agent writes a PRD: problem, solution, files affected,
acceptance criteria, non-goals, diff estimate
- **deliver:** posts PRD as issue comment + webhook notification (n8n → email).
Sets state to `waiting_human`.
- The daemon watches for human replies, judges them with a triage LLM
(APPROVE/REJECT/UNCLEAR), requeues or closes.
## Forge Interface
A thin abstraction over the forge. GitHub (`github.go`) and Gitea (`gitea.go`)
both implement it, plus a no-op (`noop.go`) used when no credential resolves —
see the degradation rule in `docs/architecture/configuration.md`. A multi-forge
deployment is configured per identity; see
`deployments/multi-forge-github-gitea.toml`.
| Method | Purpose |
| ------------------- | --------------------------------------- |
| `AcceptInvitations` | Auto-accept repo invites |
| `AssignedIssues` | Poll assigned issues (excluding PRs) |
| `IssuesWithLabel` | Poll labelled issues |
| `Comment` | Post issue/PR comment, return ID |
| `RepliesAfter` | Find human replies after a comment ID |
| `CreatePR` | Open pull request |
| `PRState` | Check merged/open/closed |
| `CloseIssue` | Close with final comment |
| `React` | Emoji reaction (pickup acknowledgement) |
| `SetStateLabel` | Replace old state label with new one |
| `VerifyPush` | Confirm push permission at startup |
State labels (`archie:queued`, `archie:working`, etc.) are created on demand,
removed when transitioning. They report state; they do not drive it. Removing
a label no longer retries a parked task -- retries are an explicit operator
action from the dashboard or chat, capped by `max_retries`.
### Webhook intake (alternative to polling)
`internal/forge/webhook` is a separate HTTP receiver, not a `Forge` method: it
verifies a GitHub webhook's HMAC signature (`internal/webhookguard.VerifyHMAC`),
decodes an `issues` event, applies the same dispatch predicate the poller
uses, and calls the same `PublishTask` the poller calls. It is opt-in via
`Forge.Intake` (`poll` default, `webhook`, `both`) and single-identity only: a
config naming `webhook`/`both` together with `[[identities]]`, or setting
intake on an identity, is rejected at validation rather than started with the
setting ignored -- the receiver binds one secret and one dispatch predicate,
so it cannot judge a second identity's events by the right rules. A
multi-identity deployment polls per identity until per-identity intake lands.
See `docs/prds/event-sources-and-reactions.md` for why this stayed a
thin decode-and-reuse layer rather than a second intake mechanism.
## Worktree Manager
Every task gets a fresh clone -- no persistent state between runs. The model
never runs git.
```
~/.local/share/archie/work/<owner>-<repo>/issue-<N>/
```
Token auth via a generated `GIT_ASKPASS` helper script -- the token never
appears in `.git/config` or process argv.
## Config
Daemon-level TOML (`~/.config/archie/config.toml`):
- `[forge]` -- type (github/gitea), host, token secret reference; `intake`
(`poll` default, `webhook`, or `both`) with `webhook_secret`/`webhook_addr`
when reacting to forge events instead of, or alongside, polling
(single-identity deployments only; rejected, not ignored, alongside
`[[identities]]` or on a `[identities.forge]` entry)
- `[dispatch]` -- trigger (assignee/label/either), labels, ack reaction
- `[[repos]]` -- owner, name, base branch, gate commands, protected paths,
ecosystem
- `[models]` -- planner, builder, triage (provider/model refs)
- `[providers]` -- LLM provider configs
- `[nats]` -- embedded or external broker deployment
- `[containers]` -- managed `archie-agent` image, limits, storage, and network
- `[budgets]` -- max steps, max tokens, wall clock, gate max failures
- `[web]` -- dashboard listen address, bound by the standalone archie-ui
process (the daemon runs no dashboard listener after the UI cutover)
- `[notify]` -- webhook URL for notifications
Secrets (forge token, provider API keys, channel tokens) are not embedded in
config files. `internal/secret` defines a pluggable `Engine` interface (`Name`,
`Version`, `Resolve(key) (string, error)`) resolved through a `Registry`; the
built-in `env` engine reads an environment variable by name and the built-in
`bws` engine caches the accessible Bitwarden project at startup. Custom engines
(sops, Vault/OpenBao) are Yaegi-interpreted `.go` plugins loaded from a
directory, following the same convention as `internal/plugin`'s daemon plugins.
Config fields reference a secret as
`{engine, key}` (`secret.SecretRef`), e.g.
`token = { engine = "env", key = "ARCHIE_GITHUB_TOKEN" }`. As of this writing
`Forge.Token` and `Provider.APIKey` use `SecretRef`; resolved provider values
are exported under private, identity-scoped environment names because the SDK
and container boundary consume credential names rather than
plaintext wire fields. The older `Provider.APIKeyEnv`, `NATSConfig.TokenEnv`, and
`TelegramConfig.TokenEnv` field names remain supported.
Per-repo gates are command lists: `[[repos.gate]] = ["go", "vet", "./..."]`. The
last command is the test runner (by convention -- TDD inverts it).
## Observability
Every lifecycle event flows through an internal event bus:
- SQLite event log (every event stamped with a row ID)
- Live dashboard via SSE (server-sent events)
- Bounded subscriber buffers -- slow consumers drop events, never block the
daemon
## Plugin engine rule (strict)
“Engine” is reserved for a typed, lifecycle-managed capability family. It is not
a synonym for a generic plugin hook. Every new plugin engine must satisfy all of
these invariants:
1. **Typed domain contract.** The engine interface exposes real
capability-specific operations in addition to identity metadata. Examples are
resolving a secret, delivering a message, or running an agent. A plugin that
only exposes `Name` and `Version` is metadata, not an engine.
2. **Owning family manager.** The capability package owns a `Registry` or
`Manager` that controls provider registration and discovery, initialization,
or access. Providers do not register arbitrary callbacks directly on the
daemon.
3. **Lifecycle and health.** When an implementation owns resources or background
work, its typed family contract must expose start, health, and stop
semantics. The owning manager invokes them and defines failure isolation and
shutdown ordering. Stateless providers may make these operations explicit
no-ops.
4. **Narrow host access.** Plugins receive the typed family registrar and only
the host services their declared capability requires. They never receive
`*daemon.Daemon`, a service locator, or an untyped hook map.
5. **Metadata-only generic plugin contract.** `plugin.Plugin` remains limited to
`Name()` and `Version()`. Domain hooks belong on typed engine interfaces;
adding capability methods to the generic interface is forbidden.
6. **Trust boundary stays explicit.** Explicitly trusted, operator-installed
plugins—including third-party secret-engine plugins the operator chooses to
trust—may run in-process and have daemon privileges. Repository-supplied code
runs in the task container. Integrations that are not trusted as daemon code
run out of process through MCP or a versioned protocol and, when a
confidentiality or integrity boundary is required, inside a real container
sandbox. A subprocess boundary alone is not a security sandbox, and calling
Yaegi-loaded code a plugin does not isolate it.
`internal/plugin/architecture_test.go` enforces the mechanically testable parts
of this rule: the generic plugin method set, the shared agent-instruction
source, and the requirement that engine interfaces have domain behavior plus an
owning registry or manager.
### Memory engine family
One engine, addressed by four typed scopes -- `ScopeGlobal`, `ScopeAgent`,
`ScopeUser`, `ScopeAgentUser` (`internal/domain/memory`, `Scope`/`ScopeKind`).
A caller names exactly the scopes it may read or write; the engine applies no
access policy of its own, so isolation falls out mechanically because
`ScopeAgent{a}` and `ScopeAgent{b}` are different storage keys. See
`docs/prds/memory-engine-unification.md` for the decision this contract
implements; it superseded `internal/memory` (`MemoryProvider`/`Manager`),
deleted in that PRD's slice 5.
`MemoryEngine` (`internal/domain/memory/contract.go`) is the typed family
contract: `Create`, `Get`, `Query`, `List`, `Update`, `Forget`, `Revisions` --
CRUD with revisions retained, not just create/read/delete. `Update` supersedes
rather than overwrites: the superseded state is appended to the scope's
`HISTORY.md` before the live block is replaced, and `Update` carries no
`Scope` field, so a record can never change scope. The family's content
scanner (prompt-injection / sensitive-data patterns) is applied inside
`Create` and `Update`, the single choke point every producer crosses.
`internal/domain/memory.Registry` is the owning family registry: registration,
start/health/stop, shutdown ordering, and failure isolation, following the
plugin engine rule (`ARCHITECTURE.md#plugin-engine-rule-strict`).
`internal/infrastructure/memory.NewBuiltinEngine` is the only registered
engine today; it persists through `internal/infrastructure/memory/builtin`,
a markdown store (`MEMORY.md`-shaped per-scope files) that owns its on-disk
format end-to-end.
Two per-turn seams in `internal/gateway` connect a chat turn to the engine,
each narrowed to exactly the operations it needs:
- **Read** (`turn_memory.go`, `MemoryStore`): `prepareTurn` runs a
scope-only `Query` over the resolved `Subject`'s readable scopes,
synchronously and before the prompt is built. It never returns an error --
a panicking or failing engine degrades to an empty `<memory>` block rather
than failing the turn -- and is bounded by a per-scope record limit and a
byte cap on the rendered block.
- **Write** (`turn_memory_tool.go`, `MemoryWriteStore`): four tools --
`memory_create`, `memory_update`, `memory_delete`, `memory_list` -- built
per turn onto the resolved `Subject`'s writable scopes, not registered once
at boot. The model chooses the scope kind; it never chooses the agent or
user id, which come from the turn's resolved `Subject` and have no schema
field to override. `ScopeGlobal` is never model-writable.
`boot.setupMemoryEngine` (`internal/app/archied/bootstrap.go`) registers and
starts the engine and must run before `setupGatewayChat`, which constructs the
only chat turn runner in production and captures the engine at construction
time (`internal/app/archied/composition_order_test.go` pins the ordering), and
before `setupCurators`, whose `Registrar` captures the engine registry. The session curator (`internal/infrastructure/sessioncurator`)
writes derived records onto `ScopeAgentUser`, making it the engine's other
live producer besides the chat write path.
## Key Design Decisions
1. **Environmental constraints over prompt rules.** The gate, test-file
protection, and diff cap are code enforcement -- not "please don't do that"
in the system prompt.
2. **The model never runs git.** Worktree operations are deterministic steps,
not agent tools.
3. **Tokens as cost, not features.** Token usage is tracked per task and emitted
in observability events but never budgeted beyond the agent loop's own
limits.
4. **Label-driven routing.** Workflow selection is a switch statement, not an
LLM call. A triage LLM is used only for the human reply judgment in
waiting_human.
5. **Shared code.** `ai-sdk/agentloop` and `ai-sdk/toolkit` provide the common
agent loop and tool set.
6. **No daemon-internal cron.** Polling is a ticker loop, not cron. The
feasibility PRD notification uses an external webhook (n8n).
7. **Fresh worktree per task.** No persistent state between runs. Crash recovery
just re-queues.
8. **Agent execution is a data boundary.** Autonomous stages receive a versioned
request and return a fenced result; SQLite, forge operations, git, workflow
sequencing, and outcome handling stay daemon-owned.
## What's Implemented vs Planned
**Implemented:**
- Daemon poll loop, enqueue, claim, process
- Forge webhook intake as a feature-flagged alternative/backstop pairing to
polling (single-identity deployments; see `internal/forge/webhook`)
- Bootstrap, implement, TDD, feasibility workflows
- GitHub and Gitea forges (issues, PRs, labels, comments, reactions)
- Worktree isolation (clone, branch, commit, push, diff, cleanup)
- SQLite store with transition audit
- Config with ecosystem support (go, python, node, rust)
- Agent stages with gates, budgets, preflight, protected paths
- Observability (event bus, SQLite log, SSE dashboard)
- Crash recovery (re-queue stale running tasks)
- Human reply judgment (triage LLM for waiting_human)
**Planned:**
- Multi-identity webhook intake (currently single-identity only)
- More ecosystems
- Notification integrations beyond n8n webhook
- Skills (SKILL.md support -- see `.archie/skills/`)
+9 -6
View File
@@ -25,11 +25,13 @@ assigned/labelled to it, works each one in an isolated git worktree through a
routed workflow (bootstrap / implement / TDD / feasibility), and opens pull
requests for human review.
See `ARCHITECTURE.md` for the package map, workflow engine design, task
lifecycle, and key design decisions. Read it before making non-trivial changes.
Key invariants: env-enforced gates, model never runs git, and agent execution
functions as a strict data boundary. Before adding a new plugin engine, satisfy
`ARCHITECTURE.md#plugin-engine-rule-strict`.
Architecture of record lives in `docs/architecture/` -- `organisation.md` for
the package map, `agent-system.md` for the workflow-execution lifecycle and its
invariants, `plugins-and-extensions.md` for engine families. Read it before
making non-trivial changes. Key invariants: env-enforced gates, model never runs
git, and agent execution functions as a strict data boundary. Before adding a new
plugin engine, satisfy
`docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`.
## Scope Discipline
@@ -257,7 +259,8 @@ structures found in legacy packages.
- Webhook intake refuses to start when `[[identities]]` is configured (one
receiver, one dispatch config). Do not silently enable it for multi-identity
deployments without first building per-identity routing -- see
`ARCHITECTURE.md`'s "Webhook intake" section.
`docs/architecture/messaging-and-work-intake.md`'s "Intake provenance"
section.
- **`internal/domain/embedding/` + `internal/infrastructure/embedding/`:**
Embedding client contract + implementation. Config-driven exactly like chat
model roles: `models.embedding = "provider/model"` plus a `[providers.*]`
+9 -1
View File
@@ -350,6 +350,10 @@ tasks:
archie-gateway and archie-ui dial it -- none owns
the DB in-process
(docs/prds/state-store-contract.md §12 step 7)
* state admin read-only PocketBase surface over archie.db on
127.0.0.1:8686 (view collections only; the gRPC
contract stays the only writer). Served by the
archie-state-store process, not a sixth one.
* archie-ui serves the dashboard on 127.0.0.1:8484 from the
embedded dist (deployments/dev.toml). After the
UI cutover (archie-core-8cda.5.4) archied runs no
@@ -380,8 +384,12 @@ tasks:
dev:state-store:
desc: archie-state-store only, no live-reload (prefer `task dev`)
# -admin-addr serves the read-only operator dashboard over the same
# archie.db this process already owns. It is a flag rather than a config
# key for the same reason -ready-addr is: the address is a property of
# how this process was launched, not of the deployment's config document.
cmds:
- go run ./cmd/archie-state-store -config-overlay deployments/dev.toml
- go run ./cmd/archie-state-store -config-overlay deployments/dev.toml -admin-addr 127.0.0.1:8686
dev:gateway:
desc: archie-gateway only, no live-reload (prefer `task dev`)
+1
View File
@@ -39,6 +39,7 @@ func run() int {
flag.StringVar(&options.Listen, "listen", "", "state store gRPC listen address (defaults to [services.state].listen, else 127.0.0.1:9090)")
flag.StringVar(&options.Token, "token", "", "bearer token required for a non-loopback listener (defaults to [services.state].target_token)")
flag.StringVar(&options.ReadyAddr, "ready-addr", "", "optional readiness HTTP listen address (e.g. 127.0.0.1:9091)")
flag.StringVar(&options.AdminAddr, "admin-addr", "", "optional read-only operator dashboard listen address (e.g. 127.0.0.1:8090)")
flag.Usage = func() {
fmt.Fprintf(flag.CommandLine.Output(), "usage: archie-state-store [flags] # serve the State Store gRPC contract\n\noffline recovery: archie-state-store <backup|restore|validate|rollback> -h\n\n")
flag.PrintDefaults()
+7 -7
View File
@@ -372,8 +372,8 @@ func TestStateStoreRealProcessSmoke(t *testing.T) {
}); err != nil {
t.Fatalf("InsertEvent config_captured: %v", err)
}
if capID, err := cl.InsertCapture(ctx, capture("sentry", `{"id":1}`), 0, 0); err != nil || capID == 0 {
t.Fatalf("InsertCapture = (%d, %v)", capID, err)
if capID, err := cl.InsertCapture(ctx, capture("sentry", `{"id":1}`), 0, 0); err != nil || capID == "" {
t.Fatalf("InsertCapture = (%q, %v)", capID, err)
}
got, err := cl.TaskByID(ctx, task.ID)
@@ -583,8 +583,8 @@ func TestStateStoreRealProcessRemoteSurfaces(t *testing.T) {
ctx := t.Context()
mappingID, err := cl.InsertMapping(ctx, mapping.Mapping{Name: "m1", Fields: []mapping.Field{{Name: "a", Path: "a", Type: mapping.TypeString}}})
if err != nil || mappingID == 0 {
t.Fatalf("InsertMapping = (%d, %v)", mappingID, err)
if err != nil || mappingID == "" {
t.Fatalf("InsertMapping = (%q, %v)", mappingID, err)
}
if got, err := cl.GetMapping(ctx, mappingID); err != nil || got == nil || got.Name != "m1" {
t.Fatalf("GetMapping = %+v, %v", got, err)
@@ -593,8 +593,8 @@ func TestStateStoreRealProcessRemoteSurfaces(t *testing.T) {
Name: "b1", Matcher: binding.Matcher{Source: "sentry"}, MappingID: mappingID,
Workflow: "implement", Secret: "0123456789abcdef0123456789abcdef",
})
if err != nil || bindingID == 0 {
t.Fatalf("InsertBinding = (%d, %v)", bindingID, err)
if err != nil || bindingID == "" {
t.Fatalf("InsertBinding = (%q, %v)", bindingID, err)
}
// A binding starts as draft and must be edited (draft -> pending_approval)
// then approved (pending_approval -> armed) before it is armed for a
@@ -613,7 +613,7 @@ func TestStateStoreRealProcessRemoteSurfaces(t *testing.T) {
t.Fatalf("ArmedBindingsForSource = %+v, %v", armed, err)
}
// Remote surfaces share error-sentinel fidelity on the wire too.
missing := mapping.Mapping{ID: mappingID + 999999, Name: "x", Fields: []mapping.Field{{Name: "a", Path: "a", Type: mapping.TypeString}}}
missing := mapping.Mapping{ID: "rabsent00000000", Name: "x", Fields: []mapping.Field{{Name: "a", Path: "a", Type: mapping.TypeString}}}
if err := cl.UpdateMapping(ctx, missing); !errors.Is(err, store.ErrMappingNotFound) {
t.Fatalf("UpdateMapping missing = %v, want ErrMappingNotFound", err)
}
+5 -2
View File
@@ -37,6 +37,7 @@ import (
"github.com/samcharles93/archie-core/internal/domain/health"
"github.com/samcharles93/archie-core/internal/domain/messaging"
"github.com/samcharles93/archie-core/internal/domain/taskactions"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/infrastructure/gatewayrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/store"
@@ -601,8 +602,9 @@ func TestUIProcessLinksMultiIdentityTaskRowsToTheirOwningForge(t *testing.T) {
}
stateTarget, stopState := serveGRPC(t, func(r grpc.ServiceRegistrar) {
eda := edastore.OpenTest(t)
staterpc.RegisterServer(r, staterpc.Deps{
Tasks: st, Captures: st, BindingDispatcher: st, ConfigSnapshots: st,
Tasks: st, Captures: eda, BindingDispatcher: eda, ConfigSnapshots: st,
Log: slog.New(slog.DiscardHandler),
})
})
@@ -657,8 +659,9 @@ func TestUIProcessServesTheDashboardAgainstLiveDependencies(t *testing.T) {
gateway := &fakeGateway{}
stateTarget, stopState := serveGRPC(t, func(r grpc.ServiceRegistrar) {
eda := edastore.OpenTest(t)
staterpc.RegisterServer(r, staterpc.Deps{
Tasks: st, Captures: st, BindingDispatcher: st, ConfigSnapshots: st,
Tasks: st, Captures: eda, BindingDispatcher: eda, ConfigSnapshots: st,
Log: slog.New(slog.DiscardHandler),
})
})
-6
View File
@@ -15,11 +15,5 @@ func main() {
if archied.IsSetupArgs(args) {
os.Exit(archied.RunSetup(args[1:], os.Stdin, os.Stdout, os.Stderr))
}
// The codesearch helper is dispatched before the daemon's flags are
// parsed: it is this same binary re-invoked as a short-lived child by
// internal/indexing, and it must not touch config, the store or NATS.
if archied.IsCodesearchHelperArgs(args) {
os.Exit(archied.RunCodesearchHelper(args[1:], os.Stdout))
}
os.Exit(archied.Run())
}
+7 -6
View File
@@ -227,12 +227,13 @@ class = "ollama"
# work_dir = "/path/to/workspace"
# parallel_tool_calls = false
# Workspace code-search index. Paths default under work_dir when unset.
[indexing]
# Code-search index files (default: <work_dir>/indexes).
# index_dir = "/var/lib/archie/indexes"
# Per-index lifecycle metadata DB (default: <work_dir>/workspace-indexes.db).
# db_path = "/var/lib/archie/workspace-indexes.db"
# Publish artifacts (review reports, design docs) to the collaborative editor.
# An empty base_url disables publishing entirely. The token is runtime config:
# set WORKSPACE_INGEST_TOKEN in the daemon's environment or use a secret ref.
[artifacts]
# base_url = "https://offloaded.dev"
# token = { engine = "env", key = "WORKSPACE_INGEST_TOKEN" }
# token_env = "WORKSPACE_INGEST_TOKEN"
# archied's own liveness surface, always on. GET /healthz and /health answer
# 200 once the daemon is serving (503 while it is still booting), and
+18
View File
@@ -124,3 +124,21 @@ indicates dual-store ownership (`docs/prds/state-store-contract.md` §12 step 7/
The readiness probe is the single-writer check the runbook has against a
degraded store: a `503` from `/health/detailed` means `archied` should be
started only after the store is healthy.
### Inspecting the store
`-admin-addr` starts a read-only operator dashboard on the same process, over
the file it already owns:
```bash
archie-state-store -admin-addr 127.0.0.1:8686 # dashboard on http://127.0.0.1:8686/_/
```
It co-tenants on `<db_path>-tasks.sqlite` rather than opening a second
database, so it does not disturb the single-writer rule above: it runs inside
the process that holds the ownership claim, and every archie table is exposed
as a read-only view collection. The routes that could write past those views
(`POST /api/sql`, `/api/backups`) are refused, so the gRPC contract stays the
only writer and `archie-state-store backup` stays the only snapshot path.
Leave the flag off and no dashboard listener is started.
+3 -6
View File
@@ -5,9 +5,9 @@ through `.5`, `.7`, `.9`-`.15`)
**Date:** 2026-09-03
This document covers the binding model and the intake threat model portion
of `archie-core-t2db.6`. It does not cover the `ARCHITECTURE.md` task-lifecycle
addendum or the operator walkthrough -- those remain separate, open pieces of
the same ticket.
of `archie-core-t2db.6`. It does not cover the task-lifecycle provenance rule
(`messaging-and-work-intake.md`'s "Intake provenance") or the operator
walkthrough -- that walkthrough remains an open piece of the same ticket.
## What a binding is
@@ -177,9 +177,6 @@ execution-time idempotency, now landed (`archie-core-t2db.17` closed): the
## What this document does not cover
- The `ARCHITECTURE.md` task-lifecycle section noting that a
playbook-originated task has different provenance from a forge-issue
task -- open, part of `archie-core-t2db.6`.
- An end-to-end operator walkthrough (point an external app at archie,
build a first playbook) -- open, part of `archie-core-t2db.6`.
- The playbook YAML / CEL / Module/Channel/Forge action-position work
+2 -4
View File
@@ -58,7 +58,7 @@ survive as a shared package, configuration domain, or renamed global model.
Its loading behaviour moves to `internal/infrastructure/configuration`. Existing
types and methods are reassigned by behaviour. Repository execution policy,
dispatch policy, retry policy, ecosystem defaults, tool policy, memory settings,
indexing settings, provider settings, and channel settings move to their
provider settings, and channel settings move to their
respective owners.
## Repository settings
@@ -163,8 +163,7 @@ existing structure combines different owners.
| webhook target | `Notify` | Notification adapter instance settings. |
| web listen address | `Web` | Web transport settings. |
| `listen` under `[services.state]` and `[services.gateway]` | `ServiceConnection` | The bind address of the process that owns the service. Read only by that process, and defaulted to the address its clients dial so the pair cannot drift. |
| index directory and database path | `Indexing` | Index infrastructure settings, derived from storage settings unless explicitly overridden. |
| forge, dispatch, models, providers, containers, memory, tools, indexing and repositories | `IdentityConfig` | Existing composite duplicates unrelated settings. Replace it with associations from an `IdentityID` to independently owned configured instances. |
| forge, dispatch, models, providers, containers, memory, tools and repositories | `IdentityConfig` | Existing composite duplicates unrelated settings. Replace it with associations from an `IdentityID` to independently owned configured instances. |
| owner, name, base, gate, protection, ecosystem, preflight, storage, retry and concurrency | `Repo` | Split across work intake, workspace, workflow execution, policy, gate, and scheduling as described in Repository settings. |
| `TaskConfig`, `TaskForge` | worker snapshot | Replace with the minimum immutable, versioned WorkflowExecution input assembled by the application. It is not a second configuration model. |
| `Extra` and arbitrary maps | root/provider config | Allowed only at an extension boundary whose owning plugin/provider supplies schema and validation. Core configuration never consumes an untyped catch-all. |
@@ -187,7 +186,6 @@ decision; exact setting names are implementation details.
| NATS worker consumer | stream `ARCHIE_TASKS`, consumer `archie-agent`, deduplication `2m`, poll `5s`, acknowledgement wait `30m`, deliveries `3`, inactive threshold `24h`, queue group and retry delay `1s` | Worker transport settings and delivery/retry policy. Shared stream defaults must have one owner rather than two copies. |
| Worker readiness | readiness timeout `20s`, retry delay `250ms` | Worker lifecycle and retry policy. |
| Execution/RPC | agent execution `30m`, task-run RPC `60s`, worktree RPC `15m` | Per-operation timeout policy. |
| Indexing | build timeout `10m`, metadata timeout `5s` | Index operation policy. |
| Plugin/tool cleanup | plugin rollback `10s`, tool-provider cleanup `10s`, MCP-provider cleanup `10s` | Lifecycle shutdown/rollback policy owned by each engine, with a shared policy vocabulary where appropriate. |
| Memory read path | per-scope record limit `20`, rendered `<memory>` block byte cap `8192` | Memory read policy owned by the engine family (`internal/domain/memory`). |
| Built-in memory store | per-file size limit `100KiB`, query result limit `100` | Split into memory resource policy, input policy, and presentation settings. |
@@ -251,6 +251,32 @@ Current intake is fragmented:
- `store.Task` collapses an external source reference and durable execution
state.
## Intake provenance
A task's origin determines which enqueue path created it, though every origin
converges on the same `tasks` table and the same `Route`/`Run` lifecycle once
queued.
- Forge-sourced work -- discovered by polling or, in single-identity
deployments, delivered by the forge webhook receiver (`internal/forge/webhook`)
-- enqueues through the same `workintake.TaskEnvelope` and `PublishTask` path.
`TaskEnvelope.IdempotencyKey` is keyed on `owner/repo/number` and not on the
delivery source, so a webhook-delivered issue and a later poll of the same
issue dedup through `PublishUnique` instead of double-enqueuing.
- Chat-spawned and playbook-binding-dispatched work enqueues through
`EnqueueChatTask`/`EnqueueBindingTask` (`internal/store`), stamped with
`binding_id`/`binding_version` for provenance.
Webhook intake is opt-in per deployment (`Forge.Intake` = `poll` (default),
`webhook`, or `both`) and single-identity only: a config naming
`webhook`/`both` alongside `[[identities]]`, or setting intake on a
`[identities.forge]` entry, is rejected at validation rather than started with
the setting ignored. One receiver binds one secret and one dispatch predicate,
so it cannot judge a second identity's events by the right rules. A
multi-identity deployment polls per identity until per-identity intake lands.
The decided contract is `docs/prds/event-sources-and-reactions.md`; the binding
model and its threat model are `docs/architecture/bindings.md`.
## Current hazards
- Treating every agent interaction as a work request would remove Archie's
+4 -3
View File
@@ -264,9 +264,10 @@ around is no longer a boundary.
returns and validates `workflow.ExecutionSettings`, so any package holding it
links the workflow domain and therefore `internal/agentexec`. This is why the
client cannot move as one package -- see open question 2.
- The schedules document, whose type is `cronstore.JobSpec` in
`internal/infrastructure`. A domain vocabulary package may not import
infrastructure.
- The schedules document, whose type is `scheduling.JobSpec` in
`internal/domain/scheduling` -- the vocabulary moved there when the legacy
file-backed cronstore was deleted, which is also what satisfies the rule
that a domain vocabulary package may not import infrastructure.
- The schema derivation, which describes the documents the server advertises.
**Placement as landed (2026-09-22, `archie-core-1ng1`).** One new package,
+94 -2
View File
@@ -23,6 +23,97 @@ versioning, definition, and execution contracts. The generic plugin domain may
provide plugin identity, discovery, compatibility, and lifecycle mechanics, but
it MUST NOT define Workflow semantics or expose untyped Workflow hooks.
## Plugin engine rule (strict)
“Engine” is reserved for a typed, lifecycle-managed capability family. It is not
a synonym for a generic plugin hook. Every new plugin engine must satisfy all of
these invariants:
1. **Typed domain contract.** The engine interface exposes real
capability-specific operations in addition to identity metadata. Examples are
resolving a secret, delivering a message, or running an agent. A plugin that
only exposes `Name` and `Version` is metadata, not an engine.
2. **Owning family manager.** The capability package owns a `Registry` or
`Manager` that controls provider registration and discovery, initialization,
or access. Providers do not register arbitrary callbacks directly on the
daemon.
3. **Lifecycle and health.** When an implementation owns resources or background
work, its typed family contract must expose start, health, and stop
semantics. The owning manager invokes them and defines failure isolation and
shutdown ordering. Stateless providers may make these operations explicit
no-ops.
4. **Narrow host access.** Plugins receive the typed family registrar and only
the host services their declared capability requires. They never receive
`*daemon.Daemon`, a service locator, or an untyped hook map.
5. **Metadata-only generic plugin contract.** `plugin.Plugin` remains limited to
`Name()` and `Version()`. Domain hooks belong on typed engine interfaces;
adding capability methods to the generic interface is forbidden.
6. **Trust boundary stays explicit.** Explicitly trusted, operator-installed
plugins—including third-party secret-engine plugins the operator chooses to
trust—may run in-process and have daemon privileges. Repository-supplied code
runs in the task container. Integrations that are not trusted as daemon code
run out of process through MCP or a versioned protocol and, when a
confidentiality or integrity boundary is required, inside a real container
sandbox. A subprocess boundary alone is not a security sandbox, and calling
Yaegi-loaded code a plugin does not isolate it.
`internal/plugin/architecture_test.go` enforces the mechanically testable parts
of this rule: the generic plugin method set, the shared agent-instruction
source, and the requirement that engine interfaces have domain behavior plus an
owning registry or manager.
### Memory engine family
One engine, addressed by four typed scopes -- `ScopeGlobal`, `ScopeAgent`,
`ScopeUser`, `ScopeAgentUser` (`internal/domain/memory`, `Scope`/`ScopeKind`).
A caller names exactly the scopes it may read or write; the engine applies no
access policy of its own, so isolation falls out mechanically because
`ScopeAgent{a}` and `ScopeAgent{b}` are different storage keys. See
`docs/prds/memory-engine-unification.md` for the decision this contract
implements; it superseded `internal/memory` (`MemoryProvider`/`Manager`),
deleted in that PRD's slice 5.
`MemoryEngine` (`internal/domain/memory/contract.go`) is the typed family
contract: `Create`, `Get`, `Query`, `List`, `Update`, `Forget`, `Revisions` --
CRUD with revisions retained, not just create/read/delete. `Update` supersedes
rather than overwrites: the superseded state is appended to the scope's
`HISTORY.md` before the live block is replaced, and `Update` carries no
`Scope` field, so a record can never change scope. The family's content
scanner (prompt-injection / sensitive-data patterns) is applied inside
`Create` and `Update`, the single choke point every producer crosses.
`internal/domain/memory.Registry` is the owning family registry: registration,
start/health/stop, shutdown ordering, and failure isolation, following the
plugin engine rule above. `internal/infrastructure/memory.NewBuiltinEngine` is
the only registered engine today; it persists through
`internal/infrastructure/memory/builtin`, a markdown store (`MEMORY.md`-shaped
per-scope files) that owns its on-disk format end-to-end.
Two per-turn seams in `internal/gateway` connect a chat turn to the engine,
each narrowed to exactly the operations it needs:
- **Read** (`turn_memory.go`, `MemoryStore`): `prepareTurn` runs a
scope-only `Query` over the resolved `Subject`'s readable scopes,
synchronously and before the prompt is built. It never returns an error --
a panicking or failing engine degrades to an empty `<memory>` block rather
than failing the turn -- and is bounded by a per-scope record limit and a
byte cap on the rendered block.
- **Write** (`turn_memory_tool.go`, `MemoryWriteStore`): four tools --
`memory_create`, `memory_update`, `memory_delete`, `memory_list` -- built
per turn onto the resolved `Subject`'s writable scopes, not registered once
at boot. The model chooses the scope kind; it never chooses the agent or
user id, which come from the turn's resolved `Subject` and have no schema
field to override. `ScopeGlobal` is never model-writable.
`boot.setupMemoryEngine` (`internal/app/archied/bootstrap.go`) registers and
starts the engine and must run before `setupGatewayChat`, which constructs the
only chat turn runner in production and captures the engine at construction
time (`internal/app/archied/composition_order_test.go` pins the ordering), and
before `setupCurators`, whose `Registrar` captures the engine registry. The
session curator (`internal/infrastructure/sessioncurator`) writes derived
records onto `ScopeAgentUser`, making it the engine's other live producer
besides the chat write path.
## Plugin implementations
First-party plugins are self-contained vertical modules under
@@ -48,8 +139,9 @@ in-process through typed capability families (this document's model) or
out-of-process like a generic `DispatchEvent` dispatcher. Decided 2026-08-22
in `docs/prds/event-sources-and-reactions.md`: in-process typed families,
confirming this document's model rather than amending it. The forge webhook
receiver (`internal/forge/webhook`, see `ARCHITECTURE.md`'s "Webhook intake"
section) and the EDA playbook engine's Module action position
receiver (`internal/forge/webhook`; see "Intake provenance" in
`docs/architecture/messaging-and-work-intake.md`) and the EDA playbook engine's
Module action position
(`docs/prds/eda-playbook-engine.md`, `internal/domain/eda/module`) are the
shipped instances of that decision -- each is a narrow typed contract with an
owning registry, not a generic event hook. The playbook engine's Channel and
+22 -12
View File
@@ -22,13 +22,13 @@
"url": "/dev/architecture/bindings/",
"title": "Playbook bindings: model and threat model",
"section": "Architecture",
"body": "# Playbook bindings: model and threat model\n\n**Status:** Approved foundation (documents shipped code: `archie-core-t2db.1`\nthrough `.5`, `.7`, `.9`-`.15`)\n**Date:** 2026-09-03\n\nThis document covers the binding model and the intake threat model portion\nof `archie-core-t2db.6`. It does not cover the `ARCHITECTURE.md` task-lifecycle\naddendum or the operator walkthrough -- those remain separate, open pieces of\nthe same ticket.\n\n## What a binding is\n\nA `binding` (`internal/domain/binding.Binding`) ties four things together:\na **matcher** (which captured events it applies to), a **mapping**\n(`internal/domain/mapping.Mapping`, how fields are pulled out of a payload),\na **workflow** (which registered workflow runs on a match), and a **state\nmachine** governing whether it is live.\n\n```go\ntype Binding struct {\n ID, Name // identity\n Matcher // { Source string } -- the webhook path segment, e.g. \"sentry\"\n MappingID // which Mapping resolves payload fields\n Workflow // registered workflow name to dispatch to\n Version // bumped on every edit\n Status // draft | pending_approval | armed\n Secret // HMAC-SHA256 shared secret, encrypted at rest, never returned by GET\n}\n```\n\nBindings live in the store (SQLite), not `config.toml` -- operators author\nthem from the dashboard while the daemon runs, without a restart.\n\n### Matching\n\n`Matcher.Source` is the only predicate today: the path segment a sender\nPOSTs to (e.g. a webhook hitting `.../sentry` matches bindings with\n`Matcher.Source == \"sentry\"`). The storage column is plain `TEXT`, so adding\npredicates later is non-breaking -- extend `Matcher`, don't invent a second\nmatcher shape.\n\n### Mapping\n\n`mapping.Resolve(fields, payload)` walks a captured JSON payload against a\nlist of typed `Field`s (name, JSON path, expected type), returning resolved\nvalues plus a `[]Failure` for anything that didn't parse or type-match. This\nis schema-by-example in practice: a mapping is authored by pointing at a\nreal captured payload in the dashboard, not by guessing a schema in advance.\n\n### Versioning\n\n`Version` increments on every `UpdateBinding` call. This exists so an\neditor's later change can never silently rewrite the historical provenance\nof a task that already fired -- a dispatched task's binding reference is\npinned to the version that was armed when it fired, not \"whatever the\nbinding currently says.\"\n\n### State machine\n\n```\ndraft ---\u003e pending_approval ---\u003e armed\n ^ ^ |\n | | |\n +--------------+-----------------+\n (any edit to an armed binding drops it back to pending_approval)\n```\n\nModeled directly on Telegram's existing `dangerousAction`/`pendingApproval`\nflow (`internal/channels/telegram/dangerous.go`, `approval.go`) -- not a new\napproval mechanism invented for this feature. **Only `armed` evaluates\nagainst incoming events** (`binding.Matches`, `internal/domain/binding/\nbinding.go`); `draft` and `pending_approval` bindings are inert. An edit to\nan already-armed binding cannot bypass approval by silent mutation -- it\ndrops back to `pending_approval` and requires an explicit `Approve` call to\nre-arm.\n\n## The threat model\n\nThis is a public, unauthenticated-by-default intake surface whose output is\nan agent with a repo, a worktree, and tool access. The design brief\n(`archie-core-t2db.5`, `docs/prds/webhook-intake-security.md`) named five\nnon-negotiables; each is verified below against the actual shipped code,\nnot the design intent.\n\n### 1. Per-source authentication\n\nReuses `internal/channels/webhook`'s existing HMAC-SHA256 scheme --\ncapture accepts unsigned events (visible in the dashboard, marked\nunauthenticated), but **only authenticated events can ever match a\nbinding**. `binding.Matches`' doc comment states this plainly: \"an\nunauthenticated event can never trigger a binding, no matter how well it\nwould otherwise match.\" Enforced structurally at two points, not just\nconvention:\n\n- `captured_events.authenticated` is a real column (`internal/store/\n captures.go`), set at capture time.\n- `dispatchBindings` (`internal/daemon/daemon.go:479`) walks **authenticated\n captures whose source has at least one armed binding** -- an\n unauthenticated capture is never even considered for dispatch.\n\n### 2. Human approval before a binding goes live\n\nCovered by the state machine above. Nothing self-arms: `pending_approval -\u003e\narmed` requires an explicit `Approve` call, and any edit to an armed\nbinding reverts it.\n\n### 3. Rate and volume limits per source\n\nCapture writes are bounded independently of the approval gate, so a flood\nof unauthenticated traffic cannot exhaust disk or drown the inspector:\n`InsertCapture` (`internal/store/captures.go`) prunes by both **retention\nwindow** (`WHERE received_Line truncated
"body": "# Playbook bindings: model and threat model\n\n**Status:** Approved foundation (documents shipped code: `archie-core-t2db.1`\nthrough `.5`, `.7`, `.9`-`.15`)\n**Date:** 2026-09-03\n\nThis document covers the binding model and the intake threat model portion\nof `archie-core-t2db.6`. It does not cover the task-lifecycle provenance rule\n(`messaging-and-work-intake.md`'s \"Intake provenance\") or the operator\nwalkthrough -- that walkthrough remains an open piece of the same ticket.\n\n## What a binding is\n\nA `binding` (`internal/domain/binding.Binding`) ties four things together:\na **matcher** (which captured events it applies to), a **mapping**\n(`internal/domain/mapping.Mapping`, how fields are pulled out of a payload),\na **workflow** (which registered workflow runs on a match), and a **state\nmachine** governing whether it is live.\n\n```go\ntype Binding struct {\n ID, Name // identity\n Matcher // { Source string } -- the webhook path segment, e.g. \"sentry\"\n MappingID // which Mapping resolves payload fields\n Workflow // registered workflow name to dispatch to\n Version // bumped on every edit\n Status // draft | pending_approval | armed\n Secret // HMAC-SHA256 shared secret, encrypted at rest, never returned by GET\n}\n```\n\nBindings live in the store (SQLite), not `config.toml` -- operators author\nthem from the dashboard while the daemon runs, without a restart.\n\n### Matching\n\n`Matcher.Source` is the only predicate today: the path segment a sender\nPOSTs to (e.g. a webhook hitting `.../sentry` matches bindings with\n`Matcher.Source == \"sentry\"`). The storage column is plain `TEXT`, so adding\npredicates later is non-breaking -- extend `Matcher`, don't invent a second\nmatcher shape.\n\n### Mapping\n\n`mapping.Resolve(fields, payload)` walks a captured JSON payload against a\nlist of typed `Field`s (name, JSON path, expected type), returning resolved\nvalues plus a `[]Failure` for anything that didn't parse or type-match. This\nis schema-by-example in practice: a mapping is authored by pointing at a\nreal captured payload in the dashboard, not by guessing a schema in advance.\n\n### Versioning\n\n`Version` increments on every `UpdateBinding` call. This exists so an\neditor's later change can never silently rewrite the historical provenance\nof a task that already fired -- a dispatched task's binding reference is\npinned to the version that was armed when it fired, not \"whatever the\nbinding currently says.\"\n\n### State machine\n\n```\ndraft ---\u003e pending_approval ---\u003e armed\n ^ ^ |\n | | |\n +--------------+-----------------+\n (any edit to an armed binding drops it back to pending_approval)\n```\n\nModeled directly on Telegram's existing `dangerousAction`/`pendingApproval`\nflow (`internal/channels/telegram/dangerous.go`, `approval.go`) -- not a new\napproval mechanism invented for this feature. **Only `armed` evaluates\nagainst incoming events** (`binding.Matches`, `internal/domain/binding/\nbinding.go`); `draft` and `pending_approval` bindings are inert. An edit to\nan already-armed binding cannot bypass approval by silent mutation -- it\ndrops back to `pending_approval` and requires an explicit `Approve` call to\nre-arm.\n\n## The threat model\n\nThis is a public, unauthenticated-by-default intake surface whose output is\nan agent with a repo, a worktree, and tool access. The design brief\n(`archie-core-t2db.5`, `docs/prds/webhook-intake-security.md`) named five\nnon-negotiables; each is verified below against the actual shipped code,\nnot the design intent.\n\n### 1. Per-source authentication\n\nReuses `internal/channels/webhook`'s existing HMAC-SHA256 scheme --\ncapture accepts unsigned events (visible in the dashboard, marked\nunauthenticated), but **only authenticated events can ever match a\nbinding**. `binding.Matches`' doc comment states this plainly: \"an\nunauthenticated event can never trigger a binding, no matter how well it\nwould otherwise match.\" Enforced structurally at two points, not just\nconvention:\n\n- `captured_events.authenticated` is a real column (`internal/store/\n captures.go`), set at capture time.\n- `dispatchBindings` (`internal/daemon/daemon.go:479`) walks **authenticated\n captures whose source has at least one armed binding** -- an\n unauthenticated capture is never even considered for dispatch.\n\n### 2. Human approval before a binding goes live\n\nCovered by the state machine above. Nothing self-arms: `pending_approval -\u003e\narmed` requires an explicit `Approve` call, and any edit to an armed\nbinding reverts it.\n\n### 3. Rate and volume limits per source\n\nCapture writes are bounded independently of the approval gate, so a flood\nof unauthenticated traffic cannot exhaust disk or drown the inspector:\n`InsertCapture` (`internal/store/captures.go`) prunLine truncated
},
{
"url": "/dev/architecture/configuration/",
"title": "Configuration",
"section": "Architecture",
"body": "# Configuration\n\n**Status:** Approved; existing-type assignment is ongoing \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Ownership\n\nConfiguration has two separate responsibilities:\n\n1. `internal/infrastructure/configuration` loads files, environment variables,\n secret references, overlays, and compatibility representations.\n2. Each domain and plugin owns the typed runtime settings required for its\n behaviour.\n\n`internal/app` translates external input into those settings, validates\ncross-capability combinations, and supplies each component only its own\nsettings.\n\nThe following are prohibited:\n\n- passing complete application configuration into a domain;\n- making a domain depend on TOML, YAML, environment-variable names, secret\n references, or a configuration library;\n- defining a central runtime configuration type imported across the system;\n- placing plugin options in an unrelated global configuration model.\n\n## Private input document\n\n`internal/infrastructure/configuration` MAY use a complete decoded document\nwhile discovering files, decoding sections, and applying overlays. It is an\ninput-boundary DTO only.\n\nThe document:\n\n- remains private to the configuration infrastructure;\n- is composed from separately defined feature sections;\n- is immediately translated by application composition;\n- is never returned as the application's runtime model;\n- is never passed through runtime components.\n\nExternal DTOs and domain runtime settings are separate types.\n\nPlugins supply external configuration definitions through a narrow extension\ncontract. The core loader MUST NOT gain a field or recognized filename for every\nplugin.\n\nInvalid configuration input MUST NOT replace active valid settings or terminate\nunaffected capabilities. Configuration changes use the candidate validation,\npromotion, audit, and recovery requirements in\n[safe-change-and-recovery.md](/dev/architecture/safe-change-and-recovery/).\n\n## Dissolution of `internal/config`\n\n`internal/config` has no target location. It MUST be dissolved and MUST NOT\nsurvive as a shared package, configuration domain, or renamed global model.\n\nIts loading behaviour moves to `internal/infrastructure/configuration`. Existing\ntypes and methods are reassigned by behaviour. Repository execution policy,\ndispatch policy, retry policy, ecosystem defaults, tool policy, memory settings,\nindexing settings, provider settings, and channel settings move to their\nrespective owners.\n\n## Repository settings\n\nThe existing `config.Repo` MUST NOT move intact into a repository domain or\nshared settings package. A managed repository is an application association:\n\n- work intake owns which repository references an identity monitors;\n- task execution owns gates, preflight, test protection, ecosystem behaviour,\n and change limits;\n- task lifecycle owns retry behaviour;\n- scheduling owns concurrency policy;\n- workspace management owns base branch and persistent-storage requirements.\n\nApplication composition correlates these settings by repository reference. No\ndomain receives a universal repository profile.\n\n## Documentation\n\nGenerated configuration reference describes external sections and their owning\ncapabilities. It MUST NOT establish a global Go configuration structure as the\npublic contract.\n\n## Value classification rule\n\nEvery value that controls behaviour MUST have an explicit owner and MUST be\nclassified as exactly one of:\n\n1. **Invariant** — changing it changes the meaning of the program, a wire\n protocol, or a domain concept. It may be a Go constant.\n2. **Runtime setting** — an operator, identity, domain, plugin, or deployment\n may legitimately choose a different value. It is typed configuration owned by\n the affected capability.\n3. **Policy value** — it expresses what is allowed, limited, routed, retried,\n protected, or gated. Its typed definition belongs to the owning domain and it\n is evaluated through the policy system.\n4. **Derived value** — it is computed from other settings and is not\n independently mutable.\n5. **Runtime state** — it changes as the application operates, such as the\n active model for a conversation. It is changed by a domain command and is not\n configuration merely because it can be changed.\n\nAn unclassified behavioural literal is a defect.\n\n`const` MUST NOT be used merely because a value currently has one default.\nConstants are reserved for actual invariants: enum discriminants, protocol\nversions, fixed compatibility tokens, and values whose alteration would change\nthe program's semantics rather than configure an installation.\n\nIn Go, a constant is a compile-time value and does not represent mutable runtime\nstorage. It generally has no address that can be observed or changed.\nArchitecturally, a constant therefore asserts that the value cannot chaLine truncated
"body": "# Configuration\n\n**Status:** Approved; existing-type assignment is ongoing \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Ownership\n\nConfiguration has two separate responsibilities:\n\n1. `internal/infrastructure/configuration` loads files, environment variables,\n secret references, overlays, and compatibility representations.\n2. Each domain and plugin owns the typed runtime settings required for its\n behaviour.\n\n`internal/app` translates external input into those settings, validates\ncross-capability combinations, and supplies each component only its own\nsettings.\n\nThe following are prohibited:\n\n- passing complete application configuration into a domain;\n- making a domain depend on TOML, YAML, environment-variable names, secret\n references, or a configuration library;\n- defining a central runtime configuration type imported across the system;\n- placing plugin options in an unrelated global configuration model.\n\n## Private input document\n\n`internal/infrastructure/configuration` MAY use a complete decoded document\nwhile discovering files, decoding sections, and applying overlays. It is an\ninput-boundary DTO only.\n\nThe document:\n\n- remains private to the configuration infrastructure;\n- is composed from separately defined feature sections;\n- is immediately translated by application composition;\n- is never returned as the application's runtime model;\n- is never passed through runtime components.\n\nExternal DTOs and domain runtime settings are separate types.\n\nPlugins supply external configuration definitions through a narrow extension\ncontract. The core loader MUST NOT gain a field or recognized filename for every\nplugin.\n\nInvalid configuration input MUST NOT replace active valid settings or terminate\nunaffected capabilities. Configuration changes use the candidate validation,\npromotion, audit, and recovery requirements in\n[safe-change-and-recovery.md](/dev/architecture/safe-change-and-recovery/).\n\n## Dissolution of `internal/config`\n\n`internal/config` has no target location. It MUST be dissolved and MUST NOT\nsurvive as a shared package, configuration domain, or renamed global model.\n\nIts loading behaviour moves to `internal/infrastructure/configuration`. Existing\ntypes and methods are reassigned by behaviour. Repository execution policy,\ndispatch policy, retry policy, ecosystem defaults, tool policy, memory settings,\nprovider settings, and channel settings move to their\nrespective owners.\n\n## Repository settings\n\nThe existing `config.Repo` MUST NOT move intact into a repository domain or\nshared settings package. A managed repository is an application association:\n\n- work intake owns which repository references an identity monitors;\n- task execution owns gates, preflight, test protection, ecosystem behaviour,\n and change limits;\n- task lifecycle owns retry behaviour;\n- scheduling owns concurrency policy;\n- workspace management owns base branch and persistent-storage requirements.\n\nApplication composition correlates these settings by repository reference. No\ndomain receives a universal repository profile.\n\n## Documentation\n\nGenerated configuration reference describes external sections and their owning\ncapabilities. It MUST NOT establish a global Go configuration structure as the\npublic contract.\n\n## Value classification rule\n\nEvery value that controls behaviour MUST have an explicit owner and MUST be\nclassified as exactly one of:\n\n1. **Invariant** — changing it changes the meaning of the program, a wire\n protocol, or a domain concept. It may be a Go constant.\n2. **Runtime setting** — an operator, identity, domain, plugin, or deployment\n may legitimately choose a different value. It is typed configuration owned by\n the affected capability.\n3. **Policy value** — it expresses what is allowed, limited, routed, retried,\n protected, or gated. Its typed definition belongs to the owning domain and it\n is evaluated through the policy system.\n4. **Derived value** — it is computed from other settings and is not\n independently mutable.\n5. **Runtime state** — it changes as the application operates, such as the\n active model for a conversation. It is changed by a domain command and is not\n configuration merely because it can be changed.\n\nAn unclassified behavioural literal is a defect.\n\n`const` MUST NOT be used merely because a value currently has one default.\nConstants are reserved for actual invariants: enum discriminants, protocol\nversions, fixed compatibility tokens, and values whose alteration would change\nthe program's semantics rather than configure an installation.\n\nIn Go, a constant is a compile-time value and does not represent mutable runtime\nstorage. It generally has no address that can be observed or changed.\nArchitecturally, a constant therefore asserts that the value cannot change for\nthe lifetiLine truncated
},
{
"url": "/dev/architecture/dashboard-agent-tools/",
@@ -58,13 +58,13 @@
"url": "/dev/architecture/messaging-and-work-intake/",
"title": "Messaging and Work Intake",
"section": "Architecture",
"body": "# Messaging and Work Intake\n\n**Status:** Current state researched; foundation decisions in progress \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchie is a personal assistant and general agentic system. Its interactions\ninclude conversation, questions, notes, reminders, goals, message handling,\ntroubleshooting, infrastructure operations, server management, source changes,\nand open-ended future capabilities.\n\nA message to an agent is therefore not inherently a work request, and invoking\nan agent is not inherently a WorkflowExecution.\n\nMessaging and work intake are separate from the Agent System's durable workflow\nexecution model:\n\n- messaging preserves interactions with people and external systems;\n- an agent turn interprets and responds to an interaction using its available\n capabilities;\n- work intake handles the optional promotion of an interaction into durable,\n workflow-backed work;\n- the Agent System owns Workflow definitions and creates WorkflowExecutions.\n\nForge issues, Jira issues, Telegram messages, emails, webhooks, and future\nchannels remain peer external message or artifact sources. None becomes an\nexecution model merely because it can lead to work.\n\n## Confirmed boundary\n\nWork intake MUST NOT create a WorkflowExecution directly.\n\nInbound interactions normally route to the Agent selected by the channel\nbinding. The selected Agent retains its own memory, user knowledge, conversation\ncontinuity, specialisation, and capability context. This continuity is\nindependent of the underlying model or provider.\n\nEach selected Agent acts through its own immutable IdentityID. Interaction and\nresulting action records preserve both that acting identity and the initiating\nuser identity when one exists.\n\nAgent memory is private by default. Interaction context MAY additionally use\nuser-wide memory and intentionally global shared memory, but another Agent's\nagent-wide memory is not part of the selected Agent's context.\n\nMultiple users may interact with the same Agent. The channel binding and inbound\nmessage MUST resolve both AgentID and initiating User IdentityID before\nconversation or memory context is loaded. Sharing an Agent MUST NOT implicitly\nshare one user's private memory with another user.\n\nAn interaction may assemble memory from four scopes: global shared, the selected\nAgent's agent-wide memory, the initiating user's authorized user-wide memory,\nand the private Agent-user relationship memory addressed by both IDs.\n\n## Conversation identity\n\nEvery conversation is identified by the complete tuple:\n\n```text\nAgentID\n+ UserIdentityID\n+ ChannelBindingID\n+ ExternalConversationID\n+ ThreadID\n```\n\n`ThreadID` has an explicit empty value for channels without threads. No element\nof this key may be replaced by a display name, configured bot username, channel\ntype, or other mutable label.\n\nAgent and user memory continue across conversations according to their memory\nscope. Immediate conversation history and delivery correlation remain isolated\nto the identified channel conversation or thread. Moving between Telegram,\nemail, or another channel does not create a new Agent or user, but it does\ncreate a distinct conversation unless an explicit linking operation relates\nthem.\n\nConversation and message events support projections for channel-entry metrics,\nuse cases, volumes, outcomes, latency, capability usage, and other product or\noperational analysis. Metrics are derived read models; they MUST NOT become\nmutable counters embedded in the conversation aggregate or change message\nrouting semantics.\n\n## Canonical messages\n\nEvery inbound, assistant, tool-call, and tool-result message has an immutable\napplication-generated MessageID. Platform identifiers are external correlation,\nnot substitutes for the canonical ID.\n\nMessages are addressable because an interaction is more than text. A message may\ncontain assistant content, reasoning data subject to its retention policy, one\nor more tool calls, a tool result with authoritative execution metadata,\nattachments, reply relationships, edits, or other typed content. Stable\nMessageIDs also allow sessions to branch from an exact prior point.\n\nThe canonical session and message implementation must supply:\n\n- application-generated stable message IDs;\n- user, assistant, and tool roles;\n- assistant tool calls and correlated tool-result messages;\n- authoritative tool-result status, timing, error, truncation, and size\n metadata;\n- session lifecycle and active-request state;\n- session-to-agent-instance and parent-session lineage;\n- durable SQLite round trips and timestamp validation.\n\nThese semantics must use the confirmed Agent, user, channel-binding, and\nconversation ownership model.\n\nThe canonical message store MUST preserve messages as immutable records and\nrepresent later edits, deletions,Line truncated
"body": "# Messaging and Work Intake\n\n**Status:** Current state researched; foundation decisions in progress \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nArchie is a personal assistant and general agentic system. Its interactions\ninclude conversation, questions, notes, reminders, goals, message handling,\ntroubleshooting, infrastructure operations, server management, source changes,\nand open-ended future capabilities.\n\nA message to an agent is therefore not inherently a work request, and invoking\nan agent is not inherently a WorkflowExecution.\n\nMessaging and work intake are separate from the Agent System's durable workflow\nexecution model:\n\n- messaging preserves interactions with people and external systems;\n- an agent turn interprets and responds to an interaction using its available\n capabilities;\n- work intake handles the optional promotion of an interaction into durable,\n workflow-backed work;\n- the Agent System owns Workflow definitions and creates WorkflowExecutions.\n\nForge issues, Jira issues, Telegram messages, emails, webhooks, and future\nchannels remain peer external message or artifact sources. None becomes an\nexecution model merely because it can lead to work.\n\n## Confirmed boundary\n\nWork intake MUST NOT create a WorkflowExecution directly.\n\nInbound interactions normally route to the Agent selected by the channel\nbinding. The selected Agent retains its own memory, user knowledge, conversation\ncontinuity, specialisation, and capability context. This continuity is\nindependent of the underlying model or provider.\n\nEach selected Agent acts through its own immutable IdentityID. Interaction and\nresulting action records preserve both that acting identity and the initiating\nuser identity when one exists.\n\nAgent memory is private by default. Interaction context MAY additionally use\nuser-wide memory and intentionally global shared memory, but another Agent's\nagent-wide memory is not part of the selected Agent's context.\n\nMultiple users may interact with the same Agent. The channel binding and inbound\nmessage MUST resolve both AgentID and initiating User IdentityID before\nconversation or memory context is loaded. Sharing an Agent MUST NOT implicitly\nshare one user's private memory with another user.\n\nAn interaction may assemble memory from four scopes: global shared, the selected\nAgent's agent-wide memory, the initiating user's authorized user-wide memory,\nand the private Agent-user relationship memory addressed by both IDs.\n\n## Conversation identity\n\nEvery conversation is identified by the complete tuple:\n\n```text\nAgentID\n+ UserIdentityID\n+ ChannelBindingID\n+ ExternalConversationID\n+ ThreadID\n```\n\n`ThreadID` has an explicit empty value for channels without threads. No element\nof this key may be replaced by a display name, configured bot username, channel\ntype, or other mutable label.\n\nAgent and user memory continue across conversations according to their memory\nscope. Immediate conversation history and delivery correlation remain isolated\nto the identified channel conversation or thread. Moving between Telegram,\nemail, or another channel does not create a new Agent or user, but it does\ncreate a distinct conversation unless an explicit linking operation relates\nthem.\n\nConversation and message events support projections for channel-entry metrics,\nuse cases, volumes, outcomes, latency, capability usage, and other product or\noperational analysis. Metrics are derived read models; they MUST NOT become\nmutable counters embedded in the conversation aggregate or change message\nrouting semantics.\n\n## Canonical messages\n\nEvery inbound, assistant, tool-call, and tool-result message has an immutable\napplication-generated MessageID. Platform identifiers are external correlation,\nnot substitutes for the canonical ID.\n\nMessages are addressable because an interaction is more than text. A message may\ncontain assistant content, reasoning data subject to its retention policy, one\nor more tool calls, a tool result with authoritative execution metadata,\nattachments, reply relationships, edits, or other typed content. Stable\nMessageIDs also allow sessions to branch from an exact prior point.\n\nThe canonical session and message implementation must supply:\n\n- application-generated stable message IDs;\n- user, assistant, and tool roles;\n- assistant tool calls and correlated tool-result messages;\n- authoritative tool-result status, timing, error, truncation, and size\n metadata;\n- session lifecycle and active-request state;\n- session-to-agent-instance and parent-session lineage;\n- durable SQLite round trips and timestamp validation.\n\nThese semantics must use the confirmed Agent, user, channel-binding, and\nconversation ownership model.\n\nThe canonical message store MUST preserve messages as immutable records and\nrepresent later edits, deletions,Line truncated
},
{
"url": "/dev/architecture/migration-decisions/",
"title": "Architecture Migration Decisions",
"section": "Architecture",
"body": "# Architecture Migration Decisions\n\n**Status:** Active migration inventory \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nThis document records what remains to be decided before Archie's current\nimplementation can be migrated into the approved domain-oriented structure.\n\nThe product and core domain model are not being redesigned. Existing Archie\nbehaviour and the existing session implementation are the migration\nbaseline. Remaining decisions MUST be grounded in current packages, consumers,\ntests, persistence, and runtime wiring.\n\n## Confirmed domain packages\n\n```text\ninternal/domain/\n identity/ (approved, not yet materialised as a package)\n agent/ (approved, not yet materialised as a package)\n workflow/\n messaging/ (canonical Message/Conversation types only; gateway migration pending, see section 2)\n workintake/\n plugin/ (approved, not yet materialised as a package)\n curator/\n mapping/\n scheduling/\n```\n\n- `identity` owns persistent actors, lifecycle, ownership, attribution, and\n audit identity.\n- `agent` owns persistent assistants, specialisation, instructions,\n model-independent continuity, capability context, and associations with\n identity and memory.\n- `workflow` is a separate domain. It owns reusable, versioned Workflow\n definitions, WorkflowSteps, WorkflowExecutions, StepExecutions, Attempts,\n outcomes, and typed Workflow plugin contracts.\n- `messaging` owns canonical Messages, Conversations, branches, channel and\n source correlation, replies, acknowledgements, delivery attempts, events, and\n projections.\n- `workintake` owns admission, validation, deterministic routing, and accepted\n work requests for the optional transition into durable workflow-backed\n execution.\n- `plugin` owns generic plugin identity, discovery, compatibility, and lifecycle\n mechanics. It remains metadata-only; capability semantics and typed\n registration belong to the owning domain.\n- `curator` owns the curator engine family: long-running agent loops that\n reason over memory and perform maintenance tasks on a declared shape\n (check-in interval, tool set, attached memory engine). See epic\n archie-core-yp9.\n- `mapping` owns the payload field-mapping vocabulary: binding named fields to\n JSON paths inside a captured webhook payload, and detecting drift. See\n `docs/prds/payload-field-mapping.md`.\n- `scheduling` owns the ticker engine: deciding when a scheduled job runs and\n with what concurrency. It declares `JobSource`/`Runner` contracts and owns\n no storage or delivery. See epic archie-core-1786637496320-249-be6d9a20.\n\n**Reconciliation (2026-08-31):** `curator`, `mapping`, and `scheduling` existed\nin the tree without being added to this list when their epics landed. They are\nconfirmed as legitimate, narrowly-scoped domains under the same rules as the\noriginal six — each owns one capability family, declares typed contracts, and\nhas no cross-domain reach. This list was out of date, not the tree.\n\nConversely, `identity`, `agent`, `messaging`, and `plugin` are approved target\ndomains that do not yet exist as packages. Their behaviour currently lives in\npre-migration packages (see \"Complete package destination map\" below for the\nfull inventory). Do **not** create empty stub packages for them ahead of an\nactual migration PR — an unused stub is dead code and gives no test coverage.\nEach materialises when its corresponding migration decision below is resolved\nand implemented in the same change.\n\nMemory's final package placement remains undecided pending focused review of the\nexisting implementation.\n\nThe following are approved shared or composition packages, not domains:\n\n```text\ninternal/eventbus/\ninternal/policy/\ninternal/infrastructure/\ninternal/app/archied/\ninternal/app/agentworker/\n```\n\n## Remaining migration decisions\n\n### 1. Complete package destination map\n\nEvery current package MUST receive one explicit disposition:\n\n- move into a confirmed domain;\n- move under infrastructure as an implementation of a domain-owned contract;\n- move into application composition;\n- merge with an identified duplicate;\n- remain as an approved shared application contract;\n- move to a plugin, extra, deployment, example, or generated artifact;\n- be deleted after its replacement is proven.\n\nThe largest unreviewed areas include memory, tools and capabilities, workspace\nand worktree behaviour, gates, storage, events, web UI, containers, RPC\npackages, release management, and scheduling.\n\nThe destination matrix MUST name the behaviour owner, target path, dependencies\nto remove, state and contracts to preserve, migration prerequisites, and\ndeletion criteria.\n\n**`internal/webui` / UI Service — RATIFIED Phase 3 disposition (2026-09-07).**\nThe current owner is the in-process HTTP/SSE dashboard: SPA hosting, operator\nauthLine truncated
"body": "# Architecture Migration Decisions\n\n**Status:** Active migration inventory \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Purpose\n\nThis document records what remains to be decided before Archie's current\nimplementation can be migrated into the approved domain-oriented structure.\n\nThe product and core domain model are not being redesigned. Existing Archie\nbehaviour and the existing session implementation are the migration\nbaseline. Remaining decisions MUST be grounded in current packages, consumers,\ntests, persistence, and runtime wiring.\n\n## Confirmed domain packages\n\n```text\ninternal/domain/\n identity/ (approved, not yet materialised as a package)\n agent/ (approved, not yet materialised as a package)\n workflow/\n messaging/ (canonical Message/Conversation types only; gateway migration pending, see section 2)\n workintake/\n plugin/ (approved, not yet materialised as a package)\n curator/\n mapping/\n scheduling/\n```\n\n- `identity` owns persistent actors, lifecycle, ownership, attribution, and\n audit identity.\n- `agent` owns persistent assistants, specialisation, instructions,\n model-independent continuity, capability context, and associations with\n identity and memory.\n- `workflow` is a separate domain. It owns reusable, versioned Workflow\n definitions, WorkflowSteps, WorkflowExecutions, StepExecutions, Attempts,\n outcomes, and typed Workflow plugin contracts.\n- `messaging` owns canonical Messages, Conversations, branches, channel and\n source correlation, replies, acknowledgements, delivery attempts, events, and\n projections.\n- `workintake` owns admission, validation, deterministic routing, and accepted\n work requests for the optional transition into durable workflow-backed\n execution.\n- `plugin` owns generic plugin identity, discovery, compatibility, and lifecycle\n mechanics. It remains metadata-only; capability semantics and typed\n registration belong to the owning domain.\n- `curator` owns the curator engine family: long-running agent loops that\n reason over memory and perform maintenance tasks on a declared shape\n (check-in interval, tool set, attached memory engine). See epic\n archie-core-yp9.\n- `mapping` owns the payload field-mapping vocabulary: binding named fields to\n JSON paths inside a captured webhook payload, and detecting drift. See\n `docs/prds/payload-field-mapping.md`.\n- `scheduling` owns the ticker engine: deciding when a scheduled job runs and\n with what concurrency. It declares `JobSource`/`Runner` contracts and owns\n no storage or delivery. See epic archie-core-1786637496320-249-be6d9a20.\n\n**Reconciliation (2026-08-31):** `curator`, `mapping`, and `scheduling` existed\nin the tree without being added to this list when their epics landed. They are\nconfirmed as legitimate, narrowly-scoped domains under the same rules as the\noriginal six — each owns one capability family, declares typed contracts, and\nhas no cross-domain reach. This list was out of date, not the tree.\n\nConversely, `identity`, `agent`, `messaging`, and `plugin` are approved target\ndomains that do not yet exist as packages. Their behaviour currently lives in\npre-migration packages (see \"Complete package destination map\" below for the\nfull inventory). Do **not** create empty stub packages for them ahead of an\nactual migration PR — an unused stub is dead code and gives no test coverage.\nEach materialises when its corresponding migration decision below is resolved\nand implemented in the same change.\n\nMemory's final package placement remains undecided pending focused review of the\nexisting implementation.\n\nThe following are approved shared or composition packages, not domains:\n\n```text\ninternal/eventbus/\ninternal/policy/\ninternal/infrastructure/\ninternal/app/archied/\ninternal/app/agentworker/\n```\n\n## Remaining migration decisions\n\n### 1. Complete package destination map\n\nEvery current package MUST receive one explicit disposition:\n\n- move into a confirmed domain;\n- move under infrastructure as an implementation of a domain-owned contract;\n- move into application composition;\n- merge with an identified duplicate;\n- remain as an approved shared application contract;\n- move to a plugin, extra, deployment, example, or generated artifact;\n- be deleted after its replacement is proven.\n\nThe largest unreviewed areas include memory, tools and capabilities, workspace\nand worktree behaviour, gates, storage, events, web UI, containers, RPC\npackages, release management, and scheduling.\n\nThe destination matrix MUST name the behaviour owner, target path, dependencies\nto remove, state and contracts to preserve, migration prerequisites, and\ndeletion criteria.\n\n**`internal/webui` / UI Service — RATIFIED Phase 3 disposition (2026-09-07).**\nThe current owner is the in-process HTTP/SSE dashboard: SPA hosting, operator\nauthLine truncated
},
{
"url": "/dev/architecture/observability/",
@@ -88,7 +88,7 @@
"url": "/dev/architecture/plugins-and-extensions/",
"title": "Plugins and Extensions",
"section": "Architecture",
"body": "# Plugins and Extensions\n\n**Status:** Approved foundation \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Plugin framework\n\nThe core plugin domain lives under `internal/domain/plugin`. It owns:\n\n- plugin identity and metadata;\n- discovery and registration contracts;\n- lifecycle rules;\n- capability registration;\n- validation and compatibility rules.\n\nThe generic plugin contract remains metadata-only. Functionality is exposed\nthrough narrow typed capability contracts with an owning registry or manager.\n\nWorkflow plugins belong to the Workflow capability family. The Workflow domain\nat `internal/domain/workflow` owns their typed registration, validation,\nversioning, definition, and execution contracts. The generic plugin domain may\nprovide plugin identity, discovery, compatibility, and lifecycle mechanics, but\nit MUST NOT define Workflow semantics or expose untyped Workflow hooks.\n\n## Plugin implementations\n\nFirst-party plugins are self-contained vertical modules under\n`plugins/\u003cplugin\u003e`.\n\nA plugin:\n\n- owns its implementation, settings, templates, assets, and tests;\n- receives only explicitly supplied contracts and services;\n- MUST NOT receive daemon internals, a service locator, or unrestricted\n application access;\n- MUST NOT import arbitrary infrastructure or application composition code;\n- MAY use approved domain and shared application contracts;\n- MUST remain removable without changes to unrelated domains.\n\nPlugins may be internally cohesive like small services. They do not become\nnetwork services without a concrete security or operational reason.\n\n## Event sources and reactions\n\n`archie-core-7d5u.1` asked whether extensions should react to events\nin-process through typed capability families (this document's model) or\nout-of-process like a generic `DispatchEvent` dispatcher. Decided 2026-08-22\nin `docs/prds/event-sources-and-reactions.md`: in-process typed families,\nconfirming this document's model rather than amending it. The forge webhook\nreceiver (`internal/forge/webhook`, see `ARCHITECTURE.md`'s \"Webhook intake\"\nsection) and the EDA playbook engine's Module action position\n(`docs/prds/eda-playbook-engine.md`, `internal/domain/eda/module`) are the\nshipped instances of that decision -- each is a narrow typed contract with an\nowning registry, not a generic event hook. The playbook engine's Channel and\nForge action positions are designed but not yet implemented (open\ninvestigation `archie-core-t2db.19`); when they land they follow the same\nmodel, not a new one.\n\n## Curator surprisal sampling\n\n**Decided 2026-08-05, wave 1 landed 2026-09-05** — the curator engine family\n(epic [#435](https://github.com/samcharles93/archie-core/issues/435)) selects which memories deserve agentic attention via\nsurprisal-based sampling: score memories by how surprising they are and\nspend the sampled reasoner's agentic budget on the most surprising items.\n\nThe prerequisites are built: the embeddings capability\n([#436](https://github.com/samcharles93/archie-core/issues/436), `internal/domain/embedding` contract +\n`internal/infrastructure/embedding` implementation, config-driven via\n`models.embedding`) and the wave-1 `Sampler` strategy seam\n(`internal/domain/sampling` -- not `internal/curator`, see\n`docs/prds/curator-sampler-wave1.md` for the settled design and tracking)\nwith four\ncheap, embedding-free strategies (recency, random, all, staleness proxy).\nNo curator consumes a `Sampler` yet, and the embeddings client is likewise\ndeliberately inert: `setupEmbeddings` constructs it at boot and holds it on\n`b.embeddings`, but no production code reads that field yet, so a resolved\n`models.embedding` role currently produces a client whose only observable\neffect is a boot log line. Both are prerequisites waiting on #437, not\nunwired defects.\n\nStill deferred: [#437](https://github.com/samcharles93/archie-core/issues/437), the embedding-backed surprisal strategy, which\ndepends on the above and on [#407](https://github.com/samcharles93/archie-core/issues/407). Requirements for that deferred work,\nso it can be picked up without re-deriving the design:\n\n- A `Sampler` implementation behind the same interface as the cheap\n strategies; selection deterministic given fixed inputs.\n- Algorithm: sample up to a candidate cap; build a k-NN graph over embedding\n vectors; row-normalize the adjacency with self-loops; power-iterate to the\n stationary distribution; surprisal = −log(stationary probability of the\n nearest point); select the highest-surprisal items.\n- Degradation: content without an embedding is skipped; an embedding failure\n aborts sampling and falls back to free exploration — a sampling failure\n must never fail or crash a curator pass.\n- Cost bound: the graph build is quadratic in the candidate cap; the cap must\n be a named constant with a documentLine truncated
"body": "# Plugins and Extensions\n\n**Status:** Approved foundation \n**Date:** 2026-07-28 \n**Tracking issue:** [#73](https://github.com/samcharles93/archie-core/issues/73)\n\n## Plugin framework\n\nThe core plugin domain lives under `internal/domain/plugin`. It owns:\n\n- plugin identity and metadata;\n- discovery and registration contracts;\n- lifecycle rules;\n- capability registration;\n- validation and compatibility rules.\n\nThe generic plugin contract remains metadata-only. Functionality is exposed\nthrough narrow typed capability contracts with an owning registry or manager.\n\nWorkflow plugins belong to the Workflow capability family. The Workflow domain\nat `internal/domain/workflow` owns their typed registration, validation,\nversioning, definition, and execution contracts. The generic plugin domain may\nprovide plugin identity, discovery, compatibility, and lifecycle mechanics, but\nit MUST NOT define Workflow semantics or expose untyped Workflow hooks.\n\n## Plugin engine rule (strict)\n\n“Engine” is reserved for a typed, lifecycle-managed capability family. It is not\na synonym for a generic plugin hook. Every new plugin engine must satisfy all of\nthese invariants:\n\n1. **Typed domain contract.** The engine interface exposes real\n capability-specific operations in addition to identity metadata. Examples are\n resolving a secret, delivering a message, or running an agent. A plugin that\n only exposes `Name` and `Version` is metadata, not an engine.\n2. **Owning family manager.** The capability package owns a `Registry` or\n `Manager` that controls provider registration and discovery, initialization,\n or access. Providers do not register arbitrary callbacks directly on the\n daemon.\n3. **Lifecycle and health.** When an implementation owns resources or background\n work, its typed family contract must expose start, health, and stop\n semantics. The owning manager invokes them and defines failure isolation and\n shutdown ordering. Stateless providers may make these operations explicit\n no-ops.\n4. **Narrow host access.** Plugins receive the typed family registrar and only\n the host services their declared capability requires. They never receive\n `*daemon.Daemon`, a service locator, or an untyped hook map.\n5. **Metadata-only generic plugin contract.** `plugin.Plugin` remains limited to\n `Name()` and `Version()`. Domain hooks belong on typed engine interfaces;\n adding capability methods to the generic interface is forbidden.\n6. **Trust boundary stays explicit.** Explicitly trusted, operator-installed\n plugins—including third-party secret-engine plugins the operator chooses to\n trust—may run in-process and have daemon privileges. Repository-supplied code\n runs in the task container. Integrations that are not trusted as daemon code\n run out of process through MCP or a versioned protocol and, when a\n confidentiality or integrity boundary is required, inside a real container\n sandbox. A subprocess boundary alone is not a security sandbox, and calling\n Yaegi-loaded code a plugin does not isolate it.\n\n`internal/plugin/architecture_test.go` enforces the mechanically testable parts\nof this rule: the generic plugin method set, the shared agent-instruction\nsource, and the requirement that engine interfaces have domain behavior plus an\nowning registry or manager.\n\n### Memory engine family\n\nOne engine, addressed by four typed scopes -- `ScopeGlobal`, `ScopeAgent`,\n`ScopeUser`, `ScopeAgentUser` (`internal/domain/memory`, `Scope`/`ScopeKind`).\nA caller names exactly the scopes it may read or write; the engine applies no\naccess policy of its own, so isolation falls out mechanically because\n`ScopeAgent{a}` and `ScopeAgent{b}` are different storage keys. See\n`docs/prds/memory-engine-unification.md` for the decision this contract\nimplements; it superseded `internal/memory` (`MemoryProvider`/`Manager`),\ndeleted in that PRD's slice 5.\n\n`MemoryEngine` (`internal/domain/memory/contract.go`) is the typed family\ncontract: `Create`, `Get`, `Query`, `List`, `Update`, `Forget`, `Revisions` --\nCRUD with revisions retained, not just create/read/delete. `Update` supersedes\nrather than overwrites: the superseded state is appended to the scope's\n`HISTORY.md` before the live block is replaced, and `Update` carries no\n`Scope` field, so a record can never change scope. The family's content\nscanner (prompt-injection / sensitive-data patterns) is applied inside\n`Create` and `Update`, the single choke point every producer crosses.\n\n`internal/domain/memory.Registry` is the owning family registry: registration,\nstart/health/stop, shutdown ordering, and failure isolation, following the\nplugin engine rule above. `internal/infrastructure/memory.NewBuiltinEngine` is\nthe only registered engine today; it persists through\n`internal/infrastructure/memory/builtin`, a markdown store (`MEMORY.md`-shaped\nper-scope files) that owns its on-diLine truncated
},
{
"url": "/dev/architecture/policy/",
@@ -126,6 +126,12 @@
"section": "PRDs",
"body": "# Agent2Agent as archie's agent-to-agent protocol — decision\n\n**Status:** Draft\n**Date:** 2026-09-22\n**Authority:** the A2A Protocol Specification v1.0.0 (`a2a-protocol.org`) for the\nprotocol, `docs/architecture/organisation.md` for package placement, and\n`docs/architecture/migration-decisions.md` §2 for session and messaging design.\n\n## Decision\n\nA2A is the protocol archie speaks to other agents. It carries agent-to-agent\nwork: discovery of what a remote agent offers, delegation of a unit of work to it,\nprogress on that work, and the artifacts it produces.\n\n## Layers\n\n**MCP stays the tool layer.** MCP connects a model to tools and data; A2A connects\nagents to agents. They are complementary and archie runs both:\n`internal/tools/provider` registers MCP providers as optional capabilities, so an\nA2A call and an MCP tool call are different kinds of traffic with different\nowners.\n\n**A2A rides archie's existing transports.** A2A's canonical data model is\nProtocol Buffer messages and it defines a gRPC binding, so archie speaks A2A over\nthe gRPC and NATS it already runs. A second wire format, a second broker, or a\nbespoke protocol for agent-to-agent traffic is a requirement not to introduce.\n\n## Where A2A stops\n\nA2A is not a general replacement for either of the two protocols archie already\nhas, and each boundary is a requirement rather than a preference.\n\n- **`internal/domain/messaging` stays the human and channel layer.** A\n Conversation is a ChannelID and a ThreadID, its participants are people on\n Telegram, email, webhook and the dashboard, and its records are what a curator\n reads. An A2A peer is not a channel participant and does not address a\n Conversation.\n- **`internal/agentexec` stays the daemon-to-worker execution protocol.** It\n carries a Request, a Result, gate and budget policy, and capture tools between\n the daemon and the agent container it owns. An A2A peer never receives that\n protocol, and A2A never carries gate, budget or protection policy.\n\nA2A is therefore an edge protocol: it exchanges work with peers outside archie's\nown process boundary, and it composes with the two protocols above rather than\nreplacing or abstracting them.\n\n## A2A Task mapping\n\n**Decision: archie's work lifecycle is mapped to A2A's, not expressed in it. An\nA2A Task is a view over archie's own task, and archie's status remains the record\nof truth.**\n\nThis is the load-bearing decision, because A2A's Task carries its own state\nmachine and archie's lifecycle is the on-disk format of its task store, whose\nstatus strings a rename migrates rather than changes.\n\nThe mapping is partial, and each gap is a case where archie knows something A2A's\nstates do not carry:\n\n- **Completion by another system.** A2A's terminal states describe what the agent\n did (`completed`, `failed`, `canceled`, `rejected`). archie's `merged` and\n `rejected` describe what a forge did to a pull request after archie finished\n its work. There is no A2A state for \"an external system accepted this\", so a\n completed A2A task cannot express the outcome that matters most to the caller.\n- **Handing work to a reviewer.** archie's `pr_open` means the work left\n archie's hands for human review. A2A's `input-required` is the nearest state,\n but it means the *client* must send more input, and a reviewer is not the\n client.\n- **Operator decisions.** `declined`, `parked` and `dead` are outcomes of\n archie's own policy and operator actions. A2A has no state for \"the operator\n refused this\" or \"a budget ran out and an operator must decide\".\n\nA WorkflowExecution is the unit an A2A task maps onto, because it is the attempt\nthat runs, streams and produces artifacts, and a follow-up attempt is a new\nexecution. A2A's own model agrees that a task is immutable once terminal and that\na refinement is a new task sharing the `contextId`, which is why archie's\nper-attempt executions and A2A's per-request tasks correspond.\n\n**Requirement:** the A2A state a peer sees is derived from archie's status by one\nmapping, and archie's status is carried on the task so a peer can read what the\nderivation lost. The mapping is total in the direction that matters -- every\narchie status has an A2A state -- and deliberately not injective, because the\ngaps above are facts A2A cannot hold.\n\n**Requirement:** adopting A2A's state names as archie's persisted task status is\nrejected. It would migrate the task store, and it would drop the forge outcomes\nthat tell a caller whether the work was accepted.\n\n## What an A2A agent is\n\n**An A2A agent is an archie Identity.** The Agent Card describes a durable,\naddressable actor with an endpoint, capabilities and authentication requirements,\nand an Identity is the actor archie persists, versions, suspends and scopes\ncredentials to.\n\nAn `agent.Persona` is not an A2A agent: a persona is prompt text selected for a\nturn, and it has no endpLine truncated
},
{
"url": "/dev/prds/artifact-sender/",
"title": "Artifact sender",
"section": "PRDs",
"body": "# Artifact sender\n\n**Status:** Approved\n**Revision:** 1\n**Authority:** this document\n\n## Purpose\n\narchie-core publishes artifacts — design documents, review reports, research briefs,\nrunnable pages — to the offloaded.dev collaborative editor. A person reads the\nartifact, comments on it, and the artifact page is the shared surface for iteration.\n\nThe receiving contract is owned by the editor: `POST /api/artifacts` with a bearer\nservice token, a closed field set, and server-side idempotency on `requestId` plus\nlast-write-wins on `sentAt`. This document owns only what archie-core sends.\n\n## Scope\n\nVersion one publishes one-way: archie-core sends, the editor receives. The first\nproducer is the pull-request adversarial review: a completed `ReviewReport` is\nrendered to markdown and published after the review completes.\n\nChat-surface tools (`create_artifact`, `update_artifact`), comment read-back, share\ntokens, and runnable-format producers (`html`, `react`, `vue`) are later slices. The\nreceiving contract already carries them; version one does not exercise them.\n\n## Configuration\n\n- New `[artifacts]` section: `base_url` (site origin), `token` (`SecretRef`) with\n `token_env` fallback naming `WORKSPACE_INGEST_TOKEN`.\n- An empty `base_url` disables the sender: no client is constructed, nothing is\n registered, and publishing degrades to a log line.\n- The token is runtime configuration. It is never baked at build time.\n\n## Sender\n\n- New package `internal/infrastructure/artifactsync` exposing one method,\n `Publish(ctx, Artifact) (string, error)`, where the string is the artifact URL.\n One-method injected seam, concrete client constructed by the boot path when the\n configuration resolves — the `crondelivery.Courier` posture.\n- The client POSTs the frozen contract body. Each publish sends the complete\n source: versions of record are full snapshots, never diffs.\n- A `503` response is retried three times with backoff. `401` and `400` are not\n retried; they are logged with the field reason the response names.\n- `requestId` is `artifact:\u003ctask-id\u003e:\u003cversion\u003e` when the producer is task-backed,\n otherwise `artifact:\u003cidentity\u003e:\u003csentAt-utc\u003e`. Sending the same `requestId` with\n an equal or older `sentAt` must be a server-side no-op, which makes sender-side\n retry safe.\n\n## Report rendering\n\n- `ReviewReport` gains a pure markdown renderer in `internal/domain/workflow`:\n the summary as the lede, one finding per subsection with file, line, verdict,\n level and the failure scenario, and the verified properties as a \"checked\n clean\" list. Findings carry their disposition when present.\n- The renderer performs no I/O and imports no infrastructure.\n\n## Attribution\n\n- `actor`: the producing identity. The reviewer publishes as `system:pr-reviewer`\n when no task identity resolves; otherwise the task's identity name.\n- `source`: the producing subsystem, `pr-review` for version one.\n- `task`: owner, repository and issue number from the task when one exists.\n\n## Failure behaviour\n\n- Publishing never fails the producing work. A failed publish is logged with its\n status and reason; the artifact is republished on the next production of the\n same artifact.\n- A successful publish records the artifact URL in the run report, so the PR\n description or task log links to the readable artifact.\n- The published URL is the editor deep link for the artifact, keyed on its\n immutable id. archie-core never mints share links; sharing is an\n account-holder decision made in the editor.\n\n## Verification\n\n- Client tests against a test server cover 201, 401, 400 with a field reason,\n and 503 retry behaviour.\n- The report renderer test covers a report with findings, checks, dispositions\n and a not-run status.\n- Configuration tests cover the disabled case, the `SecretRef` path and the\n `token_env` fallback."
},
{
"url": "/dev/prds/binding-secret-encryption/",
"title": "Binding secret at-rest encryption -- decision",
@@ -202,7 +208,7 @@
"url": "/dev/prds/deep-research-capability/",
"title": "Deep research capability — design",
"section": "PRDs",
"body": "# Deep research capability — design\n\n**Status:** Draft\n**Date:** 2026-09-14\n**Scope:** A deep-research capability in archie-core: an orchestrator that\nspawns 1..N subagents to explore and research in parallel, collates their\nfindings, and returns one cited report. Grounded in web search and web fetch.\n**Beads issue:** none yet (this document is the input to it)\n**Depends on:** nothing shipped for slice 1; slice 2 depends on slice 1.\n\n## §0 Implementation status\n\nEvery claim in this document is mapped to what exists in the tree.\n\n| Claim | Status | Where it stands |\n|---|---|---|\n| `web_fetch` tool (URL → readable text) | **Implemented** | `internal/tools/webfetch/{tool.go,webfetch.go}`; registered in `internal/app/archied/bootstrap.go` `registerStandaloneTools`, disabled-by-config returns `nil` |\n| **Web search** (query → result list) | **Aspirational — zero code** | No search provider, no search tool, no config block anywhere in the tree or in `ai-sdk` v0.1.32 |\n| `agent.Subagent` — nested agent in a fresh context window | **Implemented** (dependency) | `github.com/samcharles93/ai-sdk@v0.1.32/agent/subagent.go`. Synchronous, non-streaming, own model/system/toolset, `MaxSteps` defaults to 10, nesting depth capped at 5, returns final text only |\n| Subagent used in production | **Implemented, one consumer** | `internal/app/agentworker/review.go` — the adversarial reviewer. That is the only non-test use of `Subagent` in the repo |\n| A **delegate/subagent tool** on the chat surface | **Aspirational — zero code** | No `subagent`/`delegate` tool name is registered anywhere |\n| **Fan-out orchestration** (N missions, one collator) | **Aspirational — zero code** | `agentloop`'s package doc states it runs *one* mission: \"orchestration across missions lives with the caller\". No caller does this yet |\n| Parallel-safe tool execution | **Aspirational — zero code** | The unreachable `internal/tools/dispatch.go` dispatch/gating engine (`ExecutionClass`, `DispatchSequential`/`DispatchConcurrent`/`ClassifyEntry`) was deleted as dead code (tools-core-1); live concurrency is ai-sdk's `GenerateOptions.MaxParallelToolCalls`, which archie-core never sets |\n| Per-result caps + spill-to-disk | **Implemented** | `internal/agentexec/toolset.go` `ToolLimits{MaxResultChars,SpillDir}`, applied in `toolExecute` via `tools.CapPayload` (`limit.go`); no **aggregate** turn cap by design (`toolset.go`). `config.ToolPolicy.SpillDir`; default cap 50k chars, spill dir deliberately not defaulted |\n| Tool guardrails (repeat-failure / no-progress) | **Implemented, not on the call path** | `internal/tools/guardrail.go` engine exists. Its only non-test caller is `dispatchOne` — which is itself unreachable — so **the chat path never consults it**, and the agent path only records per *stage* (`domain/workflow/agent.go`) |\n| Structured capture tools (model reports into a typed store, not prose) | **Implemented** | `internal/app/agentworker/review.go` `record_finding` / `record_checked` — `core.NewTypedTool` appending to a caller-owned slice. This is the pattern a research source ledger should reuse |\n| Curator engine (interval, declared tool set, lifecycle) | **Implemented, tools broken** | `internal/domain/curator`; a curator declaring tools is **refused** at registration (`registry.go`, no `ToolBuilder` implementation) and `curatorLLMRunner` (`internal/app/archied/main.go`) drops `ChatRequest.Tools`. Budgets are time+concurrency only (`runtime.go`) — **no token budget** |\n| Cron / scheduling + delivery router | **Implemented, wired into no binary** | `internal/domain/scheduling`, `internal/infrastructure/cronstore` (schema v2, `Kind` discriminator), `internal/infrastructure/crondelivery` — all present and green, but `go list -deps` for every `cmd/` contains none of them, and `cron`-expression firing arithmetic is unimplemented (`schedule.go`, \"slice 2\"). `ARCHITECTURE.md` still asserts \"**No daemon-internal cron.**\" |\n| Workflow engines (multi-stage, resumable) | **Implemented, worktree-bound** | `internal/domain/workflow` — every non-triage workflow opens with `StagePrepareWorktree()` and mounts `tc.Dir` into a container; the task persists after every stage (`workflow.go`). `triage.go` documents there is \"no cheaper directory-less classification primitive\" |\n| Long-running chat-initiated work with progress | **Partial** | Chat turns run **once, synchronously**, inline in the channel's turn runner (`internal/gateway/turn.go`). Chat can spawn a *workflow task* (`task_spawn`) which is asynchronous — but that path is worktree/PR-shaped |\n| **Operator-push delivery of an async result** | **Aspirational — zero code** | Nothing pushes a finished result to a chat. No channel subscribes to the event bus (Telegram only *publishes* `KindTurnCompleted`); `crondelivery.Courier` is the intended seam but has **no implementation and no caller**; `send_file` works only inside theLine truncated
"body": "# Deep research capability — design\n\n**Status:** Draft\n**Date:** 2026-09-14\n**Scope:** A deep-research capability in archie-core: an orchestrator that\nspawns 1..N subagents to explore and research in parallel, collates their\nfindings, and returns one cited report. Grounded in web search and web fetch.\n**Beads issue:** none yet (this document is the input to it)\n**Depends on:** nothing shipped for slice 1; slice 2 depends on slice 1.\n\n## §0 Implementation status\n\nEvery claim in this document is mapped to what exists in the tree.\n\n| Claim | Status | Where it stands |\n|---|---|---|\n| `web_fetch` tool (URL → readable text) | **Implemented** | `internal/tools/webfetch/{tool.go,webfetch.go}`; registered in `internal/app/archied/bootstrap.go` `registerStandaloneTools`, disabled-by-config returns `nil` |\n| **Web search** (query → result list) | **Aspirational — zero code** | No search provider, no search tool, no config block anywhere in the tree or in `ai-sdk` v0.1.32 |\n| `agent.Subagent` — nested agent in a fresh context window | **Implemented** (dependency) | `github.com/samcharles93/ai-sdk@v0.1.32/agent/subagent.go`. Synchronous, non-streaming, own model/system/toolset, `MaxSteps` defaults to 10, nesting depth capped at 5, returns final text only |\n| Subagent used in production | **Implemented, one consumer** | `internal/app/agentworker/review.go` — the adversarial reviewer. That is the only non-test use of `Subagent` in the repo |\n| A **delegate/subagent tool** on the chat surface | **Aspirational — zero code** | No `subagent`/`delegate` tool name is registered anywhere |\n| **Fan-out orchestration** (N missions, one collator) | **Aspirational — zero code** | `agentloop`'s package doc states it runs *one* mission: \"orchestration across missions lives with the caller\". No caller does this yet |\n| Parallel-safe tool execution | **Aspirational — zero code** | The unreachable `internal/tools/dispatch.go` dispatch/gating engine (`ExecutionClass`, `DispatchSequential`/`DispatchConcurrent`/`ClassifyEntry`) was deleted as dead code (tools-core-1); live concurrency is ai-sdk's `GenerateOptions.MaxParallelToolCalls`, which archie-core never sets |\n| Per-result caps + spill-to-disk | **Implemented** | `internal/agentexec/toolset.go` `ToolLimits{MaxResultChars,SpillDir}`, applied in `toolExecute` via `tools.CapPayload` (`limit.go`); no **aggregate** turn cap by design (`toolset.go`). `config.ToolPolicy.SpillDir`; default cap 50k chars, spill dir deliberately not defaulted |\n| Tool guardrails (repeat-failure / no-progress) | **Implemented, not on the call path** | `internal/tools/guardrail.go` engine exists. Its only non-test caller is `dispatchOne` — which is itself unreachable — so **the chat path never consults it**, and the agent path only records per *stage* (`domain/workflow/agent.go`) |\n| Structured capture tools (model reports into a typed store, not prose) | **Implemented** | `internal/app/agentworker/review.go` `record_finding` / `record_checked` — `core.NewTypedTool` appending to a caller-owned slice. This is the pattern a research source ledger should reuse |\n| Curator engine (interval, declared tool set, lifecycle) | **Implemented, tools broken** | `internal/domain/curator`; a curator declaring tools is **refused** at registration (`registry.go`, no `ToolBuilder` implementation) and `curatorLLMRunner` (`internal/app/archied/main.go`) drops `ChatRequest.Tools`. Budgets are time+concurrency only (`runtime.go`) — **no token budget** |\n| Cron / scheduling + delivery router | **Implemented, wired into the daemon** | `internal/domain/scheduling` (ticker engine), `internal/infrastructure/cronstore` (schema v2, `Kind` discriminator), and `internal/infrastructure/crondelivery`; `internal/app/archied/scheduling.go` composes them and `main.go` starts the engine. |\n| Workflow engines (multi-stage, resumable) | **Implemented, worktree-bound** | `internal/domain/workflow` — every non-triage workflow opens with `StagePrepareWorktree()` and mounts `tc.Dir` into a container; the task persists after every stage (`workflow.go`). `triage.go` documents there is \"no cheaper directory-less classification primitive\" |\n| Long-running chat-initiated work with progress | **Partial** | Chat turns run **once, synchronously**, inline in the channel's turn runner (`internal/gateway/turn.go`). Chat can spawn a *workflow task* (`task_spawn`) which is asynchronous — but that path is worktree/PR-shaped |\n| **Operator-push delivery of an async result** | **Aspirational — zero code** | Nothing pushes a finished result to a chat. No channel subscribes to the event bus (Telegram only *publishes* `KindTurnCompleted`); `crondelivery.Courier` is the intended seam but has **no implementation and no caller**; `send_file` works only inside the live turn. The one out-of-band notification in the tree is feasibility's n8n webhook (`feasibility.go`) |\n| Citation storage / sourLine truncated
},
{
"url": "/dev/prds/dynamic-curators/",
@@ -220,7 +226,7 @@
"url": "/dev/prds/eda-playbook-engine/",
"title": "EDA playbook engine -- design",
"section": "PRDs",
"body": "# EDA playbook engine -- design\n\n**Status:** Draft\n**Date:** 2026-09-03\n**Beads epic:** `archie-core-t2db`\n\n## Problem\n\n\"Which workflow runs for which trigger\" is hardcoded Go: a literal\nlabel vocabulary (`workintake.labelKinds`) and a literal label-\u003eworkflow map\n(`workflow/routing.go`), extended only by adding a Go case or shipping a\nskill whose `metadata.archie.workflow` overrides one *named* built-in\nworkflow. Extending the system to a new trigger, a new action, or a new\nevent source means a feature branch, a code change in a hardcoded\npath, and new Go tests -- for every addition. That is the barrier this\ndocument exists to remove.\n\nThe goal: triggers, the actions they run, and the\nconditions that connect them should be **drop-in and load**, not **drop-in\nand touch application code**. Yaegi-evaluated Go is the unit of imperative\n\"do a thing\" logic; YAML playbooks are pure orchestration data -- no logic,\njust \"on this event, with this data, call these actions, in this order,\nunder these conditions.\" The daemon's event coordinator doesn't know what a\n`bugfix` or `image-gen` action does, it resolves the name against a\nregistry and pipes typed data through.\n\nThis is explicitly **not** scoped to code-generation workflows. A playbook\naction can be any registered capability -- run a codegen workflow, send a\nmessage on a channel, open something on a forge, generate an image -- so\nlong as it's implemented behind one of the typed engine families below.\n\n## Relationship to existing decisions\n\n- `docs/prds/event-sources-and-reactions.md` ruled\n that reactions in the two epics it covers are **producer-only** (no veto,\n no mutation of in-flight work) and explicitly rejected a single generic\n `Source` interface as \"the untyped-hook shape the plugin engine rule\n exists to prevent.\" **This document extends that scope**: playbooks can\n dispatch to multiple typed action kinds, not just \"produce a\n `TaskEnvelope`.\" It does not relitigate producer-only-ness (still no\n veto/mutation of in-flight work here) and it does not introduce one\n generic interface -- see \"Typed positions, not one generic hook\" below,\n which is how it stays compliant with the plugin engine rule instead of\n reopening that document's #4.\n- `ARCHITECTURE.md#plugin-engine-rule-strict` governs every extension point\n introduced here. Every position below is a typed domain contract with an\n owning registry, not a generic callback map. This is a hard constraint,\n confirmed against the existing rule text before writing this doc.\n- Yaegi is already a proven in-process extension mechanism\n (`internal/plugin`, `internal/domain/workflow/wfeval/yaegi.go`,\n `internal/gate/gateeval/yaegi.go`, `internal/skill/plugin.go`) -- this\n design generalizes its existing role, it does not introduce a new\n execution mechanism.\n- `internal/domain/workflow/skillbuild.BuildRegistry` already composes a\n `workflow.Registry` from a skills catalog, plugin-defined overriding\n built-in. The playbook loader described here supersedes label routing\n specifically; it does not replace `skillbuild`'s workflow-composition role,\n it becomes an additional input the daemon composes at startup.\n- Webhook event intake is a separate design. This\n document defines the router each webhook-sourced event should be able to\n reach; it does not redesign webhook receipt itself.\n\n## Shape: four logic positions, each a typed engine family\n\nEvery entrypoint kind (mechanism, action, loop, schedule, route, trigger) is itself a schema emitter: defining a new kind produces the\ngenerated contract downstream tooling consumes. The kinds slot into a fixed\nhierarchy of logic positions:\n\n```\nWorkflow -\u003e Module -\u003e Channel (telegram/whatsapp/email/...)\n -\u003e Forge (GitHub/Gitea/GitLab/...)\n```\n\nEach position is its own typed engine family per the plugin engine rule:\nidentity + capability-specific operations, an owning `Registry`/`Manager`,\nexplicit lifecycle where the implementation owns resources, narrow host\naccess. Concretely:\n\n- **Workflow** -- already exists (`internal/domain/workflow`). A playbook\n action of kind `workflow` invokes a registered `workflow.Registry` entry\n by name, unchanged from the existing execution path.\n- **Module** -- the new general-purpose action position. A `Module` is a\n Yaegi-evaluated implementation of a generated interface (args in, typed\n result out) for one declared action kind (e.g. `image-gen`, `notify`,\n `open-pr-comment`). This is the position that makes actions \"not\n restricted to code-generation.\" Full design (schema-per-kind, the\n registry shape, the trust boundary, the recommended first slice):\n `docs/prds/module-position.md`.\n- **Channel** -- already exists as `channels.Channel` (telegram, email,\n webhook). A playbook action of kind `channel` sends through a named,\n already-registered Line truncated
"body": "# EDA playbook engine -- design\n\n**Status:** Draft\n**Date:** 2026-09-03\n**Beads epic:** `archie-core-t2db`\n\n## Problem\n\n\"Which workflow runs for which trigger\" is hardcoded Go: a literal\nlabel vocabulary (`workintake.labelKinds`) and a literal label-\u003eworkflow map\n(`workflow/routing.go`), extended only by adding a Go case or shipping a\nskill whose `metadata.archie.workflow` overrides one *named* built-in\nworkflow. Extending the system to a new trigger, a new action, or a new\nevent source means a feature branch, a code change in a hardcoded\npath, and new Go tests -- for every addition. That is the barrier this\ndocument exists to remove.\n\nThe goal: triggers, the actions they run, and the\nconditions that connect them should be **drop-in and load**, not **drop-in\nand touch application code**. Yaegi-evaluated Go is the unit of imperative\n\"do a thing\" logic; YAML playbooks are pure orchestration data -- no logic,\njust \"on this event, with this data, call these actions, in this order,\nunder these conditions.\" The daemon's event coordinator doesn't know what a\n`bugfix` or `image-gen` action does, it resolves the name against a\nregistry and pipes typed data through.\n\nThis is explicitly **not** scoped to code-generation workflows. A playbook\naction can be any registered capability -- run a codegen workflow, send a\nmessage on a channel, open something on a forge, generate an image -- so\nlong as it's implemented behind one of the typed engine families below.\n\n## Relationship to existing decisions\n\n- `docs/prds/event-sources-and-reactions.md` ruled\n that reactions in the two epics it covers are **producer-only** (no veto,\n no mutation of in-flight work) and explicitly rejected a single generic\n `Source` interface as \"the untyped-hook shape the plugin engine rule\n exists to prevent.\" **This document extends that scope**: playbooks can\n dispatch to multiple typed action kinds, not just \"produce a\n `TaskEnvelope`.\" It does not relitigate producer-only-ness (still no\n veto/mutation of in-flight work here) and it does not introduce one\n generic interface -- see \"Typed positions, not one generic hook\" below,\n which is how it stays compliant with the plugin engine rule instead of\n reopening that document's #4.\n- `docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`\n governs every extension point\n introduced here. Every position below is a typed domain contract with an\n owning registry, not a generic callback map. This is a hard constraint,\n confirmed against the existing rule text before writing this doc.\n- Yaegi is already a proven in-process extension mechanism\n (`internal/plugin`, `internal/domain/workflow/wfeval/yaegi.go`,\n `internal/gate/gateeval/yaegi.go`, `internal/skill/plugin.go`) -- this\n design generalizes its existing role, it does not introduce a new\n execution mechanism.\n- `internal/domain/workflow/skillbuild.BuildRegistry` already composes a\n `workflow.Registry` from a skills catalog, plugin-defined overriding\n built-in. The playbook loader described here supersedes label routing\n specifically; it does not replace `skillbuild`'s workflow-composition role,\n it becomes an additional input the daemon composes at startup.\n- Webhook event intake is a separate design. This\n document defines the router each webhook-sourced event should be able to\n reach; it does not redesign webhook receipt itself.\n\n## Shape: four logic positions, each a typed engine family\n\nEvery entrypoint kind (mechanism, action, loop, schedule, route, trigger) is itself a schema emitter: defining a new kind produces the\ngenerated contract downstream tooling consumes. The kinds slot into a fixed\nhierarchy of logic positions:\n\n```\nWorkflow -\u003e Module -\u003e Channel (telegram/whatsapp/email/...)\n -\u003e Forge (GitHub/Gitea/GitLab/...)\n```\n\nEach position is its own typed engine family per the plugin engine rule:\nidentity + capability-specific operations, an owning `Registry`/`Manager`,\nexplicit lifecycle where the implementation owns resources, narrow host\naccess. Concretely:\n\n- **Workflow** -- already exists (`internal/domain/workflow`). A playbook\n action of kind `workflow` invokes a registered `workflow.Registry` entry\n by name, unchanged from the existing execution path.\n- **Module** -- the new general-purpose action position. A `Module` is a\n Yaegi-evaluated implementation of a generated interface (args in, typed\n result out) for one declared action kind (e.g. `image-gen`, `notify`,\n `open-pr-comment`). This is the position that makes actions \"not\n restricted to code-generation.\" Full design (schema-per-kind, the\n registry shape, the trust boundary, the recommended first slice):\n `docs/prds/module-position.md`.\n- **Channel** -- already exists as `channels.Channel` (telegram, email,\n webhook). A playbook action of kind `channel` sends through Line truncated
},
{
"url": "/dev/prds/embedded-nats/",
@@ -238,7 +244,7 @@
"url": "/dev/prds/event-sources-and-reactions/",
"title": "Event sources and reactions -- decision",
"section": "PRDs",
"body": "# Event sources and reactions -- decision\n\n**Status:** Approved\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-7d5u.1`, blocks `7d5u.2/.3/.4/.5`\n\nAnswers the four questions in `7d5u.1`'s description. Investigated against\nthe actual tree, not designed from scratch: three mechanisms already in this\ncodebase answer most of this, and the job here is picking the right one for\neach question and saying which packages change.\n\n## 1. In-process typed families, not out-of-process dispatch\n\nArchie has no out-of-process extension mechanism. The plugin engine\nrule (`ARCHITECTURE.md#plugin-engine-rule-strict`, mechanically checked by\n`internal/plugin/architecture_test.go`) keeps every capability family\nin-process behind a narrow typed registrar; NATS RPC exists for\ndaemon\u003c-\u003eagentexec, a different actor with a different trust boundary (task\nexecution, not third-party extension code), not a precedent for generic\nplugin dispatch. Neither epic's near-term payoff -- react to a labelled\nissue, bind a captured webhook to a workflow -- needs extension code in\nanother language or process. Building out-of-process dispatch now would be\ndesigning for a requirement neither epic states.\n\n`docs/architecture/plugins-and-extensions.md` does **not** need amending.\nThe line that looked like a blanket ban on hooks (`archie-core-7d5u`'s notes\nrecord the correction) is scoped to Workflow semantics specifically, and\nnothing decided here touches Workflow's ownership of its own contracts.\n\n## 2. Reaction delivery: `internal/eventbus`'s contract, a NEW stream\n\n`internal/eventbus.Publisher`/`Consumer` already has everything a reaction\nneeds: `PublishUnique(subject, idempotencyKey, payload)` for at-least-once\nwithout duplicate enqueue, `Ack`/`Nak` with \"a handler returning an error\ncauses redelivery.\" Do not design new delivery semantics; use this contract.\n\n**Do not reuse the existing `nats.Client` as-is.** `Config.StreamName` and\n`Config.Subjects` are already composition-supplied (\"the bus must not know\nwhich subjects belong to which domain,\" `config.go`), but the\nretention policy is not parameterized: `New` hardcodes\n`Retention: jetstream.WorkQueuePolicy` (`client.go`), the competing-consumer\npolicy correct for `ARCHIE_TASKS` task distribution -- one message, claimed by\nexactly one consumer -- and wrong for reactions. Two independent reactions\nboth wanting to see the same event under `WorkQueuePolicy` hit exactly the\ntrap `CLAUDE.md` already documents: overlapping filter subjects on a\nwork-queue stream mean the second consumer silently gets nothing. Reactions\nneed fan-out, not claim-once, so the client needs retention made a `Config`\nfield (defaulting to `WorkQueuePolicy` so `ARCHIE_TASKS` is\nunaffected), and a second stream (`ARCHIE_REACTIONS` or similar) configured\nunder a fan-out-capable policy, alongside the existing task-distribution\nstream -- not instead of it.\n\nThis is where `7d5u.2`'s embedded-vs-external axis attaches: whichever NATS\ninstance backs `ARCHIE_TASKS` also backs the reaction stream. NATS is not\noptional as a technology -- the choice is only\nwhether it runs as an external server or embedded in-process\n(`github.com/nats-io/nats-server/v2` is already a direct, test-proven\ndependency, just never wired into production startup). `7d5u.2` wires that;\nthis document does not re-decide it.\n\n## 3. Reactions in these two epics are producer-only\n\nNo veto, no mutation of in-flight work. Two existing shapes cover everything\neither epic actually needs, and neither is a hook that can block something\nelse:\n\n- **Cheap wake, no payload.** `internal/domain/curator/wake.go`'s\n `WakeOnPrimaryInput` subscribes to the lossy `events.Bus` and nudges a\n runtime to check sooner; a dropped event only delays a check-in, never\n loses work, because the trigger decision is a deterministic state read at\n pass time, not the event payload. Model any \"check something sooner\"\n reaction on this, unchanged.\n- **Produce new work, real payload.** `daemon.go`'s existing `pollNATS` -\u003e\n `publishTask` path is already exactly a reaction that observes forge state\n and produces a `workintake.TaskEnvelope`. A webhook source is a second\n producer into the same path, not a new capability -- see question 4.\n\nA reaction that can veto or mutate an *in-flight* task (block a PR from\nopening, alter a running stage) is a different, harder problem with its own\nordering and failure semantics, and belongs to `archie-core-h019` (adversarial\nself-review) if and when that epic needs one -- not here. Do not build a\ngeneral block-capable hook mechanism for either epic; nothing in their stated\nscope requires it, and it would duplicate whatever h019 eventually needs.\n\n## 4. No single generic intake interface; each domain keeps its own\n\nA `Source` interface generic enough to cover a forge issue, a chat message,\nand a cron tick would need an `any` payload -- theLine truncated
"body": "# Event sources and reactions -- decision\n\n**Status:** Approved\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-7d5u.1`, blocks `7d5u.2/.3/.4/.5`\n\nAnswers the four questions in `7d5u.1`'s description. Investigated against\nthe actual tree, not designed from scratch: three mechanisms already in this\ncodebase answer most of this, and the job here is picking the right one for\neach question and saying which packages change.\n\n## 1. In-process typed families, not out-of-process dispatch\n\nArchie has no out-of-process extension mechanism. The plugin engine\nrule (`docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`,\nmechanically checked by\n`internal/plugin/architecture_test.go`) keeps every capability family\nin-process behind a narrow typed registrar; NATS RPC exists for\ndaemon\u003c-\u003eagentexec, a different actor with a different trust boundary (task\nexecution, not third-party extension code), not a precedent for generic\nplugin dispatch. Neither epic's near-term payoff -- react to a labelled\nissue, bind a captured webhook to a workflow -- needs extension code in\nanother language or process. Building out-of-process dispatch now would be\ndesigning for a requirement neither epic states.\n\n`docs/architecture/plugins-and-extensions.md` does **not** need amending.\nThe line that looked like a blanket ban on hooks (`archie-core-7d5u`'s notes\nrecord the correction) is scoped to Workflow semantics specifically, and\nnothing decided here touches Workflow's ownership of its own contracts.\n\n## 2. Reaction delivery: `internal/eventbus`'s contract, a NEW stream\n\n`internal/eventbus.Publisher`/`Consumer` already has everything a reaction\nneeds: `PublishUnique(subject, idempotencyKey, payload)` for at-least-once\nwithout duplicate enqueue, `Ack`/`Nak` with \"a handler returning an error\ncauses redelivery.\" Do not design new delivery semantics; use this contract.\n\n**Do not reuse the existing `nats.Client` as-is.** `Config.StreamName` and\n`Config.Subjects` are already composition-supplied (\"the bus must not know\nwhich subjects belong to which domain,\" `config.go`), but the\nretention policy is not parameterized: `New` hardcodes\n`Retention: jetstream.WorkQueuePolicy` (`client.go`), the competing-consumer\npolicy correct for `ARCHIE_TASKS` task distribution -- one message, claimed by\nexactly one consumer -- and wrong for reactions. Two independent reactions\nboth wanting to see the same event under `WorkQueuePolicy` hit exactly the\ntrap `CLAUDE.md` already documents: overlapping filter subjects on a\nwork-queue stream mean the second consumer silently gets nothing. Reactions\nneed fan-out, not claim-once, so the client needs retention made a `Config`\nfield (defaulting to `WorkQueuePolicy` so `ARCHIE_TASKS` is\nunaffected), and a second stream (`ARCHIE_REACTIONS` or similar) configured\nunder a fan-out-capable policy, alongside the existing task-distribution\nstream -- not instead of it.\n\nThis is where `7d5u.2`'s embedded-vs-external axis attaches: whichever NATS\ninstance backs `ARCHIE_TASKS` also backs the reaction stream. NATS is not\noptional as a technology -- the choice is only\nwhether it runs as an external server or embedded in-process\n(`github.com/nats-io/nats-server/v2` is already a direct, test-proven\ndependency, just never wired into production startup). `7d5u.2` wires that;\nthis document does not re-decide it.\n\n## 3. Reactions in these two epics are producer-only\n\nNo veto, no mutation of in-flight work. Two existing shapes cover everything\neither epic actually needs, and neither is a hook that can block something\nelse:\n\n- **Cheap wake, no payload.** `internal/domain/curator/wake.go`'s\n `WakeOnPrimaryInput` subscribes to the lossy `events.Bus` and nudges a\n runtime to check sooner; a dropped event only delays a check-in, never\n loses work, because the trigger decision is a deterministic state read at\n pass time, not the event payload. Model any \"check something sooner\"\n reaction on this, unchanged.\n- **Produce new work, real payload.** `daemon.go`'s existing `pollNATS` -\u003e\n `publishTask` path is already exactly a reaction that observes forge state\n and produces a `workintake.TaskEnvelope`. A webhook source is a second\n producer into the same path, not a new capability -- see question 4.\n\nA reaction that can veto or mutate an *in-flight* task (block a PR from\nopening, alter a running stage) is a different, harder problem with its own\nordering and failure semantics, and belongs to `archie-core-h019` (adversarial\nself-review) if and when that epic needs one -- not here. Do not build a\ngeneral block-capable hook mechanism for either epic; nothing in their stated\nscope requires it, and it would duplicate whatever h019 eventually needs.\n\n## 4. No single generic intake interface; each domain keeps its own\n\nA `Source` interface generic enough to cover a forge issue, a chat message,\nand a cron tick wouldLine truncated
},
{
"url": "/dev/prds/extensions-center/",
@@ -256,7 +262,7 @@
"url": "/dev/prds/image-capability-contract/",
"title": "Image capability contract and configuration",
"section": "PRDs",
"body": "# Image capability contract and configuration\n\n**Status:** Approved\n\nEpic: archie-core-1786748942120-1-6636e629 / GitHub #496 (\"Image generation\ncapability: hosted API baseline plus local GPU workflows\").\n\nThe contract and configuration (#497) shipped first and is settled — see\n\"Package: `internal/domain/image`\" below. This document now covers the full\nfeature set: hosted provider (#498), local GPU provider (#499), `/image`\nrouting and clarification (#500), delivery through channels and web (#501).\nDocumentation (#502) and the cross-cutting test matrix (#503) are execution\nwork against the design below, not open design questions, so they are not\ngiven their own sections — each preceding section states its own test\nobligations inline.\n\n## Problem\n\nArchie needs a provider-neutral boundary for image generation/editing so a\nhosted API (gpt-image-2-class) and a future local GPU backend can both\nimplement it without either one leaking into the domain contract. Nothing\nin the repo defines this (`grep -ri image` over `internal/domain` and\n`internal/config` returns nothing image-specific).\n\n## Design\n\nFollows the plugin engine family shape already established by\n`internal/domain/curator` and `internal/domain/memory` (ARCHITECTURE.md\n\"plugin engine rule strict\"): a typed contract with real domain operations,\nan owning `Registry` (register/get/names, no start/stop lifecycle — see\nbelow), and narrow host access via a `Registrar`.\n\n### Package: `internal/domain/image`\n\n**Why no `Lifecycle` (`Start/Health/Stop`)**: curator and memory engines are\nlong-running — they hold a background cadence or a persistent connection\nworth health-checking. An image provider is call-scoped: it either serves a\n`Generate`/`Edit` request or it does not. Health here means \"is this\nprovider usable at call time\", which is exactly what `Generate`/`Edit`\nreturning a typed `ErrUnavailable` already communicates — a separate\n`Health()` poll would just duplicate that. `Registry` still owns\nregistration and lookup, so provider selection is testable with fakes per\nthe acceptance criteria, but there is no `Start`/`Stop` to orchestrate.\n\n```go\n// Capability declares what one provider supports, discoverable before a\n// request is built so a caller can reject an unsupported combination (e.g.\n// edit-with-mask on a generate-only provider) instead of sending it and\n// parsing a provider-specific error back out.\ntype Capability struct {\n Generate bool\n Edit bool // edit requires at least one input image\n Mask bool // edit supports a mask restricting the edited region\n MaxInputs int // 0 = Edit unsupported\n Sizes []string // provider-declared output sizes, e.g. \"1024x1024\"\n}\n\n// Provider is the typed contract every image backend implements — hosted\n// (gpt-image-2-class) or local (GPU recipe). No Lifecycle: a provider is\n// call-scoped, not long-running (see design doc rationale).\ntype Provider interface {\n Name() string\n // Class distinguishes hosted (spends money/quota per call) from local\n // (spends host GPU/time). Selection logic and config validation key off\n // this, never off Name, so a new provider cannot silently slip through\n // the hosted-cost gate under an unrecognized name.\n Class() ProviderClass\n Capability() Capability\n Generate(ctx context.Context, req GenerateRequest) (Result, error)\n Edit(ctx context.Context, req EditRequest) (Result, error)\n}\n\ntype ProviderClass string\n\nconst (\n ClassHosted ProviderClass = \"hosted\"\n ClassLocal ProviderClass = \"local\"\n)\n```\n\nRequest/result types (provider-neutral, no OpenAI types):\n\n```go\ntype GenerateRequest struct {\n Prompt string\n Size string // \"\" = provider default\n N int // number of images; 0 = 1\n Options map[string]string // provider-specific overflow (e.g. \"quality\"),\n // deliberately untyped so the contract does not\n // grow a field per provider\n}\n\ntype EditRequest struct {\n Prompt string\n Inputs []ImageData // 1..Capability.MaxInputs\n Mask *ImageData // nil unless Capability.Mask\n Size string\n N int\n Options map[string]string\n}\n\n// ImageData carries image bytes in-process, never a path or URL: the\n// domain layer must not assume a filesystem or a fetchable location (mirrors\n// why curator's Registrar never receives *daemon.Daemon — no ambient host\n// access smuggled in through a \"just a string\" field).\ntype ImageData struct {\n Bytes []byte\n MIMEType string\n}\n\ntype Result struct {\n Images []ImageData\n Provider string\n Model string\n CreatedAt time.Time\n}\n```\n\n### Error taxonomy\n\n```go\nvar (\n ErrUnavailable = errors.New(\"image provider unavailable\") // missing credential, backend down\n ErrUnsupporteLine truncated
"body": "# Image capability contract and configuration\n\n**Status:** Approved\n\nEpic: archie-core-1786748942120-1-6636e629 / GitHub #496 (\"Image generation\ncapability: hosted API baseline plus local GPU workflows\").\n\nThe contract and configuration (#497) shipped first and is settled — see\n\"Package: `internal/domain/image`\" below. This document now covers the full\nfeature set: hosted provider (#498), local GPU provider (#499), `/image`\nrouting and clarification (#500), delivery through channels and web (#501).\nDocumentation (#502) and the cross-cutting test matrix (#503) are execution\nwork against the design below, not open design questions, so they are not\ngiven their own sections — each preceding section states its own test\nobligations inline.\n\n## Problem\n\nArchie needs a provider-neutral boundary for image generation/editing so a\nhosted API (gpt-image-2-class) and a future local GPU backend can both\nimplement it without either one leaking into the domain contract. Nothing\nin the repo defines this (`grep -ri image` over `internal/domain` and\n`internal/config` returns nothing image-specific).\n\n## Design\n\nFollows the plugin engine family shape already established by\n`internal/domain/curator` and `internal/domain/memory`\n(`docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`): a\ntyped contract with real domain operations,\nan owning `Registry` (register/get/names, no start/stop lifecycle — see\nbelow), and narrow host access via a `Registrar`.\n\n### Package: `internal/domain/image`\n\n**Why no `Lifecycle` (`Start/Health/Stop`)**: curator and memory engines are\nlong-running — they hold a background cadence or a persistent connection\nworth health-checking. An image provider is call-scoped: it either serves a\n`Generate`/`Edit` request or it does not. Health here means \"is this\nprovider usable at call time\", which is exactly what `Generate`/`Edit`\nreturning a typed `ErrUnavailable` already communicates — a separate\n`Health()` poll would just duplicate that. `Registry` still owns\nregistration and lookup, so provider selection is testable with fakes per\nthe acceptance criteria, but there is no `Start`/`Stop` to orchestrate.\n\n```go\n// Capability declares what one provider supports, discoverable before a\n// request is built so a caller can reject an unsupported combination (e.g.\n// edit-with-mask on a generate-only provider) instead of sending it and\n// parsing a provider-specific error back out.\ntype Capability struct {\n Generate bool\n Edit bool // edit requires at least one input image\n Mask bool // edit supports a mask restricting the edited region\n MaxInputs int // 0 = Edit unsupported\n Sizes []string // provider-declared output sizes, e.g. \"1024x1024\"\n}\n\n// Provider is the typed contract every image backend implements — hosted\n// (gpt-image-2-class) or local (GPU recipe). No Lifecycle: a provider is\n// call-scoped, not long-running (see design doc rationale).\ntype Provider interface {\n Name() string\n // Class distinguishes hosted (spends money/quota per call) from local\n // (spends host GPU/time). Selection logic and config validation key off\n // this, never off Name, so a new provider cannot silently slip through\n // the hosted-cost gate under an unrecognized name.\n Class() ProviderClass\n Capability() Capability\n Generate(ctx context.Context, req GenerateRequest) (Result, error)\n Edit(ctx context.Context, req EditRequest) (Result, error)\n}\n\ntype ProviderClass string\n\nconst (\n ClassHosted ProviderClass = \"hosted\"\n ClassLocal ProviderClass = \"local\"\n)\n```\n\nRequest/result types (provider-neutral, no OpenAI types):\n\n```go\ntype GenerateRequest struct {\n Prompt string\n Size string // \"\" = provider default\n N int // number of images; 0 = 1\n Options map[string]string // provider-specific overflow (e.g. \"quality\"),\n // deliberately untyped so the contract does not\n // grow a field per provider\n}\n\ntype EditRequest struct {\n Prompt string\n Inputs []ImageData // 1..Capability.MaxInputs\n Mask *ImageData // nil unless Capability.Mask\n Size string\n N int\n Options map[string]string\n}\n\n// ImageData carries image bytes in-process, never a path or URL: the\n// domain layer must not assume a filesystem or a fetchable location (mirrors\n// why curator's Registrar never receives *daemon.Daemon — no ambient host\n// access smuggled in through a \"just a string\" field).\ntype ImageData struct {\n Bytes []byte\n MIMEType string\n}\n\ntype Result struct {\n Images []ImageData\n Provider string\n Model string\n CreatedAt time.Time\n}\n```\n\n### Error taxonomy\n\n```go\nvar (\n ErrUnavailable = errors.New(\"image provider unavailable\") // missing credential, backLine truncated
},
{
"url": "/dev/prds/inline-review/",
@@ -274,7 +280,7 @@
"url": "/dev/prds/memory-engine-unification/",
"title": "Memory engine unification: four scopes, CRUD, and one write path -- decision",
"section": "PRDs",
"body": "# Memory engine unification: four scopes, CRUD, and one write path -- decision\n\n**Status:** Approved\n**Date:** 2026-09-16\n**Findings:** `memory-4` and its dependants in the adversarial sweep\n(`.local/issue-tracker`); the real defect is the split described below, for\nwhich `memory-4` is a symptom\n**Supersedes:** the draft `memory-turn-context.md` (its synchronous-read\nargument survives and is reused; its `Manager` method and consumer-owned port do\nnot)\n**Settles:** four open items in `docs/architecture/migration-decisions.md#5`\n(listed at the end)\n\n## Problem\n\nThere are **two live memory stores for one assistant**, split by which component\nwrites them, and the chat turn reads neither.\n\n| Family | Written by | Root | Format |\n|---|---|---|---|\n| `internal/memory` (`Manager`, `builtin.Provider`) | the chat agent's `memory_edit` tool via `memorytoolprovider` (`bootstrap.go`) | `\u003cworkDir\u003e/memory` | `MEMORY.md` / `USER.md` |\n| `internal/domain/memory` + `internal/infrastructure/memory` | the session curator (`sessioncurator.go`) | `\u003cworkDir\u003e/memory-engine` | identity-keyed blocks |\n\nThey share no code, no format and no scope vocabulary. Neither reaches a prompt:\n`Manager.SystemPromptBlock` (`internal/memory/manager.go`) has no production\ncaller anywhere, so the agent can write memory through a tool and never read it\nback.\n\nThree further defects fall out of the same split:\n\n1. **The engine's contract is not CRUD.** `Store` is `Write`/`Query`/`List`/\n `Forget` — create, read, delete, and **no Update**. The retired file provider\n *does* have `handleAdd`/`handleReplace`/`handleRemove`, so the family being\n deleted is operationally richer than its replacement.\n2. **There is no scope model.** The contract carries one opaque `Identity\n string`, and the curator fills it with a **session id** (`sessioncurator.go`)\n — which is none of the four scopes `docs/architecture/agent-system.md`\n requires, and stops meaning anything when the session ends.\n3. **The chat turn has no user identity on half the channels.** Telegram and\n email populate `SenderID` with a person; the dashboard populates it with\n nothing, and a webhook populates it with nothing either — a route path names\n a source, not a person, so `webhook.go` leaves `SenderID` empty and carries\n the route in `Inbound.BudgetKey` for rate limiting. `sender_id` is persisted\n (`session_store_sqlite.go`), so whatever a channel puts there reaches the\n consumers that read it.\n\n`agent-system.md` says the write-scope selection and cross-scope sharing\nrules \"require a focused memory design decision\". This is that decision.\n\n## Decision\n\n### 1. One engine, addressed by four typed scopes\n\n`internal/domain/memory` is the only memory engine. `Observation.Identity`,\n`Query.Identity`, `Record.Identity` and `List(identity)` are deleted, not\nextended: a field whose meaning depends on the caller cannot carry a scope.\n\n```go\ntype ScopeKind string\n\nconst (\n ScopeGlobal ScopeKind = \"global\" // shared across the application\n ScopeAgent ScopeKind = \"agent\" // private to one Agent\n ScopeUser ScopeKind = \"user\" // one user, across their Agents\n ScopeAgentUser ScopeKind = \"agent-user\" // one Agent's relationship with one user\n)\n\ntype Scope struct {\n Kind ScopeKind\n Agent AgentID // required for ScopeAgent, ScopeAgentUser\n User IdentityID // required for ScopeUser, ScopeAgentUser\n}\n\nfunc (s Scope) Validate() error // rejects a component present where it is not required\nfunc (s Scope) Key() string // canonical, collision-free storage key\n```\n\n`AgentID` and `IdentityID` are opaque. This package never resolves or\ncanonicalises them.\n\nA turn's resolved identity is one value:\n\n```go\ntype Subject struct {\n AgentID AgentID\n UserID IdentityID // \"\" when the channel carries no per-person identity\n}\n\nfunc (s Subject) Scopes() []Scope // readable, most specific first\nfunc (s Subject) WritableScopes() []Scope // never includes ScopeGlobal\n```\n\n**Access control is naming a scope, and nothing else.** The read path names\nexactly the scopes of the resolved `Subject`; the engine never unions beyond what\nit is handed and applies no policy of its own. An engine that enforced policy\nwould need to know about channels, sessions and bindings, and every backend would\nreimplement it. \"Agent-wide is private to its owning Agent by default\"\n(`agent-system.md`) falls out mechanically, because `ScopeAgent{a}` and\n`ScopeAgent{b}` are different storage keys.\n\n`ScopeGlobal` is never model-writable. Writing it is an operator action; nothing\nproduces one yet, but the contract models it because the read set includes it.\n\n### 2. CRUD, with revisions retained\n\n```go\ntype Store interface {\n Create(ctx context.Context, in NewRecord) (Record, error)\n Get(ctx context.Context,Line truncated
"body": "# Memory engine unification: four scopes, CRUD, and one write path -- decision\n\n**Status:** Approved\n**Date:** 2026-09-16\n**Findings:** `memory-4` and its dependants in the adversarial sweep\n(`.local/issue-tracker`); the real defect is the split described below, for\nwhich `memory-4` is a symptom\n**Supersedes:** the draft `memory-turn-context.md` (its synchronous-read\nargument survives and is reused; its `Manager` method and consumer-owned port do\nnot)\n**Settles:** four open items in `docs/architecture/migration-decisions.md#5`\n(listed at the end)\n\n## Problem\n\nThere are **two live memory stores for one assistant**, split by which component\nwrites them, and the chat turn reads neither.\n\n| Family | Written by | Root | Format |\n|---|---|---|---|\n| `internal/memory` (`Manager`, `builtin.Provider`) | the chat agent's `memory_edit` tool via `memorytoolprovider` (`bootstrap.go`) | `\u003cworkDir\u003e/memory` | `MEMORY.md` / `USER.md` |\n| `internal/domain/memory` + `internal/infrastructure/memory` | the session curator (`sessioncurator.go`) | `\u003cworkDir\u003e/memory-engine` | identity-keyed blocks |\n\nThey share no code, no format and no scope vocabulary. Neither reaches a prompt:\n`Manager.SystemPromptBlock` (`internal/memory/manager.go`) has no production\ncaller anywhere, so the agent can write memory through a tool and never read it\nback.\n\nThree further defects fall out of the same split:\n\n1. **The engine's contract is not CRUD.** `Store` is `Write`/`Query`/`List`/\n `Forget` — create, read, delete, and **no Update**. The retired file provider\n *does* have `handleAdd`/`handleReplace`/`handleRemove`, so the family being\n deleted is operationally richer than its replacement.\n2. **There is no scope model.** The contract carries one opaque `Identity\n string`, and the curator fills it with a **session id** (`sessioncurator.go`)\n — which is none of the four scopes `docs/architecture/agent-system.md`\n requires, and stops meaning anything when the session ends.\n3. **The chat turn has no user identity on half the channels.** Telegram and\n email populate `SenderID` with a person; the dashboard populates it with\n nothing, and a webhook populates it with nothing either — a route path names\n a source, not a person, so `webhook.go` leaves `SenderID` empty and carries\n the route in `Inbound.BudgetKey` for rate limiting. `sender_id` is persisted\n (`session_store_sqlite.go`), so whatever a channel puts there reaches the\n consumers that read it.\n\n`agent-system.md` says the write-scope selection and cross-scope sharing\nrules \"require a focused memory design decision\". This is that decision.\n\n## Decision\n\n### 1. One engine, addressed by four typed scopes\n\n`internal/domain/memory` is the only memory engine. `Observation.Identity`,\n`Query.Identity`, `Record.Identity` and `List(identity)` are deleted, not\nextended: a field whose meaning depends on the caller cannot carry a scope.\n\n```go\ntype ScopeKind string\n\nconst (\n ScopeGlobal ScopeKind = \"global\" // shared across the application\n ScopeAgent ScopeKind = \"agent\" // private to one Agent\n ScopeUser ScopeKind = \"user\" // one user, across their Agents\n ScopeAgentUser ScopeKind = \"agent-user\" // one Agent's relationship with one user\n)\n\ntype Scope struct {\n Kind ScopeKind\n Agent AgentID // required for ScopeAgent, ScopeAgentUser\n User IdentityID // required for ScopeUser, ScopeAgentUser\n}\n\nfunc (s Scope) Validate() error // rejects a component present where it is not required\nfunc (s Scope) Key() string // canonical, collision-free storage key\n```\n\n`AgentID` and `IdentityID` are opaque. This package never resolves or\ncanonicalises them.\n\nA turn's resolved identity is one value:\n\n```go\ntype Subject struct {\n AgentID AgentID\n UserID IdentityID // \"\" when the channel carries no per-person identity\n}\n\nfunc (s Subject) Scopes() []Scope // readable, most specific first\nfunc (s Subject) WritableScopes() []Scope // never includes ScopeGlobal\n```\n\n**Access control is naming a scope, and nothing else.** The read path names\nexactly the scopes of the resolved `Subject`; the engine never unions beyond what\nit is handed and applies no policy of its own. An engine that enforced policy\nwould need to know about channels, sessions and bindings, and every backend would\nreimplement it. \"Agent-wide is private to its owning Agent by default\"\n(`agent-system.md`) falls out mechanically, because `ScopeAgent{a}` and\n`ScopeAgent{b}` are different storage keys.\n\n`ScopeGlobal` is never model-writable. Writing it is an operator action; nothing\nproduces one yet, but the contract models it because the read set includes it.\n\n### 2. CRUD, with revisions retained\n\n```go\ntype Store interface {\n Create(ctx context.Context, in NewRecord) (Record, error)\n Get(ctx context.Context,Line truncated
},
{
"url": "/dev/prds/messaging-service-boundary/",
@@ -364,7 +370,7 @@
"url": "/dev/prds/service-decomposition/",
"title": "Service decomposition -- decision",
"section": "PRDs",
"body": "# Service decomposition -- decision\n\n**Status:** Draft\n**Date:** 2026-09-05\n**Beads epic:** archie-core-8cda\n**Prior art:** `docs/inspiration/2026-09-05-awx-service-decomposition-and-ansible-task-model.md`,\n`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`,\n`docs/inspiration/service-discovery-research-2026-09-05.md`,\n`/work/artefacts/archie-core-current-state-archify.html`,\n`/work/artefacts/archie-core-ideal-state-archify.html`\n\nThe Phase 3 UI boundary and configuration decision is ratified in\n[`ui-service-boundary.md`](/dev/prds/ui-service-boundary/). That focused document is\nthe implementation authority for `archie-core-8cda.5.2`; this parent remains\nthe broader service-distribution proposal.\n\n## Decision\n\narchie-core moves from one Go binary (`archied`) hosting every domain\nin-process to a set of independently deployable services, each exposing\nexactly one typed contract. Consumers depend on the contract; they never\nreach into another service's internal types, session state, or config.\ngRPC is the synchronous transport; NATS remains for asynchronous fan-out\n(status, telemetry) rather than being replaced.\n\nThis is a **distribution decision**, not a rewrite of Archie's domain\nmodel. The six-domain split already approved in\n`docs/architecture/index.md` and `docs/architecture/organisation.md`\n(Identity, Agent, Messaging, Workflow, Work Intake, Plugin) is unchanged;\nthis PRD is about how the *processes* running that domain logic get\npackaged, contracted, and discovered -- not about redrawing domain\nboundaries.\n\n### Target services\n\n| Service | Contract it owns | Existing code it replaces/wraps |\n|---|---|---|\n| UI Service | thin client only; talks to Gateway and State Store contracts | `internal/webui` (SPA hosting stays; API surface becomes typed contract clients, not a peer with direct access) |\n| Gateway Service | routing / turn / session contract | `internal/gateway` |\n| Messaging Service | channel-plugin contract (Telegram, email, Discord, Teams, ...) | `internal/channels` |\n| Curator Service | curator contract; **optional**, installs as a Gateway plugin | `internal/domain/curator` |\n| Scheduler Service | cron/ticker contract | `internal/domain/scheduling` |\n| Execution Environment | execution-dispatch contract | `internal/domain/workflow`, task dispatch paths |\n| Runner | isolated worktree/container execution | `internal/container`, `internal/agentexec`, `internal/worktree` |\n| State Store | state-access contract | `internal/store` |\n\n### Discovery\n\nPer `docs/inspiration/service-discovery-research-2026-09-05.md`:\n\n- **Primary:** Kubernetes-native discovery (Service + headless-Service/DNS\n + EndpointSlice, served by CoreDNS), delivered as a Helm chart + Operator,\n on k3s for single-host installs and a real cluster for multi-host or a\n hosted offering.\n- **Fallback:** NATS-based discovery (micro `$SRV` announce + JetStream KV\n registry) for no-K8s single-host installs, with a custom gRPC resolver\n reading `host:port` from the KV bucket.\n- Both implementations sit behind one internal interface:\n\n ```go\n type ServiceRegistry interface {\n Resolve(service string) []Endpoint\n Watch(service string) \u003c-chan Event // join/leave\n }\n ```\n\n- An optional service (Curator; others later) that is not installed\n must resolve to a distinct `NotInstalled` outcome, never a `Down`/\n `Unhealthy` one. Under K8s this falls out of DNS returning no record;\n under the NATS fallback it requires an explicit \"installed\" marker in the\n KV registry, not just absence-implies-down.\n\n### Distribution model this supports\n\nBoth self-hosted (customer runs the Helm chart or the NATS-fallback\nbinaries themselves) and a managed/hosted offering (we run the same Helm\nchart) are in scope -- this is the RHEL / Ansible Automation Platform\nmodel: one architecture, sold both ways. This PRD does not decide the\nbusiness model; it decides that the architecture must not foreclose\neither shape.\n\n## Why this shape\n\n`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`\nestablished that Archie's existing domain split already does what AWX's\nservice-decomposition post argues for -- boundaries exist. What didn't\nexist was deployability: `internal/webui` was not a passive frontend. The\nre-baselined coupling inventory is below, and the ratified boundary is\n[`ui-service-boundary.md`](/dev/prds/ui-service-boundary/).\n\n- Telegram task actions now call Gateway's `ChatContract` directly through\n `chatTaskActorAdapter`; the old Telegram-to-webui HTTP detour is historical.\n- `internal/webui/api_chat.go` now holds `gateway.ChatContract`; direct\n `SessionStore`, `Router`, and `TurnHistory` access is historical.\n- The third coupling is closed. UI extraction shipped (`archie-core-8cda.5`,\n closed 2026-09-11): `cmd/archie-ui` serves the dashboard against remote\n Gateway and State Store contracts, holds Line truncated
"body": "# Service decomposition -- decision\n\n**Status:** Draft\n**Date:** 2026-09-05\n**Beads epic:** archie-core-8cda\n**Prior art:** `docs/inspiration/2026-09-05-awx-service-decomposition-and-ansible-task-model.md`,\n`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`,\n`docs/inspiration/service-discovery-research-2026-09-05.md`,\n`/work/artefacts/archie-core-current-state-archify.html`,\n`/work/artefacts/archie-core-ideal-state-archify.html`\n\nThe Phase 3 UI boundary and configuration decision is ratified in\n[`ui-service-boundary.md`](/dev/prds/ui-service-boundary/). That focused document is\nthe implementation authority for `archie-core-8cda.5.2`; this parent remains\nthe broader service-distribution proposal.\n\n## Decision\n\narchie-core moves from one Go binary (`archied`) hosting every domain\nin-process to a set of independently deployable services, each exposing\nexactly one typed contract. Consumers depend on the contract; they never\nreach into another service's internal types, session state, or config.\ngRPC is the synchronous transport; NATS remains for asynchronous fan-out\n(status, telemetry) rather than being replaced.\n\nThis is a **distribution decision**, not a rewrite of Archie's domain\nmodel. The six-domain split already approved in\n`docs/architecture/index.md` and `docs/architecture/organisation.md`\n(Identity, Agent, Messaging, Workflow, Work Intake, Plugin) is unchanged;\nthis PRD is about how the *processes* running that domain logic get\npackaged, contracted, and discovered -- not about redrawing domain\nboundaries.\n\n### Target services\n\n| Service | Contract it owns | Existing code it replaces/wraps |\n|---|---|---|\n| UI Service | thin client only; talks to Gateway and State Store contracts | `internal/webui` (SPA hosting stays; API surface becomes typed contract clients, not a peer with direct access) |\n| Gateway Service | routing / turn / session contract | `internal/gateway` |\n| Messaging Service | channel-plugin contract (Telegram, email, Discord, Teams, ...) | `internal/channels` |\n| Curator Service | curator contract; **optional**, installs as a Gateway plugin | `internal/domain/curator` |\n| Scheduler Service | cron/ticker contract | `internal/domain/scheduling` |\n| Execution Environment | execution-dispatch contract | `internal/domain/workflow`, task dispatch paths |\n| Runner | isolated worktree/container execution | `internal/container`, `internal/agentexec`, `internal/worktree` |\n| State Store | state-access contract | `internal/store` |\n\n### Discovery\n\nPer `docs/inspiration/service-discovery-research-2026-09-05.md`:\n\n- **Primary:** Kubernetes-native discovery (Service + headless-Service/DNS\n + EndpointSlice, served by CoreDNS), delivered as a Helm chart + Operator,\n on k3s for single-host installs and a real cluster for multi-host or a\n hosted offering.\n- **Fallback:** NATS-based discovery (micro `$SRV` announce + JetStream KV\n registry) for no-K8s single-host installs, with a custom gRPC resolver\n reading `host:port` from the KV bucket.\n- Both implementations sit behind one internal interface:\n\n ```go\n type ServiceRegistry interface {\n Resolve(service string) []Endpoint\n Watch(service string) \u003c-chan Event // join/leave\n }\n ```\n\n- An optional service (Curator; others later) that is not installed\n must resolve to a distinct `NotInstalled` outcome, never a `Down`/\n `Unhealthy` one. Under K8s this falls out of DNS returning no record;\n under the NATS fallback it requires an explicit \"installed\" marker in the\n KV registry, not just absence-implies-down.\n\n### Distribution model this supports\n\nBoth self-hosted (customer runs the Helm chart or the NATS-fallback\nbinaries themselves) and a managed/hosted offering (we run the same Helm\nchart) are in scope -- this is the RHEL / Ansible Automation Platform\nmodel: one architecture, sold both ways. This PRD does not decide the\nbusiness model; it decides that the architecture must not foreclose\neither shape.\n\n## Why this shape\n\n`docs/inspiration/module-reference-awx-decomposition-vs-archie.md`\nestablished that Archie's existing domain split already does what AWX's\nservice-decomposition post argues for -- boundaries exist. What didn't\nexist was deployability: `internal/webui` was not a passive frontend. The\nre-baselined coupling inventory is below, and the ratified boundary is\n[`ui-service-boundary.md`](/dev/prds/ui-service-boundary/).\n\n- Telegram task actions now call Gateway's `ChatContract` directly through\n `chatTaskActorAdapter`; the old Telegram-to-webui HTTP detour is historical.\n- `internal/webui/api_chat.go` now holds `gateway.ChatContract`; direct\n `SessionStore`, `Router`, and `TurnHistory` access is historical.\n- The third coupling is closed. UI extraction shipped (`archie-core-8cda.5`,\n closed 2026-09-11): `cmd/archie-ui` serves the dashboard against remote\n Gateway and State Store contracts, holds Line truncated
},
{
"url": "/dev/prds/service-registry/",
@@ -424,7 +430,7 @@
"url": "/dev/prds/webhook-intake-security/",
"title": "Webhook intake security -- decision",
"section": "PRDs",
"body": "# Webhook intake security -- decision\n\n**Status:** Approved\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-t2db.5`, blocks `archie-core-t2db.4`\n\n## Why this is decided ahead of the code it constrains\n\n`archie-core-t2db` (no-code playbooks) and `archie-core-7d5u` (typed event\nsources) both land a public intake surface whose eventual output is an agent\nwith tool access and a repo. Neither capture (`t2db.1`) nor the intake family\n(`7d5u.3`) has shipped yet, so this decision exists to be a contract the two\nepics build against, not a retrofit once the surface exists. A webhook URL\nthat can start an agent is remote code execution with extra steps if any of\nthe five points below is missing on day one.\n\n## 1. Per-source authentication\n\nReuse the per-route HMAC-SHA256 scheme already in\n`internal/channels/webhook/webhook.go` (`validateHMAC`), unchanged: a source\nregisters with a shared secret; the signature is checked against the raw\nbody. Do not invent a second scheme for intake.\n\n**Capture and triggering split at the signature, not at a separate flag.** An\nevent with a missing or invalid signature is still captured (visible in the\ninspector, marked unauthenticated) so an operator can diagnose a\nmisconfigured sender -- but an unauthenticated event can never be matched\nagainst a binding. Authentication is what promotes an event from \"something\narrived\" to \"something that can run a workflow.\"\n\n## 2. Human approval before a binding arms\n\nA binding has three states: `draft -\u003e pending_approval -\u003e armed`. Only\n`armed` bindings are evaluated against incoming events. Creating or editing a\nbinding's matcher, mapping, or target workflow always sets it to\n`pending_approval`, including edits to an already-armed binding -- an edit is\nnot silently re-armed, or the approval gate is decorative. Approval requires\nan authenticated dashboard session; there is no unattended approval path.\n\nModel: `internal/channels/telegram`'s dangerous-command approval flow\n(`dangerousAction` / `pendingApproval` in `dangerous.go` / `approval.go`) is\nthe existing precedent for \"a human decision gates code from running\" in this\ncodebase -- follow its shape rather than designing a new approval primitive.\n\n## 3. Rate and volume limits, per source\n\nA token-bucket limiter keyed by source ID, not global -- one noisy or hostile\nsender must not exhaust another source's capture budget. Precedent for\nper-key volume capping: `internal/pairing/store.go`'s `MaxPendingPerPlatform`.\n\nOn limit exceeded, the request is rejected (HTTP 429), not silently dropped.\nA silently dropped event is indistinguishable from a sender that never fired\n-- the same invisible-failure shape the update-verification work\n(`archie-core-522`) exists to prevent. The rejection itself is observable in\nthe inspector as a distinct outcome from \"not received.\"\n\n## 4. Blast radius bounded by code, not by prompt wording\n\nA playbook-triggered task runs through exactly the same enforcement as every\nother task: the `[[repos.gate]]` quality gate, `diff_cap_lines`, the\nsandboxed per-task container (`internal/container`), worktree isolation, and\n\"the model never runs git.\" No new relaxation is introduced for\nplaybook-triggered work, and none is needed -- the existing constraints are\nthe control.\n\nMapped payload fields are treated as ordinary user-controlled text inside an\nagent prompt, exactly like an inbound chat message never as trusted\ninstructions. \"The agent should ignore instructions in the payload\" is not a\ncontrol and must not appear as one anywhere in this feature -- per\n`ARCHITECTURE.md`'s environmental-enforcement-over-prompt-rules principle,\nthe boundary is the gate and the sandbox, not a sentence in the system\nprompt.\n\n## 5. Retention and redaction of captured payloads\n\n**Retention:** captured events (bound or not) are purged after 7 days,\nconfigurable per deployment. Unbound events exist to be mapped against; nothing\njustifies keeping them indefinitely once that window has passed, and an\nunbounded capture store is the same disk-exhaustion risk item 3 addresses at\nthe request-rate level.\n\n**Redaction:** before a captured payload is persisted, apply the same\nkey-name heuristic `internal/gateway/stream.go`'s `sensitiveParameterKey`\nalready uses for tool-call parameters (matches `token`, `secret`, `password`,\n`passwd`, `api_key`, `apikey`, `authorization`, `credential`, `cookie`,\n`private_key` against normalized JSON keys) and replace matching values with\nthe same `[redacted]` marker. This is best-effort, not a guarantee -- a\nsender that names a secret field something the heuristic misses will still\nhave it captured. Document that limitation where captured payloads are\ndisplayed, rather than implying the redaction is complete.\n\n**Visibility:** captured payloads, redacted or not, are visible only to an\nauthenticated dashboard operator -- the same authorization bounLine truncated
"body": "# Webhook intake security -- decision\n\n**Status:** Approved\n**Date:** 2026-08-22\n**Beads issue:** `archie-core-t2db.5`, blocks `archie-core-t2db.4`\n\n## Why this is decided ahead of the code it constrains\n\n`archie-core-t2db` (no-code playbooks) and `archie-core-7d5u` (typed event\nsources) both land a public intake surface whose eventual output is an agent\nwith tool access and a repo. Neither capture (`t2db.1`) nor the intake family\n(`7d5u.3`) has shipped yet, so this decision exists to be a contract the two\nepics build against, not a retrofit once the surface exists. A webhook URL\nthat can start an agent is remote code execution with extra steps if any of\nthe five points below is missing on day one.\n\n## 1. Per-source authentication\n\nReuse the per-route HMAC-SHA256 scheme already in\n`internal/channels/webhook/webhook.go` (`validateHMAC`), unchanged: a source\nregisters with a shared secret; the signature is checked against the raw\nbody. Do not invent a second scheme for intake.\n\n**Capture and triggering split at the signature, not at a separate flag.** An\nevent with a missing or invalid signature is still captured (visible in the\ninspector, marked unauthenticated) so an operator can diagnose a\nmisconfigured sender -- but an unauthenticated event can never be matched\nagainst a binding. Authentication is what promotes an event from \"something\narrived\" to \"something that can run a workflow.\"\n\n## 2. Human approval before a binding arms\n\nA binding has three states: `draft -\u003e pending_approval -\u003e armed`. Only\n`armed` bindings are evaluated against incoming events. Creating or editing a\nbinding's matcher, mapping, or target workflow always sets it to\n`pending_approval`, including edits to an already-armed binding -- an edit is\nnot silently re-armed, or the approval gate is decorative. Approval requires\nan authenticated dashboard session; there is no unattended approval path.\n\nModel: `internal/channels/telegram`'s dangerous-command approval flow\n(`dangerousAction` / `pendingApproval` in `dangerous.go` / `approval.go`) is\nthe existing precedent for \"a human decision gates code from running\" in this\ncodebase -- follow its shape rather than designing a new approval primitive.\n\n## 3. Rate and volume limits, per source\n\nA token-bucket limiter keyed by source ID, not global -- one noisy or hostile\nsender must not exhaust another source's capture budget. Precedent for\nper-key volume capping: `internal/pairing/store.go`'s `MaxPendingPerPlatform`.\n\nOn limit exceeded, the request is rejected (HTTP 429), not silently dropped.\nA silently dropped event is indistinguishable from a sender that never fired\n-- the same invisible-failure shape the update-verification work\n(`archie-core-522`) exists to prevent. The rejection itself is observable in\nthe inspector as a distinct outcome from \"not received.\"\n\n## 4. Blast radius bounded by code, not by prompt wording\n\nA playbook-triggered task runs through exactly the same enforcement as every\nother task: the `[[repos.gate]]` quality gate, `diff_cap_lines`, the\nsandboxed per-task container (`internal/container`), worktree isolation, and\n\"the model never runs git.\" No new relaxation is introduced for\nplaybook-triggered work, and none is needed -- the existing constraints are\nthe control.\n\nMapped payload fields are treated as ordinary user-controlled text inside an\nagent prompt, exactly like an inbound chat message never as trusted\ninstructions. \"The agent should ignore instructions in the payload\" is not a\ncontrol and must not appear as one anywhere in this feature -- per the\nenvironmental-enforcement principle (`docs/architecture/agent-system.md`,\n\"Existing invariants to preserve\"), the boundary is the gate and the sandbox,\nnot a sentence in the system prompt.\n\n## 5. Retention and redaction of captured payloads\n\n**Retention:** captured events (bound or not) are purged after 7 days,\nconfigurable per deployment. Unbound events exist to be mapped against; nothing\njustifies keeping them indefinitely once that window has passed, and an\nunbounded capture store is the same disk-exhaustion risk item 3 addresses at\nthe request-rate level.\n\n**Redaction:** before a captured payload is persisted, apply the same\nkey-name heuristic `internal/gateway/stream.go`'s `sensitiveParameterKey`\nalready uses for tool-call parameters (matches `token`, `secret`, `password`,\n`passwd`, `api_key`, `apikey`, `authorization`, `credential`, `cookie`,\n`private_key` against normalized JSON keys) and replace matching values with\nthe same `[redacted]` marker. This is best-effort, not a guarantee -- a\nsender that names a secret field something the heuristic misses will still\nhave it captured. Document that limitation where captured payloads are\ndisplayed, rather than implying the redaction is complete.\n\n**Visibility:** captured payloads, redacted or not, are visible only to an\nauthenticated dashboarLine truncated
}
],
"nav": [
@@ -517,6 +523,10 @@
"title": "Agent2Agent as archie's agent-to-agent protocol — decision",
"url": "/dev/prds/agent2agent-protocol/"
},
{
"title": "Artifact sender",
"url": "/dev/prds/artifact-sender/"
},
{
"title": "Binding secret at-rest encryption -- decision",
"url": "/dev/prds/binding-secret-encryption/"
+1 -1
View File
@@ -10,7 +10,7 @@
"url": "/docs/guides/first-playbook/",
"title": "Building your first playbook binding",
"section": "Guides",
"body": "# Building your first playbook binding\n\nThis walks through the whole no-code path the event-capture epic promises:\npoint an external app's webhook at archie, see the real payload it sends,\nmap the fields you care about, bind them to a workflow, and watch the next\nmatching event turn into a task -- no code, no redeploy, no restart. See\n`docs/architecture/bindings.md` for the model and threat model behind each\nstep below.\n\n## 1. Send a test event\n\nPoint whatever external app you want archie to react to at:\n\n```\nPOST http://\u003cyour-archied-host\u003e:8484/webhooks/capture/\u003csource\u003e\n```\n\n`\u003csource\u003e` is a name you choose for this sender (e.g. `sentry`, `github-ci`,\n`stripe`) -- it's just a URL path segment, not a preset list. Send one real\nevent however is easiest for a first test, e.g.:\n\n```bash\ncurl -X POST http://localhost:8484/webhooks/capture/sentry \\\n -H \"Content-Type: application/json\" \\\n -d '{\"issue\": {\"title\": \"NullPointerException in checkout\"}, \"level\": \"error\"}'\n```\n\nThis event isn't signed yet, so it's captured but marked **unauthenticated**\n-- visible for you to design against, but incapable of triggering anything\n(`docs/architecture/bindings.md`'s threat-model section explains why: an\nunauthenticated event can never match a binding, no matter how it looks).\n\n## 2. Inspect the captured payload\n\nOpen the dashboard's **Event inspector** (`/captures`). Your test event\nappears there with its real body -- this is schema-by-example: you're\nlooking at an actual payload from the actual sender, not guessing at a\nschema in advance.\n\n## 3. Map the fields you care about\n\nOpen **Field mappings** (`/mappings`) → **New mapping**. Pick your captured\nevent from the dropdown, then click through its payload tree on the right --\nevery key and value is clickable. Clicking a value binds it as a field\n(name, JSON path, and an inferred type you can change). Add whichever fields\nyour workflow needs (e.g. the issue title), mark the ones that must be\npresent as **Required**, hit **Preview** to confirm they resolve correctly\nagainst the real payload, then **Save**.\n\n## 4. Create the binding\n\nOpen **Playbook bindings** (`/bindings`) → **New binding**:\n\n- **Name** -- anything descriptive.\n- **Source** -- the same `\u003csource\u003e` segment from step 1 (e.g. `sentry`).\n- **Field mapping** -- the one you just saved.\n- **Workflow** -- which registered workflow should run on a match.\n- **Repo pin** (optional) -- leave both blank to use the single configured\n repo; set owner/repo explicitly if you run a multi-repo deployment.\n- **Shared secret** -- a real secret (16+ bytes), never `test1234`. The\n sender signs every request body with this via HMAC-SHA256 so archie can\n tell a genuine event from anyone who finds the URL.\n\nSave. The binding starts in **draft** -- nothing fires yet.\n\n## 5. Approve it\n\nA **draft** or freshly-**edited** binding needs an explicit human approval\nbefore it can ever match a live event -- this is deliberate, not a missing\nstep: an operator authoring a binding is a considered act, an event arriving\nis not, and nothing should self-arm (`docs/architecture/bindings.md`'s\nthreat-model section, point 2). Click **Approve**. Status becomes **armed**.\n\n## 6. Fire it for real\n\nSend the same kind of event again, this time **signed** with the binding's\nsecret:\n\n```bash\nBODY='{\"issue\": {\"title\": \"NullPointerException in checkout\"}, \"level\": \"error\"}'\nSECRET=\"the-secret-you-set-in-step-4\"\nSIG=$(printf '%s' \"$BODY\" | openssl dgst -sha256 -hmac \"$SECRET\" | sed 's/^.* //')\n\ncurl -X POST http://localhost:8484/webhooks/capture/sentry \\\n -H \"Content-Type: application/json\" \\\n -H \"X-Hub-Signature-256: sha256=$SIG\" \\\n -d \"$BODY\"\n```\n\nThis event is captured as **authenticated**, matched against your armed\nbinding, and dispatched into a new task through the exact same\ngate/worktree/sandbox pipeline every other task goes through -- see\n`ARCHITECTURE.md`'s Task Lifecycle section, \"Provenance by origin.\" Check\nthe **Tasks** page; a new task should appear, its body built from the fields\nyour mapping resolved.\n\n## Editing a live binding\n\nAny edit to an armed binding -- even a typo fix -- drops it back to\n**pending_approval** automatically. This can't be bypassed by editing; it\nhas to be re-approved. When editing, leave the **Shared secret** field blank\nto keep the existing secret; only fill it in if you're deliberately\nrotating it.\n"
"body": "# Building your first playbook binding\n\nThis walks through the whole no-code path the event-capture epic promises:\npoint an external app's webhook at archie, see the real payload it sends,\nmap the fields you care about, bind them to a workflow, and watch the next\nmatching event turn into a task -- no code, no redeploy, no restart. See\n`docs/architecture/bindings.md` for the model and threat model behind each\nstep below.\n\n## 1. Send a test event\n\nPoint whatever external app you want archie to react to at:\n\n```\nPOST http://\u003cyour-archied-host\u003e:8484/webhooks/capture/\u003csource\u003e\n```\n\n`\u003csource\u003e` is a name you choose for this sender (e.g. `sentry`, `github-ci`,\n`stripe`) -- it's just a URL path segment, not a preset list. Send one real\nevent however is easiest for a first test, e.g.:\n\n```bash\ncurl -X POST http://localhost:8484/webhooks/capture/sentry \\\n -H \"Content-Type: application/json\" \\\n -d '{\"issue\": {\"title\": \"NullPointerException in checkout\"}, \"level\": \"error\"}'\n```\n\nThis event isn't signed yet, so it's captured but marked **unauthenticated**\n-- visible for you to design against, but incapable of triggering anything\n(`docs/architecture/bindings.md`'s threat-model section explains why: an\nunauthenticated event can never match a binding, no matter how it looks).\n\n## 2. Inspect the captured payload\n\nOpen the dashboard's **Event inspector** (`/captures`). Your test event\nappears there with its real body -- this is schema-by-example: you're\nlooking at an actual payload from the actual sender, not guessing at a\nschema in advance.\n\n## 3. Map the fields you care about\n\nOpen **Field mappings** (`/mappings`) → **New mapping**. Pick your captured\nevent from the dropdown, then click through its payload tree on the right --\nevery key and value is clickable. Clicking a value binds it as a field\n(name, JSON path, and an inferred type you can change). Add whichever fields\nyour workflow needs (e.g. the issue title), mark the ones that must be\npresent as **Required**, hit **Preview** to confirm they resolve correctly\nagainst the real payload, then **Save**.\n\n## 4. Create the binding\n\nOpen **Playbook bindings** (`/bindings`) → **New binding**:\n\n- **Name** -- anything descriptive.\n- **Source** -- the same `\u003csource\u003e` segment from step 1 (e.g. `sentry`).\n- **Field mapping** -- the one you just saved.\n- **Workflow** -- which registered workflow should run on a match.\n- **Repo pin** (optional) -- leave both blank to use the single configured\n repo; set owner/repo explicitly if you run a multi-repo deployment.\n- **Shared secret** -- a real secret (16+ bytes), never `test1234`. The\n sender signs every request body with this via HMAC-SHA256 so archie can\n tell a genuine event from anyone who finds the URL.\n\nSave. The binding starts in **draft** -- nothing fires yet.\n\n## 5. Approve it\n\nA **draft** or freshly-**edited** binding needs an explicit human approval\nbefore it can ever match a live event -- this is deliberate, not a missing\nstep: an operator authoring a binding is a considered act, an event arriving\nis not, and nothing should self-arm (`docs/architecture/bindings.md`'s\nthreat-model section, point 2). Click **Approve**. Status becomes **armed**.\n\n## 6. Fire it for real\n\nSend the same kind of event again, this time **signed** with the binding's\nsecret:\n\n```bash\nBODY='{\"issue\": {\"title\": \"NullPointerException in checkout\"}, \"level\": \"error\"}'\nSECRET=\"the-secret-you-set-in-step-4\"\nSIG=$(printf '%s' \"$BODY\" | openssl dgst -sha256 -hmac \"$SECRET\" | sed 's/^.* //')\n\ncurl -X POST http://localhost:8484/webhooks/capture/sentry \\\n -H \"Content-Type: application/json\" \\\n -H \"X-Hub-Signature-256: sha256=$SIG\" \\\n -d \"$BODY\"\n```\n\nThis event is captured as **authenticated**, matched against your armed\nbinding, and dispatched into a new task through the exact same\ngate/worktree/sandbox pipeline every other task goes through -- see\n`docs/architecture/messaging-and-work-intake.md`'s \"Intake provenance\". Check\nthe **Tasks** page; a new task should appear, its body built from the fields\nyour mapping resolved.\n\n## Editing a live binding\n\nAny edit to an armed binding -- even a typo fix -- drops it back to\n**pending_approval** automatically. This can't be bypassed by editing; it\nhas to be re-approved. When editing, leave the **Shared secret** field blank\nto keep the existing secret; only fill it in if you're deliberately\nrotating it.\n"
}
],
"nav": [
+1 -1
View File
@@ -90,7 +90,7 @@ curl -X POST http://localhost:8484/webhooks/capture/sentry \
This event is captured as **authenticated**, matched against your armed
binding, and dispatched into a new task through the exact same
gate/worktree/sandbox pipeline every other task goes through -- see
`ARCHITECTURE.md`'s Task Lifecycle section, "Provenance by origin." Check
`docs/architecture/messaging-and-work-intake.md`'s "Intake provenance". Check
the **Tasks** page; a new task should appear, its body built from the fields
your mapping resolved.
+83
View File
@@ -0,0 +1,83 @@
# Artifact sender
**Status:** Approved
**Revision:** 1
**Authority:** this document
## Purpose
archie-core publishes artifacts — design documents, review reports, research briefs,
runnable pages — to the offloaded.dev collaborative editor. A person reads the
artifact, comments on it, and the artifact page is the shared surface for iteration.
The receiving contract is owned by the editor: `POST /api/artifacts` with a bearer
service token, a closed field set, and server-side idempotency on `requestId` plus
last-write-wins on `sentAt`. This document owns only what archie-core sends.
## Scope
Version one publishes one-way: archie-core sends, the editor receives. The first
producer is the pull-request adversarial review: a completed `ReviewReport` is
rendered to markdown and published after the review completes.
Chat-surface tools (`create_artifact`, `update_artifact`), comment read-back, share
tokens, and runnable-format producers (`html`, `react`, `vue`) are later slices. The
receiving contract already carries them; version one does not exercise them.
## Configuration
- New `[artifacts]` section: `base_url` (site origin), `token` (`SecretRef`) with
`token_env` fallback naming `WORKSPACE_INGEST_TOKEN`.
- An empty `base_url` disables the sender: no client is constructed, nothing is
registered, and publishing degrades to a log line.
- The token is runtime configuration. It is never baked at build time.
## Sender
- New package `internal/infrastructure/artifactsync` exposing one method,
`Publish(ctx, Artifact) (string, error)`, where the string is the artifact URL.
One-method injected seam, concrete client constructed by the boot path when the
configuration resolves — the `crondelivery.Courier` posture.
- The client POSTs the frozen contract body. Each publish sends the complete
source: versions of record are full snapshots, never diffs.
- A `503` response is retried three times with backoff. `401` and `400` are not
retried; they are logged with the field reason the response names.
- `requestId` is `artifact:<task-id>:<version>` when the producer is task-backed,
otherwise `artifact:<identity>:<sentAt-utc>`. Sending the same `requestId` with
an equal or older `sentAt` must be a server-side no-op, which makes sender-side
retry safe.
## Report rendering
- `ReviewReport` gains a pure markdown renderer in `internal/domain/workflow`:
the summary as the lede, one finding per subsection with file, line, verdict,
level and the failure scenario, and the verified properties as a "checked
clean" list. Findings carry their disposition when present.
- The renderer performs no I/O and imports no infrastructure.
## Attribution
- `actor`: the producing identity. The reviewer publishes as `system:pr-reviewer`
when no task identity resolves; otherwise the task's identity name.
- `source`: the producing subsystem, `pr-review` for version one.
- `task`: owner, repository and issue number from the task when one exists.
## Failure behaviour
- Publishing never fails the producing work. A failed publish is logged with its
status and reason; the artifact is republished on the next production of the
same artifact.
- A successful publish records the artifact URL in the run report, so the PR
description or task log links to the readable artifact.
- The published URL is the editor deep link for the artifact, keyed on its
immutable id. archie-core never mints share links; sharing is an
account-holder decision made in the editor.
## Verification
- Client tests against a test server cover 201, 401, 400 with a field reason,
and 503 retry behaviour.
- The report renderer test covers a report with findings, checks, dispositions
and a not-run status.
- Configuration tests cover the disabled case, the `SecretRef` path and the
`token_env` fallback.
+2 -2
View File
@@ -25,7 +25,7 @@ Every claim in this document is mapped to what exists in the tree.
| Tool guardrails (repeat-failure / no-progress) | **Implemented, not on the call path** | `internal/tools/guardrail.go` engine exists. Its only non-test caller is `dispatchOne` — which is itself unreachable — so **the chat path never consults it**, and the agent path only records per *stage* (`domain/workflow/agent.go`) |
| Structured capture tools (model reports into a typed store, not prose) | **Implemented** | `internal/app/agentworker/review.go` `record_finding` / `record_checked` — `core.NewTypedTool` appending to a caller-owned slice. This is the pattern a research source ledger should reuse |
| Curator engine (interval, declared tool set, lifecycle) | **Implemented, tools broken** | `internal/domain/curator`; a curator declaring tools is **refused** at registration (`registry.go`, no `ToolBuilder` implementation) and `curatorLLMRunner` (`internal/app/archied/main.go`) drops `ChatRequest.Tools`. Budgets are time+concurrency only (`runtime.go`) — **no token budget** |
| Cron / scheduling + delivery router | **Implemented, wired into no binary** | `internal/domain/scheduling`, `internal/infrastructure/cronstore` (schema v2, `Kind` discriminator), `internal/infrastructure/crondelivery` — all present and green, but `go list -deps` for every `cmd/` contains none of them, and `cron`-expression firing arithmetic is unimplemented (`schedule.go`, "slice 2"). `ARCHITECTURE.md` still asserts "**No daemon-internal cron.**" |
| Cron / scheduling + delivery router | **Implemented, wired into the daemon** | `internal/domain/scheduling` (ticker engine), `internal/infrastructure/cronstore` (schema v2, `Kind` discriminator), and `internal/infrastructure/crondelivery`; `internal/app/archied/scheduling.go` composes them and `main.go` starts the engine. |
| Workflow engines (multi-stage, resumable) | **Implemented, worktree-bound** | `internal/domain/workflow` — every non-triage workflow opens with `StagePrepareWorktree()` and mounts `tc.Dir` into a container; the task persists after every stage (`workflow.go`). `triage.go` documents there is "no cheaper directory-less classification primitive" |
| Long-running chat-initiated work with progress | **Partial** | Chat turns run **once, synchronously**, inline in the channel's turn runner (`internal/gateway/turn.go`). Chat can spawn a *workflow task* (`task_spawn`) which is asynchronous — but that path is worktree/PR-shaped |
| **Operator-push delivery of an async result** | **Aspirational — zero code** | Nothing pushes a finished result to a chat. No channel subscribes to the event bus (Telegram only *publishes* `KindTurnCompleted`); `crondelivery.Courier` is the intended seam but has **no implementation and no caller**; `send_file` works only inside the live turn. The one out-of-band notification in the tree is feasibility's n8n webhook (`feasibility.go`) |
@@ -198,7 +198,7 @@ Reusing it would mean every ad-hoc research run leaves a permanent job row.
| Collator | `internal/domain/research` | Model pass over `[]ChildConclusion` + source store → report body |
| Report renderer | `internal/domain/research` | Markdown + numbered citation list rendered **from the store** |
| Chat trigger | slice 2: `internal/tools/research`; slice 3: gateway tool + courier in `internal/app/archied` | — |
| Durable run + delivery (slice 3) | `internal/domain/research` + `internal/infrastructure/research` + `internal/app/archied` wiring | Follows the strict plugin-engine rule in `ARCHITECTURE.md` ("Plugin engine
| Durable run + delivery (slice 3) | `internal/domain/research` + `internal/infrastructure/research` + `internal/app/archied` wiring | Follows the strict plugin-engine rule in `docs/architecture/plugins-and-extensions.md` ("Plugin engine
rule (strict)"): typed contract, owning registry with start/health/stop, narrow typed registrar |
### Why not the curator
+2 -1
View File
@@ -42,7 +42,8 @@ long as it's implemented behind one of the typed engine families below.
generic interface -- see "Typed positions, not one generic hook" below,
which is how it stays compliant with the plugin engine rule instead of
reopening that document's #4.
- `ARCHITECTURE.md#plugin-engine-rule-strict` governs every extension point
- `docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`
governs every extension point
introduced here. Every position below is a typed domain contract with an
owning registry, not a generic callback map. This is a hard constraint,
confirmed against the existing rule text before writing this doc.
+2 -1
View File
@@ -12,7 +12,8 @@ each question and saying which packages change.
## 1. In-process typed families, not out-of-process dispatch
Archie has no out-of-process extension mechanism. The plugin engine
rule (`ARCHITECTURE.md#plugin-engine-rule-strict`, mechanically checked by
rule (`docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`,
mechanically checked by
`internal/plugin/architecture_test.go`) keeps every capability family
in-process behind a narrow typed registrar; NATS RPC exists for
daemon<->agentexec, a different actor with a different trust boundary (task
+3 -2
View File
@@ -25,8 +25,9 @@ in the repo defines this (`grep -ri image` over `internal/domain` and
## Design
Follows the plugin engine family shape already established by
`internal/domain/curator` and `internal/domain/memory` (ARCHITECTURE.md
"plugin engine rule strict"): a typed contract with real domain operations,
`internal/domain/curator` and `internal/domain/memory`
(`docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`): a
typed contract with real domain operations,
an owning `Registry` (register/get/names, no start/stop lifecycle — see
below), and narrow host access via a `Registrar`.
+2 -1
View File
@@ -332,7 +332,8 @@ Each step is independently verifiable and lands green on its own.
4. **Chat write path.** The tool adapter onto CRUD and the turn's subject on the
context. *Depends on 3, because a write is only observable through the read.*
5. **Delete `internal/memory` and the dead config/dashboard surface**, and
rewrite `ARCHITECTURE.md`'s "Memory engine family" section to describe the
rewrite the "Memory engine family" section of
`docs/architecture/plugins-and-extensions.md` to describe the
engine family rather than a manager it deleted. *Depends on 3 and 4.*
Steps 3 and 4 are separable from each other; step 1 is separable from everything.
+1 -1
View File
@@ -116,7 +116,7 @@ re-baselined coupling inventory is below, and the ratified boundary is
- Any concrete adoption of Pi (or another external agent harness) as an
execution runner. Raised in conversation alongside this decomposition,
but it is a separate decision (agent execution as a strict data
boundary, `ARCHITECTURE.md`) and does not gate or get gated by this PRD.
boundary, `docs/architecture/agent-system.md`) and does not gate or get gated by this PRD.
## Open questions
+4 -4
View File
@@ -66,10 +66,10 @@ the control.
Mapped payload fields are treated as ordinary user-controlled text inside an
agent prompt, exactly like an inbound chat message never as trusted
instructions. "The agent should ignore instructions in the payload" is not a
control and must not appear as one anywhere in this feature -- per
`ARCHITECTURE.md`'s environmental-enforcement-over-prompt-rules principle,
the boundary is the gate and the sandbox, not a sentence in the system
prompt.
control and must not appear as one anywhere in this feature -- per the
environmental-enforcement principle (`docs/architecture/agent-system.md`,
"Existing invariants to preserve"), the boundary is the gate and the sandbox,
not a sentence in the system prompt.
## 5. Retention and redaction of captured payloads
+18 -3
View File
@@ -9,27 +9,36 @@ require (
github.com/ProtonMail/go-crypto v1.4.1 // indirect
github.com/antithesishq/antithesis-sdk-go v0.8.0 // indirect
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cloudflare/circl v1.6.5 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/davidmz/go-pageant v1.0.2 // indirect
github.com/disintegration/imaging v1.6.2 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/go-connections v0.8.1 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/domodwyer/mailyak/v3 v3.6.2 // indirect
github.com/dustin/go-humanize v1.1.0 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/fatih/color v1.19.0 // indirect
github.com/felixge/httpsnoop v1.1.0 // indirect
github.com/fsnotify/fsnotify v1.10.1 // indirect
github.com/gabriel-vasile/mimetype v1.4.15 // indirect
github.com/ganigeorgiev/fexpr v0.6.0 // indirect
github.com/go-fed/httpsig v1.1.0 // indirect
github.com/go-git/gcfg/v2 v2.0.2 // indirect
github.com/go-logr/logr v1.4.4 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/google/go-querystring v1.2.0 // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/hashicorp/go-version v1.9.0 // indirect
github.com/kevinburke/ssh_config v1.6.0 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/cpuid/v2 v2.4.0 // indirect
github.com/mattn/go-colorable v0.1.15 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
@@ -41,8 +50,12 @@ require (
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/pjbgf/sha1cd v0.6.0 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/pocketbase/ozzo-validation/v4 v4.3.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/sergi/go-diff v1.4.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/cobra v1.10.2 // indirect
github.com/spf13/pflag v1.0.10 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0 // indirect
go.opentelemetry.io/otel v1.46.0 // indirect
@@ -51,13 +64,14 @@ require (
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba // indirect
golang.org/x/image v0.46.0 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/text v0.42.0 // indirect
golang.org/x/time v0.16.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260921155816-b14227669459 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260921155816-b14227669459 // indirect
modernc.org/libc v1.77.1 // indirect
modernc.org/libc v1.74.4 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.12.1 // indirect
)
@@ -71,13 +85,14 @@ require (
github.com/go-git/go-git/v6 v6.0.0-alpha.5
github.com/go-jose/go-jose/v4 v4.1.5
github.com/go-telegram/bot v1.27.0
github.com/google/codesearch v1.3.0-rc.1
github.com/google/go-github/v78 v78.0.0
github.com/google/uuid v1.6.0
github.com/moby/moby/api v1.56.0
github.com/moby/moby/client v0.6.0
github.com/nats-io/nats-server/v2 v2.15.0
github.com/nats-io/nats.go v1.54.0
github.com/pocketbase/dbx v1.12.0
github.com/pocketbase/pocketbase v0.40.4
github.com/samcharles93/ai-sdk v0.1.39
github.com/traefik/yaegi v0.16.1
golang.org/x/net v0.59.0
@@ -86,5 +101,5 @@ require (
google.golang.org/grpc v1.85.0-dev.0.20260825072537-93e31b48545e
google.golang.org/protobuf v1.36.12
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.59.0
modernc.org/sqlite v1.57.0
)
+51 -2
View File
@@ -25,6 +25,9 @@ github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYW
github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
github.com/asaskevich/govalidator v0.0.0-20200108200545-475eaeb16496/go.mod h1:oGkLhpf+kjZl6xBf758TQhh5XrAeiJv/7FRz/2spLIg=
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
github.com/bwesterb/go-ristretto v1.2.4/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
@@ -38,17 +41,22 @@ github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmC
github.com/containerd/typeurl/v2 v2.2.0/go.mod h1:8XOOxnyatxSWuG8OfsZXVnAF4iZfedjS/8UHSPJnX4g=
github.com/coreos/go-oidc/v3 v3.21.0 h1:wZo4Q9Pum8dYEj0eMUPrqR+kvuGkeUplbLpNCkBqoWM=
github.com/coreos/go-oidc/v3 v3.21.0/go.mod h1:DYCf24+ncYi+XkIH97GY1+dqoRlbaSI26KVTCI9SrY4=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davidmz/go-pageant v1.0.2 h1:bPblRCh5jGU+Uptpz6LgMZGD5hJoOt7otgT454WvHn0=
github.com/davidmz/go-pageant v1.0.2/go.mod h1:P2EDDnMqIwG5Rrp05dTRITj9z2zpGcD9efWSkTNKLIE=
github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1ei82L+c=
github.com/disintegration/imaging v1.6.2/go.mod h1:44/5580QXChDfwIclfc/PCwrr44amcmDAg8hxG0Ewe4=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/docker/go-connections v0.8.1 h1:JibmG5hULs5qXSr/cp/w3Pw5fZuStt4MOHMUExb29/M=
github.com/docker/go-connections v0.8.1/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/domodwyer/mailyak/v3 v3.6.2 h1:x3tGMsyFhTCaxp6ycgR0FE/bu5QiNp+hetUuCOBXMn8=
github.com/domodwyer/mailyak/v3 v3.6.2/go.mod h1:lOm/u9CyCVWHeaAmHIdF4RiKVxKUT/H5XX10lIKAL6c=
github.com/dustin/go-humanize v1.1.0 h1:dbKTrvD0klcbBV/h4AWJdMuZogJACoMlvWIWZ5b2xWg=
github.com/dustin/go-humanize v1.1.0/go.mod h1:hc1CvRkJMsgxqjmjMQF3QNRAZBwY8AXBAzKYoSX9sFI=
github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
@@ -57,8 +65,16 @@ github.com/envoyproxy/go-control-plane v0.14.0/go.mod h1:NcS5X47pLl/hfqxU70yPwL9
github.com/envoyproxy/go-control-plane/envoy v1.39.0/go.mod h1:5e4ylfTZO723MEEFsCpSW4ZEBWR8mwkEyXfwJBTCZ9c=
github.com/envoyproxy/go-control-plane/ratelimit v0.1.0/go.mod h1:Wk+tMFAFbCXaJPzVVHnPgRKdUdwW/KdbRt94AzgRee4=
github.com/envoyproxy/protoc-gen-validate v1.3.3/go.mod h1:TsndJ/ngyIdQRhMcVVGDDHINPLWB7C82oDArY51KfB0=
github.com/fatih/color v1.19.0 h1:Zp3PiM21/9Ld6FzSKyL5c/BULoe/ONr9KlbYVOfG8+w=
github.com/fatih/color v1.19.0/go.mod h1:zNk67I0ZUT1bEGsSGyCZYZNrHuTkJJB+r6Q9VuMi0LE=
github.com/felixge/httpsnoop v1.1.0 h1:3YtUj32ZZkqZtt3sZZsClsymw/QDuVfpNhoA31zeORc=
github.com/felixge/httpsnoop v1.1.0/go.mod h1:Zqxgdd+1Rkcz8euOqdr7lqgCRJztwr5hp9vDSi5UZCE=
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
github.com/gabriel-vasile/mimetype v1.4.15 h1:05iP/CYtZ/w455R/KZM6rZ5ieAdh99UPtd+d3YzLmaI=
github.com/gabriel-vasile/mimetype v1.4.15/go.mod h1:azpTcoLcDZRNgFou5j+APrqQx9HqVPWa6ijYQIIVswQ=
github.com/ganigeorgiev/fexpr v0.6.0 h1:Fza3O/QMBKEudUvxV862qe6GjxM60GJjjKytdp+VQus=
github.com/ganigeorgiev/fexpr v0.6.0/go.mod h1:RyGiGqmeXhEQ6+mlGdnUleLHgtzzu/VGO2WtJkF5drE=
github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c=
github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU=
github.com/go-fed/httpsig v1.1.0 h1:9M+hb0jkEICD8/cAiNqEB66R87tTINszBRTjwjQzWcI=
@@ -81,14 +97,16 @@ github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ4
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-quicktest/qt v1.101.0/go.mod h1:14Bz/f7NwaXPtdYEgzsx46kqSxVwTbzVZsDC26tQJow=
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-telegram/bot v1.27.0 h1:PG9yBDzM8O26oUerE9SZwWlHon4RpH8WcNslZ7CEZmE=
github.com/go-telegram/bot v1.27.0/go.mod h1:i2TRs7fXWIeaceF3z7KzsMt/he0TwkVC680mvdTFYeM=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/golang/glog v1.2.5/go.mod h1:6AhwSGph0fcJtXVM/PEHPqZlFeoLxhs7/t5UDAwmO+w=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/codesearch v1.3.0-rc.1 h1:vSFGuF5wS3sjimuLzqPG+PGjWQ9VvEtPFbxdZD7YuuQ=
github.com/google/codesearch v1.3.0-rc.1/go.mod h1:hKdpx7wOsvk2WLT/D6NLPxFeLT8jKCehZu21JpA2rIY=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
@@ -110,6 +128,7 @@ github.com/hashicorp/go-version v1.9.0 h1:CeOIz6k+LoN3qX9Z0tyQrPtiB1DFYRPfCIBtaX
github.com/hashicorp/go-version v1.9.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/kevinburke/ssh_config v1.6.0 h1:J1FBfmuVosPHf5GRdltRLhPJtJpTlMdKTBjRgTaQBFY=
github.com/kevinburke/ssh_config v1.6.0/go.mod h1:q2RIzfka+BXARoNexmF9gkxEX7DmvbW9P4hIVx2Kg4M=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
@@ -123,6 +142,8 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY=
github.com/mattn/go-colorable v0.1.15/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
@@ -156,17 +177,31 @@ github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmd
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pocketbase/dbx v1.12.0 h1:/oLErM+A0b4xI0PWTGPqSDVjzix48PqI/bng2l0PzoA=
github.com/pocketbase/dbx v1.12.0/go.mod h1:xXRCIAKTHMgUCyCKZm55pUOdvFziJjQfXaWKhu2vhMs=
github.com/pocketbase/ozzo-validation/v4 v4.3.0 h1:uKBDVma7bZqgR2a6AwE+k9hkuDFfiZMpBHQdZ1z3iQs=
github.com/pocketbase/ozzo-validation/v4 v4.3.0/go.mod h1:6XNjSTw/Jb2F8LOkKO3oyzIWExbrGiYoS4uVxVwz90g=
github.com/pocketbase/pocketbase v0.40.4 h1:0SvSUreR3NhUMCs9LchE59oEG53efZ3cKiMGyAGBN9U=
github.com/pocketbase/pocketbase v0.40.4/go.mod h1:2mU+80FLiY1fb13WZRg8Xx/lKg4nTjgKVJMigyRH6k0=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/russross/blackfriday v1.6.0/go.mod h1:ti0ldHuxg49ri4ksnFxlkCfN+hvslNlmVHqNRXXJNAY=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/samcharles93/ai-sdk v0.1.39 h1:gbNPYeA27MifReCruBiqMgb4fFSjCDi1Zv57a9bo0Vo=
github.com/samcharles93/ai-sdk v0.1.39/go.mod h1:u4zrQ9/Q58L5mNEut4bNQ1ks+RCKq71LN5EDz2N/0Zk=
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1/go.mod h1:uToXkOrWAZ6/Oc07xWQrPOhJotwFIyu2bBVN41fcDUY=
github.com/sergi/go-diff v1.4.0 h1:n/SP9D5ad1fORl+llWyN+D6qoUETXNZARKjyY2/KVCw=
github.com/sergi/go-diff v1.4.0/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spiffe/go-spiffe/v2 v2.8.1/go.mod h1:47Q0Q9/AqGha8QLHp+kxpH4Wca7X7EnOtlIJy3mxZ3U=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
@@ -189,6 +224,7 @@ go.opentelemetry.io/otel/sdk/metric v1.46.0 h1:0piZ26EG4RBfebb2jhDH6ERCYHoVWduc3
go.opentelemetry.io/otel/sdk/metric v1.46.0/go.mod h1:I1PbKrdVc8Qu8HYVDNtqVIwLwjNrhsV/uFuxfwg8mO4=
go.opentelemetry.io/otel/trace v1.46.0 h1:OULy7ccdJnZtJ0UDYFOIGaCmiWzJ8Vi2G/Rsu60qs1c=
go.opentelemetry.io/otel/trace v1.46.0/go.mod h1:J7GAXweO77XSFkB/rmAqk9D6ihszhFjLU+d9WuUxDLI=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
@@ -198,9 +234,13 @@ golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba h1:Ck8QetSgk912qxWLMCKxd0in+aiyBQyDSMae6e/xmpU=
golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba/go.mod h1:50RgIsmK7OwqzTTeqcSXQW8SswW0o8fRcDxmqGluJ8E=
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/image v0.46.0 h1:b1+oYj0Jbp6K5MDT4i4/eZpYlk3V8SJhhDKh6LBHAyQ=
golang.org/x/image v0.46.0/go.mod h1:3B3W05VGVQyuXucLINLjXKrqISASfi4Xj+iCVkLMwew=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues=
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
@@ -221,6 +261,7 @@ golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9sn
golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE=
golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
@@ -234,6 +275,7 @@ golang.org/x/tools/go/packages/packagestest v0.1.1-deprecated/go.mod h1:RVAQXBGN
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
google.golang.org/api v0.293.0/go.mod h1:6n5tjEB1gzwniZTepZ0g5u+wM7Bof5GeULCx/zh8ZE0=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto/googleapis/api v0.0.0-20260918162117-cecb64721679 h1:FEp7JNE32DTAwbnI/ixagnmj7Xm1eTONofGEUXFjZ4w=
google.golang.org/genproto/googleapis/api v0.0.0-20260918162117-cecb64721679/go.mod h1:52bV8FLAQ9Qmcqaq9ECLmuEHZthk+6OPV45aKBBrsNw=
google.golang.org/genproto/googleapis/api v0.0.0-20260921155816-b14227669459 h1:GS9OIt/j7c8bvBjYNgnKQysVfmV7e4jM0H8ZK95G4t8=
@@ -256,8 +298,10 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q=
gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA=
modernc.org/cc/v4 v4.29.1 h1:MKgdCV3WykTSPqpVrnxdEDS0HEd2FHpKZDzxzU5LyeI=
modernc.org/cc/v4 v4.29.7 h1:q+NXGJ0bK3b4TXFYQQVr9pYETGnmwFWkrUzJnMya/Tg=
modernc.org/cc/v4 v4.29.7/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.34.6 h1:sBgfIwyN0TQ9C5hwIeuqyeAKyMWnbvj2fvpF4L11uzU=
modernc.org/ccgo/v4 v4.36.0 h1:T5h7ZMcrw2r94oDLtBcU3YI6e1W8DiUg2SJIs2fwD2s=
modernc.org/ccgo/v4 v4.36.0/go.mod h1:rrtGc2QkS239nYb/mQNuBMyjq3/y3ZXWbBjPoV3wqzA=
modernc.org/ccgo/v4 v4.36.1 h1:ZNIUZAryN0UgnJwtyxrdEzcFc3yD4Cu4AzjfPXsLsIE=
@@ -265,10 +309,13 @@ modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.4 h1:2g65LGVSmFQrXeITAw97x7hCRvZFcyE1uDP+7Vng7JI=
modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.74.4 h1:fX1Omw4o2/1C2iRkkIsrQTasJQldLhRmuPreXLoWs9k=
modernc.org/libc v1.74.4/go.mod h1:eeQAS9W3sZeKYMFubydxJpII9ybHWshk+7or7bLG9co=
modernc.org/libc v1.77.0 h1:wV+faa9ixRoUNlakU0fHLi/1kV2YLG8D8Ksk3c2mPtY=
modernc.org/libc v1.77.0/go.mod h1:BmXDCJcMkRMRbX2EI8+Zhh4fKWlHNbCCZEy0tiZevP4=
modernc.org/libc v1.77.1 h1:Ct8j47QtiZ1Enj2DtFXQtUqrPCAjdCmPjtCuvrYQ0Hs=
@@ -281,6 +328,8 @@ modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.57.0 h1:qNQP6xnx5M0ISNtlnxoOX0+cD5bJ0/gr9aMmndFczzg=
modernc.org/sqlite v1.57.0/go.mod h1:yCJ2cmAaIkHQ25oXWrF8H4O1lIfPYPR26yCEDj2P3pQ=
modernc.org/sqlite v1.59.0 h1:X1es1GpqBlS/5T+vbM4HLUdaa8OtQx468DF2vrx+38A=
modernc.org/sqlite v1.59.0/go.mod h1:+paeT2A3iPRHkQDwG7oA6Tk0zQd5woMEI8q7orfry8k=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
@@ -30,6 +30,7 @@ import (
"github.com/samcharles93/archie-core/internal/forge"
"github.com/samcharles93/archie-core/internal/forgerpc"
agentnats "github.com/samcharles93/archie-core/internal/infrastructure/agenttransport/nats"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/infrastructure/workflowsteps"
"github.com/samcharles93/archie-core/internal/installtype"
@@ -376,9 +377,10 @@ func startStateStoreGRPC(t *testing.T, local *store.Store) string {
t.Fatal(err)
}
server := grpc.NewServer()
eda := edastore.OpenTest(t)
staterpc.RegisterServer(server, staterpc.Deps{
Tasks: local, Captures: local, Mappings: local, Bindings: local,
BindingDispatcher: local, BindingTaskCreator: local,
Tasks: local, Captures: eda, Mappings: eda, Bindings: eda,
BindingDispatcher: eda, BindingTaskCreator: local,
})
go func() { _ = server.Serve(listener) }()
t.Cleanup(func() { server.Stop(); _ = listener.Close() })
+4 -4
View File
@@ -5,15 +5,15 @@ import (
"strings"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/secret"
"github.com/samcharles93/archie-core/internal/store"
)
// bindingCipherFromConfig resolves the binding at-rest encryption key from
// config through the secret registry and returns a store cipher. An unset
// config through the secret registry and returns the EDA store's cipher. An unset
// encryption_key returns a nil cipher, keeping the legacy plaintext path (the
// store's default). See docs/prds/binding-secret-encryption.md.
func bindingCipherFromConfig(cfg config.Config, secrets *secret.Registry) (store.BindingCipher, error) {
func bindingCipherFromConfig(cfg config.Config, secrets *secret.Registry) (edastore.BindingCipher, error) {
if cfg.Bindings.EncryptionKey == (secret.SecretRef{}) {
return nil, nil
}
@@ -36,5 +36,5 @@ func bindingCipherFromConfig(cfg config.Config, secrets *secret.Registry) (store
previous = append(previous, v)
}
}
return store.NewBindingCipher(active, previous)
return edastore.NewBindingCipher(active, previous)
}
+4 -36
View File
@@ -45,8 +45,8 @@ import (
"github.com/samcharles93/archie-core/internal/forge"
forgewebhook "github.com/samcharles93/archie-core/internal/forge/webhook"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/indexing"
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
infraembedding "github.com/samcharles93/archie-core/internal/infrastructure/embedding"
"github.com/samcharles93/archie-core/internal/infrastructure/eventbus/nats"
infraMemory "github.com/samcharles93/archie-core/internal/infrastructure/memory"
@@ -65,7 +65,6 @@ import (
"github.com/samcharles93/archie-core/internal/skill"
"github.com/samcharles93/archie-core/internal/storage"
"github.com/samcharles93/archie-core/internal/tools"
toolsbuiltin "github.com/samcharles93/archie-core/internal/tools/builtin"
"github.com/samcharles93/archie-core/internal/tools/minimax"
toolprovider "github.com/samcharles93/archie-core/internal/tools/provider"
builtintoolprovider "github.com/samcharles93/archie-core/internal/tools/provider/builtin"
@@ -107,7 +106,8 @@ type boot struct {
playbooks *playbook.Store
st storecontract.TaskStore
st storecontract.TaskStore
eda *edastore.Store
// stateStore is the State Store contract adapter every daemon and gateway
// store consumer depends on. It is ALWAYS the remote *staterpc.Client
// dialed to the standalone archie-state-store gRPC service at
@@ -1115,37 +1115,6 @@ func (b *boot) setupGuardrails() {
)
}
// workspaceIndex builds the optional codesearch index that narrows grep's
// candidate files. It returns a slice so a failure is simply "no index"
// rather than a typed-nil reaching the provider.
//
// Every failure degrades instead of propagating: grep falls back to its
// direct walk, which is what it did before this was wired at all. An
// accelerator that can refuse to build must never stop the daemon starting.
//
// The context is detached from cancellation but keeps its values, because
// NewManager starts an asynchronous build whose lifetime is the process
// while the gateway path's boot context is cancelled once startup returns.
// Inheriting cancellation would kill the build mid-flight and leave grep
// permanently unindexed with nothing to show why; using Background instead
// would silently drop any values the caller's context carries.
func (b *boot) workspaceIndex(ctx context.Context, workspace string) []toolsbuiltin.GrepIndex {
cfg, log := b.cfg, b.log
idxCfg := cfg.Indexing.WithDefaults(cfg.WorkDir)
manager, err := indexing.NewManager(
context.WithoutCancel(ctx),
indexing.Config{IndexDir: idxCfg.IndexDir, DBPath: idxCfg.DBPath, Log: log},
workspace,
)
if err != nil {
log.Warn("workspace index unavailable; grep will search unindexed",
"workspace", workspace, "index_dir", idxCfg.IndexDir, "error", err)
return nil
}
log.Info("workspace index enabled", "workspace", workspace, "index_dir", idxCfg.IndexDir)
return []toolsbuiltin.GrepIndex{manager}
}
// registerTools registers the tool providers with a lifecycle: workspace
// file/shell tools and optional MCP servers. Memory tools
// (memory_create/update/delete/list) are not registered here -- they are
@@ -1159,8 +1128,7 @@ func (b *boot) registerTools(ctx context.Context) error {
// deliberate choice rather than a default.
if workspace := cfg.Chat.Workspace; workspace != "" {
unrestricted := cfg.Chat.UnrestrictedFilesystem
index := b.workspaceIndex(ctx, workspace)
if err := b.providerRegistry.Register(builtintoolprovider.New(workspace, unrestricted, index...)); err != nil {
if err := b.providerRegistry.Register(builtintoolprovider.New(workspace, unrestricted)); err != nil {
log.Error("workspace tool provider registration failed", "err", err)
return err
}
-121
View File
@@ -1,121 +0,0 @@
package archied
import (
"context"
"encoding/json"
"flag"
"fmt"
"io"
"os"
"github.com/samcharles93/archie-core/internal/indexing"
)
// flagErrorOutput receives helper diagnostics. It is a variable so tests can
// capture them, and it is deliberately not stdout: the parent parses stdout
// as JSON, so a stray usage message there would corrupt the result.
var flagErrorOutput io.Writer = os.Stderr
// codesearchHelperCommand is the argv[1] token that switches archied out of
// daemon mode and into the codesearch helper.
//
// indexing.Manager deliberately runs the google/codesearch library in a child
// process: upstream calls fatal exits on a corrupt index and leaks mmapped
// regions, neither of which may be allowed to take down a long-lived daemon.
// The child it spawns is this binary, so this name is a contract with
// internal/indexing -- renaming either side silently disables indexed search
// and falls the agent back to unindexed grep.
const codesearchHelperCommand = "workspace-codesearch"
// IsCodesearchHelperArgs reports whether args (os.Args[1:]) selects the
// helper rather than the daemon.
func IsCodesearchHelperArgs(args []string) bool {
return len(args) > 0 && args[0] == codesearchHelperCommand
}
// RunCodesearchHelper executes one helper subcommand and returns a process
// exit code. Output is the machine-readable result the parent decodes; all
// diagnostics go to stderr so they cannot corrupt it.
func RunCodesearchHelper(args []string, out io.Writer) int {
if len(args) == 0 {
helperErrorf("%s: expected a subcommand (build|candidates)\n", codesearchHelperCommand)
return 2
}
ctx := context.Background()
switch args[0] {
case "build":
fs := newHelperFlagSet("build")
root := fs.String("root", "", "workspace root to index")
index := fs.String("index", "", "path of the index sidecar to write")
if err := fs.Parse(args[1:]); err != nil {
return 2
}
if *root == "" || *index == "" {
helperErrorf("build: --root and --index are required\n")
return 2
}
if err := indexing.BuildIndex(ctx, *root, *index); err != nil {
helperErrorf("build: %v\n", err)
return 1
}
return 0
case "candidates":
return runCandidates(args[1:], out)
default:
helperErrorf("%s: unknown subcommand %q\n", codesearchHelperCommand, args[0])
return 2
}
}
func newHelperFlagSet(name string) *flag.FlagSet {
fs := flag.NewFlagSet(name, flag.ContinueOnError)
fs.SetOutput(flagErrorOutput)
return fs
}
// runCandidates handles the "candidates" subcommand, returning candidate
// file paths as a JSON array on stdout.
func runCandidates(args []string, out io.Writer) int {
fs := newHelperFlagSet("candidates")
index := fs.String("index", "", "path of the index sidecar to query")
pattern := fs.String("pattern", "", "search pattern")
literal := fs.Bool("literal", false, "treat the pattern as a literal string")
caseSensitive := fs.Bool("case-sensitive", false, "match case sensitively")
if err := fs.Parse(args); err != nil {
return 2
}
if *index == "" || *pattern == "" {
helperErrorf("candidates: --index and --pattern are required\n")
return 2
}
files, err := indexing.IndexCandidates(*index, *pattern, *literal, *caseSensitive)
if err != nil {
helperErrorf("candidates: %v\n", err)
return 1
}
// The parent decodes a JSON array and treats any decode failure as
// "no index", so a nil slice must still marshal as [] rather than null.
if files == nil {
files = []string{}
}
encoded, err := json.Marshal(files)
if err != nil {
helperErrorf("candidates: encode result: %v\n", err)
return 1
}
if _, err := out.Write(encoded); err != nil {
helperErrorf("candidates: write result: %v\n", err)
return 1
}
return 0
}
// helperErrorf writes a diagnostic to stderr. The write result is ignored
// deliberately: if stderr is gone there is nowhere left to report it, and the
// exit code already carries the outcome the parent acts on.
func helperErrorf(format string, args ...any) {
_, _ = fmt.Fprintf(flagErrorOutput, format, args...)
}
-85
View File
@@ -1,85 +0,0 @@
package archied
import (
"encoding/json"
"os"
"path/filepath"
"slices"
"strings"
"testing"
)
func TestRunCodesearchHelper(t *testing.T) {
root := t.TempDir()
if err := os.WriteFile(filepath.Join(root, "a.txt"), []byte("alpha needle omega\n"), 0o600); err != nil {
t.Fatalf("write fixture: %v", err)
}
if err := os.WriteFile(filepath.Join(root, "b.txt"), []byte("nothing here\n"), 0o600); err != nil {
t.Fatalf("write fixture: %v", err)
}
indexPath := filepath.Join(t.TempDir(), "workspace.csearch")
t.Run("build writes an index sidecar", func(t *testing.T) {
var out strings.Builder
code := RunCodesearchHelper([]string{"build", "--root", root, "--index", indexPath}, &out)
if code != 0 {
t.Fatalf("build exit code = %d, want 0", code)
}
if _, err := os.Stat(indexPath); err != nil {
t.Fatalf("index sidecar not written: %v", err)
}
})
t.Run("candidates emits the JSON contract the manager decodes", func(t *testing.T) {
var out strings.Builder
code := RunCodesearchHelper([]string{
"candidates", "--index", indexPath, "--pattern", "needle",
}, &out)
if code != 0 {
t.Fatalf("candidates exit code = %d, want 0", code)
}
var files []string
if err := json.Unmarshal([]byte(out.String()), &files); err != nil {
t.Fatalf("decode candidates output %q: %v", out.String(), err)
}
if !slices.ContainsFunc(files, func(p string) bool {
return strings.HasSuffix(p, "a.txt")
}) {
t.Errorf("candidates = %v, want the file containing the needle", files)
}
})
invalid := []struct {
name string
args []string
}{
{name: "no subcommand", args: nil},
{name: "unknown subcommand", args: []string{"frobnicate"}},
{name: "build without root", args: []string{"build", "--index", indexPath}},
{name: "build without index", args: []string{"build", "--root", root}},
{name: "candidates without index", args: []string{"candidates", "--pattern", "x"}},
{name: "candidates without pattern", args: []string{"candidates", "--index", indexPath}},
}
for _, tc := range invalid {
t.Run(tc.name+" is rejected", func(t *testing.T) {
var out strings.Builder
if code := RunCodesearchHelper(tc.args, &out); code == 0 {
t.Errorf("exit code = 0, want non-zero for %v", tc.args)
}
})
}
}
// The manager shells out to argv[0] with this exact subcommand name, so a
// rename on either side silently disables indexed search.
func TestCodesearchHelperSubcommandIsRecognised(t *testing.T) {
if !IsCodesearchHelperArgs([]string{codesearchHelperCommand, "build"}) {
t.Errorf("IsCodesearchHelperArgs() did not recognise %q", codesearchHelperCommand)
}
if IsCodesearchHelperArgs([]string{"-once"}) {
t.Error("IsCodesearchHelperArgs() claimed a normal daemon flag")
}
if IsCodesearchHelperArgs(nil) {
t.Error("IsCodesearchHelperArgs() claimed an empty argument list")
}
}
@@ -1,206 +0,0 @@
package archied
import (
"encoding/json"
"log/slog"
"os"
"path/filepath"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/app/controlplane"
controlpb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
"github.com/samcharles93/archie-core/internal/store"
)
// migrationServer builds the validating side exactly as RunStateStore composes
// it: a store over a throwaway file and openStateStoreControlPlane, the root
// helper that registers the shared step vocabulary.
func migrationServer(t *testing.T) (*controlplane.Server, *store.Store) {
t.Helper()
st, err := store.Open(t.Context(), filepath.Join(t.TempDir(), "archie.db-tasks.sqlite"))
if err != nil {
t.Fatalf("open store: %v", err)
}
t.Cleanup(func() { _ = st.Close() })
server, err := openStateStoreControlPlane(st)
if err != nil {
t.Fatalf("open control plane: %v", err)
}
return server, st
}
// seedSchedules mirrors RunStateStore's ImportConfig: the schedules resource
// starts as the empty seed at version 1, which is the state the legacy
// migration runs against.
func seedSchedules(t *testing.T, control *controlplane.Server) int64 {
t.Helper()
seed, err := json.Marshal([]cronstore.JobSpec{})
if err != nil {
t.Fatalf("encode empty schedules seed: %v", err)
}
response, err := control.Command(t.Context(), &controlpb.CommandRequest{
Kind: controlplane.SchedulesKind, Command: "replace", ValueJson: seed,
ExpectedVersion: 0, Actor: "system:migration", Source: "legacy-config", RequestId: "import:schedules",
})
if err != nil {
t.Fatalf("seed schedules: %v", err)
}
return response.GetResource().GetVersion()
}
// writeLegacyJobs writes a legacy cronstore file at the layout
// migrateLegacySchedules reads: <dir of dbPath>/cron/jobs.json, in the
// store's own on-disk envelope shape (schema v2).
func writeLegacyJobs(t *testing.T, dir string, jobs []cronstore.JobSpec) string {
t.Helper()
data, err := json.Marshal(struct {
SchemaVersion int `json:"schema_version"`
Jobs []cronstore.JobSpec `json:"jobs"`
}{SchemaVersion: 2, Jobs: jobs})
if err != nil {
t.Fatalf("encode fixture: %v", err)
}
return writeLegacyFile(t, dir, string(data))
}
func writeLegacyFile(t *testing.T, dir, content string) string {
t.Helper()
path := filepath.Join(dir, "cron", "jobs.json")
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatalf("create legacy cron directory: %v", err)
}
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatalf("write legacy jobs file: %v", err)
}
return path
}
// legacyJob is a schema-v1-shaped job the old daemon's cron engine wrote.
// An empty Kind means chat, the pre-field default (cronstore.KindChat).
func legacyJob(id, kind string, interval time.Duration) cronstore.JobSpec {
stamped := time.Date(2026, 9, 1, 9, 0, 0, 0, time.UTC)
return cronstore.JobSpec{
ID: id, Kind: kind, Detail: "legacy " + id,
Schedule: cronstore.Schedule{Kind: cronstore.ScheduleInterval, Interval: cronstore.Duration(interval)},
NextRun: stamped, Created: stamped, Updated: stamped,
}
}
// storedScheduleIDs decodes the schedules resource into the job IDs it holds.
func storedScheduleIDs(t *testing.T, st *store.Store) []string {
t.Helper()
resource, err := st.Resource(t.Context(), controlplane.SchedulesKind)
if err != nil {
t.Fatalf("read schedules resource: %v", err)
}
var jobs []cronstore.JobSpec
if err := json.Unmarshal(resource.Value, &jobs); err != nil {
t.Fatalf("decode schedules resource: %v", err)
}
ids := make([]string, 0, len(jobs))
for _, job := range jobs {
ids = append(ids, job.ID)
}
return ids
}
// TestMigrateLegacySchedulesMovesOnlyWorkflowJobs: a legacy file holding a
// chat job (the pre-field default, kind omitted) and a workflow job migrates
// the workflow job and skips -- never refuses -- the chat one. The current
// behaviour replaces the whole list and lets the schedules validator fail the
// State Store on the chat job, taking every process that dials it down with
// it (archie-core-q6vw).
func TestMigrateLegacySchedulesMovesOnlyWorkflowJobs(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "archie.db-tasks.sqlite")
writeLegacyJobs(t, dir, []cronstore.JobSpec{
legacyJob("chat-daily", "", 24*time.Hour),
legacyJob("tdd-hourly", cronstore.KindWorkflow, time.Hour),
})
server, st := migrationServer(t)
version := seedSchedules(t, server)
migrateLegacySchedules(t.Context(), server, dbPath, version, discardLog())
got := storedScheduleIDs(t, st)
if len(got) != 1 || got[0] != "tdd-hourly" {
t.Fatalf("schedules resource holds %v, want only the migratable workflow job", got)
}
if _, err := os.Stat(path_migrated(dir)); err != nil {
t.Fatalf("migrated legacy file not marked (jobs.json.migrated): %v", err)
}
if _, err := os.Stat(filepath.Join(dir, "cron", "jobs.json")); !os.IsNotExist(err) {
t.Fatalf("legacy file still at its original path after a successful migration")
}
}
// path_migrated is where a completed migration leaves the legacy file.
func path_migrated(dir string) string {
return filepath.Join(dir, "cron", "jobs.json.migrated")
}
// TestMigrateLegacySchedulesKeepsStoreRunningWhenNothingIsMigratable: a
// legacy file holding only chat jobs leaves the schedules resource untouched
// and reports -- never fails -- the skip. The store's whole job is to come up.
func TestMigrateLegacySchedulesKeepsStoreRunningWhenNothingIsMigratable(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "archie.db-tasks.sqlite")
path := writeLegacyJobs(t, dir, []cronstore.JobSpec{
legacyJob("chat-daily", "", 24*time.Hour),
legacyJob("chat-weekly", "chat", 7*24*time.Hour),
})
server, st := migrationServer(t)
version := seedSchedules(t, server)
migrateLegacySchedules(t.Context(), server, dbPath, version, discardLog())
if _, err := os.Stat(path); err != nil {
t.Fatalf("legacy file removed though nothing was migrated: %v", err)
}
if got := storedScheduleIDs(t, st); len(got) != 0 {
t.Fatalf("schedules resource holds %v, want the untouched empty seed", got)
}
}
// TestMigrateLegacySchedulesKeepsStoreRunningOnUnreadableLegacyStore: a
// corrupt legacy file is a diagnosis, not a boot blocker. The store must come
// up on the empty seed and the file must stay in place for the operator.
func TestMigrateLegacySchedulesKeepsStoreRunningOnUnreadableLegacyStore(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "archie.db-tasks.sqlite")
path := writeLegacyFile(t, dir, "{not json")
server, _ := migrationServer(t)
version := seedSchedules(t, server)
migrateLegacySchedules(t.Context(), server, dbPath, version, discardLog())
if _, err := os.Stat(path); err != nil {
t.Fatalf("legacy file removed though it was never read: %v", err)
}
}
// TestMigrateLegacySchedulesLeavesTheFileWhenTheReplaceIsRefused: even a
// workflow job the schedules validator refuses is skipped and retried on the
// next start, the way ImportConfig skips a refused seed -- never fatal.
func TestMigrateLegacySchedulesLeavesTheFileWhenTheReplaceIsRefused(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "archie.db-tasks.sqlite")
path := writeLegacyJobs(t, dir, []cronstore.JobSpec{
legacyJob("broken-interval", cronstore.KindWorkflow, 0), // interval schedule with no interval
})
server, st := migrationServer(t)
version := seedSchedules(t, server)
migrateLegacySchedules(t.Context(), server, dbPath, version, discardLog())
if _, err := os.Stat(path); err != nil {
t.Fatalf("legacy file removed though the replace never landed: %v", err)
}
if got := storedScheduleIDs(t, st); len(got) != 0 {
t.Fatalf("schedules resource holds %v, want the untouched empty seed", got)
}
}
// discardLog returns a logger that drops every record: these tests pin the
// migration's effects on the resource and the file, not its reporting.
func discardLog() *slog.Logger {
return slog.New(slog.DiscardHandler)
}
+93 -1
View File
@@ -8,6 +8,7 @@ import (
"strconv"
"strings"
"sync"
"time"
"github.com/samcharles93/archie-core/internal/agentexec"
"github.com/samcharles93/archie-core/internal/app/agentworker"
@@ -15,6 +16,7 @@ import (
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/forge"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/artifactsync"
"github.com/samcharles93/archie-core/internal/worktree"
)
@@ -29,6 +31,9 @@ type prReviewer struct {
models map[string]string
providers map[string]agentexec.Provider
maxSteps int
// artifacts publishes the completed review to the collaborative editor.
// nil = publishing is disabled (no [artifacts] base_url configured).
artifacts artifactsync.Sender
// currentMaxSteps lets the composition root apply live control-plane
// versions to reviews that have not started yet. Tests and direct
// constructions retain maxSteps as the fallback.
@@ -115,7 +120,9 @@ func (r *prReviewer) ReviewPR(ctx context.Context, identity *string, repo string
MaxSteps: maxSteps,
})
r.logReview(key, model, report)
return mapReviewResult(repo, number, model, report), nil
result := mapReviewResult(repo, number, model, report)
return r.attachArtifact(ctx, result, identity, owner, name, number, report), nil
}
// authorize enforces the identity-scoped repository allow-list. A nil
@@ -174,6 +181,62 @@ func (r *prReviewer) logReview(key, model string, report workflow.ReviewReport)
"findings", len(report.Findings), "blocking", blocking)
}
// attachArtifact publishes the report when the sender is configured and
// records the editor deep link on the result. Publishing never fails the
// review: a failure degrades to an empty URL.
func (r *prReviewer) attachArtifact(ctx context.Context, result gateway.PRReviewResult, identity *string, owner, name string, number int, report workflow.ReviewReport) gateway.PRReviewResult {
result.ArtifactURL = r.publishArtifact(ctx, reviewActor(identity), owner, name, number, report)
return result
}
// reviewActor resolves the attribution: the requesting chat identity when one
// exists, else the daemon's operator-reviewer actor.
func reviewActor(identity *string) string {
if identity != nil && *identity != "" {
return *identity
}
return "system:pr-reviewer"
}
// publishArtifact renders the report and hands it to the collaborative
// editor when artifact publishing is configured. It returns the editor deep
// link, or "" when publishing is disabled or failed: a publish failure is a
// logged degradation, never an error that fails the producing review.
func (r *prReviewer) publishArtifact(ctx context.Context, actor, owner, name string, number int, report workflow.ReviewReport) string {
if r.artifacts == nil {
return ""
}
title := fmt.Sprintf("Review of %s/%s#%d", owner, name, number)
artifact := artifactsync.Artifact{
Title: title,
Source: report.RenderMarkdown(title),
Meta: artifactsync.Meta{
Format: "markdown",
Category: "pr-review",
Visibility: "account-only",
Egress: "none",
Actor: actor,
Origin: "pr-review",
RequestID: "artifact:pr-review:" + owner + "/" + name + "#" + strconv.Itoa(number),
Version: "r1",
SentAt: time.Now().UTC(),
Task: &artifactsync.Task{Owner: owner, Repo: name, Number: number},
},
}
url, err := r.artifacts.Publish(ctx, artifact)
if err != nil {
if r.log != nil {
r.log.Warn("artifact publish failed; the review result is unaffected",
"artifact", artifact.RequestID(), "err", err)
}
return ""
}
if r.log != nil {
r.log.Info("artifact published", "artifact", artifact.RequestID(), "url", url)
}
return url
}
// splitOwnerRepo splits "owner/name" with the same validation the gateway's
// task_spawn allow-list uses.
func splitOwnerRepo(s string) (owner, repo string, err error) {
@@ -264,6 +327,7 @@ func (b *boot) prReviewer() gateway.ChatPRReviewer {
models: b.cfg.Models,
providers: executionProviders(b.cfg),
maxSteps: maxSteps,
artifacts: b.artifactSender(),
allowed: buildReviewAllowlist(b.cfg),
log: b.log,
}
@@ -273,3 +337,31 @@ func (b *boot) prReviewer() gateway.ChatPRReviewer {
b.chatPRReviewer = review
return b.chatPRReviewer
}
// artifactSender builds the collaborative-editor sender when the [artifacts]
// section configures one. An empty base_url disables publishing entirely; a
// base_url with an unresolvable token is a missing credential, which disables
// the capability with a loud warning rather than stopping the daemon
// (docs/architecture/configuration.md, "A missing credential disables a
// capability").
func (b *boot) artifactSender() artifactsync.Sender {
cfg := b.cfg.Artifacts
if cfg.BaseURL == "" {
return nil
}
token, err := b.secrets.Resolve(cfg.Token)
if err != nil {
b.log.Warn("artifact publishing disabled: the service credential could not be resolved",
"base_url", cfg.BaseURL, "token_engine", cfg.Token.Engine, "err", err)
return nil
}
if token == "" {
token = b.secrets.Getenv(cfg.TokenEnv)
}
if token == "" {
b.log.Warn("artifact publishing disabled: no service token; set the token secret or its environment variable",
"base_url", cfg.BaseURL, "token_env", cfg.TokenEnv)
return nil
}
return artifactsync.NewClient(cfg.BaseURL, token)
}
+74
View File
@@ -2,13 +2,16 @@ package archied
import (
"context"
"errors"
"log/slog"
"strings"
"testing"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/forge"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/artifactsync"
"github.com/samcharles93/archie-core/internal/tools"
"github.com/samcharles93/archie-core/internal/worktree"
)
@@ -241,3 +244,74 @@ func hasReviewPRTool(entries []tools.ToolEntry) bool {
}
return false
}
type fakeArtifactSender struct {
calls int
url string
err error
last artifactsync.Artifact
}
func (f *fakeArtifactSender) Publish(_ context.Context, a artifactsync.Artifact) (string, error) {
f.calls++
f.last = a
if f.err != nil {
return "", f.err
}
return f.url, nil
}
func TestPrReviewerPublishesTheCompletedReview(t *testing.T) {
fake := &fakeArtifactSender{url: "https://offloaded.dev/collab?document=a_1"}
r := &prReviewer{artifacts: fake, log: slog.New(slog.DiscardHandler)}
report := workflow.NewCompletedReviewReport([]workflow.ReviewFinding{{
File: "handler.go", Line: 42, Defect: "nil deref",
FailureScenario: "empty payload crashes", Verdict: workflow.ReviewVerdictConfirmed,
Level: workflow.ReviewLevelError,
}}, "one finding")
report.Checked = []workflow.ReviewCheck{{Property: "p", Evidence: "e"}}
url := r.publishArtifact(context.Background(), "system:pr-reviewer", "acme", "widget", 7, report)
if url != fake.url {
t.Fatalf("publishArtifact = %q, want the sender's url %q", url, fake.url)
}
if fake.calls != 1 {
t.Fatalf("Publish called %d times, want 1", fake.calls)
}
got := fake.last
if got.Meta.Format != "markdown" || got.Meta.Category != "pr-review" {
t.Fatalf("format/category = %s/%s, want markdown/pr-review", got.Meta.Format, got.Meta.Category)
}
if got.Meta.Actor != "system:pr-reviewer" || got.Meta.Origin != "pr-review" {
t.Fatalf("attribution = %s/%s", got.Meta.Actor, got.Meta.Origin)
}
if got.Meta.Visibility != "account-only" || got.Meta.Egress != "none" {
t.Fatalf("visibility/egress = %s/%s", got.Meta.Visibility, got.Meta.Egress)
}
if got.Title != "Review of acme/widget#7" {
t.Fatalf("title = %q", got.Title)
}
if got.Meta.RequestID != "artifact:pr-review:acme/widget#7" {
t.Fatalf("requestId = %q, want the deterministic artifact key", got.Meta.RequestID)
}
if got.Meta.Task == nil || got.Meta.Task.Owner != "acme" || got.Meta.Task.Number != 7 {
t.Fatalf("task coordinates = %+v", got.Meta.Task)
}
if !strings.Contains(got.Source, "## Findings") {
t.Fatalf("source is not the rendered report:\n%s", got.Source)
}
}
func TestPrReviewerPublishDegradesOnSenderFailure(t *testing.T) {
r := &prReviewer{artifacts: &fakeArtifactSender{err: errors.New("503 exhausted")}, log: slog.New(slog.DiscardHandler)}
if url := r.publishArtifact(context.Background(), "system:pr-reviewer", "acme", "widget", 7, workflow.NewCompletedReviewReport(nil, "s")); url != "" {
t.Fatalf("url = %q, want empty on a failed publish", url)
}
}
func TestPrReviewerPublishIsANoOpWithoutASender(t *testing.T) {
r := &prReviewer{}
if url := r.publishArtifact(context.Background(), "system:pr-reviewer", "acme", "widget", 7, workflow.NewCompletedReviewReport(nil, "s")); url != "" {
t.Fatalf("url = %q, want empty when the sender is disabled", url)
}
}
+16 -7
View File
@@ -14,7 +14,6 @@ import (
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
"github.com/samcharles93/archie-core/internal/infrastructure/crondelivery"
"github.com/samcharles93/archie-core/internal/infrastructure/cronevents"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
)
// schedulingConfig translates external input into the domain-owned engine
@@ -75,7 +74,7 @@ func (b *boot) setupScheduling() error {
if err != nil {
return fmt.Errorf("scheduling: build workflow runner: %w", err)
}
router, err := crondelivery.NewRouter(store, map[string]scheduling.Runner{cronstore.KindWorkflow: workflowRunner}, cfg.Events)
router, err := crondelivery.NewRouter(store, map[string]scheduling.Runner{scheduling.KindWorkflow: workflowRunner}, cfg.Events)
if err != nil {
return fmt.Errorf("scheduling: build router: %w", err)
}
@@ -93,17 +92,27 @@ type scheduleResourceStore struct {
client *controlplane.Client
}
func (s scheduleResourceStore) Get(ctx context.Context, id string) (cronstore.JobSpec, bool, error) {
// The schedules document behind the control plane is the production
// RouterStore: it resolves a job id to its spec and records a completed run.
// The assertions this replaces used to pin the retired file-backed store;
// crondelivery's contract comments name this type as the implementation.
var (
_ crondelivery.SpecLookup = scheduleResourceStore{}
_ crondelivery.RunRecorder = scheduleResourceStore{}
_ crondelivery.RouterStore = scheduleResourceStore{}
)
func (s scheduleResourceStore) Get(ctx context.Context, id string) (scheduling.JobSpec, bool, error) {
jobs, _, err := s.client.Schedules(ctx)
if err != nil {
return cronstore.JobSpec{}, false, err
return scheduling.JobSpec{}, false, err
}
for _, job := range jobs {
if job.ID == id {
return job, true, nil
}
}
return cronstore.JobSpec{}, false, nil
return scheduling.JobSpec{}, false, nil
}
func (s scheduleResourceStore) Due(ctx context.Context, now time.Time) ([]scheduling.Job, error) {
@@ -132,7 +141,7 @@ func (s scheduleResourceStore) MarkRun(ctx context.Context, id string, runAt tim
continue
}
found = true
if jobs[index].Schedule.Resolved().Kind == cronstore.ScheduleOnce {
if jobs[index].Schedule.Resolved().Kind == scheduling.ScheduleOnce {
jobs = append(jobs[:index], jobs[index+1:]...)
break
}
@@ -146,7 +155,7 @@ func (s scheduleResourceStore) MarkRun(ctx context.Context, id string, runAt tim
break
}
if !found {
return fmt.Errorf("%w: id %q", cronstore.ErrJobNotFound, id)
return fmt.Errorf("%w: id %q", scheduling.ErrJobNotFound, id)
}
_, err = s.client.ReplaceSchedules(ctx, jobs, version, "system:scheduler", "scheduler", fmt.Sprintf("schedule-run:%s:%d", id, runAt.UnixNano()))
if err == nil {
+97 -99
View File
@@ -13,10 +13,8 @@ import (
"context"
"encoding/json"
"fmt"
"log/slog"
"net"
"net/http"
"os"
"path/filepath"
"time"
@@ -24,12 +22,12 @@ import (
"github.com/samcharles93/archie-core/internal/app/controlplane"
"github.com/samcharles93/archie-core/internal/config"
controlpb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/health"
"github.com/samcharles93/archie-core/internal/domain/identity"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/infrastructure/readiness"
"github.com/samcharles93/archie-core/internal/infrastructure/stateadmin"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/store"
)
@@ -49,6 +47,11 @@ func configuredIdentityNames(cfg config.Config) []string {
return names
}
// edaDBPath derives the event-capture database from the configured db_path,
// beside the task database rather than inside it: the two stores have separate
// lifecycles and only meet at binding_dispatches.task_id.
func edaDBPath(configuredPath string) string { return configuredPath + "-eda.sqlite" }
// StateStoreOptions contains process inputs for the standalone State Store.
type StateStoreOptions struct {
Config string
@@ -65,6 +68,10 @@ type StateStoreOptions struct {
// address: GET /healthz (liveness) and GET /health/detailed (the
// state_db probe). Empty disables it.
ReadyAddr string
// AdminAddr, when non-empty, starts the read-only operator dashboard on
// the address. It co-tenants on the same SQLite file this process already
// owns, so it needs no lock and no database of its own. Empty disables it.
AdminAddr string
}
// RunStateStore owns the single archie.db SQLite file, registers every
@@ -107,12 +114,11 @@ func RunStateStore(ctx context.Context, options StateStoreOptions) error {
if err != nil {
return err
}
versions, skipped, err := control.ImportConfig(ctx, b.cfg)
_, skipped, err := control.ImportConfig(ctx, b.cfg)
if err != nil {
return fmt.Errorf("import control-plane resources: %w", err)
}
b.reportUnseededResources(skipped)
migrateLegacySchedules(ctx, control, b.cfg.DBPath, versions[controlplane.SchedulesKind], b.log)
token := options.Token
if token == "" {
@@ -131,10 +137,8 @@ func RunStateStore(ctx context.Context, options StateStoreOptions) error {
defer listener.Close()
b.log.Info("archie-state-store running", "addr", listener.Addr().String(), "token_required", !loopback)
if options.ReadyAddr != "" {
if err := b.startStateStoreReadiness(ctx, options.ReadyAddr); err != nil {
return err
}
if err := b.startOptionalSurfaces(ctx, options); err != nil {
return err
}
deps := b.stateStoreDeps(grants)
@@ -188,81 +192,6 @@ func openStateStoreControlPlane(resources controlplane.ResourceStore) (*controlp
return server, nil
}
// migrateLegacySchedules carries a legacy cronstore file -- the old daemon's
// <state dir>/cron/jobs.json -- into the schedules resource, once, on a fresh
// seed. It moves only the jobs the schedules resource can hold: workflow
// jobs, the one delivery the router maps. Chat and unrecognised kinds were
// already dead deliveries in the old engine (the delivery router maps
// KindWorkflow alone), so they are reported and left in the legacy file
// rather than being allowed to refuse the replace the whole store depends on.
//
// Nothing here is fatal. The migration is a legacy-data courtesy: an
// unreadable or refused sidecar is logged and left in place for the next
// start, the way ImportConfig skips a seed the resource validator refuses --
// a value this process cannot use must not stop the one process every other
// process dials (docs/prds/runtime-control-plane.md, "Bootstrap, migration,
// and recovery"). A successful replace is marked by renaming the file to
// jobs.json.migrated, so the whole legacy list -- the skipped jobs included
// -- stays on disk for the operator; until the rename lands, the replaying
// request ID keeps the replace idempotent across restarts.
func migrateLegacySchedules(ctx context.Context, control *controlplane.Server, dbPath string, version int64, log *slog.Logger) {
if version != 1 {
return
}
path := filepath.Join(filepath.Dir(dbPath), "cron", "jobs.json")
if _, err := os.Stat(path); err != nil {
if os.IsNotExist(err) {
return
}
log.Warn("legacy cron store not readable; schedules migration skipped and retried on the next start", "path", path, "err", err)
return
}
jobs, err := readLegacyJobs(ctx, path)
if err != nil {
log.Error("legacy cron store unreadable; schedules migration skipped and retried on the next start", "path", path, "err", err)
return
}
migratable := make([]cronstore.JobSpec, 0, len(jobs))
for _, job := range jobs {
if job.Kind == cronstore.KindWorkflow {
migratable = append(migratable, job)
continue
}
log.Warn("legacy cron job is not a workflow job; the schedules resource does not carry it and it stays in the legacy file", "id", job.ID, "kind", job.Kind)
}
if len(migratable) == 0 {
log.Info("legacy cron store holds no workflow jobs; nothing to migrate", "path", path, "jobs", len(jobs))
return
}
value, err := json.Marshal(migratable)
if err != nil {
log.Error("encode legacy cron jobs; schedules migration skipped and retried on the next start", "path", path, "err", err)
return
}
if _, err := control.Command(ctx, &controlpb.CommandRequest{Kind: controlplane.SchedulesKind, Command: "replace", ValueJson: value, ExpectedVersion: 1, Actor: "system:migration", Source: "legacy-cronstore", RequestId: "import:legacy-schedules"}); err != nil {
log.Error("legacy cron jobs do not satisfy the schedules resource; migration skipped and retried on the next start", "path", path, "err", err)
return
}
if err := os.Rename(path, path+".migrated"); err != nil {
log.Error("mark the migrated legacy cron file; migration retried on the next start", "path", path, "err", err)
}
}
// readLegacyJobs reads the legacy cronstore file and closes its handle,
// reporting whichever failure came first.
func readLegacyJobs(ctx context.Context, path string) ([]cronstore.JobSpec, error) {
legacy, err := cronstore.Open(path)
if err != nil {
return nil, err
}
jobs, listErr := legacy.List(ctx)
closeErr := legacy.Close()
if listErr != nil {
return nil, listErr
}
return jobs, closeErr
}
// openStateStore opens the single task-store SQLite file exactly once for
// service ownership. It resolves secrets (for the binding cipher) but does
// NOT open gateway chat sessions -- those live in the separate archie-gateway
@@ -298,7 +227,23 @@ func (b *boot) openStateStore(ctx context.Context) error {
log.Error("release state store ownership", "err", err)
}
})
st, err := openProductionTaskStore(ctx, path, store.WithBindingCipher(bindingCipher))
eda, err := edastore.Open(edastore.Config{
DBPath: edaDBPath(cfg.DBPath),
DataDir: filepath.Join(filepath.Dir(path), "eda"),
Cipher: bindingCipher,
})
if err != nil {
log.Error("open event-capture store", "err", err)
return err
}
b.eda = eda
b.addCleanup(func() {
if err := eda.Close(); err != nil {
log.Error("close event-capture store", "err", err)
}
})
st, err := openProductionTaskStore(ctx, path)
if err != nil {
log.Error("open state store", "err", err)
return err
@@ -345,20 +290,24 @@ func (b *boot) stateStoreDeps(grants *staterpc.TaskGrants) staterpc.Deps {
if b.taskLogs != nil {
deps.TaskLogs = b.taskLogs
}
if cs, ok := b.st.(storecontract.CaptureStore); ok {
deps.Captures = cs
// The event-capture contracts are served by the PocketBase store, not the
// task store: those tables moved. BindingTaskCreator stays below on the
// task store, because creating a task is the one thing it still does.
if b.eda != nil {
deps.Captures = b.eda
deps.Mappings = b.eda
deps.Bindings = b.eda
deps.BindingDispatcher = b.eda
deps.PlaybookDispatcher = b.eda
}
if ms, ok := b.st.(storecontract.MappingStore); ok {
deps.Mappings = ms
}
if bs, ok := b.st.(storecontract.BindingStore); ok {
deps.Bindings = bs
}
if bd, ok := b.st.(storecontract.BindingDispatcher); ok {
deps.BindingDispatcher = bd
}
if pd, ok := b.st.(storecontract.PlaybookDispatcher); ok {
deps.PlaybookDispatcher = pd
// tool_call events project into the tool_calls collection on the same
// event-capture store: this process legitimately owns both, so the
// projection rides on the task-store surface it decorates. With no
// event-capture store (or no task store) there is nothing to project
// through, and Tasks passes through unwrapped rather than decorating a
// nil operand.
if b.eda != nil && b.st != nil {
deps.Tasks = newToolCallProjectingTaskStore(b.st, b.eda, b.log)
}
if btc, ok := b.st.(storecontract.BindingTaskCreator); ok {
deps.BindingTaskCreator = btc
@@ -421,6 +370,55 @@ func (b *boot) startStateStoreReadiness(ctx context.Context, readyAddr string) e
return b.serveHealth(ctx, readyAddr, mux, "state store readiness")
}
// startOptionalSurfaces starts the HTTP surfaces this process serves beside
// the gRPC contract, each enabled only by its own flag.
//
// They are gathered here rather than branched inline because RunStateStore is
// at its complexity budget: a surface that is off by default belongs to its
// own decision, not to the boot sequence that triggers it -- the same reason
// reportUnseededResources is a method.
func (b *boot) startOptionalSurfaces(ctx context.Context, options StateStoreOptions) error {
if options.ReadyAddr != "" {
if err := b.startStateStoreReadiness(ctx, options.ReadyAddr); err != nil {
return err
}
}
if options.AdminAddr != "" {
if err := b.startStateAdmin(ctx, options.AdminAddr); err != nil {
return err
}
}
return nil
}
// startStateAdmin starts the read-only operator dashboard on adminAddr.
//
// It is started here, inside the process that already holds the database's
// ownership claim (openStateStore above), because that claim is what makes a
// second reader safe to co-tenant: a separate admin process would be a second
// opener of a single-owner file. The surface exposes view collections only and
// refuses the routes that could write, so the StateStoreService contract this
// process serves stays the only writer of task state.
func (b *boot) startStateAdmin(ctx context.Context, adminAddr string) error {
admin, err := stateadmin.New(stateadmin.Config{
DBPath: taskDBPath(b.cfg.DBPath),
DataDir: filepath.Join(filepath.Dir(taskDBPath(b.cfg.DBPath)), "admin"),
})
if err != nil {
return err
}
b.addCleanup(func() {
if err := admin.Close(); err != nil {
b.log.Error("close state admin", "err", err)
}
})
handler, err := admin.Handler()
if err != nil {
return fmt.Errorf("build state admin handler: %w", err)
}
return b.serveHealth(ctx, adminAddr, handler, "state store admin")
}
// stateStoreServerOpts applies the transport security boundary (§9): a
// loopback listener is served insecure with no token; any non-loopback
// address requires a bearer token, else the process fails closed. It cannot
@@ -6,6 +6,7 @@ import (
"testing"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
taskactionstore "github.com/samcharles93/archie-core/internal/infrastructure/taskactions"
"github.com/samcharles93/archie-core/internal/store"
)
@@ -15,14 +16,33 @@ import (
// b.stateStore rather than b.st: the two differ by identity, so an assertion
// that a consumer field holds storeB (not storeA) can only pass if the wiring
// reads b.stateStore.
func openSecondStore(t *testing.T) *store.Store {
// stateStoreAdapter stands in for the real adapter (*staterpc.Client), which
// fronts both halves of the contract over one connection: the task store and
// the event-capture store. A plain *store.Store no longer satisfies the
// event-capture surfaces, so a test that asserts "everything routes through
// the adapter" needs something that implements them, exactly as the client
// does in production.
// edaSide aliases the event-capture store so it can be embedded beside
// *store.Store: both are named Store, and Go allows only one embedded field
// per name.
type edaSide = edastore.Store
type stateStoreAdapter struct {
*store.Store
*edaSide
}
// Close disambiguates the two embedded stores, which both have one.
func (a *stateStoreAdapter) Close() error { return a.Store.Close() }
func openSecondStore(t *testing.T) *stateStoreAdapter {
t.Helper()
st, err := store.Open(t.Context(), filepath.Join(t.TempDir(), "state-store.db"))
if err != nil {
t.Fatalf("open second store: %v", err)
}
t.Cleanup(func() { _ = st.Close() })
return st
return &stateStoreAdapter{Store: st, edaSide: edastore.OpenTest(t)}
}
// TestBuildDaemonRoutesTaskStoreThroughStateStore proves the daemon's task
+7 -2
View File
@@ -12,6 +12,7 @@ import (
"github.com/samcharles93/archie-core/internal/config"
pb "github.com/samcharles93/archie-core/internal/contracts/state/v1"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/logging"
"github.com/samcharles93/archie-core/internal/store"
@@ -145,12 +146,16 @@ func TestStateStoreDepsServePlaybookDispatcher(t *testing.T) {
b := newBootstrap()
b.st = st
// The playbook ledger moved to the event-capture store with the rest of
// the dispatch tables; the task store no longer serves it.
eda := edastore.OpenTest(t)
b.eda = eda
deps := b.stateStoreDeps(&staterpc.TaskGrants{})
if deps.PlaybookDispatcher == nil {
t.Fatal("stateStoreDeps leaves PlaybookDispatcher nil; the standalone State Store is the only production server for the playbook dispatch ledger")
}
if deps.PlaybookDispatcher != storecontract.PlaybookDispatcher(st) {
t.Fatalf("PlaybookDispatcher = %T, want the opened *store.Store", deps.PlaybookDispatcher)
if deps.PlaybookDispatcher != storecontract.PlaybookDispatcher(eda) {
t.Fatalf("PlaybookDispatcher = %T, want the opened *edastore.Store", deps.PlaybookDispatcher)
}
// A boot without a store must not fabricate one: nil keeps the RPCs honest
// as unavailable rather than depending on a nil receiver.
@@ -0,0 +1,87 @@
package archied
import (
"context"
"log/slog"
"sync/atomic"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
)
// toolCallWriter is the edastore surface the projection writes through. Kept
// narrow so a test can fail it without touching the real store.
type toolCallWriter interface {
InsertToolCall(ctx context.Context, tc edastore.ToolCall) error
}
// toolCallProjectingTaskStore decorates the State Store's task store so every
// persisted tool_call event also lands in the tool_calls collection on the
// event-capture store, which this same process owns. The task event log stays
// the source of truth; the collection is the queryable transcript of it.
//
// A projection failure is NEVER returned: event persistence is load-bearing,
// the collection is observability, and failing a task write because its
// transcript copy failed would trade the fact for its shadow. The failure is
// instead counted and logged with the running count, so it is visible without
// ever changing InsertEvent's outcome.
type toolCallProjectingTaskStore struct {
storecontract.TaskStore
toolCalls toolCallWriter
log *slog.Logger
projectionFailures atomic.Int64
}
func newToolCallProjectingTaskStore(inner storecontract.TaskStore, toolCalls toolCallWriter, log *slog.Logger) *toolCallProjectingTaskStore {
if log == nil {
log = slog.Default()
}
return &toolCallProjectingTaskStore{TaskStore: inner, toolCalls: toolCalls, log: log}
}
// toolCallProjectionFailures reports how many tool_call projections have been
// dropped since this process started. Named and exported to the test surface
// rather than buried in a log field alone: a silent swallow is the bug this
// decorator exists not to have.
func (s *toolCallProjectingTaskStore) toolCallProjectionFailures() int64 {
return s.projectionFailures.Load()
}
func (s *toolCallProjectingTaskStore) InsertEvent(ctx context.Context, e events.Event) (int64, error) {
id, err := s.TaskStore.InsertEvent(ctx, e)
if err != nil || e.Kind != events.KindToolCall {
return id, err
}
if perr := s.toolCalls.InsertToolCall(ctx, toolCallFromEvent(e)); perr != nil {
failures := s.projectionFailures.Add(1)
s.log.Error("tool_call projection failed; the event persisted but the tool_calls row did not",
"err", perr, "task_id", e.TaskID, "attempt", e.Attempt,
"tool", e.Data["tool"], "projection_failures", failures)
}
return id, nil
}
// toolCallFromEvent adapts a tool_call event to the existing tool_calls
// collection. The event carries the tool name and the failure flag in data and
// the one-line outcome in detail: a succeeded call's summary is its result, a
// failed call's summary is its error. The collection's args and duration_ms
// fields stay unwritten -- the event reports neither -- so they are schema
// reserved for a producer that one day reports them, not promises this
// projection makes.
func toolCallFromEvent(e events.Event) edastore.ToolCall {
tool, _ := e.Data["tool"].(string)
failed, _ := e.Data["failed"].(bool)
tc := edastore.ToolCall{
TaskID: e.TaskID,
Attempt: e.Attempt,
Tool: tool,
CalledAt: e.At,
}
if failed {
tc.Error = e.Detail
} else {
tc.Result = e.Detail
}
return tc
}
@@ -0,0 +1,192 @@
package archied
import (
"bytes"
"context"
"errors"
"log/slog"
"strings"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/store"
)
func toolCallEvent(taskID int64, attempt int) events.Event {
return events.Event{
Kind: events.KindToolCall,
TaskID: taskID,
Attempt: attempt,
Stage: "implement",
Detail: "all checks passed",
Data: map[string]any{"tool": "shell", "failed": false},
At: time.Now().UTC(),
Workflow: "tdd",
}
}
// TestInsertEventProjectsToolCall pins the projection (test a): one tool_call
// event through the decorated store persists the event AND exactly one
// tool_calls row, with the row's fields taken from the event -- the tool name
// from data, the output summary in result for a succeeded call.
func TestInsertEventProjectsToolCall(t *testing.T) {
st := store.OpenTest(t)
eda := edastore.OpenTest(t)
ts := newToolCallProjectingTaskStore(st, eda, slog.Default())
id, err := ts.InsertEvent(t.Context(), toolCallEvent(42, 2))
if err != nil {
t.Fatalf("InsertEvent() error = %v", err)
}
if id == 0 {
t.Fatal("InsertEvent() returned id 0; the event itself must still persist")
}
rows, err := eda.TaskToolCalls(t.Context(), 42)
if err != nil {
t.Fatalf("TaskToolCalls() error = %v", err)
}
if len(rows) != 1 {
t.Fatalf("TaskToolCalls() returned %d rows, want exactly one per tool_call event: %+v", len(rows), rows)
}
row := rows[0]
if row.TaskID != 42 || row.Attempt != 2 {
t.Errorf("row task/attempt = %d/%d, want 42/2", row.TaskID, row.Attempt)
}
if row.Tool != "shell" {
t.Errorf("row tool = %q, want \"shell\" from the event's data", row.Tool)
}
if row.Result != "all checks passed" {
t.Errorf("row result = %q, want the event detail of a succeeded call", row.Result)
}
if row.Error != "" {
t.Errorf("row error = %q, want empty for a succeeded call", row.Error)
}
if row.CalledAt.IsZero() {
t.Error("row called_at is zero; it must carry the event time")
}
}
// TestInsertEventProjectsFailedToolCall pins the failure column: a tool_call
// event whose call failed stores its detail under error, not result, so the
// transcript can distinguish the two without re-parsing text.
func TestInsertEventProjectsFailedToolCall(t *testing.T) {
st := store.OpenTest(t)
eda := edastore.OpenTest(t)
ts := newToolCallProjectingTaskStore(st, eda, slog.Default())
e := toolCallEvent(42, 2)
e.Detail = "error: no such file"
e.Data = map[string]any{"tool": "read_file", "failed": true}
if _, err := ts.InsertEvent(t.Context(), e); err != nil {
t.Fatalf("InsertEvent() error = %v", err)
}
rows, err := eda.TaskToolCalls(t.Context(), 42)
if err != nil {
t.Fatalf("TaskToolCalls() error = %v", err)
}
if len(rows) != 1 {
t.Fatalf("TaskToolCalls() returned %d rows, want 1", len(rows))
}
if rows[0].Error != "error: no such file" {
t.Errorf("row error = %q, want the failed call's detail", rows[0].Error)
}
if rows[0].Result != "" {
t.Errorf("row result = %q, want empty for a failed call", rows[0].Result)
}
}
// TestInsertEventLeavesOtherKindsUnprojected pins the kind gate: only
// tool_call events reach the collection; the task timeline's other kinds must
// not grow tool_calls rows.
func TestInsertEventLeavesOtherKindsUnprojected(t *testing.T) {
st := store.OpenTest(t)
eda := edastore.OpenTest(t)
ts := newToolCallProjectingTaskStore(st, eda, slog.Default())
if _, err := ts.InsertEvent(t.Context(), events.Event{
Kind: events.KindStageFinish, TaskID: 42, Attempt: 2,
Data: map[string]any{"duration_ms": 12.0},
}); err != nil {
t.Fatalf("InsertEvent() error = %v", err)
}
rows, err := eda.TaskToolCalls(t.Context(), 42)
if err != nil {
t.Fatalf("TaskToolCalls() error = %v", err)
}
if len(rows) != 0 {
t.Fatalf("TaskToolCalls() returned %d rows for a stage_finish event, want 0", len(rows))
}
}
// failingToolCallWriter stands in for a broken PocketBase side, so the
// never-returned-failure contract can be tested without sabotaging the real
// store.
type failingToolCallWriter struct{ err error }
func (f failingToolCallWriter) InsertToolCall(context.Context, edastore.ToolCall) error {
return f.err
}
// TestInsertEventSucceedsWhenProjectionFails pins the load-bearing half of the
// contract: a tool_calls write failure is never returned from InsertEvent --
// event persistence is the contract, the projection is observability -- and
// the failure is recorded, not swallowed silently.
func TestInsertEventSucceedsWhenProjectionFails(t *testing.T) {
st := store.OpenTest(t)
var buf bytes.Buffer
ts := newToolCallProjectingTaskStore(st, failingToolCallWriter{
err: errors.New("disk on fire"),
}, slog.New(slog.NewTextHandler(&buf, nil)))
id, err := ts.InsertEvent(t.Context(), toolCallEvent(42, 2))
if err != nil {
t.Fatalf("InsertEvent() error = %v; a projection failure must never fail the event write", err)
}
if id == 0 {
t.Fatal("InsertEvent() returned id 0; the event must still persist")
}
if got := ts.toolCallProjectionFailures(); got != 1 {
t.Errorf("toolCallProjectionFailures() = %d, want 1: the failure must be counted, not swallowed", got)
}
if !strings.Contains(buf.String(), "disk on fire") {
t.Errorf("log output %q does not carry the projection failure; it was swallowed silently", buf.String())
}
// The event row is what the whole surface exists for -- it must be there.
persisted, err := st.TaskEvents(t.Context(), 42)
if err != nil {
t.Fatalf("TaskEvents() error = %v", err)
}
if len(persisted) != 1 {
t.Fatalf("TaskEvents() returned %d events, want 1: the event persisted despite the projection failure", len(persisted))
}
}
// TestStateStoreDepsProjectToolCalls pins the wire-in: with both stores open,
// the composition serves Tasks as the projecting decorator; with no
// event-capture store, Tasks is passed through untouched rather than wrapped
// around a nil writer.
func TestStateStoreDepsProjectToolCalls(t *testing.T) {
st := store.OpenTest(t)
eda := edastore.OpenTest(t)
b := newBootstrap()
b.st = st
b.eda = eda
deps := b.stateStoreDeps(&staterpc.TaskGrants{})
if _, ok := deps.Tasks.(*toolCallProjectingTaskStore); !ok {
t.Fatalf("deps.Tasks = %T, want the tool_call projecting decorator", deps.Tasks)
}
plain := newBootstrap()
plain.st = st
plainDeps := plain.stateStoreDeps(&staterpc.TaskGrants{})
if _, ok := plainDeps.Tasks.(*toolCallProjectingTaskStore); ok {
t.Fatal("deps.Tasks is decorated with no event-capture store on the boot; the projection would write through a nil writer")
}
}
@@ -13,6 +13,7 @@ import (
"google.golang.org/grpc"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/logging"
"github.com/samcharles93/archie-core/internal/store"
@@ -165,8 +166,9 @@ func composeUIProcess(t *testing.T) (*webui.Server, int64) {
logs := logging.NewTaskRegistry(filepath.Join(t.TempDir(), "logs", "tasks"), logging.NewFeed(10), logging.TaskSinkOptions{})
target, stop := serveGRPC(t, func(r grpc.ServiceRegistrar) {
eda := edastore.OpenTest(t)
staterpc.RegisterServer(r, staterpc.Deps{
Tasks: st, Captures: st, BindingDispatcher: st, ConfigSnapshots: st,
Tasks: st, Captures: eda, BindingDispatcher: eda, ConfigSnapshots: st,
TaskLogs: logs, Log: slog.New(slog.DiscardHandler),
})
})
+3 -1
View File
@@ -21,6 +21,7 @@ import (
"github.com/samcharles93/archie-core/internal/domain/taskactions"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/gateway"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/infrastructure/gatewayrpc"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/store"
@@ -151,7 +152,8 @@ func TestUIServesDashboardAgainstRemoteContracts(t *testing.T) {
}
stateTarget, stopState := serveGRPC(t, func(r grpc.ServiceRegistrar) {
staterpc.RegisterServer(r, staterpc.Deps{Tasks: st, Captures: st, BindingDispatcher: st, ConfigSnapshots: st, Log: slog.New(slog.DiscardHandler)})
eda := edastore.OpenTest(t)
staterpc.RegisterServer(r, staterpc.Deps{Tasks: st, Captures: eda, BindingDispatcher: eda, ConfigSnapshots: st, Log: slog.New(slog.DiscardHandler)})
})
defer stopState()
chat := &fakeChat{sessions: []gateway.SessionContext{
+3 -1
View File
@@ -13,6 +13,7 @@ import (
"google.golang.org/grpc"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/logging"
"github.com/samcharles93/archie-core/internal/store"
@@ -178,8 +179,9 @@ func composeUIProcessWithLog(t *testing.T) (*webui.Server, int64, int) {
}
target, stop := serveGRPC(t, func(r grpc.ServiceRegistrar) {
eda := edastore.OpenTest(t)
staterpc.RegisterServer(r, staterpc.Deps{
Tasks: st, Captures: st, BindingDispatcher: st, ConfigSnapshots: st,
Tasks: st, Captures: eda, BindingDispatcher: eda, ConfigSnapshots: st,
TaskLogs: logs, Log: slog.New(slog.DiscardHandler),
})
})
+5 -5
View File
@@ -11,9 +11,9 @@ import (
pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1"
"github.com/samcharles93/archie-core/internal/domain/agent"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/infrastructure/controlplanerpc"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
)
// Client is the control-plane client for every resource that does not resolve
@@ -25,7 +25,7 @@ import (
// Client is the control-plane client a process that serves the store uses. It
// embeds the Messaging Service's client -- the generic resource read path and
// the channel-settings projection -- and adds the resource-specific reads and
// watches, which need the workflow and cronstore engines that such a process
// watches, which need the workflow and scheduling engines that such a process
// already links (archie-core-1ng1).
type Client struct {
*controlplanerpc.Client
@@ -213,19 +213,19 @@ func sendUpdate[T any](ctx context.Context, out chan<- T, update T) bool {
}
}
func (c *Client) Schedules(ctx context.Context) ([]cronstore.JobSpec, int64, error) {
func (c *Client) Schedules(ctx context.Context) ([]scheduling.JobSpec, int64, error) {
response, err := c.rpc.Query(ctx, &pb.QueryRequest{Kind: SchedulesKind})
if err != nil {
return nil, 0, controlplanerpc.ClientError(err)
}
var jobs []cronstore.JobSpec
var jobs []scheduling.JobSpec
if err := json.Unmarshal(response.Resource.ValueJson, &jobs); err != nil {
return nil, 0, err
}
return jobs, response.Resource.Version, nil
}
func (c *Client) ReplaceSchedules(ctx context.Context, jobs []cronstore.JobSpec, expectedVersion int64, actor, source, requestID string) (int64, error) {
func (c *Client) ReplaceSchedules(ctx context.Context, jobs []scheduling.JobSpec, expectedVersion int64, actor, source, requestID string) (int64, error) {
value, err := json.Marshal(jobs)
if err != nil {
return 0, err
+5 -6
View File
@@ -8,7 +8,6 @@ import (
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
)
const SchedulesKind = "schedules"
@@ -17,9 +16,9 @@ func scheduleDefinition() Definition {
return Definition{
Kind: SchedulesKind,
Title: "Schedules",
Document: []cronstore.JobSpec{},
Document: []scheduling.JobSpec{},
ApplyMode: "live",
Seed: func(config.Config) any { return []cronstore.JobSpec{} },
Seed: func(config.Config) any { return []scheduling.JobSpec{} },
Validate: func(input []byte) error {
return validateAs(input, validateSchedules)
},
@@ -27,7 +26,7 @@ func scheduleDefinition() Definition {
}
}
func validateSchedules(jobs []cronstore.JobSpec) error {
func validateSchedules(jobs []scheduling.JobSpec) error {
seen := make(map[string]struct{}, len(jobs))
for _, job := range jobs {
if strings.TrimSpace(job.ID) == "" {
@@ -37,7 +36,7 @@ func validateSchedules(jobs []cronstore.JobSpec) error {
return fmt.Errorf("duplicate schedule %q", job.ID)
}
seen[job.ID] = struct{}{}
if job.Kind != cronstore.KindWorkflow {
if job.Kind != scheduling.KindWorkflow {
return fmt.Errorf("schedule %q has unsupported kind %q", job.ID, job.Kind)
}
if err := job.Schedule.Validate(); err != nil {
@@ -51,7 +50,7 @@ func validateSchedules(jobs []cronstore.JobSpec) error {
}
func normalizeSchedules(input []byte) ([]byte, error) {
var jobs []cronstore.JobSpec
var jobs []scheduling.JobSpec
if err := json.Unmarshal(input, &jobs); err != nil {
return nil, err
}
+3 -3
View File
@@ -5,14 +5,14 @@ import (
"testing"
"time"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
)
func TestScheduleDefinitionComputesFirstRunAndRejectsDuplicateIDs(t *testing.T) {
t.Parallel()
definition := scheduleDefinition()
jobs := []cronstore.JobSpec{{ID: "daily", Kind: cronstore.KindWorkflow, Schedule: cronstore.Schedule{Kind: cronstore.ScheduleInterval, Interval: cronstore.Duration(time.Hour)}}}
jobs := []scheduling.JobSpec{{ID: "daily", Kind: scheduling.KindWorkflow, Schedule: scheduling.Schedule{Kind: scheduling.ScheduleInterval, Interval: scheduling.Duration(time.Hour)}}}
input, err := json.Marshal(jobs)
if err != nil {
t.Fatal(err)
@@ -21,7 +21,7 @@ func TestScheduleDefinitionComputesFirstRunAndRejectsDuplicateIDs(t *testing.T)
if err != nil {
t.Fatal(err)
}
var normalized []cronstore.JobSpec
var normalized []scheduling.JobSpec
if err := json.Unmarshal(got, &normalized); err != nil {
t.Fatal(err)
}
+2 -2
View File
@@ -17,7 +17,7 @@ const bareObjectSchema = `{"type":"object"}`
var timeType = reflect.TypeFor[time.Time]()
// stdDuration is the marker this tree's string-form duration types satisfy:
// config.Duration, channelDuration and cronstore.Duration each expose the
// config.Duration, channelDuration and scheduling.Duration each expose the
// standard duration for arithmetic.
var stdDuration = reflect.TypeOf((*interface{ Std() time.Duration })(nil)).Elem()
@@ -28,7 +28,7 @@ var stdDuration = reflect.TypeOf((*interface{ Std() time.Duration })(nil)).Elem(
// stale the moment a fourth duration type appears and the failure is silent in
// the worst direction: the field derives as an integer and a client renders a
// number box for a duration. That is exactly what the enumerated version of this
// function did to cronstore.Duration when the schedules interval became one.
// function did to scheduling.Duration when the schedules interval became one.
func durationLike(t reflect.Type) bool {
if !t.Implements(stdDuration) {
return false
+18 -2
View File
@@ -533,8 +533,10 @@ type Config struct {
// Tools holds MCP server and tool policy configuration (from config.tools.yaml).
Tools ToolsConfig `toml:"tools" yaml:"tools"`
// Indexing locates the workspace codesearch index.
Indexing IndexingConfig `toml:"indexing" yaml:"indexing"`
// Artifacts configures publishing artifacts to the collaborative editor.
// An empty BaseURL disables the sender entirely.
Artifacts ArtifactsConfig `toml:"artifacts" yaml:"artifacts"`
// Identities declares multi-identity configurations. When non-empty,
// each identity runs its own poll loop with its own forge client,
@@ -559,6 +561,20 @@ type Config struct {
Extra map[string]any `toml:"-" yaml:"-" json:"extra,omitempty"`
}
// ArtifactsConfig locates the collaborative editor that receives artifacts.
// An absent section is valid and disables publishing: the capability degrades
// to a log line rather than stopping the daemon.
type ArtifactsConfig struct {
// BaseURL is the editor origin, e.g. https://offloaded.dev. Empty
// disables artifact publishing.
BaseURL string `toml:"base_url" yaml:"base_url"`
// Token is the service credential the editor's ingest route requires.
Token SecretRef `toml:"token" yaml:"token"`
// TokenEnv names the environment variable holding the token when Token
// is unset. Defaults to WORKSPACE_INGEST_TOKEN.
TokenEnv string `toml:"token_env" yaml:"token_env"`
}
// IdentityConfig is a per-identity configuration subset. Each identity
// gets its own poll loop goroutine with isolated forge, repo, model,
// and dispatch configuration.
-29
View File
@@ -1,29 +0,0 @@
package config
import "path/filepath"
// IndexingConfig locates the workspace codesearch index.
//
// The index is a large, fully rebuildable artifact, so both paths are
// settable to allow putting it on different storage from the task work tree.
type IndexingConfig struct {
// IndexDir holds the codesearch sidecar files.
IndexDir string `toml:"index_dir" yaml:"index_dir" json:"index_dir,omitempty"`
// DBPath is the SQLite database holding index lifecycle metadata.
DBPath string `toml:"db_path" yaml:"db_path" json:"db_path,omitempty"`
}
// WithDefaults fills unset paths from the daemon's work directory.
//
// Deriving from workDir rather than a shared data home is deliberate: two
// archied instances run on one host, each with its own work_dir, so this
// keeps their indexes separate without either having to configure anything.
func (c IndexingConfig) WithDefaults(workDir string) IndexingConfig {
if c.IndexDir == "" {
c.IndexDir = filepath.Join(workDir, "indexes")
}
if c.DBPath == "" {
c.DBPath = filepath.Join(workDir, "workspace-indexes.db")
}
return c
}
+79 -73
View File
@@ -1288,7 +1288,7 @@ type Task struct {
WatchCommentId int64 `protobuf:"varint,20,opt,name=watch_comment_id,json=watchCommentId,proto3" json:"watch_comment_id,omitempty"`
Source string `protobuf:"bytes,21,opt,name=source,proto3" json:"source,omitempty"`
Identity string `protobuf:"bytes,22,opt,name=identity,proto3" json:"identity,omitempty"`
BindingId int64 `protobuf:"varint,23,opt,name=binding_id,json=bindingId,proto3" json:"binding_id,omitempty"`
BindingId string `protobuf:"bytes,23,opt,name=binding_id,json=bindingId,proto3" json:"binding_id,omitempty"`
BindingVersion int64 `protobuf:"varint,24,opt,name=binding_version,json=bindingVersion,proto3" json:"binding_version,omitempty"`
CreatedAt *timestamppb.Timestamp `protobuf:"bytes,25,opt,name=created_at,json=createdAt,proto3" json:"created_at,omitempty"`
UpdatedAt *timestamppb.Timestamp `protobuf:"bytes,26,opt,name=updated_at,json=updatedAt,proto3" json:"updated_at,omitempty"`
@@ -1490,11 +1490,11 @@ func (x *Task) GetIdentity() string {
return ""
}
func (x *Task) GetBindingId() int64 {
func (x *Task) GetBindingId() string {
if x != nil {
return x.BindingId
}
return 0
return ""
}
func (x *Task) GetBindingVersion() int64 {
@@ -1711,7 +1711,7 @@ func (x *Event) GetPrincipalId() string {
// CapturedEvent mirrors internal/store.CapturedEvent.
type CapturedEvent struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
ReceivedAt *timestamppb.Timestamp `protobuf:"bytes,2,opt,name=received_at,json=receivedAt,proto3" json:"received_at,omitempty"`
Source string `protobuf:"bytes,3,opt,name=source,proto3" json:"source,omitempty"`
RemoteAddr string `protobuf:"bytes,4,opt,name=remote_addr,json=remoteAddr,proto3" json:"remote_addr,omitempty"`
@@ -1753,11 +1753,11 @@ func (*CapturedEvent) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{26}
}
func (x *CapturedEvent) GetId() int64 {
func (x *CapturedEvent) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
func (x *CapturedEvent) GetReceivedAt() *timestamppb.Timestamp {
@@ -1881,7 +1881,7 @@ func (x *MappingField) GetRequired() bool {
// Mapping mirrors internal/domain/mapping.Mapping.
type Mapping struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
SourceHint string `protobuf:"bytes,3,opt,name=source_hint,json=sourceHint,proto3" json:"source_hint,omitempty"`
Fields []*MappingField `protobuf:"bytes,4,rep,name=fields,proto3" json:"fields,omitempty"`
@@ -1921,11 +1921,11 @@ func (*Mapping) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{28}
}
func (x *Mapping) GetId() int64 {
func (x *Mapping) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
func (x *Mapping) GetName() string {
@@ -2011,10 +2011,10 @@ func (x *BindingMatcher) GetSource() string {
// Binding mirrors internal/domain/binding.Binding.
type Binding struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
Matcher *BindingMatcher `protobuf:"bytes,3,opt,name=matcher,proto3" json:"matcher,omitempty"`
MappingId int64 `protobuf:"varint,4,opt,name=mapping_id,json=mappingId,proto3" json:"mapping_id,omitempty"`
MappingId string `protobuf:"bytes,4,opt,name=mapping_id,json=mappingId,proto3" json:"mapping_id,omitempty"`
Workflow string `protobuf:"bytes,5,opt,name=workflow,proto3" json:"workflow,omitempty"`
Owner string `protobuf:"bytes,6,opt,name=owner,proto3" json:"owner,omitempty"`
Repo string `protobuf:"bytes,7,opt,name=repo,proto3" json:"repo,omitempty"`
@@ -2057,11 +2057,11 @@ func (*Binding) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{30}
}
func (x *Binding) GetId() int64 {
func (x *Binding) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
func (x *Binding) GetName() string {
@@ -2078,11 +2078,11 @@ func (x *Binding) GetMatcher() *BindingMatcher {
return nil
}
func (x *Binding) GetMappingId() int64 {
func (x *Binding) GetMappingId() string {
if x != nil {
return x.MappingId
}
return 0
return ""
}
func (x *Binding) GetWorkflow() string {
@@ -4133,9 +4133,14 @@ func (x *InsertEventResponse) GetId() int64 {
}
type EventsSinceRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
SinceId int64 `protobuf:"varint,1,opt,name=since_id,json=sinceId,proto3" json:"since_id,omitempty"`
Limit int64 `protobuf:"varint,2,opt,name=limit,proto3" json:"limit,omitempty"`
state protoimpl.MessageState `protogen:"open.v1"`
// since_id is the opaque resume cursor returned by the previous page (the SSE
// id: field, echoed back as Last-Event-ID). Empty means "from the
// beginning". The field keeps its name and number so the wire name stays
// put; only its type widened, from the SQLite rowid to the fixed-width
// "<at>|<row id>" cursor.
SinceId string `protobuf:"bytes,1,opt,name=since_id,json=sinceId,proto3" json:"since_id,omitempty"`
Limit int64 `protobuf:"varint,2,opt,name=limit,proto3" json:"limit,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -4170,11 +4175,11 @@ func (*EventsSinceRequest) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{70}
}
func (x *EventsSinceRequest) GetSinceId() int64 {
func (x *EventsSinceRequest) GetSinceId() string {
if x != nil {
return x.SinceId
}
return 0
return ""
}
func (x *EventsSinceRequest) GetLimit() int64 {
@@ -5763,7 +5768,7 @@ func (x *InsertCaptureRequest) GetMaxEvents() int64 {
type InsertCaptureResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -5798,11 +5803,11 @@ func (*InsertCaptureResponse) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{101}
}
func (x *InsertCaptureResponse) GetId() int64 {
func (x *InsertCaptureResponse) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
type ListCapturesRequest struct {
@@ -6028,7 +6033,7 @@ func (x *InsertMappingRequest) GetMapping() *Mapping {
type InsertMappingResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -6063,16 +6068,16 @@ func (*InsertMappingResponse) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{107}
}
func (x *InsertMappingResponse) GetId() int64 {
func (x *InsertMappingResponse) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
type GetMappingRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -6107,11 +6112,11 @@ func (*GetMappingRequest) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{108}
}
func (x *GetMappingRequest) GetId() int64 {
func (x *GetMappingRequest) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
type GetMappingResponse struct {
@@ -6328,7 +6333,7 @@ func (*UpdateMappingResponse) Descriptor() ([]byte, []int) {
type DeleteMappingRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -6363,11 +6368,11 @@ func (*DeleteMappingRequest) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{114}
}
func (x *DeleteMappingRequest) GetId() int64 {
func (x *DeleteMappingRequest) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
type DeleteMappingResponse struct {
@@ -6452,7 +6457,7 @@ func (x *InsertBindingRequest) GetBinding() *Binding {
type InsertBindingResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -6487,16 +6492,16 @@ func (*InsertBindingResponse) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{117}
}
func (x *InsertBindingResponse) GetId() int64 {
func (x *InsertBindingResponse) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
type GetBindingRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -6531,11 +6536,11 @@ func (*GetBindingRequest) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{118}
}
func (x *GetBindingRequest) GetId() int64 {
func (x *GetBindingRequest) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
type GetBindingResponse struct {
@@ -6752,7 +6757,7 @@ func (*UpdateBindingResponse) Descriptor() ([]byte, []int) {
type DeleteBindingRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -6787,11 +6792,11 @@ func (*DeleteBindingRequest) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{124}
}
func (x *DeleteBindingRequest) GetId() int64 {
func (x *DeleteBindingRequest) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
type DeleteBindingResponse struct {
@@ -6832,7 +6837,7 @@ func (*DeleteBindingResponse) Descriptor() ([]byte, []int) {
type ApproveBindingRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -6867,11 +6872,11 @@ func (*ApproveBindingRequest) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{126}
}
func (x *ApproveBindingRequest) GetId() int64 {
func (x *ApproveBindingRequest) GetId() string {
if x != nil {
return x.Id
}
return 0
return ""
}
type ApproveBindingResponse struct {
@@ -7000,12 +7005,13 @@ func (x *ArmedBindingsForSourceResponse) GetBindings() []*Binding {
type RecordDispatchRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
BindingId int64 `protobuf:"varint,1,opt,name=binding_id,json=bindingId,proto3" json:"binding_id,omitempty"`
BindingId string `protobuf:"bytes,1,opt,name=binding_id,json=bindingId,proto3" json:"binding_id,omitempty"`
BindingVersion int64 `protobuf:"varint,2,opt,name=binding_version,json=bindingVersion,proto3" json:"binding_version,omitempty"`
CaptureId int64 `protobuf:"varint,3,opt,name=capture_id,json=captureId,proto3" json:"capture_id,omitempty"`
TaskId int64 `protobuf:"varint,4,opt,name=task_id,json=taskId,proto3" json:"task_id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
CaptureId string `protobuf:"bytes,3,opt,name=capture_id,json=captureId,proto3" json:"capture_id,omitempty"`
// task_id addresses the SQLite-owned task tables, which are not migrating.
TaskId int64 `protobuf:"varint,4,opt,name=task_id,json=taskId,proto3" json:"task_id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *RecordDispatchRequest) Reset() {
@@ -7038,11 +7044,11 @@ func (*RecordDispatchRequest) Descriptor() ([]byte, []int) {
return file_state_v1_state_proto_rawDescGZIP(), []int{130}
}
func (x *RecordDispatchRequest) GetBindingId() int64 {
func (x *RecordDispatchRequest) GetBindingId() string {
if x != nil {
return x.BindingId
}
return 0
return ""
}
func (x *RecordDispatchRequest) GetBindingVersion() int64 {
@@ -7052,11 +7058,11 @@ func (x *RecordDispatchRequest) GetBindingVersion() int64 {
return 0
}
func (x *RecordDispatchRequest) GetCaptureId() int64 {
func (x *RecordDispatchRequest) GetCaptureId() string {
if x != nil {
return x.CaptureId
}
return 0
return ""
}
func (x *RecordDispatchRequest) GetTaskId() int64 {
@@ -7487,7 +7493,7 @@ type EnqueueBindingTaskRequest struct {
Body string `protobuf:"bytes,4,opt,name=body,proto3" json:"body,omitempty"`
Workflow string `protobuf:"bytes,5,opt,name=workflow,proto3" json:"workflow,omitempty"`
Identity string `protobuf:"bytes,6,opt,name=identity,proto3" json:"identity,omitempty"`
BindingId int64 `protobuf:"varint,7,opt,name=binding_id,json=bindingId,proto3" json:"binding_id,omitempty"`
BindingId string `protobuf:"bytes,7,opt,name=binding_id,json=bindingId,proto3" json:"binding_id,omitempty"`
BindingVersion int64 `protobuf:"varint,8,opt,name=binding_version,json=bindingVersion,proto3" json:"binding_version,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
@@ -7565,11 +7571,11 @@ func (x *EnqueueBindingTaskRequest) GetIdentity() string {
return ""
}
func (x *EnqueueBindingTaskRequest) GetBindingId() int64 {
func (x *EnqueueBindingTaskRequest) GetBindingId() string {
if x != nil {
return x.BindingId
}
return 0
return ""
}
func (x *EnqueueBindingTaskRequest) GetBindingVersion() int64 {
@@ -7734,7 +7740,7 @@ const file_state_v1_state_proto_rawDesc = "" +
"\x06source\x18\x15 \x01(\tR\x06source\x12\x1a\n" +
"\bidentity\x18\x16 \x01(\tR\bidentity\x12\x1d\n" +
"\n" +
"binding_id\x18\x17 \x01(\x03R\tbindingId\x12'\n" +
"binding_id\x18\x17 \x01(\tR\tbindingId\x12'\n" +
"\x0fbinding_version\x18\x18 \x01(\x03R\x0ebindingVersion\x129\n" +
"\n" +
"created_at\x18\x19 \x01(\v2\x1a.google.protobuf.TimestampR\tcreatedAt\x129\n" +
@@ -7762,7 +7768,7 @@ const file_state_v1_state_proto_rawDesc = "" +
"actor_kind\x18\r \x01(\tR\tactorKind\x12!\n" +
"\fprincipal_id\x18\x0e \x01(\tR\vprincipalId\"\x8c\x02\n" +
"\rCapturedEvent\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\x12;\n" +
"\x02id\x18\x01 \x01(\tR\x02id\x12;\n" +
"\vreceived_at\x18\x02 \x01(\v2\x1a.google.protobuf.TimestampR\n" +
"receivedAt\x12\x16\n" +
"\x06source\x18\x03 \x01(\tR\x06source\x12\x1f\n" +
@@ -7778,7 +7784,7 @@ const file_state_v1_state_proto_rawDesc = "" +
"\x04type\x18\x03 \x01(\tR\x04type\x12\x1a\n" +
"\brequired\x18\x04 \x01(\bR\brequired\"\xf4\x01\n" +
"\aMapping\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\x12\x12\n" +
"\x02id\x18\x01 \x01(\tR\x02id\x12\x12\n" +
"\x04name\x18\x02 \x01(\tR\x04name\x12\x1f\n" +
"\vsource_hint\x18\x03 \x01(\tR\n" +
"sourceHint\x12.\n" +
@@ -7790,11 +7796,11 @@ const file_state_v1_state_proto_rawDesc = "" +
"\x0eBindingMatcher\x12\x16\n" +
"\x06source\x18\x01 \x01(\tR\x06source\"\x86\x03\n" +
"\aBinding\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\x12\x12\n" +
"\x02id\x18\x01 \x01(\tR\x02id\x12\x12\n" +
"\x04name\x18\x02 \x01(\tR\x04name\x122\n" +
"\amatcher\x18\x03 \x01(\v2\x18.state.v1.BindingMatcherR\amatcher\x12\x1d\n" +
"\n" +
"mapping_id\x18\x04 \x01(\x03R\tmappingId\x12\x1a\n" +
"mapping_id\x18\x04 \x01(\tR\tmappingId\x12\x1a\n" +
"\bworkflow\x18\x05 \x01(\tR\bworkflow\x12\x14\n" +
"\x05owner\x18\x06 \x01(\tR\x05owner\x12\x12\n" +
"\x04repo\x18\a \x01(\tR\x04repo\x12\x18\n" +
@@ -7926,7 +7932,7 @@ const file_state_v1_state_proto_rawDesc = "" +
"\x13InsertEventResponse\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\"E\n" +
"\x12EventsSinceRequest\x12\x19\n" +
"\bsince_id\x18\x01 \x01(\x03R\asinceId\x12\x14\n" +
"\bsince_id\x18\x01 \x01(\tR\asinceId\x12\x14\n" +
"\x05limit\x18\x02 \x01(\x03R\x05limit\">\n" +
"\x13EventsSinceResponse\x12'\n" +
"\x06events\x18\x01 \x03(\v2\x0f.state.v1.EventR\x06events\",\n" +
@@ -8024,7 +8030,7 @@ const file_state_v1_state_proto_rawDesc = "" +
"\n" +
"max_events\x18\x03 \x01(\x03R\tmaxEvents\"'\n" +
"\x15InsertCaptureResponse\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\"+\n" +
"\x02id\x18\x01 \x01(\tR\x02id\"+\n" +
"\x13ListCapturesRequest\x12\x14\n" +
"\x05limit\x18\x01 \x01(\x03R\x05limit\"K\n" +
"\x14ListCapturesResponse\x123\n" +
@@ -8036,9 +8042,9 @@ const file_state_v1_state_proto_rawDesc = "" +
"\x14InsertMappingRequest\x12+\n" +
"\amapping\x18\x01 \x01(\v2\x11.state.v1.MappingR\amapping\"'\n" +
"\x15InsertMappingResponse\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\"#\n" +
"\x02id\x18\x01 \x01(\tR\x02id\"#\n" +
"\x11GetMappingRequest\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\"W\n" +
"\x02id\x18\x01 \x01(\tR\x02id\"W\n" +
"\x12GetMappingResponse\x12+\n" +
"\amapping\x18\x01 \x01(\v2\x11.state.v1.MappingR\amapping\x12\x14\n" +
"\x05found\x18\x02 \x01(\bR\x05found\"\x15\n" +
@@ -8049,14 +8055,14 @@ const file_state_v1_state_proto_rawDesc = "" +
"\amapping\x18\x01 \x01(\v2\x11.state.v1.MappingR\amapping\"\x17\n" +
"\x15UpdateMappingResponse\"&\n" +
"\x14DeleteMappingRequest\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\"\x17\n" +
"\x02id\x18\x01 \x01(\tR\x02id\"\x17\n" +
"\x15DeleteMappingResponse\"C\n" +
"\x14InsertBindingRequest\x12+\n" +
"\abinding\x18\x01 \x01(\v2\x11.state.v1.BindingR\abinding\"'\n" +
"\x15InsertBindingResponse\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\"#\n" +
"\x02id\x18\x01 \x01(\tR\x02id\"#\n" +
"\x11GetBindingRequest\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\"W\n" +
"\x02id\x18\x01 \x01(\tR\x02id\"W\n" +
"\x12GetBindingResponse\x12+\n" +
"\abinding\x18\x01 \x01(\v2\x11.state.v1.BindingR\abinding\x12\x14\n" +
"\x05found\x18\x02 \x01(\bR\x05found\"\x15\n" +
@@ -8067,10 +8073,10 @@ const file_state_v1_state_proto_rawDesc = "" +
"\abinding\x18\x01 \x01(\v2\x11.state.v1.BindingR\abinding\"\x17\n" +
"\x15UpdateBindingResponse\"&\n" +
"\x14DeleteBindingRequest\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\"\x17\n" +
"\x02id\x18\x01 \x01(\tR\x02id\"\x17\n" +
"\x15DeleteBindingResponse\"'\n" +
"\x15ApproveBindingRequest\x12\x0e\n" +
"\x02id\x18\x01 \x01(\x03R\x02id\"\x18\n" +
"\x02id\x18\x01 \x01(\tR\x02id\"\x18\n" +
"\x16ApproveBindingResponse\"7\n" +
"\x1dArmedBindingsForSourceRequest\x12\x16\n" +
"\x06source\x18\x01 \x01(\tR\x06source\"O\n" +
@@ -8078,10 +8084,10 @@ const file_state_v1_state_proto_rawDesc = "" +
"\bbindings\x18\x01 \x03(\v2\x11.state.v1.BindingR\bbindings\"\x97\x01\n" +
"\x15RecordDispatchRequest\x12\x1d\n" +
"\n" +
"binding_id\x18\x01 \x01(\x03R\tbindingId\x12'\n" +
"binding_id\x18\x01 \x01(\tR\tbindingId\x12'\n" +
"\x0fbinding_version\x18\x02 \x01(\x03R\x0ebindingVersion\x12\x1d\n" +
"\n" +
"capture_id\x18\x03 \x01(\x03R\tcaptureId\x12\x17\n" +
"capture_id\x18\x03 \x01(\tR\tcaptureId\x12\x17\n" +
"\atask_id\x18\x04 \x01(\x03R\x06taskId\"\x18\n" +
"\x16RecordDispatchResponse\"\xa3\x01\n" +
"\x1dRecordPlaybookDispatchRequest\x12\x1f\n" +
@@ -8113,7 +8119,7 @@ const file_state_v1_state_proto_rawDesc = "" +
"\bworkflow\x18\x05 \x01(\tR\bworkflow\x12\x1a\n" +
"\bidentity\x18\x06 \x01(\tR\bidentity\x12\x1d\n" +
"\n" +
"binding_id\x18\a \x01(\x03R\tbindingId\x12'\n" +
"binding_id\x18\a \x01(\tR\tbindingId\x12'\n" +
"\x0fbinding_version\x18\b \x01(\x03R\x0ebindingVersion\"@\n" +
"\x1aEnqueueBindingTaskResponse\x12\"\n" +
"\x04task\x18\x01 \x01(\v2\x0e.state.v1.TaskR\x04task2\xe1)\n" +
+48 -2
View File
@@ -678,7 +678,7 @@ func (d *Daemon) dispatchOneBinding(ctx context.Context, b binding.Binding, c st
if hasBlockingFailure(m.Fields, failures) {
_, _ = d.Store.InsertEvent(ctx, events.Event{
Kind: "binding_dispatch_failure",
Detail: fmt.Sprintf("binding %d (capture %d): required field failed", b.ID, c.ID),
Detail: fmt.Sprintf("binding %q (capture %q): required field failed", b.ID, c.ID),
Data: map[string]any{
"binding_id": b.ID,
"binding_version": b.Version,
@@ -746,7 +746,7 @@ func hasBlockingFailure(fields []mapping.Field, failures []mapping.Failure) bool
// provenance to hand back to the operator.
func renderBindingBody(values map[string]any, c storecontract.CapturedEvent) string {
if len(values) == 0 {
return fmt.Sprintf("(no fields resolved from capture %d)", c.ID)
return fmt.Sprintf("(no fields resolved from capture %q)", c.ID)
}
parts := make([]string, 0, len(values))
for k, v := range values {
@@ -994,6 +994,14 @@ func (d *Daemon) processNATSTask(ctx context.Context, msg eventbus.Message) {
return
}
if ok, reason := d.identityMayAct(ctx, tm.Identity); !ok {
d.Log.Error("nats intake rejected: "+reason, "subject", msg.Subject())
if err := msg.Ack(); err != nil {
d.Log.Warn("ack failed", "err", err)
}
return
}
inserted, err := d.Store.EnqueueIssue(ctx, tm.Owner, tm.Repo, tm.Number, tm.Title, tm.Body, strings.Join(tm.Labels, ","), tm.Identity)
if err != nil {
d.Log.Error("nats enqueue failed", "err", err)
@@ -1126,6 +1134,10 @@ func (d *Daemon) process(ctx context.Context, task *workflow.Task) {
defer finished()
defer d.openTaskLog(task)()
if d.parkIfIdentityUnresolvable(ctx, task) {
return
}
trees := d.treesFor(task)
repo, ok := d.repoFor(task)
if !ok {
@@ -1770,6 +1782,40 @@ func (d *Daemon) identityFor(task *workflow.Task) *IdentityRunner {
return nil
}
// identityMayAct reports whether the identity a task carries may be acted on.
// An empty identity is the single-identity root and always may act. A
// non-empty identity must resolve to a configured runner and pass the
// lifecycle check; otherwise the returned reason says why it may not. This is
// the single task-side fail-closed point, mirroring identity.Resolve's
// credential-side behaviour: a task naming an identity archie no longer knows
// -- renamed, retired, or deleted in the control plane -- must park, never
// fall back to the root forge's credential.
func (d *Daemon) identityMayAct(ctx context.Context, name string) (bool, string) {
if name == "" {
return true, ""
}
runner := d.identityFor(&workflow.Task{Identity: name})
if runner == nil {
return false, "identity " + name + " no longer resolves"
}
if !d.identityActive(ctx, runner.ID) {
return false, "identity " + runner.Name + " may not act"
}
return true, ""
}
// parkIfIdentityUnresolvable parks task when its non-empty identity cannot be
// acted on, reporting whether it did. An empty identity is the root
// single-identity path and never parks here.
func (d *Daemon) parkIfIdentityUnresolvable(ctx context.Context, task *workflow.Task) bool {
ok, reason := d.identityMayAct(ctx, task.Identity)
if ok {
return false
}
d.parkRunningTask(ctx, task.ID, reason)
return true
}
func (d *Daemon) identityActive(ctx context.Context, id identity.IdentityID) bool {
if d.IdentityRepository == nil {
return true
+56 -33
View File
@@ -11,6 +11,7 @@ import (
"github.com/samcharles93/archie-core/internal/domain/binding"
"github.com/samcharles93/archie-core/internal/domain/mapping"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/store"
)
@@ -35,7 +36,7 @@ func TestDispatchBindingsCreatesTaskFromArmedCapture(t *testing.T) {
}
got := tasks[0]
if got.BindingID != bindingID {
t.Errorf("task.BindingID = %d, want %d", got.BindingID, bindingID)
t.Errorf("task.BindingID = %q, want %q", got.BindingID, bindingID)
}
if got.BindingVersion != bindingVersion {
t.Errorf("task.BindingVersion = %d, want %d", got.BindingVersion, bindingVersion)
@@ -170,7 +171,7 @@ func TestDispatchBindingsSkipsWhenResolveFails(t *testing.T) {
if len(tasks) != 0 {
t.Fatalf("required-field failure spawned %d task(s), want 0", len(tasks))
}
events, err := s.EventsSince(t.Context(), 0, 100)
events, err := s.EventsSince(t.Context(), "", 100)
if err != nil {
t.Fatalf("EventsSince: %v", err)
}
@@ -230,9 +231,9 @@ func TestDispatchBindingsNilDispatcherIsNoOp(t *testing.T) {
d := &Daemon{
Cfg: config.NewHolder(config.Config{Repos: []config.Repo{{Owner: "acme", Name: "widget"}}}),
Store: s,
Mappings: s,
Bindings: s,
Store: s.Store,
Mappings: s.EdaStore,
Bindings: s.EdaStore,
// BindingDispatcher and BindingTaskCreator intentionally nil.
Log: slog.New(slog.NewTextHandler(os.Stderr, nil)),
}
@@ -248,22 +249,31 @@ func TestDispatchBindingsNilDispatcherIsNoOp(t *testing.T) {
// production uses (CREATE TABLE + ALTER TABLE migrations), so
// binding_id / binding_version / binding_dispatches all exist on the
// store by the time a test runs.
func openDispatchTestStore(t *testing.T) *store.Store {
// dispatchStores pairs the two stores the dispatcher spans: tasks stay on
// SQLite, event capture is the PocketBase store. They are embedded so a seed
// helper still reads as one handle; the method sets no longer overlap, which
// is exactly what the split bought.
type dispatchStores struct {
*store.Store
EdaStore *edastore.Store
}
func openDispatchTestStore(t *testing.T) *dispatchStores {
t.Helper()
s, err := store.Open(t.Context(), filepath.Join(t.TempDir(), "dispatch.db"))
if err != nil {
t.Fatalf("store.Open: %v", err)
}
t.Cleanup(func() { _ = s.Close() })
return s
return &dispatchStores{Store: s, EdaStore: edastore.OpenTest(t)}
}
// seedMapping inserts a mapping with the given fields and returns its id.
// The Name field is "sentry-mapping" by default; callers pass fields
// inline.
func seedMapping(t *testing.T, s *store.Store, sourceHint string, fields ...mapping.Field) int64 {
func seedMapping(t *testing.T, s *dispatchStores, sourceHint string, fields ...mapping.Field) string {
t.Helper()
id, err := s.InsertMapping(t.Context(), mapping.Mapping{
id, err := s.EdaStore.InsertMapping(t.Context(), mapping.Mapping{
Name: sourceHint + "-mapping",
SourceHint: sourceHint,
Fields: fields,
@@ -281,7 +291,7 @@ func seedMapping(t *testing.T, s *store.Store, sourceHint string, fields ...mapp
// advances through pending_approval via the store's test-only DB
// accessor -- production callers must use the structured methods
// (bindings.go) rather than touching the SQL directly.
func seedArmedBinding(t *testing.T, s *store.Store, source string, mappingID int64) (int64, int) {
func seedArmedBinding(t *testing.T, s *dispatchStores, source, mappingID string) (string, int) {
t.Helper()
return seedArmedBindingWithRepo(t, s, source, mappingID, "", "")
}
@@ -289,9 +299,9 @@ func seedArmedBinding(t *testing.T, s *store.Store, source string, mappingID int
// seedArmedBindingWithRepo is seedArmedBinding with an explicit
// owner/repo pin (pass "", "" for the unpinned case seedArmedBinding
// covers).
func seedArmedBindingWithRepo(t *testing.T, s *store.Store, source string, mappingID int64, owner, repo string) (int64, int) {
func seedArmedBindingWithRepo(t *testing.T, s *dispatchStores, source, mappingID, owner, repo string) (string, int) {
t.Helper()
id, err := s.InsertBinding(t.Context(), binding.Binding{
id, err := s.EdaStore.InsertBinding(t.Context(), binding.Binding{
Name: "test " + source,
Matcher: binding.Matcher{Source: source},
MappingID: mappingID,
@@ -303,13 +313,25 @@ func seedArmedBindingWithRepo(t *testing.T, s *store.Store, source string, mappi
if err != nil {
t.Fatalf("InsertBinding: %v", err)
}
if _, err := s.DB().ExecContext(t.Context(), `UPDATE bindings SET status='pending_approval' WHERE id=?`, id); err != nil {
t.Fatalf("force pending_approval: %v", err)
// Edit moves a draft to pending_approval through the real lifecycle,
// which is what makes it approvable. The old raw-SQL shortcut reached
// into a table the task store no longer owns.
if err := s.EdaStore.UpdateBinding(t.Context(), binding.Binding{
ID: id,
Name: "test " + source,
Matcher: binding.Matcher{Source: source},
MappingID: mappingID,
Workflow: "implement",
Owner: owner,
Repo: repo,
Secret: "0123456789abcdef0123456789abcdef",
}); err != nil {
t.Fatalf("UpdateBinding to pending_approval: %v", err)
}
if err := s.ApproveBinding(t.Context(), id); err != nil {
if err := s.EdaStore.ApproveBinding(t.Context(), id); err != nil {
t.Fatalf("ApproveBinding: %v", err)
}
got, err := s.GetBinding(t.Context(), id)
got, err := s.EdaStore.GetBinding(t.Context(), id)
if err != nil || got == nil {
t.Fatalf("GetBinding after approve: %+v, %v", got, err)
}
@@ -321,9 +343,9 @@ func seedArmedBindingWithRepo(t *testing.T, s *store.Store, source string, mappi
// the per-insert pruning the production path uses, so a test that
// only needs a single row in the table is not at the mercy of
// maxEvents=0's "no cap" reading.
func seedCapture(t *testing.T, s *store.Store, source string, authenticated bool, body string) {
func seedCapture(t *testing.T, s *dispatchStores, source string, authenticated bool, body string) {
t.Helper()
_, err := s.InsertCapture(t.Context(), store.CapturedEvent{
_, err := s.EdaStore.InsertCapture(t.Context(), store.CapturedEvent{
Source: source,
Body: body,
Authenticated: authenticated,
@@ -341,7 +363,7 @@ func seedCapture(t *testing.T, s *store.Store, source string, authenticated bool
// single-repo fallback accepts the dispatch; with zero or many
// configured repos the loop would log and skip, masking the test's
// positive assertions.
func newDispatchDaemon(t *testing.T, s *store.Store) *Daemon {
func newDispatchDaemon(t *testing.T, s *dispatchStores) *Daemon {
t.Helper()
return newDispatchDaemonWithRepos(t, s, []config.Repo{{Owner: "acme", Name: "widget"}})
}
@@ -349,36 +371,37 @@ func newDispatchDaemon(t *testing.T, s *store.Store) *Daemon {
// newDispatchDaemonWithRepos is newDispatchDaemon with an explicit repo
// list, for tests exercising resolveBindingRepo's owner/repo-pin path
// against zero, one, or several configured repos.
func newDispatchDaemonWithRepos(t *testing.T, s *store.Store, repos []config.Repo) *Daemon {
func newDispatchDaemonWithRepos(t *testing.T, s *dispatchStores, repos []config.Repo) *Daemon {
t.Helper()
cfg := config.Config{Repos: repos}
return &Daemon{
Cfg: config.NewHolder(cfg),
Store: s,
Mappings: s,
Bindings: s,
BindingDispatcher: s,
BindingTaskCreator: s,
Store: s.Store,
Mappings: s.EdaStore,
Bindings: s.EdaStore,
BindingDispatcher: s.EdaStore,
BindingTaskCreator: s.Store,
Log: slog.New(slog.NewTextHandler(os.Stderr, nil)),
}
}
// assertDispatchRecorded reads the dedup ledger directly: the
// (binding_id, capture_id) primary key is what guarantees no double
// (binding, capture) unique index is what guarantees no double
// dispatch. Going through the SQL table rather than the public
// BindingStore surface keeps the assertion focused on the contract
// Phase E actually writes to, not on a ListDispatches helper that
// may not exist.
func assertDispatchRecorded(t *testing.T, s *store.Store, bindingID, taskID int64) {
func assertDispatchRecorded(t *testing.T, s *dispatchStores, bindingID string, taskID int64) {
t.Helper()
var gotBindingID, gotTaskID int64
if err := s.DB().QueryRowContext(
t.Context(),
`SELECT binding_id, task_id FROM binding_dispatches WHERE binding_id=?`, bindingID,
).Scan(&gotBindingID, &gotTaskID); err != nil {
var gotBindingID string
var gotTaskID int64
if err := s.EdaStore.App().DB().NewQuery(
"SELECT binding, task_id FROM binding_dispatches WHERE binding = {:binding}").
Bind(map[string]any{"binding": bindingID}).
Row(&gotBindingID, &gotTaskID); err != nil {
t.Fatalf("binding_dispatches row missing: %v", err)
}
if gotBindingID != bindingID || gotTaskID != taskID {
t.Fatalf("binding_dispatches row = (%d, %d), want (%d, %d)", gotBindingID, gotTaskID, bindingID, taskID)
t.Fatalf("binding_dispatches row = (%q, %d), want (%q, %d)", gotBindingID, gotTaskID, bindingID, taskID)
}
}
+163
View File
@@ -0,0 +1,163 @@
package daemon
import (
"context"
"log/slog"
"strings"
"testing"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/domain/identity"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/domain/workintake"
"github.com/samcharles93/archie-core/internal/eventbus"
"github.com/samcharles93/archie-core/internal/store"
"github.com/samcharles93/archie-core/internal/worktree"
)
// fakeMessage is a minimal eventbus.Message for driving processNATSTask
// without standing up a broker.
type fakeMessage struct {
data []byte
subject string
acked bool
naked bool
}
func (m *fakeMessage) Data() []byte { return m.data }
func (m *fakeMessage) Subject() string { return m.subject }
func (m *fakeMessage) Ack() error { m.acked = true; return nil }
func (m *fakeMessage) Nak() error { m.naked = true; return nil }
var _ eventbus.Message = (*fakeMessage)(nil)
// TestProcessNATSTaskRejectsUnresolvableIdentity pins the intake half of the
// fail-closed relation: a task envelope naming an identity this daemon does
// not know must not be written to the store at all. Today it is enqueued and
// only fails open later, when forgeFor/repoFor fall back to the root forge.
func TestProcessNATSTaskRejectsUnresolvableIdentity(t *testing.T) {
s := store.OpenTest(t)
d := &Daemon{
Store: s,
Cfg: config.NewHolder(config.Config{}),
Trees: &worktree.Manager{},
Log: slog.New(slog.DiscardHandler),
}
data, err := workintake.TaskEnvelope{
Owner: "acme", Repo: "widget", Number: 1, Title: "ghost task",
Identity: "ghost",
}.Encode()
if err != nil {
t.Fatal(err)
}
msg := &fakeMessage{data: data, subject: workintake.SubjectTaskDefault}
d.processNATSTask(t.Context(), msg)
if msg.naked {
t.Fatal("unresolvable identity intake was nacked; a redelivery can never resolve it")
}
if !msg.acked {
t.Fatal("unresolvable identity intake was not dropped (acked)")
}
task, err := s.TaskByIssue(t.Context(), "acme", "widget", 1)
if err != nil {
t.Fatal(err)
}
if task != nil {
t.Fatalf("task %d was created with an unresolvable identity %q", task.ID, task.Identity)
}
}
// TestProcessParksTaskWhoseIdentityNoLongerResolves pins the dispatch half of
// the fail-closed relation: a task whose non-empty identity no longer has a
// configured runner must park, never fall back to the root forge's credential.
func TestProcessParksTaskWhoseIdentityNoLongerResolves(t *testing.T) {
d, s, rootFg, _, _ := twoIdentityDaemon(t)
d.Log = slog.New(slog.DiscardHandler)
ctx := context.Background()
if _, err := s.EnqueueIssue(ctx, "acme", "shared", 42, "t", "b", "", "ghost"); err != nil {
t.Fatal(err)
}
task, err := s.ClaimNext(ctx)
if err != nil || task == nil {
t.Fatalf("ClaimNext = (%+v, %v)", task, err)
}
d.process(ctx, task)
got, err := s.TaskByID(ctx, task.ID)
if err != nil || got == nil {
t.Fatalf("TaskByID = (%+v, %v)", got, err)
}
if got.Status != workflow.StatusParked {
t.Fatalf("status = %q, want parked", got.Status)
}
if !strings.Contains(got.ParkReason, "no longer resolves") {
t.Errorf("ParkReason = %q, want it to name the unresolvable identity", got.ParkReason)
}
// The root forge must not have been touched for this task: parking on an
// identity that no longer resolves is the fail-closed outcome, not a
// fallback to root credentials.
if calls := len(rootFg.stateLabels) + len(rootFg.closedIssues) + len(rootFg.comments); calls != 0 {
t.Errorf("root forge was called %d time(s) for a task with an unresolvable identity", calls)
}
}
// TestProcessParksRetiredIdentity pins the lifecycle half: an identity that
// still has a runner but has been retired may not act, so its tasks park
// rather than running under a credential the control plane has retired.
func TestProcessParksRetiredIdentity(t *testing.T) {
s := store.OpenTest(t)
ctx := context.Background()
log := slog.New(slog.DiscardHandler)
id := identity.StableID("worker")
created, err := identity.New(id, identity.KindBot, "worker")
if err != nil {
t.Fatal(err)
}
created, err = s.Create(ctx, created, identity.Audit{ActorID: identity.SystemID, Source: "test", RequestID: "create"})
if err != nil {
t.Fatal(err)
}
if _, err := s.Apply(ctx, created.ID, created.Version, identity.Command{Type: identity.CommandRetire}, identity.Audit{ActorID: identity.SystemID, Source: "test", RequestID: "retire"}); err != nil {
t.Fatal(err)
}
repo := config.Repo{Owner: "acme", Name: "widget"}
d := &Daemon{
Cfg: config.NewHolder(config.Config{Repos: []config.Repo{repo}}),
Store: s,
Log: log,
IdentityRepository: s,
Identities: []*IdentityRunner{{
Name: "worker", ID: id,
Forge: &testForge{}, Trees: &worktree.Manager{},
Repos: []config.Repo{repo}, Cfg: config.IdentityConfig{Name: "worker"}, Log: log,
}},
}
if _, err := s.EnqueueIssue(ctx, "acme", "widget", 7, "t", "b", "", string(id)); err != nil {
t.Fatal(err)
}
task, err := s.ClaimNext(ctx)
if err != nil || task == nil {
t.Fatalf("ClaimNext = (%+v, %v)", task, err)
}
d.process(ctx, task)
got, err := s.TaskByID(ctx, task.ID)
if err != nil || got == nil {
t.Fatalf("TaskByID = (%+v, %v)", got, err)
}
if got.Status != workflow.StatusParked {
t.Fatalf("status = %q, want parked", got.Status)
}
if !strings.Contains(got.ParkReason, "may not act") {
t.Errorf("ParkReason = %q, want it to name the retired identity", got.ParkReason)
}
}
+4 -4
View File
@@ -43,10 +43,10 @@ type Matcher struct {
// for an event to be marked authenticated; it is never returned by
// GET handlers.
type Binding struct {
ID int64 `json:"id"`
ID string `json:"id"`
Name string `json:"name"`
Matcher Matcher `json:"matcher"`
MappingID int64 `json:"mapping_id"`
MappingID string `json:"mapping_id"`
Workflow string `json:"workflow"`
// Owner and Repo pin a binding to a specific configured repo, so a
// multi-repo deployment can dispatch correctly. Both empty means "no
@@ -110,8 +110,8 @@ func (b Binding) validateCommon() error {
if strings.TrimSpace(b.Matcher.Source) == "" {
return fmt.Errorf("binding: matcher.source is required")
}
if b.MappingID <= 0 {
return fmt.Errorf("binding: mapping_id must be positive")
if strings.TrimSpace(b.MappingID) == "" {
return fmt.Errorf("binding: mapping_id is required")
}
if strings.TrimSpace(b.Workflow) == "" {
return fmt.Errorf("binding: workflow is required")
+1 -1
View File
@@ -6,7 +6,7 @@ func validBinding() Binding {
return Binding{
Name: "sentry alerts",
Matcher: Matcher{Source: "sentry"},
MappingID: 1,
MappingID: "m1",
Workflow: "implement",
Secret: "0123456789abcdef0123456789abcdef",
}
+1 -1
View File
@@ -50,7 +50,7 @@ type Field struct {
// through one example captured event but not pinned to that event or its
// source -- t2db.4's matcher decides which events a binding applies to.
type Mapping struct {
ID int64 `json:"id"`
ID string `json:"id"`
Name string `json:"name"`
SourceHint string `json:"source_hint"`
Fields []Field `json:"fields"`
+112
View File
@@ -0,0 +1,112 @@
package scheduling
import (
"errors"
"time"
)
// Kind values for JobSpec.Kind: the discriminator that decides which delivery
// runner handles a job.
//
// They live beside the field that carries them because they are part of the
// schedules vocabulary an operator writes into the schedules document, a strict
// decoder reads back, and a build that never heard of a kind must reject loudly
// rather than deliver the wrong way. Which runner serves a kind is not this
// package's business: the router in internal/infrastructure/crondelivery owns
// that mapping, which is why an unrecognised kind is stored verbatim rather
// than refused at write time.
const (
// KindChat delivers the job's Payload.Text as a chat message through the
// deployment's channel. It is the default: an empty Kind means KindChat,
// so every job written before this field existed keeps delivering the
// way it did when it was created.
KindChat = "chat"
// KindWorkflow submits the job as a unit of work through the work-intake
// path, so the existing task lifecycle owns scheduling, retries, gates
// and PR opening.
KindWorkflow = "workflow"
)
// JobSpec is one scheduled job as the schedules document persists it, plus
// the bookkeeping fields the scheduler computes (NextRun, LastRun, Created,
// Updated). The wire shape is the public API; the scheduling.Job returned
// to the engine is a stripped subset.
type JobSpec struct {
// ID is the unique identifier. Required, non-empty, and the merge
// key for run recording.
ID string `json:"id"`
// Pool is "parallel" or "sequential". Empty means sequential — the
// safer default, matching Pool.resolve().
Pool string `json:"pool,omitempty"`
// Detail is a human-readable label rendered on events.
Detail string `json:"detail,omitempty"`
// Kind selects which delivery runner handles this job. Empty means
// KindChat, so a job written before this field existed keeps
// delivering as a chat message. This package does not interpret the
// value: an unrecognised kind is stored verbatim and refused by the
// delivery router, which is the one place the mapping from kind to
// runner lives.
Kind string `json:"kind,omitempty"`
// Schedule decides when this job is due. See schedule.go for the
// supported kinds and their arithmetic.
Schedule Schedule `json:"schedule"`
// Target and Payload describe where the job's output goes and what it
// carries. They live on the wire so jobs created ahead of the delivery
// runners that read them do not need a shape migration.
Target Target `json:"target"`
Payload Payload `json:"payload"`
// NextRun is the absolute moment the job will next fire. Computed
// at write time so the engine's hot path is a single comparison.
NextRun time.Time `json:"next_run"`
// LastRun is the last run time reported by the run-recording step,
// nil if the job has never run. The schedule does not know whether
// the run succeeded — that judgement belongs to the runner.
LastRun *time.Time `json:"last_run,omitempty"`
// Created and Updated are audit metadata written by the document's
// owner; the engine does not read them.
Created time.Time `json:"created"`
Updated time.Time `json:"updated"`
}
// Target identifies where the job's output goes. Empty Target is valid —
// the delivery runner chooses a default.
type Target struct {
// ChatID is the chat channel id (Telegram chat id, email address,
// webhook URL — depends on the channel). Empty defers to a default
// chosen by the delivery runner.
ChatID string `json:"chat_id,omitempty"`
}
// Payload is the content sent at run time.
type Payload struct {
// Text is the literal message body. Templating is a future concern
// and is deliberately absent here.
Text string `json:"text,omitempty"`
}
// Sentinels the schedule vocabulary and its callers match with errors.Is.
var (
// ErrJobNotFound is returned by a job source when the requested id
// is absent from its document.
ErrJobNotFound = errors.New("scheduling: job not found")
// ErrInvalidSpec is returned when a JobSpec or Schedule fails
// structural validation (empty id, unknown schedule kind,
// non-positive interval).
ErrInvalidSpec = errors.New("scheduling: invalid job spec")
// ErrScheduleUnsupported is returned when a job's schedule kind
// cannot compute its next run time. Interval arithmetic is fully
// implemented; cron and once return this so callers can surface a
// clear error rather than silently advancing.
ErrScheduleUnsupported = errors.New("scheduling: schedule kind not implemented")
)
@@ -1,9 +1,11 @@
package cronstore
package scheduling
import (
"encoding/json"
"fmt"
"time"
"github.com/pocketbase/pocketbase/tools/cron"
)
// Duration is a schedule interval as the document writes it: the human string
@@ -77,21 +79,22 @@ type Schedule struct {
Start time.Time `json:"start"`
}
// Schedule kinds the store recognises. Slice 1 fully implements
// Interval; Once is accepted at Create / fires at Due exactly once, but
// has no next-run arithmetic for MarkRun (one-shot by definition). Cron
// is parsed at Create but has no firing arithmetic yet — Slice 2 will
// add the parser. MarkRun on a kind this build does not implement
// returns ErrScheduleUnsupported rather than silently advancing next_run.
// Schedule kinds the store recognises. Interval and Cron have full
// next-run arithmetic; Once is accepted at Create / fires at Due exactly
// once, but has no next-run arithmetic for MarkRun (one-shot by
// definition). MarkRun on a kind this build does not implement returns
// ErrScheduleUnsupported rather than silently advancing next_run.
const (
// ScheduleInterval fires every Interval after the last successful run
// (or after Start, if never run). The simplest recurring schedule.
ScheduleInterval = "interval"
// ScheduleCron fires at the next moment matching a standard 5-field
// cron expression after the last successful run. Slice 2 will add
// the parser; until then, MarkRun on a cron job returns
// ErrScheduleUnsupported.
// ScheduleCron fires at the next moment matching a 5-field cron
// expression (or one of the @ macros) after the last successful
// run, computed in the zone of that run time. Expressions are parsed
// with pocketbase tools/cron, which ANDs day-of-month with day-of-week
// instead of Vixie cron's OR: "0 0 1 * 1" means only Monday the 1st,
// not "the 1st or any Monday".
ScheduleCron = "cron"
// ScheduleOnce fires once at the configured At time. Subsequent
@@ -110,8 +113,8 @@ func (s Schedule) Validate() error {
return fmt.Errorf("%w: interval must be positive, got %v", ErrInvalidSpec, s.Interval)
}
case ScheduleCron:
if s.Cron == "" {
return fmt.Errorf("%w: cron expression must be non-empty", ErrInvalidSpec)
if _, err := parseCronSpec(s.Cron); err != nil {
return err
}
case ScheduleOnce:
if s.At == nil {
@@ -136,8 +139,8 @@ func (s Schedule) resolve() Schedule {
// firstRun returns when the job should fire the very first time it has
// never run before. For Interval, it is Start + Interval (or now + Interval
// when Start is zero, so a freshly-created job with no Start still has a
// finite next-run). For Once, it is At. For Cron, it is left as a future
// responsibility — Slice 1 will not synthesise a value it cannot honour.
// finite next-run). For Once, it is At. For Cron, it is the first matching
// minute strictly after Start (or after now when Start is zero).
func (s Schedule) firstRun(now time.Time) (time.Time, error) {
switch s.resolve().Kind {
case ScheduleInterval:
@@ -151,12 +154,62 @@ func (s Schedule) firstRun(now time.Time) (time.Time, error) {
}
return *s.At, nil
case ScheduleCron:
return time.Time{}, fmt.Errorf("%w: cron parsing not yet implemented (slice 2)", ErrScheduleUnsupported)
sched, err := parseCronSpec(s.Cron)
if err != nil {
return time.Time{}, err
}
anchor := now
if !s.Start.IsZero() {
anchor = s.Start
}
return nextCronRun(sched, anchor)
default:
return time.Time{}, fmt.Errorf("%w: unknown kind %q", ErrScheduleUnsupported, s.Kind)
}
}
// parseCronSpec parses expr with pocketbase tools/cron — the project's
// chosen cron dialect: standard 5-field expressions plus the @yearly,
// @monthly, @weekly, @daily and @hourly macros — and wraps any parse
// failure in ErrInvalidSpec so callers need only the one sentinel.
func parseCronSpec(expr string) (*cron.Schedule, error) {
sched, err := cron.NewSchedule(expr)
if err != nil {
return nil, fmt.Errorf("%w: cron expression %q: %w", ErrInvalidSpec, expr, err)
}
return sched, nil
}
// cronHorizon bounds the next-run scan. A satisfiable expression always
// fires well within it (the sparsest realistic case, Feb 29, recurs every
// four years); only a spec that can never fire — e.g. "0 0 30 2 *", which
// the parser's field ranges cannot reject — exhausts the scan.
const cronHorizon = 5 * 365 * 24 * time.Hour
// nextCronRun returns the first wall-clock minute in the location of from
// at which sched is due, strictly after the minute containing from — the
// minute that just fired is never its own successor. Whole months and
// hours that cannot match are skipped, so even a sparse expression costs
// a handful of iterations per year scanned.
func nextCronRun(sched *cron.Schedule, from time.Time) (time.Time, error) {
t := from.Truncate(time.Minute).Add(time.Minute)
for limit := from.Add(cronHorizon); t.Before(limit); {
if sched.IsDue(cron.NewMoment(t)) {
return t, nil
}
if _, ok := sched.Months[int(t.Month())]; !ok {
t = time.Date(t.Year(), t.Month(), 1, 0, 0, 0, 0, t.Location()).AddDate(0, 1, 0)
continue
}
if _, ok := sched.Hours[t.Hour()]; !ok {
t = t.Add(time.Hour).Truncate(time.Hour)
continue
}
t = t.Add(time.Minute)
}
return time.Time{}, fmt.Errorf("%w: cron expression does not match any minute within the next five years", ErrInvalidSpec)
}
func (s Schedule) FirstRun(now time.Time) (time.Time, error) { return s.firstRun(now) }
func (s Schedule) NextRun(lastRun time.Time) (time.Time, error) { return s.nextRun(lastRun) }
@@ -165,9 +218,9 @@ func (s Schedule) Resolved() Schedule { return s.resolve() }
// nextRun returns when the job should fire after a successful run at
// lastRun. It is the engine-side counterpart to firstRun: every MarkRun
// advances next_run through this function. Interval is fully implemented;
// Once and Cron deliberately fail so a silently-wrong value cannot reach
// the ticker.
// advances next_run through this function. Interval and Cron are fully
// implemented; Once deliberately fails (there is no recurring next run)
// so a silently-wrong value cannot reach the ticker.
func (s Schedule) nextRun(lastRun time.Time) (time.Time, error) {
switch s.resolve().Kind {
case ScheduleInterval:
@@ -175,7 +228,11 @@ func (s Schedule) nextRun(lastRun time.Time) (time.Time, error) {
case ScheduleOnce:
return time.Time{}, fmt.Errorf("%w: once schedule has no next run", ErrScheduleUnsupported)
case ScheduleCron:
return time.Time{}, fmt.Errorf("%w: cron arithmetic not yet implemented (slice 2)", ErrScheduleUnsupported)
sched, err := parseCronSpec(s.Cron)
if err != nil {
return time.Time{}, err
}
return nextCronRun(sched, lastRun)
default:
return time.Time{}, fmt.Errorf("%w: unknown kind %q", ErrScheduleUnsupported, s.Kind)
}
+237
View File
@@ -0,0 +1,237 @@
package scheduling
import (
"errors"
"testing"
"time"
)
// testZone is a fixed +02:00 zone: explicit, DST-free, and distinct from UTC
// so any arithmetic that silently computes in UTC fails. Every schedule test
// builds its times here, making the suite independent of the host's TZ.
var testZone = time.FixedZone("TEST", 2*60*60)
// zoned builds a wall-clock time in the fixed test zone.
func zoned(year int, month time.Month, day, hour, minute, sec int) time.Time {
return time.Date(year, month, day, hour, minute, sec, 0, testZone)
}
func TestScheduleCronNextRun(t *testing.T) {
tests := []struct {
name string
expr string
lastRun time.Time
want time.Time
}{
{
name: "minute later in the same hour",
expr: "30 14 * * *",
lastRun: zoned(2026, 8, 24, 13, 0, 0),
want: zoned(2026, 8, 24, 14, 30, 0),
},
{
name: "hour boundary skips the remainder of the hour",
expr: "*/15 * * * *",
lastRun: zoned(2026, 8, 24, 12, 7, 0),
want: zoned(2026, 8, 24, 12, 15, 0),
},
{
// pins the pocketbase AND semantics: Vixie cron would give the
// next Monday (2026-08-31); tools/cron gives Monday the 1st.
name: "dual-restricted day-of-month and day-of-week AND, not Vixie OR",
expr: "0 0 1 * 1",
lastRun: zoned(2026, 8, 24, 12, 15, 30),
want: zoned(2027, 2, 1, 0, 0, 0),
},
{
name: "run exactly on a matching minute advances to the next one",
expr: "0 * * * *",
lastRun: zoned(2026, 8, 24, 12, 0, 0),
want: zoned(2026, 8, 24, 13, 0, 0),
},
{
name: "across the hour boundary",
expr: "5,35 * * * *",
lastRun: zoned(2026, 8, 24, 12, 36, 0),
want: zoned(2026, 8, 24, 13, 5, 0),
},
{
name: "range with step across hours",
expr: "0 9-17/2 * * *",
lastRun: zoned(2026, 8, 24, 10, 0, 0),
want: zoned(2026, 8, 24, 11, 0, 0),
},
{
name: "across midnight",
expr: "0 0 * * *",
lastRun: zoned(2026, 8, 24, 23, 59, 0),
want: zoned(2026, 8, 25, 0, 0, 0),
},
{
name: "across the month boundary",
expr: "0 0 1 * *",
lastRun: zoned(2026, 8, 31, 12, 0, 0),
want: zoned(2026, 9, 1, 0, 0, 0),
},
{
name: "weekday rolls to next Monday",
expr: "0 0 * * 1",
lastRun: zoned(2026, 8, 21, 10, 0, 0),
want: zoned(2026, 8, 24, 0, 0, 0),
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
s := Schedule{Kind: ScheduleCron, Cron: tc.expr}
got, err := s.NextRun(tc.lastRun)
if err != nil {
t.Fatalf("NextRun(%v): %v", tc.lastRun, err)
}
if !got.Equal(tc.want) {
t.Fatalf("NextRun(%v) = %v (%s), want %v (%s)",
tc.lastRun, got, got.Format("2006-01-02 15:04:05 MST"),
tc.want, tc.want.Format("2006-01-02 15:04:05 MST"))
}
})
}
}
func TestScheduleCronNextRunMacros(t *testing.T) {
// lastRun is Monday 2026-08-24 12:15:30 TEST.
lastRun := zoned(2026, 8, 24, 12, 15, 30)
tests := []struct {
name string
expr string
want time.Time
}{
{name: "@hourly", expr: "@hourly", want: zoned(2026, 8, 24, 13, 0, 0)},
{name: "@daily", expr: "@daily", want: zoned(2026, 8, 25, 0, 0, 0)},
{name: "@midnight", expr: "@midnight", want: zoned(2026, 8, 25, 0, 0, 0)},
{name: "@weekly", expr: "@weekly", want: zoned(2026, 8, 30, 0, 0, 0)},
{name: "@monthly", expr: "@monthly", want: zoned(2026, 9, 1, 0, 0, 0)},
{name: "@yearly", expr: "@yearly", want: zoned(2027, 1, 1, 0, 0, 0)},
{name: "@annually", expr: "@annually", want: zoned(2027, 1, 1, 0, 0, 0)},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
s := Schedule{Kind: ScheduleCron, Cron: tc.expr}
got, err := s.NextRun(lastRun)
if err != nil {
t.Fatalf("NextRun(%v): %v", lastRun, err)
}
if !got.Equal(tc.want) {
t.Fatalf("NextRun(%v) with %q = %v, want %v", lastRun, tc.expr, got, tc.want)
}
})
}
}
func TestScheduleCronComputesInThePassedZone(t *testing.T) {
s := Schedule{Kind: ScheduleCron, Cron: "0 6 * * *"}
// lastRun 05:30 in +02:00: 06:00 local is 04:00 UTC.
got, err := s.NextRun(zoned(2026, 8, 24, 5, 30, 0))
if err != nil {
t.Fatalf("NextRun: %v", err)
}
if want := time.Date(2026, 8, 24, 4, 0, 0, 0, time.UTC); !got.Equal(want) {
t.Fatalf("fixed-zone NextRun = %v (%s), want %v UTC", got, got.Format("15:04 MST"), want)
}
// The same wall-clock spec anchored in UTC must land on 06:00 UTC instead.
got, err = s.NextRun(time.Date(2026, 8, 24, 5, 30, 0, 0, time.UTC))
if err != nil {
t.Fatalf("NextRun: %v", err)
}
if want := time.Date(2026, 8, 24, 6, 0, 0, 0, time.UTC); !got.Equal(want) {
t.Fatalf("UTC NextRun = %v, want %v UTC", got, want)
}
}
func TestScheduleCronFirstRun(t *testing.T) {
t.Run("anchored on Start", func(t *testing.T) {
s := Schedule{
Kind: ScheduleCron,
Cron: "0 12 * * *",
Start: zoned(2026, 8, 24, 9, 0, 0),
}
want := zoned(2026, 8, 24, 12, 0, 0)
got, err := s.FirstRun(time.Date(2026, 8, 24, 6, 0, 0, 0, testZone))
if err != nil {
t.Fatalf("FirstRun: %v", err)
}
if !got.Equal(want) {
t.Fatalf("FirstRun = %v, want %v", got, want)
}
})
t.Run("zero Start anchors on now", func(t *testing.T) {
s := Schedule{Kind: ScheduleCron, Cron: "0 12 * * *"}
now := zoned(2026, 8, 24, 12, 30, 0)
want := zoned(2026, 8, 25, 12, 0, 0)
got, err := s.FirstRun(now)
if err != nil {
t.Fatalf("FirstRun: %v", err)
}
if !got.Equal(want) {
t.Fatalf("FirstRun(%v) = %v, want %v", now, got, want)
}
})
}
func TestScheduleCronValidate(t *testing.T) {
t.Run("empty cron is rejected", func(t *testing.T) {
err := Schedule{Kind: ScheduleCron}.Validate()
if !errors.Is(err, ErrInvalidSpec) {
t.Fatalf("Validate() = %v, want ErrInvalidSpec", err)
}
})
t.Run("valid expressions pass", func(t *testing.T) {
for _, expr := range []string{"*/15 * * * *", "0 9-17/2 * * *", "@daily", "@yearly"} {
if err := (Schedule{Kind: ScheduleCron, Cron: expr}).Validate(); err != nil {
t.Fatalf("Validate(%q) = %v, want nil", expr, err)
}
}
})
}
func TestScheduleCronInvalidExpressions(t *testing.T) {
tests := []struct {
name string
expr string
}{
{name: "minute out of range", expr: "61 * * * *"},
{name: "hour out of range", expr: "* 24 * * *"},
{name: "day of month out of range", expr: "0 0 32 * *"},
{name: "month out of range", expr: "0 0 * 13 *"},
{name: "day of week out of range", expr: "0 0 * * 7"},
{name: "too few fields", expr: "* * * *"},
{name: "too many fields", expr: "* * * * * *"},
{name: "non-numeric field", expr: "banana * * * *"},
{name: "zero step", expr: "*/0 * * * *"},
{name: "empty", expr: ""},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
s := Schedule{Kind: ScheduleCron, Cron: tc.expr}
if err := s.Validate(); !errors.Is(err, ErrInvalidSpec) {
t.Errorf("Validate() = %v, want ErrInvalidSpec", err)
}
_, err := s.FirstRun(zoned(2026, 8, 24, 12, 0, 0))
if !errors.Is(err, ErrInvalidSpec) {
t.Errorf("FirstRun err = %v, want ErrInvalidSpec", err)
}
_, err = s.NextRun(zoned(2026, 8, 24, 12, 0, 0))
if !errors.Is(err, ErrInvalidSpec) {
t.Errorf("NextRun err = %v, want ErrInvalidSpec", err)
}
})
}
}
+54
View File
@@ -0,0 +1,54 @@
package storecontract
import (
"fmt"
"strconv"
"strings"
"time"
)
// EventCursorLayout is the fixed-width UTC layout every persisted event's at
// column and every EventsSince cursor is built with. Fixed width is the whole
// point: time.RFC3339Nano trims trailing zeros, so a whole-second row
// ("...05Z") and a full-precision row ("...05.123456789Z") would compare out
// of chronological order as strings. With nine fractional digits always
// present, lexicographic order is chronological order. It mirrors edastore's
// capturedAtLayout, which exists for the same reason; the two stores now share
// this rule.
const EventCursorLayout = "2006-01-02T15:04:05.000000000Z"
// cursorSeparator joins the sort key and the row id inside a cursor. The sort
// key never contains it, so splitting on the last occurrence is unambiguous.
const cursorSeparator = "|"
// cursorIDWidth zero-pads the row id so the whole cursor is lexicographically
// sortable: the fixed-width sort key is followed by a fixed-width tie-break,
// and string comparison of two cursors is the total order (key, id).
const cursorIDWidth = 20
// EventCursor returns the opaque resume cursor for an event that happened at
// at with row id id. The cursor is a string so EventSource can carry it
// verbatim in its id: field and echo it back as the Last-Event-ID header; it
// is opaque to every consumer except EventsSince.
func EventCursor(at time.Time, id int64) string {
return at.UTC().Format(EventCursorLayout) + cursorSeparator + fmt.Sprintf("%0*d", cursorIDWidth, id)
}
// ParseEventCursor splits a cursor back into its sort key (the fixed-width at
// string) and row id. ok is false for an empty or malformed cursor, which
// means "from the beginning" -- the store must not treat a bad resume point as
// an error worth failing a feed over.
func ParseEventCursor(cursor string) (at string, id int64, ok bool) {
if cursor == "" {
return "", 0, false
}
i := strings.LastIndex(cursor, cursorSeparator)
if i < 0 {
return "", 0, false
}
id, err := strconv.ParseInt(cursor[i+1:], 10, 64)
if err != nil {
return "", 0, false
}
return cursor[:i], id, true
}
+15 -13
View File
@@ -80,7 +80,7 @@ type TaskQueries interface {
// TaskEvents groups observability and lifecycle methods.
type TaskEvents interface {
InsertEvent(ctx context.Context, e events.Event) (int64, error)
EventsSince(ctx context.Context, sinceID int64, limit int) ([]events.Event, error)
EventsSince(ctx context.Context, cursor string, limit int) ([]events.Event, error)
TaskEvents(ctx context.Context, taskID int64) ([]events.Event, error)
WorkflowStats(ctx context.Context) ([]WorkflowStat, error)
StageStats(ctx context.Context) ([]StageStat, error)
@@ -94,7 +94,7 @@ type TaskEvents interface {
// should not acquire the full task-lifecycle surface, mirroring why
// TaskArchiver is split out above. See docs/prds/event-capture-storage.md.
type CaptureStore interface {
InsertCapture(ctx context.Context, c CapturedEvent, retention time.Duration, maxEvents int) (int64, error)
InsertCapture(ctx context.Context, c CapturedEvent, retention time.Duration, maxEvents int) (string, error)
ListCaptures(ctx context.Context, limit int) ([]CapturedEvent, error)
}
@@ -142,11 +142,11 @@ type ApplyStatusStore interface {
// surface, not the full task or capture APIs. See
// docs/prds/payload-field-mapping.md.
type MappingStore interface {
InsertMapping(ctx context.Context, m mapping.Mapping) (int64, error)
GetMapping(ctx context.Context, id int64) (*mapping.Mapping, error)
InsertMapping(ctx context.Context, m mapping.Mapping) (string, error)
GetMapping(ctx context.Context, id string) (*mapping.Mapping, error)
ListMappings(ctx context.Context) ([]mapping.Mapping, error)
UpdateMapping(ctx context.Context, m mapping.Mapping) error
DeleteMapping(ctx context.Context, id int64) error
DeleteMapping(ctx context.Context, id string) error
}
// BindingStore persists playbook bindings (t2db.4 Phase B): CRUD and the
@@ -156,12 +156,12 @@ type MappingStore interface {
// BindingDispatcher; the daemon depends on both. See
// docs/prds/playbook-binding.md.
type BindingStore interface {
InsertBinding(ctx context.Context, b binding.Binding) (int64, error)
GetBinding(ctx context.Context, id int64) (*binding.Binding, error)
InsertBinding(ctx context.Context, b binding.Binding) (string, error)
GetBinding(ctx context.Context, id string) (*binding.Binding, error)
ListBindings(ctx context.Context) ([]binding.Binding, error)
UpdateBinding(ctx context.Context, b binding.Binding) error
DeleteBinding(ctx context.Context, id int64) error
ApproveBinding(ctx context.Context, id int64) error
DeleteBinding(ctx context.Context, id string) error
ApproveBinding(ctx context.Context, id string) error
}
// BindingDispatcher is the dispatch-loop surface over the bindings store:
@@ -174,9 +174,11 @@ type BindingDispatcher interface {
ArmedBindingsForSource(ctx context.Context, source string) ([]binding.Binding, error)
RecordDispatch(
ctx context.Context,
bindingID int64,
bindingID string,
bindingVersion int64,
captureID int64,
captureID string,
// taskID addresses the SQLite-owned task tables, which are not
// migrating: it stays an integer on purpose.
taskID int64,
) error
ListUndispatchedCaptures(ctx context.Context, sources []string, limit int) ([]CapturedEvent, error)
@@ -187,7 +189,7 @@ type BindingDispatcher interface {
// keeps the lifecycle surface narrow (8 methods, the interfacebloat limit)
// and keeps the binding-specific shape on the binding interfaces.
type BindingTaskCreator interface {
EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity string, bindingID int64, bindingVersion int) (*task.Task, error)
EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity, bindingID string, bindingVersion int) (*task.Task, error)
}
// PlaybookDispatcher is the idempotency-ledger surface for side-effecting
@@ -231,7 +233,7 @@ type TaskLogStore interface {
// no forge/task association -- just what arrived, from where, and when.
// See docs/prds/event-capture-storage.md.
type CapturedEvent struct {
ID int64 `json:"id"`
ID string `json:"id"`
ReceivedAt time.Time `json:"received_at"`
Source string `json:"source"`
RemoteAddr string `json:"remote_addr"`
@@ -0,0 +1,56 @@
package workflow
import (
"fmt"
"strings"
)
// RenderMarkdown renders an adversarial review report as the artifact source
// the collaborative editor publishes. It is a pure function over the report:
// no I/O, no infrastructure, so the rendered form is testable in isolation.
//
// A completed review renders its summary, every finding as its own subsection
// (file, line, verdict, level, failure scenario, and the mechanical suggestion
// and disposition when present) and the verified properties as a "checked
// clean" list. A review that did not run renders only its reason, because a
// report that never executed has no findings or checks to stand behind.
func (r ReviewReport) RenderMarkdown(title string) string {
var b strings.Builder
fmt.Fprintf(&b, "# %s\n\n", title)
if r.Status == ReviewStatusNotRun || (r.SkipReason != "" && len(r.Findings) == 0 && len(r.Checked) == 0) {
fmt.Fprintf(&b, "> This review did not run: %s\n", r.SkipReason)
return strings.TrimRight(b.String(), "\n") + "\n"
}
if r.Summary != "" {
b.WriteString(r.Summary)
b.WriteString("\n\n")
}
if len(r.Findings) > 0 {
b.WriteString("## Findings\n\n")
for _, f := range r.Findings {
fmt.Fprintf(&b, "### %s:%d — %s (%s)\n\n", f.File, f.Line, f.Level, f.Verdict)
fmt.Fprintf(&b, "**Defect:** %s\n\n", f.Defect)
if f.FailureScenario != "" {
fmt.Fprintf(&b, "**Failure scenario:** %s\n\n", f.FailureScenario)
}
if f.Suggestion != "" {
fmt.Fprintf(&b, "Suggested fix: %s\n\n", f.Suggestion)
}
if f.Disposition != "" {
fmt.Fprintf(&b, "Disposition: %s\n\n", f.Disposition)
}
}
}
if len(r.Checked) > 0 {
b.WriteString("## Checked clean\n\n")
for _, c := range r.Checked {
fmt.Fprintf(&b, "- %s — %s\n", c.Property, c.Evidence)
}
}
return strings.TrimRight(b.String(), "\n") + "\n"
}
@@ -0,0 +1,78 @@
package workflow
import (
"strings"
"testing"
)
func mdFixture() ReviewReport {
disposition := ReviewDisposition("fixed")
return ReviewReport{
Status: ReviewStatusCompleted,
Summary: "The nil-deref path is reachable from a malformed payload.",
Findings: []ReviewFinding{
{
File: "handler.go", Line: 42, Defect: "nil deref on empty payload",
FailureScenario: "POST with a body lacking the field crashes the handler.",
Suggestion: "guard the field before use", Verdict: ReviewVerdictConfirmed,
Level: ReviewLevelError, Category: "correctness", Disposition: disposition,
},
{
File: "store.go", Line: 9, Defect: "query loses the filter",
FailureScenario: "paged reads return rows from other scopes",
Verdict: ReviewVerdictPlausible, Level: ReviewLevelWarn,
},
},
Checked: []ReviewCheck{
{Property: "nil-safety of Foo callers", Evidence: "read all 4 callers; each nil-checks"},
},
}
}
func TestReviewReportRendersFindingsAsMarkdown(t *testing.T) {
got := mdFixture().RenderMarkdown("Review of acme/widget#42")
for _, want := range []string{
"# Review of acme/widget#42",
"The nil-deref path is reachable",
"### handler.go:42",
"**Defect:** nil deref on empty payload",
"**Failure scenario:** POST with a body lacking the field crashes the handler.",
"error (confirmed)",
"Suggested fix: guard the field before use",
"Disposition: fixed",
"### store.go:9",
"warn (plausible)",
"## Checked clean",
"- nil-safety of Foo callers — read all 4 callers; each nil-checks",
} {
if !strings.Contains(got, want) {
t.Fatalf("rendered report missing %q:\n%s", want, got)
}
}
}
func TestReviewReportRendersNotRunStatus(t *testing.T) {
report := NewNotRunReviewReport("no reviewer model configured")
got := report.RenderMarkdown("Review of acme/widget#42")
if !strings.Contains(got, "no reviewer model configured") {
t.Fatalf("skip reason missing:\n%s", got)
}
if strings.Contains(got, "## Findings") || strings.Contains(got, "## Checked clean") {
t.Fatalf("a not-run report must not render finding or check sections:\n%s", got)
}
}
func TestReviewReportRendersACleanReview(t *testing.T) {
report := ReviewReport{
Status: ReviewStatusCompleted,
Summary: "No defects found.",
Checked: []ReviewCheck{{Property: "p", Evidence: "e"}},
}
got := report.RenderMarkdown("Review")
if strings.Contains(got, "### ") {
t.Fatalf("a zero-finding review must not render finding sections:\n%s", got)
}
if !strings.Contains(got, "No defects found.") || !strings.Contains(got, "- p — e") {
t.Fatalf("summary and checks missing:\n%s", got)
}
}
+2 -2
View File
@@ -82,8 +82,8 @@ type Task struct {
// from a playbook binding dispatch (t2db.4 Phase B). They record
// provenance: which binding fired this task and at what version, so
// later edits to the binding cannot silently rewrite history.
BindingID int64 `json:"binding_id"`
BindingVersion int `json:"binding_version"`
BindingID string `json:"binding_id"`
BindingVersion int `json:"binding_version"`
// ReviewPayload is the JSON-encoded review unit (the forge review's
// actionable comments) the remediate workflow's current run must
// address. The daemon's reaction consumer injects it before queuing a
+9
View File
@@ -105,6 +105,15 @@ const (
// trigger. data: session, channel.
KindTurnCompleted = "turn_completed"
// EDA edit activity (bindings and mappings on the captures store). Each
// successful write on the State Store's PocketBase tables emits one
// event through the same task-DB path capture arrival uses, so a
// dashboard edit is visible on the activity stream without a manual
// refresh. data: id (the edited record), action (create, update,
// approve, delete).
KindBindingChanged = "binding_changed"
KindMappingChanged = "mapping_changed"
// KindUpdateReport carries the phase-2 outcome of a dashboard-initiated
// update -- whether the restarted daemon came back up healthy and on
// the version it claimed, relayed once on the boot that finds the
+3
View File
@@ -24,6 +24,9 @@ type PRReviewResult struct {
// Reason explains why a not_run/skipped review did not execute.
Reason string `json:"reason,omitempty"`
Findings []PRReviewFinding `json:"findings"`
// ArtifactURL is the editor deep link for the published review artifact,
// empty when artifact publishing is disabled or failed.
ArtifactURL string `json:"artifact_url,omitempty"`
}
// PRReviewFinding is one defect from an operator-triggered review.
-557
View File
@@ -1,557 +0,0 @@
// Package indexing provides language-neutral workspace text indexing.
package indexing
import (
"context"
"crypto/sha256"
"database/sql"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"log/slog"
"os"
"os/exec"
"path/filepath"
"regexp"
"regexp/syntax"
"slices"
"strings"
"time"
"github.com/google/codesearch/index"
"github.com/google/uuid"
_ "modernc.org/sqlite"
)
const (
// MaxCandidateFiles bounds helper output and grep argv construction.
MaxCandidateFiles = 2000
maxCandidateArgBytes = 24 * 1024
buildTimeout = 10 * time.Minute
metadataTimeout = 5 * time.Second
)
// CandidateProvider returns files which may match a grep query. A false
// handled result tells the caller to use its normal unindexed search path.
type CandidateProvider interface {
Candidates(context.Context, string, bool, bool) ([]string, bool)
}
// Manager owns the workspace index lifecycle. The codesearch implementation
// runs in a child process so upstream fatal exits and mmap lifetime cannot
// terminate or leak inside the interactive agent process.
type Manager struct {
root string
indexPath string
dbPath string
runner helperRunner
log *slog.Logger
}
// logger is nil-safe: a Manager may be constructed as a struct literal
// (the tests do), so defaulting at the point of use rather than in the
// constructor is what actually holds.
func (m *Manager) logger() *slog.Logger {
if m == nil || m.log == nil {
return slog.Default()
}
return m.log
}
type helperRunner interface {
Build(context.Context, string, string) error
Candidates(context.Context, string, string, bool, bool) ([]string, error)
}
type processRunner struct{}
// Config locates the index storage for a Manager.
//
// Archied resolves these in its config layer ([indexing] in config.toml,
// defaulted off work_dir by config.finalize) and passes them down. Defaulting
// deliberately does NOT live here so this package stays free of archie-core's
// config.
type Config struct {
// IndexDir holds the mmap-friendly codesearch sidecar files.
IndexDir string
// DBPath is the SQLite database holding index lifecycle metadata.
DBPath string
// Log receives index build failures. Without it a failed build is
// invisible: the manager degrades to unindexed search and the reason
// is only ever written to a database column nobody reads.
Log *slog.Logger
}
// NewManager prepares lifecycle storage and starts an asynchronous atomic
// refresh. An existing index remains available while the refresh runs.
func NewManager(ctx context.Context, cfg Config, root string) (*Manager, error) {
if strings.TrimSpace(cfg.IndexDir) == "" {
return nil, fmt.Errorf("indexing: IndexDir is required")
}
if strings.TrimSpace(cfg.DBPath) == "" {
return nil, fmt.Errorf("indexing: DBPath is required")
}
absRoot, err := filepath.Abs(root)
if err != nil {
return nil, fmt.Errorf("resolve workspace root: %w", err)
}
if err := os.MkdirAll(cfg.IndexDir, 0o700); err != nil {
return nil, fmt.Errorf("create workspace index directory: %w", err)
}
m := &Manager{
root: filepath.Clean(absRoot),
indexPath: filepath.Join(cfg.IndexDir, workspaceKey(absRoot)+".csearch"),
dbPath: cfg.DBPath,
runner: processRunner{},
log: cfg.Log,
}
if err := m.ensureSchema(ctx); err != nil {
return nil, err
}
m.refreshAsync(ctx)
return m, nil
}
// Candidates implements the tools.GrepIndex contract.
func (m *Manager) Candidates(ctx context.Context, pattern string, literal, caseSensitive bool) ([]string, bool) {
state, err := m.state(ctx)
if err != nil || state.IndexedAt.IsZero() {
return nil, false
}
if _, err := os.Stat(state.IndexPath); err != nil {
return nil, false
}
files, err := m.runner.Candidates(ctx, state.IndexPath, pattern, literal, caseSensitive)
if err != nil {
return nil, false
}
// The sidecar is a snapshot. Include files changed or created since the
// build began so indexed search cannot miss live workspace edits.
merged, ok := m.mergeCandidateFiles(ctx, files, state)
if !ok {
return nil, false
}
slices.Sort(merged)
return merged, true
}
// mergeCandidateFiles combines sidecar results with workspace files that
// were changed or created since the index was built.
func (m *Manager) mergeCandidateFiles(ctx context.Context, files []string, state indexState) ([]string, bool) {
workspaceFiles, err := WorkspaceFiles(ctx, m.root)
if err != nil {
return nil, false
}
changedFiles := changedGitFiles(ctx, m.root)
seen := make(map[string]struct{}, len(files))
merged := make([]string, 0, len(files))
argumentBytes := 0
add := func(path string) {
path, ok := confinedWorkspaceFile(m.root, path)
if !ok {
return
}
if _, ok := seen[path]; ok {
return
}
seen[path] = struct{}{}
merged = append(merged, path)
argumentBytes += len(path) + 1
}
for _, path := range files {
add(path)
}
for _, path := range workspaceFiles {
info, statErr := os.Stat(path)
_, changed := changedFiles[path]
if statErr == nil && (changed || !info.ModTime().Before(state.IndexedAt)) {
add(path)
}
}
if len(merged) == 0 || len(merged) > MaxCandidateFiles || argumentBytes > maxCandidateArgBytes {
return nil, false
}
return merged, true
}
func changedGitFiles(ctx context.Context, root string) map[string]struct{} {
changed := map[string]struct{}{}
commands := [][]string{
{"-C", root, "ls-files", "-m", "-o", "--exclude-standard", "-z"},
{"-C", root, "diff", "--cached", "--name-only", "--diff-filter=ACMR", "-z"},
}
for _, args := range commands {
out, err := exec.CommandContext(ctx, "git", args...).Output()
if err != nil {
return map[string]struct{}{}
}
for name := range strings.SplitSeq(string(out), "\x00") {
if name != "" {
path := filepath.Join(root, filepath.FromSlash(name))
if confined, ok := confinedWorkspaceFile(root, path); ok {
changed[confined] = struct{}{}
}
}
}
}
return changed
}
type indexState struct {
IndexPath string
IndexedAt time.Time
}
func (m *Manager) refreshAsync(parent context.Context) {
go func() {
ctx, cancel := context.WithTimeout(parent, buildTimeout)
defer cancel()
startedAt := time.Now().UTC()
buildID := uuid.NewString()
previous, _ := m.state(ctx)
claimed, err := m.claimBuild(ctx, startedAt, buildID)
if err != nil {
// Degrading to unindexed search is safe but slow, and until
// this is logged the only record of why is a database column
// with no reader.
m.logger().Warn("workspace index build not claimed", "root", m.root, "error", err)
return
}
if !claimed {
// Another process holds the build claim; not an error.
return
}
finalPath := m.indexPath + "." + buildID
tmp := finalPath + ".tmp"
_ = os.Remove(tmp)
if err := m.runner.Build(ctx, m.root, tmp); err != nil {
_ = os.Remove(tmp)
m.logger().Warn("workspace index build failed; falling back to unindexed search",
"root", m.root, "error", err)
m.finishBuild(parent, buildID, "error", time.Time{}, "", err.Error())
return
}
if err := os.Rename(tmp, finalPath); err != nil {
_ = os.Remove(tmp)
m.logger().Warn("workspace index could not be published",
"root", m.root, "error", err)
m.finishBuild(parent, buildID, "error", time.Time{}, "", err.Error())
return
}
if !m.finishBuild(parent, buildID, "ready", startedAt, finalPath, "") {
_ = os.Remove(finalPath)
m.logger().Warn("workspace index build superseded before publication", "root", m.root)
return
}
m.logger().Info("workspace index ready", "root", m.root, "index", finalPath)
if previous.IndexPath != "" && previous.IndexPath != finalPath &&
(previous.IndexPath == m.indexPath || strings.HasPrefix(previous.IndexPath, m.indexPath+".")) {
_ = os.Remove(previous.IndexPath)
}
}()
}
func (m *Manager) finishBuild(parent context.Context, buildID, status string, indexedAt time.Time, indexPath, lastError string) bool {
ctx, cancel := context.WithTimeout(context.WithoutCancel(parent), metadataTimeout)
defer cancel()
db, err := openIndexDB(m.dbPath)
if err != nil {
return false
}
// Each helper opens its own single-connection handle and the write has
// already been committed by the time this runs, so a Close error carries
// no recoverable information.
defer func() { _ = db.Close() }()
indexed := ""
if !indexedAt.IsZero() {
indexed = indexedAt.Format(time.RFC3339Nano)
}
result, err := db.ExecContext(ctx, `UPDATE workspace_indexes SET
index_path=CASE WHEN ? = 'ready' THEN ? ELSE index_path END,
status=?, indexed_at=CASE WHEN ? = 'ready' THEN ? ELSE indexed_at END,
last_error=?, updated_at=?, generation=generation + CASE WHEN ? = 'ready' THEN 1 ELSE 0 END
WHERE root=? AND build_id=?`, status, indexPath, status, status, indexed,
lastError, time.Now().UTC().Format(time.RFC3339Nano), status, m.root, buildID)
if err != nil {
return false
}
rows, err := result.RowsAffected()
return err == nil && rows == 1
}
func (m *Manager) claimBuild(ctx context.Context, startedAt time.Time, buildID string) (bool, error) {
db, err := openIndexDB(m.dbPath)
if err != nil {
return false, err
}
// Each helper opens its own single-connection handle and the write has
// already been committed by the time this runs, so a Close error carries
// no recoverable information.
defer func() { _ = db.Close() }()
cutoff := startedAt.Add(-buildTimeout).Format(time.RFC3339Nano)
result, err := db.ExecContext(ctx, `INSERT INTO workspace_indexes (root, index_path, status, indexed_at, build_started_at, build_id, last_error, updated_at)
VALUES (?, ?, 'building', '', ?, ?, '', ?)
ON CONFLICT(root) DO UPDATE SET status='building',
build_started_at=excluded.build_started_at, build_id=excluded.build_id, last_error='', updated_at=excluded.updated_at
WHERE workspace_indexes.status != 'building' OR workspace_indexes.updated_at < ?`,
m.root, m.indexPath, startedAt.Format(time.RFC3339Nano), buildID, startedAt.Format(time.RFC3339Nano), cutoff)
if err != nil {
return false, fmt.Errorf("claim workspace index build: %w", err)
}
rows, err := result.RowsAffected()
if err != nil {
return false, fmt.Errorf("inspect workspace index build claim: %w", err)
}
return rows > 0, nil
}
func (m *Manager) ensureSchema(ctx context.Context) error {
db, err := openIndexDB(m.dbPath)
if err != nil {
return err
}
// Each helper opens its own single-connection handle and the write has
// already been committed by the time this runs, so a Close error carries
// no recoverable information.
defer func() { _ = db.Close() }()
_, err = db.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS workspace_indexes (
root TEXT PRIMARY KEY,
index_path TEXT NOT NULL,
status TEXT NOT NULL,
format_version TEXT NOT NULL DEFAULT 'codesearch-v2',
generation INTEGER NOT NULL DEFAULT 0,
indexed_at TEXT NOT NULL DEFAULT '',
build_started_at TEXT NOT NULL DEFAULT '',
build_id TEXT NOT NULL DEFAULT '',
last_error TEXT NOT NULL DEFAULT '',
updated_at TEXT NOT NULL
)`)
if err != nil {
return fmt.Errorf("create workspace index metadata: %w", err)
}
for _, migration := range []string{
`ALTER TABLE workspace_indexes ADD COLUMN format_version TEXT NOT NULL DEFAULT 'codesearch-v2'`,
`ALTER TABLE workspace_indexes ADD COLUMN generation INTEGER NOT NULL DEFAULT 0`,
`ALTER TABLE workspace_indexes ADD COLUMN build_started_at TEXT NOT NULL DEFAULT ''`,
`ALTER TABLE workspace_indexes ADD COLUMN build_id TEXT NOT NULL DEFAULT ''`,
} {
if _, migrationErr := db.ExecContext(ctx, migration); migrationErr != nil && !strings.Contains(migrationErr.Error(), "duplicate column name") {
return fmt.Errorf("migrate workspace index metadata: %w", migrationErr)
}
}
return nil
}
func (m *Manager) state(ctx context.Context) (indexState, error) {
db, err := openIndexDB(m.dbPath)
if err != nil {
return indexState{}, err
}
// Each helper opens its own single-connection handle and the write has
// already been committed by the time this runs, so a Close error carries
// no recoverable information.
defer func() { _ = db.Close() }()
var indexed, indexPath string
err = db.QueryRowContext(ctx, `SELECT index_path, indexed_at FROM workspace_indexes WHERE root = ?`, m.root).Scan(&indexPath, &indexed)
if errors.Is(err, sql.ErrNoRows) {
return indexState{}, nil
}
if err != nil {
return indexState{}, fmt.Errorf("read workspace index metadata: %w", err)
}
at, _ := time.Parse(time.RFC3339Nano, indexed)
return indexState{IndexPath: indexPath, IndexedAt: at}, nil
}
func openIndexDB(path string) (*sql.DB, error) {
// modernc.org/sqlite only applies pragmas passed via repeated _pragma
// params; unrecognized keys like _journal_mode/_busy_timeout are
// silently ignored, which previously left the busy timeout at 0 and
// caused SQLITE_BUSY on any lock contention instead of a bounded wait.
db, err := sql.Open("sqlite", path+"?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)")
if err != nil {
return nil, fmt.Errorf("open workspace index metadata: %w", err)
}
db.SetMaxOpenConns(1)
return db, nil
}
func workspaceKey(root string) string {
sum := sha256.Sum256([]byte(filepath.Clean(root)))
return hex.EncodeToString(sum[:12])
}
func (processRunner) Build(ctx context.Context, root, target string) error {
_, err := runHelper(ctx, "build", "--root", root, "--index", target)
return err
}
func (processRunner) Candidates(ctx context.Context, indexPath, pattern string, literal, caseSensitive bool) ([]string, error) {
args := []string{"candidates", "--index", indexPath, "--pattern", pattern}
if literal {
args = append(args, "--literal")
}
if caseSensitive {
args = append(args, "--case-sensitive")
}
out, err := runHelper(ctx, args...)
if err != nil {
return nil, err
}
var files []string
if err := json.Unmarshal(out, &files); err != nil {
return nil, fmt.Errorf("decode codesearch candidates: %w", err)
}
return files, nil
}
func runHelper(ctx context.Context, args ...string) ([]byte, error) {
executable, err := os.Executable()
if err != nil {
return nil, fmt.Errorf("locate archied executable: %w", err)
}
cmd := exec.CommandContext(ctx, executable, append([]string{"workspace-codesearch"}, args...)...)
out, err := cmd.Output()
if err != nil {
if exitErr, ok := errors.AsType[*exec.ExitError](err); ok {
return nil, fmt.Errorf("codesearch helper: %s", strings.TrimSpace(string(exitErr.Stderr)))
}
return nil, fmt.Errorf("run codesearch helper: %w", err)
}
return out, nil
}
// BuildIndex creates a codesearch sidecar for root. It is called only inside
// the isolated helper process.
func BuildIndex(ctx context.Context, root, target string) error {
files, err := WorkspaceFiles(ctx, root)
if err != nil {
return err
}
w := index.Create(target)
w.AddRoots([]index.Path{index.MakePath(root)})
for _, path := range files {
if err := w.AddFile(path); err != nil {
continue
}
}
w.Flush()
return nil
}
// IndexCandidates returns the conservative trigram candidate set. It is
// called only inside the isolated helper process.
func IndexCandidates(indexPath, pattern string, literal, caseSensitive bool) (files []string, err error) {
defer func() {
if recovered := recover(); recovered != nil {
files = nil
err = fmt.Errorf("open codesearch index: %v", recovered)
}
}()
if literal {
pattern = regexp.QuoteMeta(pattern)
}
if !caseSensitive && !hasUppercase(pattern) {
pattern = "(?i:" + pattern + ")"
}
re, err := syntax.Parse(pattern, syntax.Perl)
if err != nil {
return nil, fmt.Errorf("parse search pattern: %w", err)
}
ix := index.Open(indexPath)
ids := ix.PostingQuery(index.RegexpQuery(re))
files = make([]string, 0, len(ids))
for _, id := range ids {
files = append(files, ix.Name(id).String())
}
return files, nil
}
// WorkspaceFiles lists searchable workspace files, preferring Git's ignore
// semantics and falling back to a conservative filesystem walk.
func WorkspaceFiles(ctx context.Context, root string) ([]string, error) {
cmd := exec.CommandContext(ctx, "git", "-C", root, "ls-files", "-co", "--exclude-standard", "-z")
out, err := cmd.Output()
if err == nil {
parts := strings.Split(string(out), "\x00")
files := make([]string, 0, len(parts))
for _, part := range parts {
if part != "" {
path := filepath.Join(root, filepath.FromSlash(part))
if confined, ok := confinedWorkspaceFile(root, path); ok {
files = append(files, confined)
}
}
}
slices.Sort(files)
return files, nil
}
var files []string
err = filepath.WalkDir(root, func(path string, entry os.DirEntry, walkErr error) error {
if walkErr != nil {
return nil //nolint:nilerr // an unreadable entry skips that path and continues the walk; a best-effort index must not abort on one bad file
}
if path != root && strings.HasPrefix(entry.Name(), ".") {
if entry.IsDir() {
return filepath.SkipDir
}
return nil
}
if !entry.IsDir() && entry.Type().IsRegular() {
if confined, ok := confinedWorkspaceFile(root, path); ok {
files = append(files, confined)
}
}
return nil
})
if err != nil {
return nil, fmt.Errorf("walk workspace files: %w", err)
}
slices.Sort(files)
return files, nil
}
func confinedWorkspaceFile(root, path string) (string, bool) {
absRoot, err := filepath.Abs(root)
if err != nil {
return "", false
}
absPath, err := filepath.Abs(path)
if err != nil {
return "", false
}
// Reject a symlink at the final component before resolving parent links.
info, err := os.Lstat(absPath)
if err != nil || !info.Mode().IsRegular() {
return "", false
}
resolvedRoot, err := filepath.EvalSymlinks(absRoot)
if err != nil {
return "", false
}
resolvedPath, err := filepath.EvalSymlinks(absPath)
if err != nil {
return "", false
}
rel, err := filepath.Rel(resolvedRoot, resolvedPath)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return "", false
}
resolvedInfo, err := os.Lstat(resolvedPath)
if err != nil || !resolvedInfo.Mode().IsRegular() {
return "", false
}
return filepath.Clean(resolvedPath), true
}
func hasUppercase(value string) bool {
for _, r := range value {
if r >= 'A' && r <= 'Z' {
return true
}
}
return false
}
-224
View File
@@ -1,224 +0,0 @@
package indexing
import (
"context"
"os"
"os/exec"
"path/filepath"
"slices"
"testing"
"time"
)
func TestBuildIndexAndCandidates(t *testing.T) {
root := t.TempDir()
writeTestFile(t, root, "a.txt", "alpha needle omega\n")
writeTestFile(t, root, "b.txt", "nothing here\n")
indexPath := filepath.Join(t.TempDir(), "workspace.csearch")
if err := BuildIndex(context.Background(), root, indexPath); err != nil {
t.Fatalf("BuildIndex() error = %v", err)
}
files, err := IndexCandidates(indexPath, "needle", false, false)
if err != nil {
t.Fatalf("IndexCandidates() error = %v", err)
}
if len(files) != 1 || files[0] != filepath.Join(root, "a.txt") {
t.Fatalf("candidates = %v", files)
}
}
func TestWorkspaceFilesRespectsGitIgnore(t *testing.T) {
root := t.TempDir()
runGit(t, root, "init", "-q")
writeTestFile(t, root, ".gitignore", "ignored.txt\n")
writeTestFile(t, root, "tracked.txt", "tracked\n")
writeTestFile(t, root, "untracked.txt", "untracked\n")
writeTestFile(t, root, "ignored.txt", "ignored\n")
runGit(t, root, "add", ".gitignore", "tracked.txt")
files, err := WorkspaceFiles(context.Background(), root)
if err != nil {
t.Fatalf("WorkspaceFiles() error = %v", err)
}
if slices.Contains(files, filepath.Join(root, "ignored.txt")) {
t.Fatalf("ignored file returned: %v", files)
}
for _, name := range []string{"tracked.txt", "untracked.txt"} {
if !slices.Contains(files, filepath.Join(root, name)) {
t.Fatalf("%s missing from %v", name, files)
}
}
}
func TestWorkspaceFilesRejectsTrackedSymlinkOutsideRoot(t *testing.T) {
root := t.TempDir()
runGit(t, root, "init", "-q")
outside := filepath.Join(t.TempDir(), "secret.txt")
if err := os.WriteFile(outside, []byte("secret"), 0o600); err != nil {
t.Fatal(err)
}
link := filepath.Join(root, "link.txt")
if err := os.Symlink(outside, link); err != nil {
t.Fatal(err)
}
runGit(t, root, "add", "link.txt")
files, err := WorkspaceFiles(context.Background(), root)
if err != nil {
t.Fatal(err)
}
_, accepted := confinedWorkspaceFile(root, link)
if slices.Contains(files, link) || accepted {
t.Fatalf("outside symlink accepted: %v", files)
}
}
func TestConfinedWorkspaceFileRejectsIntermediateSymlinkEscape(t *testing.T) {
root := t.TempDir()
outside := t.TempDir()
writeTestFile(t, outside, "secret.txt", "secret")
if err := os.Symlink(outside, filepath.Join(root, "linkdir")); err != nil {
t.Fatal(err)
}
if path, ok := confinedWorkspaceFile(root, filepath.Join(root, "linkdir", "secret.txt")); ok {
t.Fatalf("intermediate symlink escape accepted as %s", path)
}
}
func TestManagerRefreshBuildsAtomicallyAndRecordsGeneration(t *testing.T) {
root := t.TempDir()
dir := t.TempDir()
m := &Manager{
root: root, indexPath: filepath.Join(dir, "workspace.csearch"),
dbPath: filepath.Join(dir, "indexes.db"), runner: writingRunner{},
}
if err := m.ensureSchema(context.Background()); err != nil {
t.Fatal(err)
}
ctx := t.Context()
m.refreshAsync(ctx)
// Poll for the async build to complete. Use a deadline longer than SQLite's
// busy timeout (5 s) so a contended CI runner doesn't trigger a false timeout.
deadline := time.Now().Add(8 * time.Second)
var installedPath string
for {
state, stateErr := m.state(context.Background())
if stateErr == nil && !state.IndexedAt.IsZero() {
if _, err := os.Stat(state.IndexPath); err == nil {
installedPath = state.IndexPath
break
}
}
// If the goroutine errored, fail fast with the last error.
if stateErr == nil && state.IndexedAt.IsZero() {
// Check if status indicates an error.
db, dbErr := openIndexDB(m.dbPath)
if dbErr == nil {
var status, lastErr string
_ = db.QueryRowContext(context.Background(),
`SELECT status, last_error FROM workspace_indexes WHERE root = ?`, m.root).Scan(&status, &lastErr)
_ = db.Close()
if status == "error" {
t.Fatalf("build failed: %s", lastErr)
}
}
}
if time.Now().After(deadline) {
t.Fatal("index was not atomically installed")
}
time.Sleep(10 * time.Millisecond)
}
if installedPath == m.indexPath {
t.Fatalf("build did not install a generation-specific sidecar: %s", installedPath)
}
db, err := openIndexDB(m.dbPath)
if err != nil {
t.Fatal(err)
}
defer func() { _ = db.Close() }()
var status string
var generation int
if err := db.QueryRowContext(context.Background(), `SELECT status, generation FROM workspace_indexes WHERE root = ?`, root).Scan(&status, &generation); err != nil {
t.Fatal(err)
}
if status != "ready" || generation != 1 {
t.Fatalf("state = %q generation %d", status, generation)
}
}
func TestStaleBuilderCannotCompleteNewerLease(t *testing.T) {
dir := t.TempDir()
m := &Manager{root: t.TempDir(), indexPath: filepath.Join(dir, "workspace.csearch"), dbPath: filepath.Join(dir, "indexes.db")}
if err := m.ensureSchema(context.Background()); err != nil {
t.Fatal(err)
}
oldStart := time.Now().UTC().Add(-2 * buildTimeout)
claimed, err := m.claimBuild(context.Background(), oldStart, "build-a")
if err != nil || !claimed {
t.Fatalf("first claim = %v, %v", claimed, err)
}
newStart := time.Now().UTC()
claimed, err = m.claimBuild(context.Background(), newStart, "build-b")
if err != nil || !claimed {
t.Fatalf("replacement claim = %v, %v", claimed, err)
}
if m.finishBuild(context.Background(), "build-a", "ready", oldStart, filepath.Join(dir, "a"), "") {
t.Fatal("stale builder completed replacement lease")
}
pathB := filepath.Join(dir, "b")
if !m.finishBuild(context.Background(), "build-b", "ready", newStart, pathB, "") {
t.Fatal("current builder could not complete lease")
}
state, err := m.state(context.Background())
if err != nil || state.IndexPath != pathB {
t.Fatalf("state = %#v, %v", state, err)
}
}
type writingRunner struct{}
func (writingRunner) Build(_ context.Context, _, target string) error {
return os.WriteFile(target, []byte("index"), 0o600)
}
func (writingRunner) Candidates(context.Context, string, string, bool, bool) ([]string, error) {
return nil, nil
}
func writeTestFile(t *testing.T, dir, name, content string) {
t.Helper()
if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o600); err != nil {
t.Fatal(err)
}
}
func runGit(t *testing.T, dir string, args ...string) {
t.Helper()
cmd := exec.CommandContext(context.Background(), "git", append([]string{"-C", dir}, args...)...)
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("git %v: %v: %s", args, err, out)
}
}
// A Manager built as a struct literal has no logger. Logging on the build
// path must not panic, since that path runs in a background goroutine where
// a panic takes the whole daemon down.
func TestManagerLoggerIsNilSafe(t *testing.T) {
tests := []struct {
name string
m *Manager
}{
{name: "zero manager", m: &Manager{}},
{name: "nil manager", m: nil},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := tc.m.logger(); got == nil {
t.Fatal("logger() returned nil")
}
tc.m.logger().Warn("must not panic")
})
}
}
@@ -0,0 +1,181 @@
// Package artifactsync publishes artifacts to the offloaded.dev collaborative
// editor: archie-core generates and sends, the editor hosts, renders and
// shares. It is infrastructure in the Courier posture — one method, injected
// by the composition root, absent when configuration does not resolve.
package artifactsync
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"time"
)
const contractFormat = "offloaded-collab-artifact"
// Artifact is one publishable artifact: the rendered source plus the metadata
// the receiving contract carries.
type Artifact struct {
Title string
Source string // CommonMark for prose, raw source for runnable formats
Meta Meta
}
// Meta is the artifact block of the receiving contract: shape, attribution and
// idempotency fields. Origin is named "source" on the wire; the top-level
// Source field carries the content.
type Meta struct {
Format string `json:"format"`
Category string `json:"category"`
Entry string `json:"entry,omitempty"` // required for html/react/vue, one file per artifact
Visibility string `json:"visibility"`
Egress string `json:"egress"`
Actor string `json:"actor"`
Origin string `json:"source"` // the producing subsystem, e.g. "pr-review"
RequestID string `json:"requestId"`
Version string `json:"version,omitempty"`
SentAt time.Time `json:"sentAt"`
Task *Task `json:"task,omitempty"`
}
// Task carries the forge coordinates the artifact was produced for.
type Task struct {
Owner string `json:"owner"`
Repo string `json:"repo"`
Number int `json:"number"`
Task string `json:"task,omitempty"`
}
// Sender publishes one artifact and returns the editor deep link for it.
type Sender interface {
Publish(ctx context.Context, a Artifact) (string, error)
}
// Client is the Sender against a deployed editor.
type Client struct {
baseURL string
token string
client *http.Client
// wait is the base delay between 503 retries, doubled per attempt.
wait time.Duration
}
// NewClient builds a Client against the editor origin. It returns nil when
// baseURL is empty, which is the disabled posture: the composition root skips
// wiring the sender entirely rather than carrying one that cannot be called.
func NewClient(baseURL, token string) *Client {
if baseURL == "" {
return nil
}
return &Client{
baseURL: strings.TrimRight(baseURL, "/"),
token: token,
client: &http.Client{Timeout: 20 * time.Second},
wait: 250 * time.Millisecond,
}
}
// publishBody is the frozen receiving contract. Its shape is the public API;
// changing it is a contract change with the editor, not a refactor.
type publishBody struct {
Format string `json:"format"`
Title string `json:"title"`
Source string `json:"source"`
Artifact Meta `json:"artifact"`
}
type publishResponse struct {
OK bool `json:"ok"`
ID string `json:"id"`
URL string `json:"url"`
}
type errorResponse struct {
Error string `json:"error"`
}
// Publish posts the artifact once. The full source is always sent: versions of
// record are full snapshots stored per publish, never diffs.
func (c *Client) Publish(ctx context.Context, a Artifact) (string, error) {
if c == nil {
return "", fmt.Errorf("artifactsync: sender is not configured")
}
body := publishBody{Format: contractFormat, Title: a.Title, Source: a.Source, Artifact: a.Meta}
payload, err := json.Marshal(body)
if err != nil {
return "", fmt.Errorf("artifactsync: encode artifact: %w", err)
}
const attempts = 4 // the initial attempt plus three retries
var lastErr error
for attempt := range attempts {
if attempt > 0 {
select {
case <-ctx.Done():
return "", fmt.Errorf("artifactsync: publish %q: %w", a.RequestID(), ctx.Err())
case <-time.After(c.wait << (attempt - 1)):
}
}
url, retryable, err := c.postOnce(ctx, payload)
if err == nil {
return url, nil
}
lastErr = err
if !retryable {
return "", err
}
}
return "", fmt.Errorf("artifactsync: publish %q: %w", a.RequestID(), lastErr)
}
// RequestID is the artifact's own idempotency key, for logging and for callers
// that compose one from the producer's natural key.
func (a Artifact) RequestID() string { return a.Meta.RequestID }
// postOnce performs one HTTP attempt. retryable marks failures worth another
// attempt: 503 and transport errors. Client errors are configuration or shape
// mistakes and are surfaced immediately with the field reason.
func (c *Client) postOnce(ctx context.Context, payload []byte) (url string, retryable bool, err error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+"/api/artifacts", bytes.NewReader(payload))
if err != nil {
return "", false, fmt.Errorf("artifactsync: build request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+c.token)
resp, err := c.client.Do(req)
if err != nil {
return "", true, fmt.Errorf("artifactsync: post artifact: %w", err)
}
defer resp.Body.Close()
switch resp.StatusCode {
case http.StatusCreated:
var out publishResponse
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
return "", false, fmt.Errorf("artifactsync: decode publish response: %w", err)
}
if out.URL == "" {
return "", false, fmt.Errorf("artifactsync: publish response carries no url")
}
return out.URL, false, nil
case http.StatusServiceUnavailable:
return "", true, fmt.Errorf("artifactsync: editor store unavailable (503)")
case http.StatusUnauthorized:
return "", false, fmt.Errorf("artifactsync: editor rejected the service token (401); check [artifacts].token")
case http.StatusBadRequest:
reason := "unknown"
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 4096))
var e errorResponse
if json.Unmarshal(raw, &e) == nil && e.Error != "" {
reason = e.Error
}
return "", false, fmt.Errorf("artifactsync: editor rejected the artifact (400): %s", reason)
default:
return "", true, fmt.Errorf("artifactsync: unexpected publish status %d", resp.StatusCode)
}
}
@@ -0,0 +1,159 @@
package artifactsync
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"time"
)
func sampleArtifact() Artifact {
return Artifact{
Title: "Review of PR #7",
Source: "# Review\n\none finding, one check",
Meta: Meta{
Format: "markdown",
Category: "pr-review",
Visibility: "account-only",
Egress: "none",
Actor: "system:pr-reviewer",
Origin: "pr-review",
RequestID: "artifact:pr-review:acme/widget#7",
Version: "r1",
SentAt: time.Date(2026, 9, 22, 16, 45, 0, 0, time.UTC),
Task: &Task{Owner: "acme", Repo: "widget", Number: 7},
},
}
}
func TestPublishPostsTheContractBody(t *testing.T) {
var gotPath, gotAuth string
var gotBody map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath, gotAuth = r.URL.Path, r.Header.Get("Authorization")
body, _ := io.ReadAll(r.Body)
_ = json.Unmarshal(body, &gotBody)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusCreated)
_, _ = w.Write([]byte(`{"ok":true,"id":"a_1","url":"https://offloaded.dev/collab?document=a_1"}`))
}))
defer srv.Close()
c := NewClient(srv.URL, "secret-token")
url, err := c.Publish(context.Background(), sampleArtifact())
if err != nil {
t.Fatalf("Publish: %v", err)
}
if url != "https://offloaded.dev/collab?document=a_1" {
t.Fatalf("url = %q, want the editor deep link", url)
}
if gotPath != "/api/artifacts" {
t.Fatalf("path = %q, want /api/artifacts", gotPath)
}
if gotAuth != "Bearer secret-token" {
t.Fatalf("Authorization = %q, want the bearer token", gotAuth)
}
if gotBody["format"] != "offloaded-collab-artifact" {
t.Fatalf("top-level format = %v, want the contract envelope", gotBody["format"])
}
if gotBody["title"] != "Review of PR #7" || gotBody["source"] != sampleArtifact().Source {
t.Fatalf("title/source did not round-trip: %v", gotBody)
}
artifact, ok := gotBody["artifact"].(map[string]any)
if !ok {
t.Fatalf("artifact block missing: %v", gotBody)
}
if artifact["format"] != "markdown" || artifact["category"] != "pr-review" {
t.Fatalf("artifact format/category = %v", artifact)
}
if artifact["actor"] != "system:pr-reviewer" || artifact["source"] != "pr-review" {
t.Fatalf("attribution = %v", artifact)
}
if artifact["requestId"] != "artifact:pr-review:acme/widget#7" || artifact["version"] != "r1" {
t.Fatalf("idempotency fields = %v", artifact)
}
if artifact["visibility"] != "account-only" || artifact["egress"] != "none" {
t.Fatalf("visibility/egress = %v", artifact)
}
task, ok := artifact["task"].(map[string]any)
if !ok || task["owner"] != "acme" || task["repo"] != "widget" {
t.Fatalf("task coordinates = %v", artifact["task"])
}
}
func TestPublishDoesNotRetryAClientError(t *testing.T) {
for _, status := range []int{http.StatusUnauthorized, http.StatusBadRequest} {
var attempts atomic.Int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
attempts.Add(1)
w.WriteHeader(status)
_, _ = w.Write([]byte(`{"error":"source is required"}`))
}))
_, err := NewClient(srv.URL, "tok").Publish(context.Background(), sampleArtifact())
srv.Close()
if err == nil {
t.Fatalf("status %d: Publish error = nil, want failure", status)
}
if attempts.Load() != 1 {
t.Fatalf("status %d attempts = %d, want exactly one attempt", status, attempts.Load())
}
}
}
func TestPublishSurfacesTheFieldReasonOn400(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusBadRequest)
_, _ = w.Write([]byte(`{"error":"source is required"}`))
}))
defer srv.Close()
_, err := NewClient(srv.URL, "tok").Publish(context.Background(), sampleArtifact())
if err == nil || !strings.Contains(err.Error(), "source is required") {
t.Fatalf("error = %v, want it to carry the field reason", err)
}
}
func TestPublishRetries503UntilSuccess(t *testing.T) {
var attempts atomic.Int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if attempts.Add(1) < 3 {
w.WriteHeader(http.StatusServiceUnavailable)
return
}
w.WriteHeader(http.StatusCreated)
_, _ = w.Write([]byte(`{"ok":true,"id":"a_2","url":"u"}`))
}))
defer srv.Close()
c := NewClient(srv.URL, "tok")
c.wait = time.Millisecond
url, err := c.Publish(context.Background(), sampleArtifact())
if err != nil || url != "u" {
t.Fatalf("Publish = (%q, %v), want success after two 503s", url, err)
}
if attempts.Load() != 3 {
t.Fatalf("attempts = %d, want 3", attempts.Load())
}
}
func TestPublishFailsWhen503Persists(t *testing.T) {
var attempts atomic.Int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
attempts.Add(1)
w.WriteHeader(http.StatusServiceUnavailable)
}))
defer srv.Close()
c := NewClient(srv.URL, "tok")
c.wait = time.Millisecond
if _, err := c.Publish(context.Background(), sampleArtifact()); err == nil {
t.Fatal("Publish error = nil, want failure after exhausting retries")
}
if attempts.Load() != 4 {
t.Fatalf("attempts = %d, want the initial attempt plus three retries", attempts.Load())
}
}
@@ -9,13 +9,13 @@ import (
"log/slog"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/domain/binding"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/infrastructure/edastore"
"github.com/samcharles93/archie-core/internal/store"
"github.com/samcharles93/archie-core/internal/webhookguard"
)
@@ -37,7 +37,9 @@ func (r *recordingPublisher) Publish(_ context.Context, e events.Event) {
// wrote.
type testReceiver struct {
*Receiver
store *store.Store
// store is the event-capture store the receiver writes through, kept so a
// test asserts on persisted rows rather than on a mock's recollection.
store *edastore.Store
}
// Handler serves the receiver's route the way its host process mounts it,
@@ -58,11 +60,7 @@ func captureTestServerWithoutStore(t *testing.T) testReceiver {
func captureTestServer(t *testing.T) testReceiver {
t.Helper()
s, err := store.Open(t.Context(), filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = s.Close() })
s := edastore.OpenTest(t)
return testReceiver{
Receiver: &Receiver{
Log: slog.New(slog.DiscardHandler),
@@ -125,9 +123,9 @@ func (s *stubBindingStore) ApproveBinding(_ context.Context, _ int64) error {
func (s *stubBindingStore) RecordDispatch(
_ context.Context,
_ string,
_ int64,
_ int64,
_ int64,
_ string,
_ int64,
) error {
return nil
@@ -148,10 +146,10 @@ func captureBindingServer(t *testing.T, secret, source string) testReceiver {
srv.Bindings = &stubBindingStore{
armedBySource: map[string][]binding.Binding{
source: {{
ID: 1,
ID: "rbind0000000001",
Name: "test",
Matcher: binding.Matcher{Source: source},
MappingID: 1,
MappingID: "rmap00000000001",
Workflow: "implement",
Version: 1,
Status: binding.StatusArmed,
@@ -425,8 +423,8 @@ type captureInsertErrorStore struct {
store.CaptureStore
}
func (captureInsertErrorStore) InsertCapture(context.Context, store.CapturedEvent, time.Duration, int) (int64, error) {
return 0, errors.New("boom")
func (captureInsertErrorStore) InsertCapture(context.Context, store.CapturedEvent, time.Duration, int) (string, error) {
return "", errors.New("boom")
}
func TestHandleCaptureStorageFailureIs500(t *testing.T) {
@@ -543,8 +541,8 @@ func TestHandleCaptureWithMultipleArmedBindingsReturns409(t *testing.T) {
srv.Bindings = &stubBindingStore{
armedBySource: map[string][]binding.Binding{
"github": {
{ID: 1, Matcher: binding.Matcher{Source: "github"}, Status: binding.StatusArmed, Secret: "abcdefghijklmnop"},
{ID: 2, Matcher: binding.Matcher{Source: "github"}, Status: binding.StatusArmed, Secret: "qrstuvwxyz123456"},
{ID: "rbind0000000001", Matcher: binding.Matcher{Source: "github"}, Status: binding.StatusArmed, Secret: "abcdefghijklmnop"},
{ID: "rbind0000000002", Matcher: binding.Matcher{Source: "github"}, Status: binding.StatusArmed, Secret: "qrstuvwxyz123456"},
},
},
}
@@ -0,0 +1,52 @@
package configuration
import (
"os"
"path/filepath"
"testing"
"github.com/samcharles93/archie-core/internal/config"
)
func TestArtifactsConfigDefaults(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.toml")
contents := "bot_user = \"widget\"\n" +
"[forge]\ntype = \"gitea\"\nhost = \"https://git.example.test\"\ntoken_env = \"T\"\n"
if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
t.Fatal(err)
}
cfg, err := loadFile(path)
if err != nil {
t.Fatal(err)
}
if cfg.Artifacts.BaseURL != "" {
t.Errorf("BaseURL = %q, want empty so the sender stays disabled", cfg.Artifacts.BaseURL)
}
if cfg.Artifacts.TokenEnv != "WORKSPACE_INGEST_TOKEN" {
t.Errorf("TokenEnv = %q, want the documented fallback", cfg.Artifacts.TokenEnv)
}
}
func TestArtifactsConfigPreservesExplicitValues(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.toml")
contents := "bot_user = \"widget\"\n" +
"[artifacts]\nbase_url = \"https://offloaded.dev\"\n" +
"token = { engine = \"env\", key = \"ARCHIE_GITHUB_TOKEN\" }\n" +
"token_env = \"OTHER_TOKEN\"\n"
if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
t.Fatal(err)
}
cfg, err := loadFile(path)
if err != nil {
t.Fatal(err)
}
if cfg.Artifacts.BaseURL != "https://offloaded.dev" {
t.Errorf("BaseURL = %q, want the explicit value", cfg.Artifacts.BaseURL)
}
if want := (config.SecretRef{Engine: "env", Key: "ARCHIE_GITHUB_TOKEN"}); cfg.Artifacts.Token != want {
t.Errorf("Token = %+v, want %+v", cfg.Artifacts.Token, want)
}
if cfg.Artifacts.TokenEnv != "OTHER_TOKEN" {
t.Errorf("TokenEnv = %q, want the explicit value", cfg.Artifacts.TokenEnv)
}
}
@@ -76,6 +76,7 @@ func (l *Loader) applyDefaults(cfg *config.Config) {
}
l.applyGeneralDefaults(cfg)
applyForgeDefaults(cfg)
applyArtifactsDefaults(cfg)
applyDispatchDefaults(cfg)
applyIdentityDefaults(cfg)
applyContainerDefaults(cfg)
@@ -231,7 +232,6 @@ func (l *Loader) applyGeneralDefaults(cfg *config.Config) {
if cfg.DBPath == "" {
cfg.DBPath = filepath.Join(l.dataHome(), "archie", "archie.db")
}
cfg.Indexing = cfg.Indexing.WithDefaults(cfg.WorkDir)
if cfg.PollInterval == 0 {
cfg.PollInterval = config.Duration(defaultPollInterval)
}
@@ -280,6 +280,17 @@ func applyForgeDefaults(cfg *config.Config) {
}
}
// applyArtifactsDefaults fills the artifacts section, promoting the
// token_env fallback to a secret reference when no explicit one was given.
func applyArtifactsDefaults(cfg *config.Config) {
if cfg.Artifacts.TokenEnv == "" {
cfg.Artifacts.TokenEnv = "WORKSPACE_INGEST_TOKEN"
}
if cfg.Artifacts.Token == (config.SecretRef{}) {
cfg.Artifacts.Token = config.SecretRef{Engine: "env", Key: cfg.Artifacts.TokenEnv}
}
}
// applyNATSDefaults resolves the nats mode from URL when the operator left it
// unset. URL set implies external; URL empty implies embedded.
func applyNATSDefaults(cfg *config.Config) {
@@ -904,55 +904,6 @@ func TestLoadBytesDoesNotExist(t *testing.T) {
}
}
func TestFinalizeIndexingDefaults(t *testing.T) {
tests := []struct {
name string
extra string
wantIndex string
wantDBIsSet bool
}{
{
name: "defaults derive from work_dir",
// Both paths must land under the daemon's own work_dir so two
// archied instances on one host cannot share an index.
extra: "\nwork_dir = \"/srv/archie/work\"\n",
wantIndex: filepath.Join("/srv/archie/work", "indexes"),
wantDBIsSet: true,
},
{
name: "explicit index_dir is preserved",
extra: "\nwork_dir = \"/srv/archie/work\"\n" +
"[indexing]\nindex_dir = \"/mnt/fast/idx\"\n",
wantIndex: "/mnt/fast/idx",
wantDBIsSet: true,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.toml")
// tc.extra precedes the table sections: a top-level key written
// after [[repos]] would be parsed as a member of that table.
contents := "bot_user = \"widget\"\n" + tc.extra +
"[forge]\ntype = \"gitea\"\nhost = \"https://git.example.test\"\ntoken_env = \"T\"\n" +
"[[repos]]\nowner = \"acme\"\nname = \"app\"\n"
if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
t.Fatal(err)
}
cfg, err := loadFile(path)
if err != nil {
t.Fatal(err)
}
if cfg.Indexing.IndexDir != tc.wantIndex {
t.Errorf("Indexing.IndexDir = %q, want %q", cfg.Indexing.IndexDir, tc.wantIndex)
}
if tc.wantDBIsSet && cfg.Indexing.DBPath == "" {
t.Error("Indexing.DBPath is empty; want a default derived from work_dir")
}
})
}
}
func TestIdentitiesConfigParsesTwoIdentities(t *testing.T) {
cfg, err := loadBytes([]byte(`
bot_user = "default-legacy"
@@ -155,7 +155,7 @@ func TestApply_Idempotent_SecondRunLeavesOtherEditByteIdentical(t *testing.T) {
}
}
// [models], [indexing], [web], and [notify] in the real template all carry
// [models], [web], and [notify] in the real template all carry
// a trailing "# ..." comment on the header line itself
// ("[models] # role -> ..."). The header regexes were anchored with no
// allowance for that, so Apply's table-tracking never recognised these as
@@ -29,24 +29,23 @@ import (
"time"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
)
// SpecLookup resolves a job id to its persisted spec.
//
// It is declared here (consumer-owned) rather than imported from cronstore as
// a concrete type so this package depends on the one method it uses.
// *cronstore.Store satisfies it as written, so a deployment passes the store
// straight in.
// It is declared here (consumer-owned) rather than imported from the document's
// owner as a concrete type so this package depends on the one method it uses.
// scheduleResourceStore in internal/app/archied satisfies it as written, so a
// deployment passes that in.
type SpecLookup interface {
// Get returns the job and whether it was found; the error is reserved
// for I/O failures, matching cronstore.Store.Get.
Get(ctx context.Context, id string) (cronstore.JobSpec, bool, error)
// for I/O failures, matching scheduleResourceStore.Get.
Get(ctx context.Context, id string) (scheduling.JobSpec, bool, error)
}
// RunRecorder advances a job's schedule once a run has completed. It is
// declared here (consumer-owned) for the same reason SpecLookup is, and
// *cronstore.Store satisfies it as written.
// scheduleResourceStore satisfies it as written.
//
// MarkRun is the only step that moves a job's NextRun, so a store whose Due
// keeps reporting an already-completed job keeps handing it back on every
@@ -98,20 +97,18 @@ var errSpecMissing = errors.New("crondelivery: job spec not found")
// hydrate loads the job's persisted spec, wrapping both a store error and the
// not-found case so a caller can match on one sentinel.
func hydrate(ctx context.Context, specs SpecLookup, job scheduling.Job) (cronstore.JobSpec, error) {
func hydrate(ctx context.Context, specs SpecLookup, job scheduling.Job) (scheduling.JobSpec, error) {
spec, ok, err := specs.Get(ctx, job.ID)
if err != nil {
return cronstore.JobSpec{}, fmt.Errorf("crondelivery: load job %q: %w", job.ID, err)
return scheduling.JobSpec{}, fmt.Errorf("crondelivery: load job %q: %w", job.ID, err)
}
if !ok {
return cronstore.JobSpec{}, fmt.Errorf("%w: id %q", errSpecMissing, job.ID)
return scheduling.JobSpec{}, fmt.Errorf("%w: id %q", errSpecMissing, job.ID)
}
return spec, nil
}
// compile-time check: the store is the lookup a deployment hands in unchanged.
var (
_ SpecLookup = (*cronstore.Store)(nil)
_ RunRecorder = (*cronstore.Store)(nil)
_ RouterStore = (*cronstore.Store)(nil)
)
// Compile-time check that the production store satisfies the contract. It
// lives in internal/app/archied, which imports this package, so the assertion
// is made there (scheduling.go) rather than here, where it would be an import
// cycle.
@@ -6,13 +6,12 @@ package crondelivery
import (
"context"
"path/filepath"
"fmt"
"sync"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
)
// --- fakes --------------------------------------------------------------------
@@ -88,24 +87,24 @@ func (n *naiveRunner) count() int {
// fakeLookup serves specs from a map and ignores ctx entirely, like
// naiveCourier.
//
// The cancellation tests need it because the real store is itself
// cancellation-aware: *cronstore.Store.Get returns ctx.Err() before it reads,
// so with a real store the runner's own guard is masked — hydrate fails and the
// run returns the context error whether or not the runner checked. A lookup
// that ignores ctx removes that mask, leaving the runner's guard as the only
// thing that can prevent the side effect.
// The cancellation tests need it because a production store is expected to be
// cancellation-aware: a source whose Get returns ctx.Err() before it reads
// masks the runner's own guard — hydrate fails and the run returns the context
// error whether or not the runner checked. A lookup that ignores ctx removes
// that mask, leaving the runner's guard as the only thing that can prevent the
// side effect.
type fakeLookup struct {
specs map[string]cronstore.JobSpec
specs map[string]scheduling.JobSpec
}
func (f *fakeLookup) Get(_ context.Context, id string) (cronstore.JobSpec, bool, error) {
func (f *fakeLookup) Get(_ context.Context, id string) (scheduling.JobSpec, bool, error) {
spec, ok := f.specs[id]
return spec, ok, nil
}
// MarkRun satisfies RouterStore. It deliberately does nothing: these tests
// exercise the runner's and the router's own dispatch guards, and the record
// step is covered against the real store in
// step is covered against a store that really advances in
// TestRouterAdvancesTheScheduleAfterASuccessfulRun.
func (f *fakeLookup) MarkRun(_ context.Context, _ string, _ time.Time) error { return nil }
@@ -247,20 +246,80 @@ func (c *countingRunner) count() int {
// --- helpers ------------------------------------------------------------------
// newStore opens a real cronstore on a temp path and closes it on cleanup.
func newStore(t *testing.T) *cronstore.Store {
t.Helper()
s, err := cronstore.Open(filepath.Join(t.TempDir(), "jobs.json"))
if err != nil {
t.Fatalf("cronstore.Open: %v", err)
}
t.Cleanup(func() { _ = s.Close() })
return s
// memStore is the test's schedules document: a map with the three faces the
// production store shows the package — SpecLookup (Get), RunRecorder (MarkRun
// with real schedule arithmetic), and JobSource (Due). It replaces the file-
// backed store this package's fixtures used to open, which died with the
// legacy jobs.json store; the document now lives behind the control plane in
// production, and what these tests need from persistence is semantics, not a
// file.
type memStore struct {
mu sync.Mutex
jobs map[string]scheduling.JobSpec
}
// createJob persists a spec through the real store and returns the scheduling
// job the engine would hand a runner for it.
func createJob(t *testing.T, s *cronstore.Store, spec cronstore.JobSpec) scheduling.Job {
func newStore(t *testing.T) *memStore {
t.Helper()
return &memStore{jobs: make(map[string]scheduling.JobSpec)}
}
// Create persists a spec verbatim. Fixtures that want a due job set NextRun
// themselves (createJob does it when unset).
func (m *memStore) Create(_ context.Context, spec scheduling.JobSpec) error {
m.mu.Lock()
defer m.mu.Unlock()
if _, exists := m.jobs[spec.ID]; exists {
return fmt.Errorf("memStore: id %q already exists", spec.ID)
}
m.jobs[spec.ID] = spec
return nil
}
func (m *memStore) Get(_ context.Context, id string) (scheduling.JobSpec, bool, error) {
m.mu.Lock()
defer m.mu.Unlock()
spec, ok := m.jobs[id]
return spec, ok, nil
}
// Due returns the jobs whose NextRun is at or before now, the same contract
// the production source keeps.
func (m *memStore) Due(_ context.Context, now time.Time) ([]scheduling.Job, error) {
m.mu.Lock()
defer m.mu.Unlock()
due := make([]scheduling.Job, 0, len(m.jobs))
for _, spec := range m.jobs {
if !spec.NextRun.After(now) {
due = append(due, scheduling.Job{ID: spec.ID, Pool: scheduling.Pool(spec.Pool), Detail: spec.Detail})
}
}
return due, nil
}
// MarkRun advances an interval job's NextRun the way the production
// recording step does, and reports a once schedule as unsupported — the
// sentinel the router swallows.
func (m *memStore) MarkRun(_ context.Context, id string, runAt time.Time) error {
m.mu.Lock()
defer m.mu.Unlock()
spec, ok := m.jobs[id]
if !ok {
return fmt.Errorf("%w: id %q", scheduling.ErrJobNotFound, id)
}
next, err := spec.Schedule.NextRun(runAt)
if err != nil {
return err
}
at := runAt.UTC()
spec.LastRun = &at
spec.NextRun = next
m.jobs[id] = spec
return nil
}
// createJob persists a spec and returns the scheduling job the engine would
// hand a runner for it.
func createJob(t *testing.T, s *memStore, spec scheduling.JobSpec) scheduling.Job {
t.Helper()
if spec.NextRun.IsZero() {
// Due now: the engine only ever hands a runner a job its source
@@ -268,30 +327,30 @@ func createJob(t *testing.T, s *cronstore.Store, spec cronstore.JobSpec) schedul
spec.NextRun = time.Now().UTC().Add(-time.Minute)
}
if err := s.Create(t.Context(), spec); err != nil {
t.Fatalf("cronstore.Create(%q): %v", spec.ID, err)
t.Fatalf("Create(%q): %v", spec.ID, err)
}
return scheduling.Job{ID: spec.ID, Pool: scheduling.Pool(spec.Pool), Detail: spec.Detail}
}
// chatSpec is a minimal chat job.
func chatSpec(id, chatID, text string) cronstore.JobSpec {
return cronstore.JobSpec{
func chatSpec(id, chatID, text string) scheduling.JobSpec {
return scheduling.JobSpec{
ID: id,
Detail: "test job " + id,
Kind: cronstore.KindChat,
Schedule: cronstore.Schedule{Kind: cronstore.ScheduleInterval, Interval: cronstore.Duration(time.Hour)},
Target: cronstore.Target{ChatID: chatID},
Payload: cronstore.Payload{Text: text},
Kind: scheduling.KindChat,
Schedule: scheduling.Schedule{Kind: scheduling.ScheduleInterval, Interval: scheduling.Duration(time.Hour)},
Target: scheduling.Target{ChatID: chatID},
Payload: scheduling.Payload{Text: text},
}
}
// workflowSpec is a minimal workflow job.
func workflowSpec(id, detail, body string) cronstore.JobSpec {
return cronstore.JobSpec{
func workflowSpec(id, detail, body string) scheduling.JobSpec {
return scheduling.JobSpec{
ID: id,
Detail: detail,
Kind: cronstore.KindWorkflow,
Schedule: cronstore.Schedule{Kind: cronstore.ScheduleInterval, Interval: cronstore.Duration(time.Hour)},
Payload: cronstore.Payload{Text: body},
Kind: scheduling.KindWorkflow,
Schedule: scheduling.Schedule{Kind: scheduling.ScheduleInterval, Interval: scheduling.Duration(time.Hour)},
Payload: scheduling.Payload{Text: body},
}
}
@@ -9,7 +9,6 @@ import (
"github.com/samcharles93/archie-core/internal/domain/scheduling"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
)
// phaseDispatch names the step that failed on a KindJobError the Router emits.
@@ -43,7 +42,7 @@ type Router struct {
// emission — the router still refuses, it is just unobservable, the same
// tolerance the engine has for its own nil sink.
//
// An empty kind resolves to cronstore.KindChat, so a deployment only needs a
// An empty kind resolves to scheduling.KindChat, so a deployment only needs a
// mapping for the kinds it actually uses.
func NewRouter(specs RouterStore, runners map[string]scheduling.Runner, sink scheduling.Sink) (*Router, error) {
if specs == nil {
@@ -67,7 +66,7 @@ func (r *Router) Run(ctx context.Context, job scheduling.Job) error {
kind := spec.Kind
if kind == "" {
kind = cronstore.KindChat
kind = scheduling.KindChat
}
runner, ok := r.runners[kind]
if !ok {
@@ -86,12 +85,12 @@ func (r *Router) Run(ctx context.Context, job scheduling.Job) error {
// per tick instead.
//
// A schedule kind with no recurring next run -- a one-shot -- reports
// cronstore.ErrScheduleUnsupported, which is the job's definition rather than a
// scheduling.ErrScheduleUnsupported, which is the job's definition rather than a
// run failure, so it is swallowed. Anything else is returned: the run happened
// but the bookkeeping did not, and hiding that would leave the job silently
// re-firing.
func (r *Router) recordRun(ctx context.Context, job scheduling.Job) error {
if err := r.specs.MarkRun(ctx, job.ID, time.Now()); err != nil && !errors.Is(err, cronstore.ErrScheduleUnsupported) {
if err := r.specs.MarkRun(ctx, job.ID, time.Now()); err != nil && !errors.Is(err, scheduling.ErrScheduleUnsupported) {
return fmt.Errorf("crondelivery: record run of job %q: %w", job.ID, err)
}
return nil
@@ -8,7 +8,6 @@ import (
"github.com/samcharles93/archie-core/internal/domain/scheduling"
"github.com/samcharles93/archie-core/internal/events"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
)
// --- Router -------------------------------------------------------------------
@@ -19,17 +18,17 @@ import (
func routerFixture(t *testing.T, kind string) (*Router, *countingRunner, *countingRunner, *recordingSink) {
t.Helper()
s := newStore(t)
createJob(t, s, cronstore.JobSpec{
createJob(t, s, scheduling.JobSpec{
ID: "routed",
Detail: "routed job",
Kind: kind,
Schedule: cronstore.Schedule{Kind: cronstore.ScheduleInterval, Interval: cronstore.Duration(time.Hour)},
Schedule: scheduling.Schedule{Kind: scheduling.ScheduleInterval, Interval: scheduling.Duration(time.Hour)},
})
chat, workflow := &countingRunner{}, &countingRunner{}
sink := &recordingSink{}
r, err := NewRouter(s, map[string]scheduling.Runner{
cronstore.KindChat: chat,
cronstore.KindWorkflow: workflow,
scheduling.KindChat: chat,
scheduling.KindWorkflow: workflow,
}, sink)
if err != nil {
t.Fatalf("NewRouter: %v", err)
@@ -44,8 +43,8 @@ func TestRouterDispatchesByKind(t *testing.T) {
wantChat int
wantWorkflow int
}{
{"chat kind reaches the chat runner", cronstore.KindChat, 1, 0},
{"workflow kind reaches the workflow runner", cronstore.KindWorkflow, 0, 1},
{"chat kind reaches the chat runner", scheduling.KindChat, 1, 0},
{"workflow kind reaches the workflow runner", scheduling.KindWorkflow, 0, 1},
{"empty kind defaults to the chat runner", "", 1, 0},
}
for _, tc := range tests {
@@ -117,15 +116,15 @@ func TestRouterUnknownKindIsRefusedAtTheGate(t *testing.T) {
// tolerance: a nil sink disables emission without disabling the refusal.
func TestRouterNilSinkDoesNotPanicOnUnknownKind(t *testing.T) {
s := newStore(t)
createJob(t, s, cronstore.JobSpec{
createJob(t, s, scheduling.JobSpec{
ID: "routed",
Kind: "teleport",
Schedule: cronstore.Schedule{Kind: cronstore.ScheduleInterval, Interval: cronstore.Duration(time.Hour)},
Schedule: scheduling.Schedule{Kind: scheduling.ScheduleInterval, Interval: scheduling.Duration(time.Hour)},
})
chat, workflow := &countingRunner{}, &countingRunner{}
r, err := NewRouter(s, map[string]scheduling.Runner{
cronstore.KindChat: chat,
cronstore.KindWorkflow: workflow,
scheduling.KindChat: chat,
scheduling.KindWorkflow: workflow,
}, nil)
if err != nil {
t.Fatalf("NewRouter: %v", err)
@@ -143,7 +142,7 @@ func TestRouterNilSinkDoesNotPanicOnUnknownKind(t *testing.T) {
func TestRouterMissingSpecReturnsError(t *testing.T) {
s := newStore(t)
r, err := NewRouter(s, map[string]scheduling.Runner{
cronstore.KindChat: &countingRunner{},
scheduling.KindChat: &countingRunner{},
}, &recordingSink{})
if err != nil {
t.Fatalf("NewRouter: %v", err)
@@ -161,7 +160,7 @@ func TestRouterMissingSpecReturnsError(t *testing.T) {
// the job the engine built, not a re-derived one: identity and pool are the
// engine's, and the runner's event attribution depends on them surviving.
func TestRouterPassesTheEngineJobThrough(t *testing.T) {
r, chat, _, _ := routerFixture(t, cronstore.KindChat)
r, chat, _, _ := routerFixture(t, scheduling.KindChat)
job := scheduling.Job{ID: "routed", Pool: scheduling.PoolSequential, Detail: "detail from engine"}
if err := r.Run(t.Context(), job); err != nil {
@@ -185,7 +184,7 @@ func TestRouterPassesTheEngineJobThrough(t *testing.T) {
// honours cancellation would make this pass whether or not the runner checks,
// which is the failure mode the criterion exists to prevent.
func TestEveryRunnerHonoursCancellation(t *testing.T) {
lookup := &fakeLookup{specs: map[string]cronstore.JobSpec{
lookup := &fakeLookup{specs: map[string]scheduling.JobSpec{
"chat-job": chatSpec("chat-job", "ops-room", "hello"),
"wf-job": workflowSpec("wf-job", "workflow job", "body"),
}}
@@ -204,8 +203,8 @@ func TestEveryRunnerHonoursCancellation(t *testing.T) {
// router's own guard: a cancelled run must not reach the dispatch.
forwarded := &naiveRunner{}
router, err := NewRouter(lookup, map[string]scheduling.Runner{
cronstore.KindChat: forwarded,
cronstore.KindWorkflow: workflow,
scheduling.KindChat: forwarded,
scheduling.KindWorkflow: workflow,
}, &recordingSink{})
if err != nil {
t.Fatalf("NewRouter: %v", err)
@@ -246,27 +245,27 @@ func TestEveryRunnerHonoursCancellation(t *testing.T) {
const dailyStatusSummary = "Daily status summary: all workers healthy, no tasks parked."
// TestDailyStatusSummaryEndToEnd is the epic's headline scenario exercised
// through the whole chain a deployment uses: a real job persisted in a real
// cronstore is reported due by the store's own JobSource, handed to the router
// through the whole chain a deployment uses: a scheduled job in the schedules
// document is reported due by its own JobSource, handed to the router
// the engine is given, dispatched to the chat runner, and delivered as the
// composed message.
func TestDailyStatusSummaryEndToEnd(t *testing.T) {
s := newStore(t)
const chatID = "ops-room"
spec := cronstore.JobSpec{
spec := scheduling.JobSpec{
ID: "daily-status",
Detail: "daily status summary",
Kind: cronstore.KindChat,
Schedule: cronstore.Schedule{Kind: cronstore.ScheduleInterval, Interval: cronstore.Duration(24 * time.Hour)},
Target: cronstore.Target{ChatID: chatID},
Payload: cronstore.Payload{Text: dailyStatusSummary},
Kind: scheduling.KindChat,
Schedule: scheduling.Schedule{Kind: scheduling.ScheduleInterval, Interval: scheduling.Duration(24 * time.Hour)},
Target: scheduling.Target{ChatID: chatID},
Payload: scheduling.Payload{Text: dailyStatusSummary},
// Due now: create it with a next_run in the past, the state a
// long-idle daemon wakes up to.
NextRun: time.Now().UTC().Add(-time.Minute),
}
if err := s.Create(t.Context(), spec); err != nil {
t.Fatalf("cronstore.Create: %v", err)
t.Fatalf("Create: %v", err)
}
// The engine's source: the store itself, answering what is due.
@@ -284,7 +283,7 @@ func TestDailyStatusSummaryEndToEnd(t *testing.T) {
t.Fatalf("NewChatCourier: %v", err)
}
router, err := NewRouter(s, map[string]scheduling.Runner{
cronstore.KindChat: courierRunner,
scheduling.KindChat: courierRunner,
}, &recordingSink{})
if err != nil {
t.Fatalf("NewRouter: %v", err)
@@ -334,7 +333,7 @@ func TestConstructorsRejectNilDependencies(t *testing.T) {
}},
{"Router without a spec lookup", func() error {
_, err := NewRouter(nil, map[string]scheduling.Runner{
cronstore.KindChat: &countingRunner{},
scheduling.KindChat: &countingRunner{},
}, nil)
return err
}},
@@ -363,7 +362,7 @@ func TestImplementsSchedulingRunner(t *testing.T) {
}
// TestRouterAdvancesTheScheduleAfterASuccessfulRun is the acceptance criterion
// for a recurring job's timing: cronstore.Store.MarkRun is the only step that
// for a recurring job's timing: the recording step is the only thing that
// moves a job's NextRun, and the Router is the single Runner the engine hands
// every job. With no caller the store keeps reporting the job due on every
// tick, so a job scheduled every hour fires once per tick instead.
@@ -380,7 +379,7 @@ func TestRouterAdvancesTheScheduleAfterASuccessfulRun(t *testing.T) {
}
r, err := NewRouter(s, map[string]scheduling.Runner{
cronstore.KindChat: &countingRunner{},
scheduling.KindChat: &countingRunner{},
}, &recordingSink{})
if err != nil {
t.Fatalf("NewRouter: %v", err)
@@ -404,23 +403,23 @@ func TestRouterAdvancesTheScheduleAfterASuccessfulRun(t *testing.T) {
// TestRouterToleratesAOnceScheduleWithoutANextRun pins the other half: a
// one-shot job has no recurring next run by definition, so MarkRun reports
// cronstore.ErrScheduleUnsupported. That is not a run failure -- the job ran --
// scheduling.ErrScheduleUnsupported. That is not a run failure -- the job ran --
// and returning it would make the engine emit KindJobError for a job that
// succeeded.
func TestRouterToleratesAOnceScheduleWithoutANextRun(t *testing.T) {
s := newStore(t)
at := time.Now().Add(-time.Hour)
job := createJob(t, s, cronstore.JobSpec{
job := createJob(t, s, scheduling.JobSpec{
ID: "one-shot",
Detail: "one-shot job",
Kind: cronstore.KindChat,
Schedule: cronstore.Schedule{Kind: cronstore.ScheduleOnce, At: &at},
Target: cronstore.Target{ChatID: "chat-1"},
Payload: cronstore.Payload{Text: "once"},
Kind: scheduling.KindChat,
Schedule: scheduling.Schedule{Kind: scheduling.ScheduleOnce, At: &at},
Target: scheduling.Target{ChatID: "chat-1"},
Payload: scheduling.Payload{Text: "once"},
})
r, err := NewRouter(s, map[string]scheduling.Runner{
cronstore.KindChat: &countingRunner{},
scheduling.KindChat: &countingRunner{},
}, &recordingSink{})
if err != nil {
t.Fatalf("NewRouter: %v", err)
@@ -7,7 +7,6 @@ import (
"time"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
"github.com/samcharles93/archie-core/internal/infrastructure/cronstore"
)
// --- ChatCourier --------------------------------------------------------------
@@ -83,7 +82,7 @@ func TestChatCourierHonoursCancellation(t *testing.T) {
t.Run("cancelled before the run sends nothing", func(t *testing.T) {
// Both the lookup and the courier ignore ctx, so only the
// runner's own guard can stop the send.
lookup := &fakeLookup{specs: map[string]cronstore.JobSpec{
lookup := &fakeLookup{specs: map[string]scheduling.JobSpec{
"cancel-me": chatSpec("cancel-me", "ops-room", "morning"),
}}
courier := &naiveCourier{}
@@ -224,7 +223,7 @@ func TestWorkflowTaskHonoursCancellation(t *testing.T) {
t.Run("cancelled before the run submits nothing", func(t *testing.T) {
// Both the lookup and the submitter ignore ctx, so only the
// runner's own guard can stop the submit.
lookup := &fakeLookup{specs: map[string]cronstore.JobSpec{
lookup := &fakeLookup{specs: map[string]scheduling.JobSpec{
"cancel-me": workflowSpec("cancel-me", "cancel me", "body"),
}}
sub := &naiveSubmitter{}
-41
View File
@@ -1,41 +0,0 @@
// Package cronstore is the file-backed job store that backs the scheduling
// ticker engine. It is infrastructure: it owns the on-disk layout, the
// cross-process lock, and the version-stamped JSON schema. It does not own
// timing semantics — those live in internal/domain/scheduling.
//
// Two surfaces:
//
// - scheduling.JobSource: Due(ctx, now) — what the engine consumes.
// - CRUD on JobSpec — what the CLI and dashboard consume.
//
// Both share the same Store value, the same lock, and the same on-disk
// file. The engine applies no schedule arithmetic of its own; whatever
// "due" means is the store's decision, including future kinds the store
// grows into.
//
// Concurrency model: every read and write acquires an OS-level flock on a
// sibling lock file, so a CLI edit and a daemon writer cannot tear the file.
// Conflict resolution is last-writer-wins: the second writer overwrites the
// first's whole document. The lock makes that serial rather than racy.
// In-process, two Store values for the same path are serialised by a
// package-level mutex keyed by lock-file path — flock is per file
// description on Linux and would not block them otherwise.
//
// File format (versioned; bump schemaVersion on ANY shape change
// including additive ones, since DisallowUnknownFields rejects them):
//
// {
// "schema_version": 2,
// "jobs": [ JobSpec, ... ]
// }
//
// Version 2 added JobSpec.Kind, the delivery discriminator. A version-1
// file still loads — Kind reads back empty, which the delivery router
// resolves to the chat default — so jobs created before the field existed
// keep firing the way they always did.
//
// JobSpec carries the engine-facing fields it always needed (ID, Pool,
// Detail) plus everything the store computes for it (NextRun, LastRun,
// Created, Updated). Merge semantics on Update mean a nil field in a Patch
// leaves the corresponding persisted field untouched.
package cronstore
@@ -1,39 +0,0 @@
package cronstore
import "errors"
// Sentinel errors callers can match with errors.Is.
var (
// ErrUnsupportedSchema is returned by Open when the persisted file
// declares a schema version this build does not understand. The
// store treats that as an explicit refusal to read: a forward
// version may have shape we cannot safely render back out, and
// silently downgrading would lose data.
ErrUnsupportedSchema = errors.New("cronstore: unsupported schema version")
// ErrCorruptFile is returned by Open when the file exists but does
// not decode as a valid envelope: truncated, garbage, or fields of
// the wrong shape. The store never panics on bad input, so the
// engine can surface this as phase="source" KindJobError.
ErrCorruptFile = errors.New("cronstore: corrupt file")
// ErrJobExists is returned by Create when the requested id is
// already present. Callers that want upsert semantics should
// follow up with Update.
ErrJobExists = errors.New("cronstore: job already exists")
// ErrJobNotFound is returned by Get, Update and MarkRun when the
// id is absent. Delete is the exception: it is idempotent.
ErrJobNotFound = errors.New("cronstore: job not found")
// ErrInvalidSpec is returned by Create and Update when the supplied
// JobSpec / Patch fails the store's structural validation (empty id,
// unknown schedule kind, non-positive interval).
ErrInvalidSpec = errors.New("cronstore: invalid job spec")
// ErrScheduleUnsupported is returned by MarkRun when the job's
// schedule kind cannot compute its next run time. Slice 1 only
// knows interval arithmetic; cron / once return this so callers
// can surface a clear error rather than silently advancing.
ErrScheduleUnsupported = errors.New("cronstore: schedule kind not implemented")
)
-24
View File
@@ -1,24 +0,0 @@
package cronstore
// Kind values for JobSpec.Kind: the discriminator that decides which delivery
// runner handles a job.
//
// They live here, beside the field that carries them, because they are part of
// the on-disk vocabulary — an operator writes one in jobs.json, the strict
// decoder reads it back, and a file written by a build that never heard of it
// must fail loudly rather than deliver the wrong way. Which runner serves a
// kind is not the store's business: the router in
// internal/infrastructure/crondelivery owns that mapping, which is why an
// unrecognised kind is left on disk rather than refused at Create.
const (
// KindChat delivers the job's Payload.Text as a chat message through the
// deployment's channel. It is the default: an empty Kind means KindChat,
// so every job written before this field existed (schema version 1)
// keeps delivering the way it did when it was created.
KindChat = "chat"
// KindWorkflow submits the job as a unit of work through the work-intake
// path, so the existing task lifecycle owns scheduling, retries, gates
// and PR opening.
KindWorkflow = "workflow"
)
@@ -1,148 +0,0 @@
// Tests for the JobSpec.Kind discriminator added in the cron-delivery slice.
//
// The store owns the persisted vocabulary; the runner mapping belongs to
// internal/infrastructure/crondelivery. These tests pin the store-side
// properties that mapping depends on: the field round-trips, an empty value
// reads back as the chat default, and a file written before the field existed
// still loads.
package cronstore
import (
"bytes"
"context"
"os"
"path/filepath"
"testing"
"time"
)
// TestPersistedFileStampsCurrentSchemaVersion guards the additive-change rule
// documented on schemaVersion: adding Kind to JobSpec forces a version bump,
// because DisallowUnknownFields means an older build must reject a file
// carrying a field it does not know rather than silently dropping it.
func TestPersistedFileStampsCurrentSchemaVersion(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
s, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer s.Close()
mustCreate(t, s, fixedIntervalSpec("x", time.Minute))
if err := s.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
if !bytes.Contains(data, []byte(`"schema_version":2`)) {
t.Errorf("file does not carry the bumped schema_version:2 stamp: %s", data)
}
}
// TestKindRoundTrips pins that the discriminator survives a write and read on
// the real file, not just in memory: Create serialises it, a fresh Open with
// the strict decoder must read it back.
func TestKindRoundTrips(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
s, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
spec := fixedIntervalSpec("nightly-build", time.Hour)
spec.Kind = KindWorkflow
mustCreate(t, s, spec)
if err := s.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
// A second handle proves the value is on disk and survives the strict
// decoder, rather than only living in the writer's in-memory mirror.
reopened, err := Open(path)
if err != nil {
t.Fatalf("reopen: %v", err)
}
defer reopened.Close()
got, ok, err := reopened.Get(context.Background(), "nightly-build")
if err != nil {
t.Fatalf("Get: %v", err)
}
if !ok {
t.Fatal("Get returned ok=false after reopen")
}
if got.Kind != KindWorkflow {
t.Errorf("Kind = %q, want %q", got.Kind, KindWorkflow)
}
}
// TestEmptyKindLoadsAsChatDefault pins the compatibility rule the router
// relies on: a job that names no kind delivers as a chat message. Asserting
// the empty string (rather than KindChat) is deliberate — the store must not
// rewrite what the operator wrote; resolving empty to "chat" is the router's
// job, and this test pins that the store hands it through untouched.
func TestEmptyKindLoadsAsChatDefault(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("no-kind", time.Minute))
got, ok, err := s.Get(context.Background(), "no-kind")
if err != nil {
t.Fatalf("Get: %v", err)
}
if !ok {
t.Fatal("Get returned ok=false")
}
if got.Kind != "" {
t.Errorf("Kind = %q, want empty (the chat default)", got.Kind)
}
}
// TestVersion1FileWithoutKindStillLoads is the migration guard for the schema
// bump. A file written by Slice 1 has schema_version 1 and no kind field; the
// bumped build must load it rather than rejecting it, or every existing cron
// job stops firing the moment this ships.
func TestVersion1FileWithoutKindStillLoads(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
past := time.Now().Add(-time.Minute).UTC().Format(time.RFC3339Nano)
contents := `{"schema_version":1,"jobs":[{"id":"legacy","pool":"sequential",` +
`"detail":"written before kind existed",` +
`"schedule":{"kind":"interval","interval":60000000000},` +
`"target":{"chat_id":"ops"},"payload":{"text":"legacy payload"},` +
`"next_run":"` + past + `","created":"` + past + `","updated":"` + past + `"}]}`
if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
t.Fatalf("WriteFile: %v", err)
}
s, err := Open(path)
if err != nil {
t.Fatalf("Open on a version-1 file after the bump = %v, want nil", err)
}
defer s.Close()
got, ok, err := s.Get(context.Background(), "legacy")
if err != nil {
t.Fatalf("Get: %v", err)
}
if !ok {
t.Fatal("Get returned ok=false for a version-1 job")
}
if got.Kind != "" {
t.Errorf("Kind = %q, want empty for a version-1 job", got.Kind)
}
if got.Payload.Text != "legacy payload" {
t.Errorf("Payload.Text = %q, want %q", got.Payload.Text, "legacy payload")
}
// It must still be dispatchable: the engine reads due jobs through Due.
due, err := s.Due(context.Background(), time.Now())
if err != nil {
t.Fatalf("Due: %v", err)
}
if len(due) != 1 || due[0].ID != "legacy" {
t.Errorf("Due on a version-1 file = %+v, want the legacy job", due)
}
}
@@ -1,68 +0,0 @@
//go:build unix
package cronstore
import (
"fmt"
"os"
"syscall"
)
// lockedFile is an OS-level flock wrapper around a sidecar file. The
// store acquires it before reading or writing the JSON, so a CLI edit
// and a daemon writer cannot tear the same jobs.json.
//
// flock is per-open-file-description, which means a single Store holds
// the lock for as long as it lives; concurrent processes opening the
// same sidecar queue behind it.
type lockedFile struct {
f *os.File
}
// openLock creates (or opens) the lock file and returns it unlocked.
// The lock is acquired separately via Lock, so Open can hydrate without
// holding it.
func openLock(path string) (*lockedFile, error) {
f, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE, 0o644)
if err != nil {
return nil, fmt.Errorf("open lock file %s: %w", path, err)
}
return &lockedFile{f: f}, nil
}
// Lock blocks until the OS-level exclusive flock is acquired. flock is
// reentrant on Linux per-fd, so calling Lock twice on the same lockedFile
// from the same goroutine is fine; calling it from two goroutines still
// serialises.
func (l *lockedFile) Lock() error {
if l == nil || l.f == nil {
return fmt.Errorf("cronstore: lock not initialised")
}
if err := syscall.Flock(int(l.f.Fd()), syscall.LOCK_EX); err != nil {
return fmt.Errorf("flock LOCK_EX: %w", err)
}
return nil
}
// Unlock releases the exclusive flock. Idempotent in the sense that a
// second call after Close simply returns nil because the fd is closed.
func (l *lockedFile) Unlock() error {
if l == nil || l.f == nil {
return nil
}
if err := syscall.Flock(int(l.f.Fd()), syscall.LOCK_UN); err != nil {
return fmt.Errorf("flock LOCK_UN: %w", err)
}
return nil
}
// Close closes the sidecar fd. The flock is released automatically by
// the kernel.
func (l *lockedFile) Close() error {
if l == nil || l.f == nil {
return nil
}
err := l.f.Close()
l.f = nil
return err
}
@@ -1,53 +0,0 @@
//go:build windows
// flock is a POSIX primitive; Windows does not implement syscall.Flock
// the same way. Windows deployments of archied are not the supported
// surface (the deployment model in CLAUDE.md and deployments/*.toml is
// Linux + macOS), so this file deliberately fails closed: every Lock
// returns an error rather than silently degrading to "no lock".
//
// If a Windows deployment ever becomes supported, swap this for a real
// LockFileEx / UnlockFileEx implementation; the rest of the store does
// not need to change.
package cronstore
import (
"errors"
"fmt"
"os"
)
// lockedFile is a stub on Windows. The store still opens the sidecar
// file so the on-disk layout matches across platforms, but every Lock
// returns an explicit error.
type lockedFile struct {
f *os.File
}
func openLock(path string) (*lockedFile, error) {
f, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE, 0o644)
if err != nil {
return nil, fmt.Errorf("open lock file %s: %w", path, err)
}
return &lockedFile{f: f}, nil
}
// errLockUnsupported is wrapped in every Lock call so callers can match
// on it without parsing strings.
var errLockUnsupported = errors.New("cronstore: cross-process locking is unsupported on Windows")
func (l *lockedFile) Lock() error { return errLockUnsupported }
func (l *lockedFile) Unlock() error { return nil }
func (l *lockedFile) Close() error {
if l == nil || l.f == nil {
return nil
}
err := l.f.Close()
l.f = nil
return err
}
// Keep imports alive when this file is built in isolation.
var _ = fmt.Sprintf
@@ -1,33 +0,0 @@
package cronstore
import "sync"
// processLocks serialises Stores within one process. It exists because
// Linux flock is per file description, not per inode: two Store
// instances opening the same jobs.json.lock each get their own fd and
// their own independent flock state, so they would not block each
// other inside a single process. The cross-process lock handles the
// CLI-vs-daemon case; this map handles the in-process case.
//
// Keyed by absolute lock-file path so two unrelated cronstores in the
// same process do not contend. Entries are created lazily and live for
// the life of the process; the map is append-only and small in
// practice (one entry per distinct config dir).
var (
processLocksMu sync.Mutex
processLocks = map[string]*sync.Mutex{}
)
// processLockFor returns the per-lock-path mutex, creating it on first
// use. Callers must hold it for the duration of an operation that has
// already acquired the cross-process flock.
func processLockFor(path string) *sync.Mutex {
processLocksMu.Lock()
defer processLocksMu.Unlock()
if m, ok := processLocks[path]; ok {
return m
}
m := &sync.Mutex{}
processLocks[path] = m
return m
}
@@ -1,91 +0,0 @@
package cronstore
import (
"encoding/json"
"strings"
"testing"
"time"
)
// TestScheduleIntervalWritesADurationString pins the document form: the interval
// an operator edits must read as a duration, not as the nanosecond count that
// asked them to type 1800000000000.
func TestScheduleIntervalWritesADurationString(t *testing.T) {
encoded, err := json.Marshal(Schedule{Kind: ScheduleInterval, Interval: Duration(30 * time.Minute)})
if err != nil {
t.Fatalf("marshal: %v", err)
}
if got := string(encoded); !strings.Contains(got, `"interval":"30m0s"`) {
t.Errorf("marshalled schedule = %s, want interval as the string \"30m0s\"", got)
}
if strings.Contains(string(encoded), "1800000000000") {
t.Errorf("marshalled schedule = %s, want no nanosecond count", encoded)
}
}
// TestScheduleIntervalReadsEveryStoredForm covers what the store already holds.
// Every schedule written before this shape existed carries a nanosecond count, in
// both the control plane's schedules resource and this package's own jobs table,
// and both have to keep working without being re-saved first.
func TestScheduleIntervalReadsEveryStoredForm(t *testing.T) {
for _, tt := range []struct {
name string
document string
expected time.Duration
}{
{"legacy nanosecond count", `{"kind":"interval","interval":1800000000000}`, 30 * time.Minute},
{"duration string", `{"kind":"interval","interval":"30m"}`, 30 * time.Minute},
{"duration string, canonical", `{"kind":"interval","interval":"30m0s"}`, 30 * time.Minute},
} {
t.Run(tt.name, func(t *testing.T) {
var schedule Schedule
if err := json.Unmarshal([]byte(tt.document), &schedule); err != nil {
t.Fatalf("unmarshal %s: %v", tt.document, err)
}
if schedule.Interval.Std() != tt.expected {
t.Errorf("Interval = %v, want %v", schedule.Interval, tt.expected)
}
if err := schedule.Validate(); err != nil {
t.Errorf("Validate() = %v, want the decoded schedule to be usable", err)
}
})
}
}
// TestScheduleIntervalRefusesWhatItCannotUse keeps the tolerance above from
// becoming indifference: a duration that does not parse must fail loudly rather
// than decode as zero, because a zero interval is a job that never fires.
func TestScheduleIntervalRefusesWhatItCannotUse(t *testing.T) {
for _, tt := range []struct{ name, document string }{
{"an unparseable duration", `{"kind":"interval","interval":"soon"}`},
{"a fractional count", `{"kind":"interval","interval":1.5}`},
{"a null interval", `{"kind":"interval","interval":null}`},
} {
t.Run(tt.name, func(t *testing.T) {
var schedule Schedule
if err := json.Unmarshal([]byte(tt.document), &schedule); err == nil {
t.Fatalf("unmarshal(%s) = nil, want a refusal: a silent zero interval never fires", tt.document)
}
})
}
}
// TestScheduleIntervalRoundTripsThroughTheStore pins the shape the jobs table
// keeps, since it persists the spec as JSON rather than as columns.
func TestScheduleIntervalRoundTripsThroughTheStore(t *testing.T) {
spec := JobSpec{ID: "nightly", Schedule: Schedule{Kind: ScheduleInterval, Interval: Duration(90 * time.Minute)}}
encoded, err := json.Marshal(spec)
if err != nil {
t.Fatalf("marshal: %v", err)
}
if !strings.Contains(string(encoded), `"interval":"1h30m0s"`) {
t.Fatalf("marshalled spec = %s, want the interval as \"1h30m0s\"", encoded)
}
var decoded JobSpec
if err := json.Unmarshal(encoded, &decoded); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if decoded.Schedule.Interval.Std() != 90*time.Minute {
t.Errorf("round-tripped Interval = %v, want 1h30m0s", decoded.Schedule.Interval)
}
}
-665
View File
@@ -1,665 +0,0 @@
package cronstore
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
)
// schemaVersion is stamped on every persisted file. Bump it on any
// breaking change to JobSpec, Schedule, Payload or envelope shape, AND
// on any additive change: DisallowUnknownFields means a future build
// that adds a new field to JobSpec must also bump schemaVersion so
// older builds keep rejecting the file rather than silently dropping
// the new field.
const schemaVersion = 2
// filePerm is the mode applied to the jobs.json file on write. Payloads
// and chat ids can carry operator-identifying information, so 0o600 is
// the default; operators who want it group-readable can chmod after.
const filePerm = 0o600
// JobSpec is one persisted cron job as the engine's view sees it, plus
// the bookkeeping fields the store computes (NextRun, LastRun, Created,
// Updated). The wire shape is the public API; the scheduling.Job returned
// to the engine is a stripped subset.
type JobSpec struct {
// ID is the unique identifier. Required, non-empty, and the merge
// key for Update and MarkRun.
ID string `json:"id"`
// Pool is "parallel" or "sequential". Empty means sequential — the
// safer default, matching scheduling.Pool.resolve().
Pool string `json:"pool,omitempty"`
// Detail is a human-readable label rendered on events.
Detail string `json:"detail,omitempty"`
// Kind selects which delivery runner handles this job. Empty means
// KindChat, so a job written before this field existed (schema
// version 1) keeps delivering as a chat message. The store does not
// interpret the value: an unrecognised kind is persisted verbatim and
// refused by the delivery router, which is the one place the mapping
// from kind to runner lives.
Kind string `json:"kind,omitempty"`
// Schedule decides when this job is due. See schedule.go for the
// supported kinds and their arithmetic.
Schedule Schedule `json:"schedule"`
// Target and Payload are placeholders for Slice 2 (delivery runner).
// They live on the wire now so jobs created ahead of Slice 2 do not
// need a schema migration when delivery lands.
Target Target `json:"target"`
Payload Payload `json:"payload"`
// NextRun is the absolute moment the job will next fire. Computed
// at write time so the engine's hot path is a single comparison.
NextRun time.Time `json:"next_run"`
// LastRun is the last run time reported by MarkRun, nil if the job
// has never run. Mutated only by MarkRun. The store does not know
// whether the run succeeded — that judgement belongs to the runner.
LastRun *time.Time `json:"last_run,omitempty"`
// Created and Updated are stamped by Create / Update respectively.
// They are audit metadata; the engine does not read them.
Created time.Time `json:"created"`
Updated time.Time `json:"updated"`
}
// Target identifies where the job's output goes (Slice 2 fills this in).
// Empty Target is valid — Slice 1 stores the field but no consumer reads
// it yet.
type Target struct {
// ChatID is the chat channel id (Telegram chat id, email address,
// webhook URL — depends on the channel). Empty defers to a default
// chosen by the delivery runner.
ChatID string `json:"chat_id,omitempty"`
}
// Payload is the content sent at run time. Slice 2 fills this in; Slice 1
// stores the text verbatim so the engine does not need to know about it.
type Payload struct {
// Text is the literal message body. Templating (cron blueprints) is
// a Slice 3 concern and is deliberately absent here.
Text string `json:"text,omitempty"`
}
// Patch is the partial JobSpec Update merges over the existing record. A
// nil pointer field leaves the corresponding persisted field untouched;
// the non-nil pointer or non-zero value overwrites it.
//
// Schedule is replaced wholesale when the Patch's Schedule pointer is
// non-nil — schedules are not deep-merged, because each field of
// Schedule carries semantics (Kind, Interval, Cron, At) that have no
// meaningful "preserve" answer on their own.
type Patch struct {
Pool *string `json:"pool,omitempty"`
Detail *string `json:"detail,omitempty"`
Schedule *Schedule `json:"schedule,omitempty"`
Target *Target `json:"target,omitempty"`
Payload *Payload `json:"payload,omitempty"`
}
// envelope is the on-disk shape. Kept private so the only public surface
// is JobSpec / Patch.
type envelope struct {
SchemaVersion int `json:"schema_version"`
Jobs []JobSpec `json:"jobs"`
}
// Store is one open handle to the job file. It is safe for concurrent
// callers in the same process, and safe against concurrent CLI / daemon
// processes through the cross-process lock.
//
// All exported methods take a context for cancellation; reads and writes
// are short, so the ctx is plumbed through but rarely observed.
//
// After Close, every other method returns ErrClosed. Close itself is
// safe to call multiple times.
type Store struct {
path string // jobs.json
lock *lockedFile
// closed is set by Close so concurrent callers fail fast rather than
// dereferencing a closed lock fd.
closed atomic.Bool
// mu guards the in-memory copy that mirrors the file. Acquire it
// before reading or writing jobs; the cross-process lock guards
// the file itself. They must always be taken in the same order:
// cross-process lock first, then mu.
mu sync.Mutex
buf []JobSpec // in-memory mirror of the on-disk jobs array
}
// ErrClosed is returned by every method after Close. It lets callers
// distinguish "I gave up" from a real error.
var ErrClosed = errors.New("cronstore: store is closed")
// Open constructs a Store for the given jobs.json path. The file is
// created lazily on first write, but the parent directory is created
// immediately so a subsequent Create does not race against mkdir.
//
// Open validates the persisted file's schema version: a future version
// the build does not know returns ErrUnsupportedSchema rather than a
// silent best-effort decode. A corrupt or truncated file returns
// ErrCorruptFile.
func Open(path string) (*Store, error) {
if path == "" {
return nil, fmt.Errorf("%w: path must be non-empty", ErrInvalidSpec)
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return nil, fmt.Errorf("cronstore: create parent directory: %w", err)
}
lock, err := openLock(path + ".lock")
if err != nil {
return nil, fmt.Errorf("cronstore: open lock: %w", err)
}
s := &Store{
path: path,
lock: lock,
}
// Best-effort hydrate. An absent file is fine: the store will
// materialise it on first Create.
if err := s.hydrate(); err != nil {
_ = lock.Close()
return nil, err
}
return s, nil
}
// Path returns the file the store reads and writes. Exposed for
// diagnostics and for the CLI to print the location it is editing.
func (s *Store) Path() string { return s.path }
// Close releases the cross-process lock. After Close, every other
// method returns ErrClosed. Close is safe to call more than once.
func (s *Store) Close() error {
if !s.closed.CompareAndSwap(false, true) {
return nil
}
if s.lock == nil {
return nil
}
err := s.lock.Close()
s.lock = nil
return err
}
// checkOpen returns ErrClosed when the store has been closed. Every
// public method calls this first so a Close mid-operation surfaces as a
// real error rather than a panic on a nil lock fd.
func (s *Store) checkOpen() error {
if s.closed.Load() {
return ErrClosed
}
return nil
}
// Create adds a new job. Fails with ErrJobExists on a duplicate id, with
// ErrInvalidSpec on a structurally invalid spec. The store stamps
// Created / Updated to now and computes NextRun from the schedule.
func (s *Store) Create(ctx context.Context, spec JobSpec) error {
if err := ctx.Err(); err != nil {
return err
}
if err := s.checkOpen(); err != nil {
return err
}
if strings.TrimSpace(spec.ID) == "" {
return fmt.Errorf("%w: id must be non-empty", ErrInvalidSpec)
}
if err := spec.Schedule.Validate(); err != nil {
return err
}
if err := scheduling.Pool(spec.Pool).Validate(); err != nil {
return err
}
if err := s.withLock(func() error {
for _, existing := range s.buf {
if existing.ID == spec.ID {
return fmt.Errorf("%w: id %q", ErrJobExists, spec.ID)
}
}
now := time.Now().UTC()
// Honor a caller-supplied NextRun when present: the test suite
// needs to set up past/future due states deterministically, and
// the CLI may want to schedule a one-off for a specific time.
// Zero NextRun falls back to the schedule's first-run arithmetic.
if spec.NextRun.IsZero() {
next, err := spec.Schedule.firstRun(now)
if err != nil {
return err
}
spec.NextRun = next.UTC()
}
spec.Created = now
spec.Updated = now
spec.Schedule = spec.Schedule.resolve()
s.buf = append(s.buf, spec)
return s.writeLocked()
}); err != nil {
return err
}
return nil
}
// Get returns the job by id. The second return value is false when the
// id is absent; the error is reserved for I/O failures. The returned
// JobSpec is a deep copy: callers can mutate LastRun and other fields
// without disturbing the store's in-memory mirror.
func (s *Store) Get(ctx context.Context, id string) (JobSpec, bool, error) {
if err := ctx.Err(); err != nil {
return JobSpec{}, false, err
}
if err := s.checkOpen(); err != nil {
return JobSpec{}, false, err
}
var (
got JobSpec
ok bool
)
err := s.withLock(func() error {
for _, j := range s.buf {
if j.ID == id {
got = deepCopyJob(j)
ok = true
return nil
}
}
return nil
})
return got, ok, err
}
// List returns a copy of every persisted job. Order is the on-disk
// order, which is also the creation order. Callers may mutate the
// returned slice and its elements without disturbing the store's
// in-memory mirror.
func (s *Store) List(ctx context.Context) ([]JobSpec, error) {
if err := ctx.Err(); err != nil {
return nil, err
}
if err := s.checkOpen(); err != nil {
return nil, err
}
var out []JobSpec
err := s.withLock(func() error {
out = make([]JobSpec, len(s.buf))
for i := range s.buf {
out[i] = deepCopyJob(s.buf[i])
}
return nil
})
return out, err
}
// Update applies a Patch over the existing record. Fields the Patch
// leaves nil are preserved verbatim; non-nil pointers and the Schedule
// are wholesale replacements. Schedule changes trigger a NextRun
// recomputation, since NextRun is derived from Schedule and LastRun.
//
// Returns ErrJobNotFound when the id is absent.
func (s *Store) Update(ctx context.Context, id string, patch Patch) error {
if err := ctx.Err(); err != nil {
return err
}
if err := s.checkOpen(); err != nil {
return err
}
if id == "" {
return fmt.Errorf("%w: id must be non-empty", ErrInvalidSpec)
}
if patch.Pool != nil {
if err := scheduling.Pool(*patch.Pool).Validate(); err != nil {
return err
}
}
if patch.Schedule != nil {
if err := patch.Schedule.Validate(); err != nil {
return err
}
}
return s.withLock(func() error {
for i := range s.buf {
if s.buf[i].ID != id {
continue
}
if err := applyPatch(&s.buf[i], patch); err != nil {
return err
}
s.buf[i].Updated = time.Now().UTC()
return s.writeLocked()
}
return fmt.Errorf("%w: id %q", ErrJobNotFound, id)
})
}
// applyPatch overwrites record's non-nil Patch fields in place, recomputing
// NextRun when Schedule changes.
func applyPatch(record *JobSpec, patch Patch) error {
if patch.Pool != nil {
record.Pool = *patch.Pool
}
if patch.Detail != nil {
record.Detail = *patch.Detail
}
if patch.Schedule != nil {
if err := applyScheduleChange(record, patch.Schedule); err != nil {
return err
}
}
if patch.Target != nil {
record.Target = *patch.Target
}
if patch.Payload != nil {
record.Payload = *patch.Payload
}
return nil
}
// applyScheduleChange sets record's Schedule and recomputes NextRun, which
// is stale once Schedule changes. Recomputed from the last run if there is
// one, else as a first run from now — keeping it strictly future-of-now
// would re-fire a job the operator just shortened.
func applyScheduleChange(record *JobSpec, schedule *Schedule) error {
record.Schedule = schedule.resolve()
var next time.Time
var err error
if record.LastRun != nil {
next, err = record.Schedule.nextRun(*record.LastRun)
} else {
next, err = record.Schedule.firstRun(time.Now().UTC())
}
if err != nil {
return err
}
record.NextRun = next.UTC()
return nil
}
// Delete removes the job by id. Idempotent: deleting a missing id
// returns nil rather than ErrJobNotFound, so a CLI retry after a
// partial failure cannot stall the operator.
func (s *Store) Delete(ctx context.Context, id string) error {
if err := ctx.Err(); err != nil {
return err
}
if err := s.checkOpen(); err != nil {
return err
}
if id == "" {
return fmt.Errorf("%w: id must be non-empty", ErrInvalidSpec)
}
return s.withLock(func() error {
for i := range s.buf {
if s.buf[i].ID != id {
continue
}
s.buf = append(s.buf[:i], s.buf[i+1:]...)
return s.writeLocked()
}
return nil
})
}
// MarkRun records that id ran at runAt and advances NextRun to the
// schedule's next firing. It is the engine-side counterpart to Create:
// once the ticker has dispatched a job, the runner (or the engine
// itself, in tests) calls MarkRun to keep next_run fresh.
//
// Returns ErrJobNotFound when the id is absent and
// ErrScheduleUnsupported when the schedule kind has no defined
// "next after this run" semantics.
func (s *Store) MarkRun(ctx context.Context, id string, runAt time.Time) error {
if err := ctx.Err(); err != nil {
return err
}
if err := s.checkOpen(); err != nil {
return err
}
if id == "" {
return fmt.Errorf("%w: id must be non-empty", ErrInvalidSpec)
}
return s.withLock(func() error {
for i := range s.buf {
if s.buf[i].ID != id {
continue
}
next, err := s.buf[i].Schedule.nextRun(runAt)
if err != nil {
return err
}
t := runAt.UTC()
s.buf[i].LastRun = &t
s.buf[i].NextRun = next.UTC()
s.buf[i].Updated = time.Now().UTC()
return s.writeLocked()
}
return fmt.Errorf("%w: id %q", ErrJobNotFound, id)
})
}
// Due implements scheduling.JobSource. It returns the jobs whose
// NextRun is at or before now, in creation order. The returned
// scheduling.Job is the engine's projection of the persisted JobSpec:
// only the fields the engine relies on (ID, Pool, Detail) cross the
// boundary. Schedule / Payload / Target are deliberately not exposed:
// the engine does no arithmetic on them and has no business knowing
// they exist.
//
// A job with an unrecognised pool is skipped: Create / Update enforce
// the set, but a file edited by hand could carry an invalid value, and
// the engine cannot reason about a pool it has never heard of. The
// store does not crash on bad input — the same way it does not panic
// on a truncated file.
func (s *Store) Due(ctx context.Context, now time.Time) ([]scheduling.Job, error) {
if err := ctx.Err(); err != nil {
return nil, err
}
if err := s.checkOpen(); err != nil {
return nil, err
}
var due []scheduling.Job
err := s.withLock(func() error {
for _, j := range s.buf {
if j.NextRun.After(now) {
continue
}
pool := scheduling.Pool(j.Pool)
if err := pool.Validate(); err != nil {
continue
}
due = append(due, scheduling.Job{
ID: j.ID,
Pool: pool,
Detail: j.Detail,
})
}
return nil
})
return due, err
}
// withLock serialises access to the file. Callers receive a closure
// that runs with the cross-process lock held and the in-memory buffer
// freshly hydrated from disk; the closure's return value becomes
// withLock's return value.
//
// Three locks stack, always in this order:
//
// 1. processMu: keyed by lock-file path. Serialises every Store that
// points at the same jobs.json within one process. Necessary
// because Linux flock is per file description: two Store instances
// in the same process have independent fds and would otherwise
// proceed in parallel. fcntl / POSIX locks would not have this
// property — they would block — but flock is the documented
// primitive on Linux + macOS, so we compensate at this layer.
// 2. cross-process flock: serialises Store instances across
// processes (CLI vs daemon). flock alone is insufficient for
// in-process coordination (see #1).
// 3. s.mu: guards s.buf against same-Store concurrent goroutines.
// Redundant under #1 + #2 but cheap, and it documents the
// invariant for any reader trying to follow the locking story.
//
// After all three are held, the in-memory buffer is re-read from disk
// so a writer that just acquired the locks sees every change the
// previous writer committed. Without this, a Store that opened before
// any writes happened would carry an empty view forever and silently
// overwrite everything.
//
// Re-reading on every operation is acceptable for cron workloads: the
// file holds O(dozens) of jobs, not O(thousands), and operations are
// human-paced, not high-throughput.
func (s *Store) withLock(fn func() error) error {
processMu := processLockFor(s.path + ".lock")
processMu.Lock()
defer processMu.Unlock()
if s.closed.Load() {
return ErrClosed
}
if err := s.lock.Lock(); err != nil {
return fmt.Errorf("cronstore: acquire lock: %w", err)
}
defer func() { _ = s.lock.Unlock() }()
s.mu.Lock()
defer s.mu.Unlock()
if err := s.hydrate(); err != nil {
return err
}
return fn()
}
// hydrate reads the file into the in-memory buffer. Called from Open
// (no lock needed; nothing else can see the Store yet) and from
// withLock (with both locks held).
func (s *Store) hydrate() error {
data, err := os.ReadFile(s.path)
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
return nil
}
return fmt.Errorf("%w: read %s: %w", ErrCorruptFile, s.path, err)
}
if len(bytes.TrimSpace(data)) == 0 {
return nil
}
var env envelope
dec := json.NewDecoder(bytes.NewReader(data))
// Reject unknown fields so a typo on the operator's side does not
// silently land in a future field and shadow a real value. The
// cost is that any additive change to JobSpec MUST bump
// schemaVersion — see the comment on that constant.
dec.DisallowUnknownFields()
if err := dec.Decode(&env); err != nil {
return fmt.Errorf("%w: decode %s: %w", ErrCorruptFile, s.path, err)
}
if env.SchemaVersion == 0 {
return fmt.Errorf("%w: missing schema_version in %s", ErrCorruptFile, s.path)
}
if env.SchemaVersion > schemaVersion {
return fmt.Errorf("%w: file is version %d, this build understands up to %d",
ErrUnsupportedSchema, env.SchemaVersion, schemaVersion)
}
s.buf = env.Jobs
return nil
}
// writeLocked persists the in-memory buffer to the file. Must be called
// with s.mu held and the cross-process lock held — both guarantees
// follow from withLock. Atomic write via temp-file + rename, so a
// crash mid-write leaves the previous file intact rather than a
// truncated mess the next Open would have to clean up.
func (s *Store) writeLocked() error {
env := envelope{
SchemaVersion: schemaVersion,
Jobs: s.buf,
}
var buf bytes.Buffer
enc := json.NewEncoder(&buf)
// operator-readable file: do not escape <, >, &. The default
// encoding/json behaviour writes these as < etc, which is
// round-trip-safe but unreadable when a human opens jobs.json to
// understand why a job failed to fire.
enc.SetEscapeHTML(false)
if err := enc.Encode(env); err != nil {
return fmt.Errorf("cronstore: encode: %w", err)
}
if err := atomicWriteFile(s.path, buf.Bytes(), filePerm); err != nil {
return fmt.Errorf("cronstore: write %s: %w", s.path, err)
}
return nil
}
// atomicWriteFile writes data to path via a sibling temp file and
// rename. Rename within a single filesystem is atomic on POSIX, so a
// concurrent reader either sees the previous file in full or the new
// one in full — never a half-written body. Sync before rename gives a
// durability guarantee: the inode's data is on disk before the
// directory entry switches over.
func atomicWriteFile(path string, data []byte, perm os.FileMode) error {
dir := filepath.Dir(path)
tmp, err := os.CreateTemp(dir, filepath.Base(path)+".tmp-*")
if err != nil {
return err
}
tmpName := tmp.Name()
defer func() {
// Best-effort cleanup on any failure path. The rename has
// either happened (and tmpName is gone) or it has not (and
// the temp file still needs to go).
_ = os.Remove(tmpName)
}()
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
return err
}
if err := tmp.Sync(); err != nil {
_ = tmp.Close()
return err
}
if err := tmp.Close(); err != nil {
return err
}
if err := os.Chmod(tmpName, perm); err != nil {
return err
}
return os.Rename(tmpName, path)
}
// deepCopyJob returns a JobSpec value independent of the source's
// pointer fields. JobSpec carries one pointer (LastRun); a shallow
// copy aliases it and lets callers mutate the store's in-memory state
// by writing through the returned value.
func deepCopyJob(j JobSpec) JobSpec {
out := j
if j.LastRun != nil {
t := *j.LastRun
out.LastRun = &t
}
if j.Schedule.At != nil {
t := *j.Schedule.At
out.Schedule.At = &t
}
return out
}
@@ -1,485 +0,0 @@
// Additional tests added in response to the Slice-1 adversarial review.
// They live in a separate file so the original spec file stays focused
// on the contract-level behaviour the engine depends on, and these
// cases stay grouped where the reviewer can re-read them as a unit.
package cronstore
import (
"bytes"
"context"
"errors"
"os"
"os/exec"
"path/filepath"
"sync"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
)
// --- C2 + P8: ErrCorruptFile is the right sentinel for bad files -------
func TestOpenTruncatedFileReturnsErrCorruptFile(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
if err := os.WriteFile(path, []byte(`{"schema_version":1,"jobs":[{"id":"x","`), 0o600); err != nil {
t.Fatalf("WriteFile: %v", err)
}
_, err := Open(path)
if !errors.Is(err, ErrCorruptFile) {
t.Fatalf("Open on truncated file = %v, want ErrCorruptFile", err)
}
}
func TestOpenGarbageFileReturnsErrCorruptFile(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
if err := os.WriteFile(path, []byte("not json at all"), 0o600); err != nil {
t.Fatalf("WriteFile: %v", err)
}
_, err := Open(path)
if !errors.Is(err, ErrCorruptFile) {
t.Fatalf("Open on garbage file = %v, want ErrCorruptFile", err)
}
}
func TestOpenEmptyFileIsTreatedAsNoJobs(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
if err := os.WriteFile(path, []byte(" \n\t "), 0o600); err != nil {
t.Fatalf("WriteFile: %v", err)
}
s, err := Open(path)
if err != nil {
t.Fatalf("Open on whitespace-only file = %v, want nil", err)
}
defer s.Close()
jobs, _ := s.List(context.Background())
if len(jobs) != 0 {
t.Errorf("List on whitespace-only file = %d jobs, want 0", len(jobs))
}
}
// --- C6 / P8: DisallowUnknownFields rejects hand-edited extras ---------
func TestOpenRejectsUnknownFieldAtSameSchemaVersion(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
// A future build might add a "tags" field to JobSpec. An older
// build with DisallowUnknownFields set must reject the file rather
// than silently dropping the field — that's the contract behind
// bumping schemaVersion on additive changes.
contents := `{"schema_version":1,"jobs":[{"id":"x","pool":"parallel","schedule":{"kind":"interval","interval":60000000000},"target":{},"payload":{},"next_run":"2026-08-28T12:00:00Z","created":"2026-08-28T12:00:00Z","updated":"2026-08-28T12:00:00Z","tags":["ops"]}]}`
if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
t.Fatalf("WriteFile: %v", err)
}
_, err := Open(path)
if !errors.Is(err, ErrCorruptFile) {
t.Fatalf("Open on file with unknown field = %v, want ErrCorruptFile", err)
}
}
// --- P2: Get / List return deep copies -------------------------------
func TestGetReturnsDeepCopyOfLastRun(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("x", time.Hour))
first, _, _ := s.Get(context.Background(), "x")
if first.LastRun != nil {
t.Fatalf("LastRun on fresh create = %v, want nil", first.LastRun)
}
runAt := time.Now().UTC().Add(-time.Minute)
if err := s.MarkRun(context.Background(), "x", runAt); err != nil {
t.Fatalf("MarkRun: %v", err)
}
got, _, _ := s.Get(context.Background(), "x")
if got.LastRun == nil {
t.Fatal("LastRun is nil after MarkRun")
}
// Caller mutates the returned LastRun. The store's in-memory
// value must NOT change — a future Get must still report the
// original runAt.
*got.LastRun = time.Date(2099, 1, 1, 0, 0, 0, 0, time.UTC)
got2, _, _ := s.Get(context.Background(), "x")
if !got2.LastRun.Equal(runAt) {
t.Errorf("after caller mutation, Get returned LastRun=%v, want %v", got2.LastRun, runAt)
}
}
func TestListReturnsDeepCopies(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("x", time.Hour))
mustCreate(t, s, fixedIntervalSpec("y", time.Hour))
list1, _ := s.List(context.Background())
if len(list1) != 2 {
t.Fatalf("List returned %d jobs, want 2", len(list1))
}
// Caller mutates the returned slice (deletes an entry) and a job's
// string fields. Neither must touch the store's in-memory mirror.
list1[0].Detail = "tampered"
list2, _ := s.List(context.Background())
if len(list2) != 2 {
t.Errorf("after caller truncated list, second List returned %d jobs, want 2", len(list2))
}
for _, j := range list2 {
if j.Detail == "tampered" {
t.Errorf("caller's Detail mutation leaked into the store: %+v", j)
}
}
}
// --- P3: pool validation in Create / Update ---------------------------
func TestCreateRejectsInvalidPool(t *testing.T) {
s := newTestStore(t)
spec := fixedIntervalSpec("x", time.Minute)
spec.Pool = "paralle" // typo
if err := s.Create(context.Background(), spec); err == nil {
t.Fatal("Create with invalid pool succeeded; want error")
}
}
func TestUpdateRejectsInvalidPool(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("x", time.Minute))
if err := s.Update(context.Background(), "x", Patch{Pool: new("paralle")}); err == nil {
t.Fatal("Update with invalid pool succeeded; want error")
}
}
// --- P5 / C1 / P6: Once schedule behaviour + MarkRun sentinel ---------
func TestCreateAcceptsOnceSchedule(t *testing.T) {
s := newTestStore(t)
at := time.Now().Add(time.Hour)
spec := JobSpec{
ID: "one-off",
Schedule: Schedule{
Kind: ScheduleOnce,
At: &at,
},
}
if err := s.Create(context.Background(), spec); err != nil {
t.Fatalf("Create with Once schedule: %v", err)
}
}
func TestDueFiresOnceAtConfiguredTime(t *testing.T) {
s := newTestStore(t)
at := time.Date(2026, 8, 28, 12, 0, 0, 0, time.UTC)
spec := JobSpec{
ID: "one-off",
Schedule: Schedule{
Kind: ScheduleOnce,
At: &at,
},
}
if err := s.Create(context.Background(), spec); err != nil {
t.Fatalf("Create: %v", err)
}
due, err := s.Due(context.Background(), at)
if err != nil {
t.Fatalf("Due: %v", err)
}
if len(due) != 1 || due[0].ID != "one-off" {
t.Errorf("Due(at) = %+v, want one-off", due)
}
due, _ = s.Due(context.Background(), at.Add(-time.Second))
if len(due) != 0 {
t.Errorf("Due(at - 1s) = %+v, want empty", due)
}
}
func TestMarkRunOnOnceReturnsErrScheduleUnsupported(t *testing.T) {
s := newTestStore(t)
at := time.Now().Add(time.Hour)
spec := JobSpec{
ID: "one-off",
Schedule: Schedule{
Kind: ScheduleOnce,
At: &at,
},
}
if err := s.Create(context.Background(), spec); err != nil {
t.Fatalf("Create: %v", err)
}
err := s.MarkRun(context.Background(), "one-off", time.Now())
if !errors.Is(err, ErrScheduleUnsupported) {
t.Fatalf("MarkRun on once = %v, want ErrScheduleUnsupported", err)
}
}
// --- P8: Due drops unknown pool values silently -----------------------
func TestDueSilentlySkipsUnknownPool(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
s, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer s.Close()
// Hand-edit the file: bypass Create's validation by writing a
// job with an invalid pool directly to disk.
past := time.Now().Add(-time.Minute).UTC().Format(time.RFC3339Nano)
contents := `{"schema_version":1,"jobs":[{"id":"badpool","pool":"paralle","schedule":{"kind":"interval","interval":60000000000},"target":{},"payload":{},"next_run":"` + past + `","created":"` + past + `","updated":"` + past + `"}]}`
if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
t.Fatalf("WriteFile: %v", err)
}
// Drop the in-memory mirror so the next read picks up the file.
if err := s.hydrate(); err != nil {
t.Fatalf("hydrate: %v", err)
}
due, err := s.Due(context.Background(), time.Now())
if err != nil {
t.Fatalf("Due: %v", err)
}
if len(due) != 0 {
t.Errorf("Due returned %d jobs for an invalid pool, want 0", len(due))
}
// But Get still sees the job — Create/Update validation is the
// gate that prevents this in normal operation; a hand-edited
// file remains visible through CRUD.
if _, ok, _ := s.Get(context.Background(), "badpool"); !ok {
t.Error("Get returned ok=false for a hand-edited job with bad pool")
}
}
// --- P10: UTF-8 + quote + newline round-trip --------------------------
func TestCreateAndGetRoundTripWithUnicodeAndQuotes(t *testing.T) {
s := newTestStore(t)
spec := JobSpec{
ID: "ops-standup",
Pool: string(scheduling.PoolSequential),
Detail: "Hello\n\"世界\"\\back",
Schedule: Schedule{
Kind: ScheduleInterval,
Interval: Duration(30 * time.Minute),
},
Payload: Payload{Text: "<script>alert(1)</script> & 中文"},
}
mustCreate(t, s, spec)
got, ok, err := s.Get(context.Background(), spec.ID)
if err != nil {
t.Fatalf("Get: %v", err)
}
if !ok {
t.Fatal("Get returned ok=false")
}
if got.Detail != spec.Detail {
t.Errorf("Detail = %q, want %q", got.Detail, spec.Detail)
}
if got.Payload.Text != spec.Payload.Text {
t.Errorf("Payload.Text = %q, want %q", got.Payload.Text, spec.Payload.Text)
}
}
// --- C3: operator-readable file does not HTML-escape -------------------
func TestFileIsOperatorReadableNoHTMLEscape(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
s, _ := Open(path)
defer s.Close()
spec := fixedIntervalSpec("x", time.Minute)
spec.Payload.Text = "<a&b>"
mustCreate(t, s, spec)
_ = s.Close()
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
// With SetEscapeHTML(false), < > & appear as the literal characters.
// The escaped form would be <, >, & — none of which
// is the printable payload. Assert the literal is there.
if !bytes.Contains(data, []byte(`"<a&b>"`)) {
t.Errorf("payload was HTML-escaped; want operator-readable literal: %s", data)
}
}
// --- C4: file is 0o600, not 0o644 --------------------------------------
func TestFileIsCreatedWithOwnerOnlyPermissions(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
s, _ := Open(path)
defer s.Close()
mustCreate(t, s, fixedIntervalSpec("x", time.Minute))
_ = s.Close()
info, err := os.Stat(path)
if err != nil {
t.Fatalf("Stat: %v", err)
}
perm := info.Mode().Perm()
if perm != 0o600 {
t.Errorf("jobs.json perm = %#o, want 0o600", perm)
}
}
// --- C5: cross-process flock via os/exec ------------------------------
// TestCrossProcessFlockBlocksParentWhileChildHoldsLock runs a child
// process that grabs the cross-process flock and sleeps; the parent
// then tries to take the same lock and must block for at least the
// child's sleep duration. Without the flock, the parent's Create
// would return immediately and the test would fail.
func TestCrossProcessFlockBlocksParentWhileChildHoldsLock(t *testing.T) {
if testing.Short() {
t.Skip("subprocess test")
}
if _, err := exec.LookPath("go"); err != nil {
t.Skip("no go toolchain available for subprocess test")
}
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
ctx := t.Context()
// Build a tiny helper that opens the same file and holds the
// flock. The helper is built once per run; the cost is paid only
// when this test runs.
helperSrc := filepath.Join(dir, "helper.go")
if err := os.WriteFile(helperSrc, []byte(`
package main
import (
"fmt"
"os"
"syscall"
"time"
)
func main() {
f, err := os.OpenFile(os.Args[1]+".lock", os.O_RDWR|os.O_CREATE, 0o644)
if err != nil { fmt.Println("open:", err); os.Exit(2) }
if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX); err != nil {
fmt.Println("flock:", err); os.Exit(3)
}
fmt.Println("locked")
// Signal readiness, then hold the lock long enough for the
// parent to observe the block.
time.Sleep(2 * time.Second)
_ = f.Close()
}
`), 0o644); err != nil {
t.Fatalf("WriteFile helper: %v", err)
}
helperBin := filepath.Join(dir, "helper")
if out, err := exec.CommandContext(ctx, "go", "build", "-o", helperBin, helperSrc).CombinedOutput(); err != nil {
t.Fatalf("go build helper: %v\n%s", err, out)
}
// Launch the child in the background.
child := exec.CommandContext(ctx, helperBin, path)
child.Stdout = os.Stdout
child.Stderr = os.Stderr
if err := child.Start(); err != nil {
t.Fatalf("child Start: %v", err)
}
defer func() {
_ = child.Process.Kill()
_ = child.Wait()
}()
// Give the child a moment to acquire the lock.
time.Sleep(300 * time.Millisecond)
// From the parent, open the same Store and call Create. Without
// flock, Create would return instantly; with flock, it must
// block until the child releases.
parent, err := Open(path)
if err != nil {
t.Fatalf("parent Open: %v", err)
}
defer parent.Close()
start := time.Now()
if err := parent.Create(context.Background(), fixedIntervalSpec("from-parent", time.Minute)); err != nil {
t.Fatalf("parent Create: %v", err)
}
elapsed := time.Since(start)
// The child held the lock for ~2s. The parent's Create must have
// blocked for at least the overlap between its start and the
// child's release — conservatively, >1s.
if elapsed < time.Second {
t.Errorf("parent Create returned in %v; expected to block on the child's flock", elapsed)
}
}
// --- P1: Close mid-operation returns ErrClosed ------------------------
func TestMethodsReturnErrClosedAfterClose(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("x", time.Minute))
if err := s.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
// Every public method must surface ErrClosed, not panic on the
// nil lock fd.
tests := []struct {
name string
fn func() error
}{
{"Create", func() error { return s.Create(context.Background(), fixedIntervalSpec("y", time.Minute)) }},
{"Get", func() error { _, _, err := s.Get(context.Background(), "x"); return err }},
{"List", func() error { _, err := s.List(context.Background()); return err }},
{"Update", func() error { return s.Update(context.Background(), "x", Patch{Detail: new("z")}) }},
{"Delete", func() error { return s.Delete(context.Background(), "x") }},
{"MarkRun", func() error { return s.MarkRun(context.Background(), "x", time.Now()) }},
{"Due", func() error { _, err := s.Due(context.Background(), time.Now()); return err }},
}
// Close itself is documented as idempotent (returns nil on the
// second call), so it does NOT appear in the ErrClosed list above.
if err := s.Close(); err != nil {
t.Errorf("second Close = %v, want nil", err)
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
err := tc.fn()
if !errors.Is(err, ErrClosed) {
t.Errorf("%s after Close = %v, want ErrClosed", tc.name, err)
}
})
}
}
// --- P8: same-path stores share the in-process mutex ------------------
func TestTwoStoresForSamePathSerialiseInProcess(t *testing.T) {
// This is the test that fails without process_lock.go. With it,
// 50/50 writes from each side land; without it, half the data
// disappears. Mirrors TestConcurrentWritersAreSerialised but in
// its own right so the regression is named.
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
a, _ := Open(path)
defer a.Close()
b, _ := Open(path)
defer b.Close()
var wg sync.WaitGroup
for i := range 25 {
wg.Add(2)
go func(i int) {
defer wg.Done()
_ = a.Create(context.Background(), fixedIntervalSpec("a"+itoa(i), time.Minute))
}(i)
go func(i int) {
defer wg.Done()
_ = b.Create(context.Background(), fixedIntervalSpec("b"+itoa(i), time.Minute))
}(i)
}
wg.Wait()
jobs, _ := a.List(context.Background())
if len(jobs) != 50 {
t.Errorf("after 50 concurrent writes from two stores, %d persisted, want 50", len(jobs))
}
}
@@ -1,512 +0,0 @@
// Package cronstore is the persistent job store backing the scheduling
// ticker engine. Tests in this file are the spec: an implementing agent
// must make every test in this file pass WITHOUT modifying the tests
// themselves.
//
// The store's only contract with the engine is scheduling.JobSource: one
// file-backed JSON document, cross-process locked, with version-stamped
// schema so future slices can extend it without breaking old files.
package cronstore
import (
"context"
"errors"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/domain/scheduling"
)
// helpers
func newTestStore(t *testing.T) *Store {
t.Helper()
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
s, err := Open(path)
if err != nil {
t.Fatalf("Open(%s): %v", path, err)
}
t.Cleanup(func() { _ = s.Close() })
return s
}
func mustCreate(t *testing.T, s *Store, spec JobSpec) {
t.Helper()
if err := s.Create(context.Background(), spec); err != nil {
t.Fatalf("Create(%q): %v", spec.ID, err)
}
}
func fixedIntervalSpec(id string, every time.Duration) JobSpec {
return JobSpec{
ID: id,
Pool: string(scheduling.PoolParallel),
Detail: "test job " + id,
Schedule: Schedule{
Kind: ScheduleInterval,
Interval: Duration(every),
},
Payload: Payload{Text: "hello " + id},
}
}
// --- open / close ------------------------------------------------------------
func TestOpenMissingFileReturnsEmpty(t *testing.T) {
dir := t.TempDir()
s, err := Open(filepath.Join(dir, "jobs.json"))
if err != nil {
t.Fatalf("Open: %v", err)
}
defer s.Close()
jobs, err := s.List(context.Background())
if err != nil {
t.Fatalf("List: %v", err)
}
if len(jobs) != 0 {
t.Errorf("List on missing file = %d jobs, want 0", len(jobs))
}
}
func TestOpenCreatesParentDirectory(t *testing.T) {
dir := t.TempDir()
deep := filepath.Join(dir, "a", "b", "c", "jobs.json")
s, err := Open(deep)
if err != nil {
t.Fatalf("Open with missing parent: %v", err)
}
defer s.Close()
mustCreate(t, s, fixedIntervalSpec("x", time.Minute))
if _, err := os.Stat(deep); err != nil {
t.Fatalf("expected file to exist at %s: %v", deep, err)
}
}
// --- schema versioning -------------------------------------------------------
func TestPersistedFileHasVersionStamp(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
s, err := Open(path)
if err != nil {
t.Fatalf("Open: %v", err)
}
mustCreate(t, s, fixedIntervalSpec("x", time.Minute))
_ = s.Close()
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
// Asserted against the constant, not a literal: the stamp must track
// schemaVersion through every bump (Kind was the first), and a
// hardcoded value here would fail on the bump as a false alarm
// instead of guarding the property it names.
want := `"schema_version":` + strconv.Itoa(schemaVersion)
if !strings.Contains(string(data), want) {
t.Errorf("file does not contain %s stamp: %s", want, data)
}
}
func TestOpenRejectsUnknownSchemaVersion(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
if err := os.WriteFile(path, []byte(`{"schema_version":999,"jobs":[]}`), 0o644); err != nil {
t.Fatalf("WriteFile: %v", err)
}
if _, err := Open(path); err == nil {
t.Fatal("Open with future schema version succeeded; want error")
} else if !errors.Is(err, ErrUnsupportedSchema) {
t.Errorf("error = %v, want ErrUnsupportedSchema", err)
}
}
// --- create / read -----------------------------------------------------------
func TestCreateAndGetRoundTrip(t *testing.T) {
s := newTestStore(t)
want := JobSpec{
ID: "morning-status",
Pool: string(scheduling.PoolSequential),
Detail: "morning status",
Schedule: Schedule{
Kind: ScheduleInterval,
Interval: Duration(30 * time.Minute),
},
Payload: Payload{Text: "status please"},
}
mustCreate(t, s, want)
got, ok, err := s.Get(context.Background(), want.ID)
if err != nil {
t.Fatalf("Get: %v", err)
}
if !ok {
t.Fatal("Get returned ok=false")
}
if got.ID != want.ID {
t.Errorf("ID = %q, want %q", got.ID, want.ID)
}
if got.Pool != want.Pool {
t.Errorf("Pool = %q, want %q", got.Pool, want.Pool)
}
if got.Detail != want.Detail {
t.Errorf("Detail = %q, want %q", got.Detail, want.Detail)
}
if got.Schedule.Kind != want.Schedule.Kind {
t.Errorf("Schedule.Kind = %q, want %q", got.Schedule.Kind, want.Schedule.Kind)
}
if got.Schedule.Interval != want.Schedule.Interval {
t.Errorf("Schedule.Interval = %v, want %v", got.Schedule.Interval, want.Schedule.Interval)
}
if got.Payload.Text != want.Payload.Text {
t.Errorf("Payload.Text = %q, want %q", got.Payload.Text, want.Payload.Text)
}
if got.NextRun.IsZero() {
t.Error("NextRun is zero; Create must compute it for an interval job")
}
if got.Created.IsZero() {
t.Error("Created is zero; Create must stamp it")
}
if got.Updated.IsZero() {
t.Error("Updated is zero; Create must stamp it")
}
if !got.Created.Equal(got.Updated) {
t.Errorf("Created %v != Updated %v on a fresh create", got.Created, got.Updated)
}
}
func TestGetMissingReturnsNotFound(t *testing.T) {
s := newTestStore(t)
_, ok, err := s.Get(context.Background(), "nope")
if err != nil {
t.Fatalf("Get: %v", err)
}
if ok {
t.Error("Get of missing id returned ok=true")
}
}
func TestCreateRejectsEmptyID(t *testing.T) {
s := newTestStore(t)
if err := s.Create(context.Background(), fixedIntervalSpec("", time.Minute)); err == nil {
t.Fatal("Create with empty id succeeded; want error")
}
}
func TestCreateRejectsUnknownScheduleKind(t *testing.T) {
s := newTestStore(t)
spec := fixedIntervalSpec("x", time.Minute)
spec.Schedule.Kind = "magic"
if err := s.Create(context.Background(), spec); err == nil {
t.Fatal("Create with unknown schedule kind succeeded; want error")
}
}
func TestCreateRejectsDuplicateID(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("dup", time.Minute))
if err := s.Create(context.Background(), fixedIntervalSpec("dup", time.Hour)); err == nil {
t.Fatal("Create with duplicate id succeeded; want error")
}
}
// --- update with merge semantics --------------------------------------------
func TestUpdatePreservesFieldsNotInPatch(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, JobSpec{
ID: "patch-me",
Pool: string(scheduling.PoolParallel),
Detail: "original detail",
Schedule: Schedule{
Kind: ScheduleInterval,
Interval: Duration(10 * time.Minute),
},
Payload: Payload{Text: "original payload"},
})
if err := s.Update(context.Background(), "patch-me", Patch{
Detail: new("updated detail"),
}); err != nil {
t.Fatalf("Update: %v", err)
}
got, _, _ := s.Get(context.Background(), "patch-me")
if got.Detail != "updated detail" {
t.Errorf("Detail = %q, want %q", got.Detail, "updated detail")
}
// Untouched fields must survive.
if got.Pool != string(scheduling.PoolParallel) {
t.Errorf("Pool = %q, want %q (merge must preserve)", got.Pool, scheduling.PoolParallel)
}
if got.Schedule.Interval.Std() != 10*time.Minute {
t.Errorf("Schedule.Interval = %v, want %v (merge must preserve)", got.Schedule.Interval, 10*time.Minute)
}
if got.Payload.Text != "original payload" {
t.Errorf("Payload.Text = %q, want %q (merge must preserve)", got.Payload.Text, "original payload")
}
}
func TestUpdateAdvancesUpdatedTimestamp(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("ts", time.Minute))
first, _, _ := s.Get(context.Background(), "ts")
// Sleep long enough that Updated must move forward even on coarse
// clocks. mtime resolution on some filesystems is the second.
time.Sleep(1100 * time.Millisecond)
if err := s.Update(context.Background(), "ts", Patch{Detail: new("new")}); err != nil {
t.Fatalf("Update: %v", err)
}
got, _, _ := s.Get(context.Background(), "ts")
if !got.Updated.After(first.Updated) {
t.Errorf("Updated did not advance: first=%v got=%v", first.Updated, got.Updated)
}
}
func TestUpdateRecomputesNextRunWhenScheduleChanges(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("rs", time.Hour))
first, _, _ := s.Get(context.Background(), "rs")
// Change the interval; NextRun must move.
if err := s.Update(context.Background(), "rs", Patch{
Schedule: &Schedule{Kind: ScheduleInterval, Interval: Duration(2 * time.Minute)},
}); err != nil {
t.Fatalf("Update: %v", err)
}
got, _, _ := s.Get(context.Background(), "rs")
if got.Schedule.Interval.Std() != 2*time.Minute {
t.Errorf("Schedule.Interval = %v, want %v", got.Schedule.Interval, 2*time.Minute)
}
if !got.NextRun.Before(first.NextRun) {
t.Errorf("NextRun did not move earlier after shortening interval: first=%v got=%v", first.NextRun, got.NextRun)
}
}
func TestUpdateMissingReturnsNotFound(t *testing.T) {
s := newTestStore(t)
if err := s.Update(context.Background(), "ghost", Patch{Detail: new("x")}); err == nil {
t.Fatal("Update of missing id succeeded; want error")
}
}
// --- delete ------------------------------------------------------------------
func TestDeleteRemovesJob(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("bye", time.Minute))
if err := s.Delete(context.Background(), "bye"); err != nil {
t.Fatalf("Delete: %v", err)
}
if _, ok, _ := s.Get(context.Background(), "bye"); ok {
t.Error("Get after Delete returned ok=true")
}
}
func TestDeleteMissingIsIdempotent(t *testing.T) {
s := newTestStore(t)
// First delete is a no-op on a missing id; second delete must not panic
// and must not return an error. CLI retries should be safe.
if err := s.Delete(context.Background(), "never-existed"); err != nil {
t.Errorf("Delete(missing) = %v, want nil", err)
}
if err := s.Delete(context.Background(), "never-existed"); err != nil {
t.Errorf("Delete(missing) again = %v, want nil", err)
}
}
// --- list --------------------------------------------------------------------
func TestListReturnsAllJobs(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("a", time.Minute))
mustCreate(t, s, fixedIntervalSpec("b", time.Minute))
mustCreate(t, s, fixedIntervalSpec("c", time.Minute))
jobs, err := s.List(context.Background())
if err != nil {
t.Fatalf("List: %v", err)
}
if len(jobs) != 3 {
t.Errorf("List returned %d jobs, want 3", len(jobs))
}
}
// --- JobSource.Due -----------------------------------------------------------
func TestDueReturnsOnlyJobsWhoseNextRunIsInThePast(t *testing.T) {
s := newTestStore(t)
now := time.Date(2026, 8, 28, 12, 0, 0, 0, time.UTC)
// past: due
past := fixedIntervalSpec("past", 30*time.Minute)
past.NextRun = now.Add(-time.Minute)
// future: not due
future := fixedIntervalSpec("future", 30*time.Minute)
future.NextRun = now.Add(time.Hour)
mustCreate(t, s, past)
mustCreate(t, s, future)
got, err := s.Due(context.Background(), now)
if err != nil {
t.Fatalf("Due: %v", err)
}
if len(got) != 1 {
t.Fatalf("Due returned %d jobs, want 1: %+v", len(got), got)
}
if got[0].ID != "past" {
t.Errorf("due job = %q, want %q", got[0].ID, "past")
}
if got[0].Pool != scheduling.PoolParallel {
t.Errorf("due job pool = %q, want %q", got[0].Pool, scheduling.PoolParallel)
}
if got[0].Detail != "test job past" {
t.Errorf("due job detail = %q, want %q", got[0].Detail, "test job past")
}
}
func TestDueOnEmptyStoreReturnsEmptySlice(t *testing.T) {
s := newTestStore(t)
got, err := s.Due(context.Background(), time.Now())
if err != nil {
t.Fatalf("Due: %v", err)
}
if len(got) != 0 {
t.Errorf("Due on empty = %d jobs, want 0", len(got))
}
}
func TestImplementsSchedulingJobSource(t *testing.T) {
// Static, compile-time check: *Store must satisfy scheduling.JobSource
// so the ticker engine can poll it without knowing the file layout.
var _ scheduling.JobSource = (*Store)(nil)
}
func TestMarkRunAdvancesNextRun(t *testing.T) {
s := newTestStore(t)
mustCreate(t, s, fixedIntervalSpec("recurring", time.Hour))
first, _, _ := s.Get(context.Background(), "recurring")
runAt := first.NextRun.Add(time.Minute) // pretend engine ran a bit late
if err := s.MarkRun(context.Background(), "recurring", runAt); err != nil {
t.Fatalf("MarkRun: %v", err)
}
got, _, _ := s.Get(context.Background(), "recurring")
if got.LastRun == nil {
t.Fatal("LastRun is nil after MarkRun")
}
if !got.LastRun.Equal(runAt) {
t.Errorf("LastRun = %v, want %v", got.LastRun, runAt)
}
wantNext := runAt.Add(time.Hour)
if !got.NextRun.Equal(wantNext) {
t.Errorf("NextRun = %v, want %v (last_run + interval)", got.NextRun, wantNext)
}
}
func TestMarkRunMissingReturnsNotFound(t *testing.T) {
s := newTestStore(t)
if err := s.MarkRun(context.Background(), "ghost", time.Now()); err == nil {
t.Fatal("MarkRun of missing id succeeded; want error")
}
}
// --- partial-write handling --------------------------------------------------
func TestOpenTruncatedFileReturnsExplicitError(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
// Header present, body cut mid-array. Must not panic on read; the
// engine surfaces this as phase="source" KindJobError, not a crash.
if err := os.WriteFile(path, []byte(`{"schema_version":1,"jobs":[{"id":"x","`), 0o644); err != nil {
t.Fatalf("WriteFile: %v", err)
}
_, err := Open(path)
if err == nil {
t.Fatal("Open on truncated file succeeded; want explicit error")
}
if errors.Is(err, ErrUnsupportedSchema) {
t.Errorf("error = ErrUnsupportedSchema; want a corrupt-file error")
}
}
func TestOpenGarbageFileReturnsExplicitError(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
if err := os.WriteFile(path, []byte("not json at all"), 0o644); err != nil {
t.Fatalf("WriteFile: %v", err)
}
if _, err := Open(path); err == nil {
t.Fatal("Open on garbage file succeeded; want explicit error")
}
}
// --- cross-process locking ---------------------------------------------------
func TestConcurrentWritersAreSerialised(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "jobs.json")
a, err := Open(path)
if err != nil {
t.Fatalf("Open a: %v", err)
}
defer a.Close()
b, err := Open(path)
if err != nil {
t.Fatalf("Open b: %v", err)
}
defer b.Close()
// 50 writes from each writer, on the same file, interleaved. Without
// the lock, two writers would race and the file would end up either
// truncated or holding a non-deterministic subset. The lock makes
// each writer wait, so all 100 distinct ids end up persisted.
var wg sync.WaitGroup
for i := range 50 {
wg.Add(2)
go func(i int) {
defer wg.Done()
id := "a" + itoa(i)
_ = a.Create(context.Background(), fixedIntervalSpec(id, time.Minute))
}(i)
go func(i int) {
defer wg.Done()
id := "b" + itoa(i)
_ = b.Create(context.Background(), fixedIntervalSpec(id, time.Minute))
}(i)
}
wg.Wait()
jobs, err := a.List(context.Background())
if err != nil {
t.Fatalf("List: %v", err)
}
if len(jobs) != 100 {
t.Errorf("after 100 concurrent writes, %d jobs persisted, want 100", len(jobs))
}
}
// itoa avoids strconv import in tests.
func itoa(n int) string {
if n == 0 {
return "0"
}
var buf [20]byte
i := len(buf)
for n > 0 {
i--
buf[i] = byte('0' + n%10)
n /= 10
}
return string(buf[i:])
}
@@ -1,4 +1,4 @@
package store
package edastore
import (
"crypto/aes"
@@ -1,4 +1,4 @@
package store
package edastore
import (
"encoding/base64"
@@ -121,122 +121,112 @@ func TestBindingCipherDerivesDistinctKeys(t *testing.T) {
}
}
func TestStoreBindingSecretEncryptedAtRest(t *testing.T) {
s := openTestWithBindingCipher(t)
b := testBinding("sentry")
id, err := s.InsertBinding(t.Context(), b)
if err != nil {
t.Fatalf("InsertBinding: %v", err)
}
// The raw column must not be the plaintext secret.
var stored string
if err := s.db.QueryRowContext(t.Context(), `SELECT secret FROM bindings WHERE id=?`, id).Scan(&stored); err != nil {
t.Fatalf("read raw secret: %v", err)
}
if stored == b.Secret {
t.Fatal("bindings.secret stored as plaintext")
}
if !strings.HasPrefix(stored, "arcie-binding:v1:") {
t.Fatalf("stored secret = %q, want arcie-binding:v1: envelope", stored)
}
// GetBinding returns the plaintext secret.
got, err := s.GetBinding(t.Context(), id)
if err != nil || got == nil {
t.Fatalf("GetBinding: %+v, %v", got, err)
}
if got.Secret != b.Secret {
t.Fatalf("GetBinding.Secret = %q, want %q", got.Secret, b.Secret)
}
}
func TestStoreListBindingsDecryptsSecrets(t *testing.T) {
s := openTestWithBindingCipher(t)
if _, err := s.InsertBinding(t.Context(), testBinding("sentry")); err != nil {
t.Fatalf("InsertBinding: %v", err)
}
got, err := s.ListBindings(t.Context())
if err != nil {
t.Fatalf("ListBindings: %v", err)
}
if len(got) != 1 {
t.Fatalf("ListBindings = %d rows, want 1", len(got))
}
if got[0].Secret != testBinding("sentry").Secret {
t.Fatalf("ListBindings.Secret not decrypted")
}
}
func TestStoreBindingCipherNoKeyIsPlaintext(t *testing.T) {
s := openTest(t) // no cipher -> legacy plaintext
b := testBinding("sentry")
id, err := s.InsertBinding(t.Context(), b)
if err != nil {
t.Fatalf("InsertBinding: %v", err)
}
got, err := s.GetBinding(t.Context(), id)
if err != nil || got == nil {
t.Fatalf("GetBinding: %+v, %v", got, err)
}
if got.Secret != b.Secret {
t.Fatalf("plaintext store round-trip = %q, want %q", got.Secret, b.Secret)
}
}
// TestStoreEncryptedReadLeavesLegacyPlaintextAlone pins that enabling a
// cipher does not break rows written before it was configured: a non-envelope
// secret must remain readable as plaintext (encryption is not retroactive).
func TestStoreEncryptedReadLeavesLegacyPlaintextAlone(t *testing.T) {
path := filepath.Join(t.TempDir(), "legacy.db")
plain, err := Open(t.Context(), path)
if err != nil {
t.Fatalf("Open (plaintext): %v", err)
}
b := testBinding("sentry")
id, err := plain.InsertBinding(t.Context(), b)
if err != nil {
t.Fatalf("InsertBinding (plaintext): %v", err)
}
if err := plain.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
c, _ := NewBindingCipher(testBindingKey, nil)
encrypted, err := Open(t.Context(), path, WithBindingCipher(c))
if err != nil {
t.Fatalf("Open (cipher): %v", err)
}
t.Cleanup(func() { _ = encrypted.Close() })
got, err := encrypted.GetBinding(t.Context(), id)
if err != nil {
t.Fatalf("GetBinding under cipher: %v", err)
}
if got == nil {
t.Fatal("GetBinding returned nil")
}
if got.Secret != b.Secret {
t.Fatalf("legacy plaintext secret = %q, want %q (must stay readable)", got.Secret, b.Secret)
}
}
func openTestWithBindingCipher(t *testing.T) *Store {
// openWithCipher builds a store whose binding secrets are encrypted at rest.
func openWithCipher(t *testing.T, cipher BindingCipher) *Store {
t.Helper()
c, err := NewBindingCipher(testBindingKey, nil)
dir := t.TempDir()
s, err := Open(Config{
DBPath: filepath.Join(dir, "eda.sqlite"),
DataDir: filepath.Join(dir, "pb_data"),
Cipher: cipher,
})
if err != nil {
t.Fatalf("NewBindingCipher: %v", err)
}
s, err := Open(t.Context(), filepath.Join(t.TempDir(), "test.db"), WithBindingCipher(c))
if err != nil {
t.Fatalf("Open: %v", err)
t.Fatalf("Open() error = %v", err)
}
t.Cleanup(func() { _ = s.Close() })
return s
}
// tamperEnvelope decodes the payload, flips one byte inside the sealed
// ciphertext (after the 12-byte nonce), and re-encodes it, so the auth tag
// must fail. It returns the envelope with a corrupt ciphertext.
func sampleBinding() bindingInput {
return bindingInput{Name: "n", Source: "sentry", Workflow: "implement", Secret: "supersecretvalue0123"}
}
// TestBindingSecretIsEncryptedAtRest pins that the plaintext secret never
// reaches the database file. Reading the column directly is the point: a
// round-trip through the store would decrypt and prove nothing.
func TestBindingSecretIsEncryptedAtRest(t *testing.T) {
cipher, err := NewBindingCipher(testBindingKey, nil)
if err != nil {
t.Fatalf("NewBindingCipher() error = %v", err)
}
s := openWithCipher(t, cipher)
in := sampleBinding()
id := insertBinding(t, s, in)
var stored string
if err := s.App().DB().NewQuery("SELECT secret FROM " + CollBindings + " WHERE id = {:id}").
Bind(map[string]any{"id": id}).Row(&stored); err != nil {
t.Fatalf("read raw secret: %v", err)
}
if stored == in.Secret {
t.Fatal("secret is stored as plaintext; it must be encrypted at rest")
}
if !strings.Contains(stored, ":") {
t.Errorf("stored secret %q does not look like a cipher envelope", stored)
}
}
// TestBindingSecretDecryptsOnRead pins the other half: what was encrypted
// comes back as the original plaintext.
func TestBindingSecretDecryptsOnRead(t *testing.T) {
cipher, _ := NewBindingCipher(testBindingKey, nil)
s := openWithCipher(t, cipher)
in := sampleBinding()
id := insertBinding(t, s, in)
got, err := s.GetBinding(t.Context(), id)
if err != nil {
t.Fatalf("GetBinding() error = %v", err)
}
if got.Secret != in.Secret {
t.Errorf("Secret = %q, want %q", got.Secret, in.Secret)
}
}
// TestNoCipherKeepsPlaintext pins the behaviour that predates encryption: a
// deployment with no configured key still works, storing plaintext.
func TestNoCipherKeepsPlaintext(t *testing.T) {
s := openWithCipher(t, nil)
in := sampleBinding()
id := insertBinding(t, s, in)
var stored string
if err := s.App().DB().NewQuery("SELECT secret FROM " + CollBindings + " WHERE id = {:id}").
Bind(map[string]any{"id": id}).Row(&stored); err != nil {
t.Fatalf("read raw secret: %v", err)
}
if stored != in.Secret {
t.Errorf("stored secret = %q, want the plaintext %q with no cipher configured", stored, in.Secret)
}
}
// TestRotatedKeyStillReadsOldRows pins key rotation: a secret written under
// the previous key must stay readable once a new active key is installed,
// otherwise a rotation silently bricks every armed binding.
func TestRotatedKeyStillReadsOldRows(t *testing.T) {
const oldKey = "fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210"
oldCipher, _ := NewBindingCipher(oldKey, nil)
s := openWithCipher(t, oldCipher)
in := sampleBinding()
id := insertBinding(t, s, in)
rotated, err := NewBindingCipher(testBindingKey, []string{oldKey})
if err != nil {
t.Fatalf("NewBindingCipher(rotated) error = %v", err)
}
s.cipher = rotated
got, err := s.GetBinding(t.Context(), id)
if err != nil {
t.Fatalf("GetBinding() after rotation error = %v", err)
}
if got.Secret != in.Secret {
t.Errorf("Secret after rotation = %q, want %q: the previous key must still decrypt", got.Secret, in.Secret)
}
}
// tamperEnvelope flips one byte inside the ciphertext region so Decrypt must
// fail authentication rather than return altered plaintext.
func tamperEnvelope(t *testing.T, envelope string) string {
t.Helper()
parts := strings.SplitN(envelope, ":", 4)
@@ -0,0 +1,190 @@
package edastore
import (
"context"
"database/sql"
"errors"
"fmt"
"strings"
"time"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
"github.com/samcharles93/archie-core/internal/domain/storecontract"
)
// capturedAtLayout is fixed-width UTC with nanosecond precision, so string
// comparison is time comparison. time.RFC3339Nano trims trailing zeros, which
// would break that.
const capturedAtLayout = "2006-01-02T15:04:05.000000000Z"
// This store is the implementation of every event-capture contract. The task
// lifecycle contracts stay on the SQLite store; the two meet only at
// binding_dispatches.task_id.
var (
_ storecontract.CaptureStore = (*Store)(nil)
_ storecontract.MappingStore = (*Store)(nil)
_ storecontract.BindingStore = (*Store)(nil)
_ storecontract.BindingDispatcher = (*Store)(nil)
_ storecontract.PlaybookDispatcher = (*Store)(nil)
)
func (s *Store) record(collection, id string) (*core.Record, error) {
r, err := s.app.FindRecordById(collection, id)
if errors.Is(err, sql.ErrNoRows) {
return nil, nil
}
return r, err
}
// InsertCapture stores one inbound event verbatim. Redaction happens upstream
// in webhookguard; this persists exactly what it is given, because field
// mapping is designed against the payload as received.
//
// retention and maxEvents are accepted for contract compatibility and applied
// as a prune after the insert.
func (s *Store) InsertCapture(ctx context.Context, c storecontract.CapturedEvent, retention time.Duration, maxEvents int) (string, error) {
collection, err := s.app.FindCollectionByNameOrId(CollCaptures)
if err != nil {
return "", err
}
r := core.NewRecord(collection)
r.Set("source", c.Source)
r.Set("remote_addr", c.RemoteAddr)
r.Set("content_type", c.ContentType)
r.Set("headers", c.Headers)
r.Set("body", c.Body)
r.Set("authenticated", c.Authenticated)
received := c.ReceivedAt
if received.IsZero() {
received = time.Now()
}
r.Set("received_at", received.UTC().Format(capturedAtLayout))
if err := s.app.Save(r); err != nil {
return "", fmt.Errorf("edastore: insert capture: %w", err)
}
if err := s.pruneCaptures(ctx, retention, maxEvents); err != nil {
return "", err
}
return r.Id, nil
}
// pruneCaptures enforces the retention window and count cap. A capture exists
// to be mapped against while it is recent; keeping every event forever would
// make an unauthenticated intake surface a disk-growth vector.
func (s *Store) pruneCaptures(_ context.Context, retention time.Duration, maxEvents int) error {
if retention > 0 {
cutoff := time.Now().Add(-retention).UTC().Format(time.RFC3339)
if _, err := s.app.DB().NewQuery(
"DELETE FROM " + CollCaptures + " WHERE received_at < {:cutoff}").
Bind(map[string]any{"cutoff": cutoff}).Execute(); err != nil {
return fmt.Errorf("edastore: prune captures by age: %w", err)
}
}
if maxEvents > 0 {
if _, err := s.app.DB().NewQuery(
"DELETE FROM " + CollCaptures + " WHERE id NOT IN (SELECT id FROM " + CollCaptures +
" ORDER BY received_at DESC LIMIT {:keep})").
Bind(map[string]any{"keep": maxEvents}).Execute(); err != nil {
return fmt.Errorf("edastore: prune captures by count: %w", err)
}
}
return nil
}
// parseCapturedAt reads the stored timestamp, tolerating the RFC3339 form a
// caller-supplied value may carry.
func parseCapturedAt(raw string) time.Time {
for _, layout := range []string{capturedAtLayout, time.RFC3339Nano, time.RFC3339} {
if t, err := time.Parse(layout, raw); err == nil {
return t
}
}
return time.Time{}
}
func captureValue(r *core.Record) storecontract.CapturedEvent {
return storecontract.CapturedEvent{
ID: r.Id,
ReceivedAt: parseCapturedAt(r.GetString("received_at")),
Source: r.GetString("source"),
RemoteAddr: r.GetString("remote_addr"),
ContentType: r.GetString("content_type"),
Headers: r.GetString("headers"),
Body: r.GetString("body"),
Authenticated: r.GetBool("authenticated"),
}
}
// ListCaptures returns the newest captures first.
func (s *Store) ListCaptures(_ context.Context, limit int) ([]storecontract.CapturedEvent, error) {
records, err := s.app.FindRecordsByFilter(CollCaptures, "", "-received_at", limit, 0)
if err != nil {
return nil, fmt.Errorf("edastore: list captures: %w", err)
}
out := make([]storecontract.CapturedEvent, 0, len(records))
for _, r := range records {
out = append(out, captureValue(r))
}
return out, nil
}
// ListUndispatchedCaptures returns recent captures for the given sources that
// no binding has dispatched yet.
//
// This is raw SQL rather than a record filter: the anti-join against the
// dispatch ledger is a subquery, and PocketBase's filter DSL is not SQL (it
// has no IN (SELECT ...) form). A read fires no hooks either way, so nothing
// is lost by dropping to the query builder here.
func (s *Store) ListUndispatchedCaptures(_ context.Context, sources []string, limit int) ([]storecontract.CapturedEvent, error) {
if len(sources) == 0 || limit <= 0 {
return nil, nil
}
placeholders := make([]string, 0, len(sources))
params := dbx.Params{"limit": limit}
for i, src := range sources {
key := fmt.Sprintf("s%d", i)
placeholders = append(placeholders, "{:"+key+"}")
params[key] = src
}
query := "SELECT id, received_at, source, remote_addr, content_type, headers, body, authenticated FROM " +
CollCaptures + " WHERE source IN (" + strings.Join(placeholders, ",") +
") AND id NOT IN (SELECT capture FROM " + CollBindingDispatches + ") ORDER BY received_at DESC LIMIT {:limit}"
var rows []captureRow
if err := s.app.DB().NewQuery(query).Bind(params).All(&rows); err != nil {
return nil, fmt.Errorf("edastore: list undispatched captures: %w", err)
}
out := make([]storecontract.CapturedEvent, 0, len(rows))
for _, r := range rows {
out = append(out, r.value())
}
return out, nil
}
// captureRow is the raw-SQL shape of a capture, used only where a record
// filter cannot express the query.
type captureRow struct {
ID string `db:"id"`
ReceivedAt string `db:"received_at"`
Source string `db:"source"`
RemoteAddr string `db:"remote_addr"`
ContentType string `db:"content_type"`
Headers string `db:"headers"`
Body string `db:"body"`
Authenticated bool `db:"authenticated"`
}
func (r captureRow) value() storecontract.CapturedEvent {
return storecontract.CapturedEvent{
ID: r.ID,
ReceivedAt: parseCapturedAt(r.ReceivedAt),
Source: r.Source,
RemoteAddr: r.RemoteAddr,
ContentType: r.ContentType,
Headers: r.Headers,
Body: r.Body,
Authenticated: r.Authenticated,
}
}
@@ -0,0 +1,393 @@
package edastore
import (
"context"
"encoding/json"
"fmt"
"strings"
"github.com/pocketbase/pocketbase/core"
"github.com/samcharles93/archie-core/internal/domain/binding"
"github.com/samcharles93/archie-core/internal/domain/mapping"
"github.com/samcharles93/archie-core/internal/events"
)
// --- mappings ---
func mappingValue(r *core.Record) (mapping.Mapping, error) {
var fields []mapping.Field
if raw := r.GetString("fields"); raw != "" {
if err := json.Unmarshal([]byte(raw), &fields); err != nil {
return mapping.Mapping{}, fmt.Errorf("edastore: decode mapping fields: %w", err)
}
}
return mapping.Mapping{
ID: r.Id,
Name: r.GetString("name"),
SourceHint: r.GetString("source_hint"),
Fields: fields,
CreatedAt: r.GetDateTime("created_at").Time(),
UpdatedAt: r.GetDateTime("updated_at").Time(),
}, nil
}
func (s *Store) applyMapping(r *core.Record, m mapping.Mapping) error {
encoded, err := json.Marshal(m.Fields)
if err != nil {
return fmt.Errorf("edastore: encode mapping fields: %w", err)
}
r.Set("name", m.Name)
r.Set("source_hint", m.SourceHint)
r.Set("fields", string(encoded))
return nil
}
func (s *Store) InsertMapping(_ context.Context, m mapping.Mapping) (string, error) {
collection, err := s.app.FindCollectionByNameOrId(CollMappings)
if err != nil {
return "", err
}
r := core.NewRecord(collection)
if err := s.applyMapping(r, m); err != nil {
return "", err
}
if err := s.app.Save(r); err != nil {
return "", fmt.Errorf("edastore: insert mapping: %w", err)
}
s.notifyWrite(events.KindMappingChanged, "mapping", "create", r.Id)
return r.Id, nil
}
func (s *Store) GetMapping(_ context.Context, id string) (*mapping.Mapping, error) {
r, err := s.record(CollMappings, id)
if err != nil || r == nil {
return nil, err
}
m, err := mappingValue(r)
if err != nil {
return nil, err
}
return &m, nil
}
func (s *Store) ListMappings(_ context.Context) ([]mapping.Mapping, error) {
records, err := s.app.FindRecordsByFilter(CollMappings, "", "-created_at", 0, 0)
if err != nil {
return nil, fmt.Errorf("edastore: list mappings: %w", err)
}
out := make([]mapping.Mapping, 0, len(records))
for _, r := range records {
m, err := mappingValue(r)
if err != nil {
return nil, err
}
out = append(out, m)
}
return out, nil
}
func (s *Store) UpdateMapping(_ context.Context, m mapping.Mapping) error {
r, err := s.record(CollMappings, m.ID)
if err != nil {
return err
}
if r == nil {
return ErrMappingNotFound
}
if err := s.applyMapping(r, m); err != nil {
return err
}
if err := s.app.Save(r); err != nil {
return err
}
s.notifyWrite(events.KindMappingChanged, "mapping", "update", m.ID)
return nil
}
func (s *Store) DeleteMapping(_ context.Context, id string) error {
r, err := s.record(CollMappings, id)
if err != nil {
return err
}
if r == nil {
return ErrMappingNotFound
}
if err := s.app.Delete(r); err != nil {
return err
}
s.notifyWrite(events.KindMappingChanged, "mapping", "delete", id)
return nil
}
// --- bindings ---
func (s *Store) bindingValue(r *core.Record) (binding.Binding, error) {
b := binding.Binding{
ID: r.Id,
Name: r.GetString("name"),
Matcher: binding.Matcher{Source: r.GetString("source")},
MappingID: r.GetString("mapping"),
Workflow: r.GetString("workflow"),
Owner: r.GetString("owner"),
Repo: r.GetString("repo"),
Version: r.GetInt("version"),
Status: binding.Status(r.GetString("status")),
Secret: r.GetString("secret"),
CreatedAt: r.GetDateTime("created_at").Time(),
UpdatedAt: r.GetDateTime("updated_at").Time(),
}
if s.cipher != nil && b.Secret != "" {
plain, err := s.cipher.Decrypt(b.Secret)
if err != nil {
return binding.Binding{}, fmt.Errorf("edastore: decrypt binding secret: %w", err)
}
b.Secret = plain
}
return b, nil
}
func (s *Store) applyBinding(r *core.Record, b binding.Binding) error {
r.Set("name", b.Name)
r.Set("source", b.Matcher.Source)
r.Set("mapping", b.MappingID)
r.Set("workflow", b.Workflow)
r.Set("owner", b.Owner)
r.Set("repo", b.Repo)
// An empty secret means "keep the stored one". An edit form that does not
// echo the secret back must not silently disarm the binding by erasing it.
if b.Secret == "" {
return nil
}
secret := b.Secret
if s.cipher != nil {
encrypted, err := s.cipher.Encrypt(secret)
if err != nil {
return fmt.Errorf("edastore: encrypt binding secret: %w", err)
}
secret = encrypted
}
r.Set("secret", secret)
return nil
}
// InsertBinding stores a new binding as a draft. The caller's status is
// ignored on purpose: a binding cannot go live without explicit approval, so
// ApproveBinding is the only path to armed.
//
// A source may carry at most one binding, so an insert against a source that
// already has one is refused rather than creating an ambiguous second matcher.
func (s *Store) InsertBinding(ctx context.Context, b binding.Binding) (string, error) {
existing, err := s.bindingsByFilter("source = {:source}", map[string]any{"source": b.Matcher.Source})
if err != nil {
return "", err
}
if len(existing) > 0 {
return "", ErrBindingOverlap
}
collection, err := s.app.FindCollectionByNameOrId(CollBindings)
if err != nil {
return "", err
}
r := core.NewRecord(collection)
if err := s.applyBinding(r, b); err != nil {
return "", err
}
r.Set("version", 1)
r.Set("status", string(binding.StatusDraft))
if err := s.app.Save(r); err != nil {
return "", fmt.Errorf("edastore: insert binding: %w", err)
}
_ = ctx
s.notifyWrite(events.KindBindingChanged, "binding", "create", r.Id)
return r.Id, nil
}
func (s *Store) GetBinding(_ context.Context, id string) (*binding.Binding, error) {
r, err := s.record(CollBindings, id)
if err != nil || r == nil {
return nil, err
}
b, err := s.bindingValue(r)
if err != nil {
return nil, err
}
return &b, nil
}
func (s *Store) ListBindings(_ context.Context) ([]binding.Binding, error) {
return s.bindingsByFilter("", nil)
}
// ArmedBindingsForSource returns the armed bindings a source can dispatch. A
// draft or pending_approval binding is never returned: that is the gate.
func (s *Store) ArmedBindingsForSource(_ context.Context, source string) ([]binding.Binding, error) {
return s.bindingsByFilter("source = {:source} && status = {:status}",
map[string]any{"source": source, "status": string(binding.StatusArmed)})
}
func (s *Store) bindingsByFilter(filter string, params map[string]any) ([]binding.Binding, error) {
var records []*core.Record
var err error
if params == nil {
records, err = s.app.FindRecordsByFilter(CollBindings, filter, "-created_at", 0, 0)
} else {
records, err = s.app.FindRecordsByFilter(CollBindings, filter, "-created_at", 0, 0, params)
}
if err != nil {
return nil, fmt.Errorf("edastore: list bindings: %w", err)
}
out := make([]binding.Binding, 0, len(records))
for _, r := range records {
b, err := s.bindingValue(r)
if err != nil {
return nil, err
}
out = append(out, b)
}
return out, nil
}
// UpdateBinding rewrites a binding's editable fields, bumps its version, and
// sends it back to pending_approval. An edit always requires re-approval: an
// armed binding whose matcher or workflow changed is a different binding, and
// re-arming it silently would defeat the gate.
func (s *Store) UpdateBinding(_ context.Context, b binding.Binding) error {
r, err := s.record(CollBindings, b.ID)
if err != nil {
return err
}
if r == nil {
return ErrBindingNotFound
}
others, err := s.bindingsByFilter("source = {:source} && id != {:id}",
map[string]any{"source": b.Matcher.Source, "id": b.ID})
if err != nil {
return err
}
if len(others) > 0 {
return ErrBindingOverlap
}
if err := s.applyBinding(r, b); err != nil {
return err
}
r.Set("status", string(binding.StatusPendingApproval))
r.Set("version", r.GetInt("version")+1)
if err := s.app.Save(r); err != nil {
return err
}
s.notifyWrite(events.KindBindingChanged, "binding", "update", b.ID)
return nil
}
func (s *Store) DeleteBinding(_ context.Context, id string) error {
r, err := s.record(CollBindings, id)
if err != nil {
return err
}
if r == nil {
return ErrBindingNotFound
}
if err := s.app.Delete(r); err != nil {
return err
}
s.notifyWrite(events.KindBindingChanged, "binding", "delete", id)
return nil
}
// ApproveBinding is the only transition that arms a binding. It moves
// pending_approval to armed; any other current state is refused, so a draft
// cannot skip review and an armed binding cannot be re-approved.
//
// A source may have only one armed binding, so approval is refused when
// another binding on the same source is already armed. Without that guard,
// one capture could match two armed matchers and dispatch twice.
func (s *Store) ApproveBinding(_ context.Context, id string) error {
r, err := s.record(CollBindings, id)
if err != nil {
return err
}
if r == nil {
return ErrBindingNotFound
}
armed, err := s.bindingsByFilter("source = {:source} && status = {:status} && id != {:id}",
map[string]any{"source": r.GetString("source"), "status": string(binding.StatusArmed), "id": id})
if err != nil {
return err
}
if len(armed) > 0 {
return ErrBindingOverlap
}
if r.GetString("status") != string(binding.StatusPendingApproval) {
return ErrBindingTransition
}
r.Set("status", string(binding.StatusArmed))
if err := s.app.Save(r); err != nil {
return err
}
s.notifyWrite(events.KindBindingChanged, "binding", "approve", id)
return nil
}
// --- dispatch ledgers ---
// RecordDispatch writes one at-most-once binding dispatch. Uniqueness is the
// collection's index, so a replay is refused by the schema; the caller sees
// ErrAlreadyDispatched rather than a silent second run.
func (s *Store) RecordDispatch(_ context.Context, bindingID string, bindingVersion int64, captureID string, taskID int64) error {
collection, err := s.app.FindCollectionByNameOrId(CollBindingDispatches)
if err != nil {
return err
}
r := core.NewRecord(collection)
r.Set("binding", bindingID)
r.Set("binding_version", bindingVersion)
r.Set("capture", captureID)
r.Set("task_id", taskID)
return ledgerWrite(s.app.Save(r), "edastore: record dispatch")
}
func (s *Store) RecordPlaybookDispatch(_ context.Context, playbookID, playbookVersion, eventID, actionID string) error {
collection, err := s.app.FindCollectionByNameOrId(CollPlaybookDispatches)
if err != nil {
return err
}
r := core.NewRecord(collection)
r.Set("playbook_id", playbookID)
r.Set("playbook_version", playbookVersion)
r.Set("event_id", eventID)
r.Set("action_id", actionID)
return ledgerWrite(s.app.Save(r), "edastore: record playbook dispatch")
}
func (s *Store) DeletePlaybookDispatches(_ context.Context, playbookID string) error {
_, err := s.app.DB().NewQuery(
"DELETE FROM " + CollPlaybookDispatches + " WHERE playbook_id = {:id}").
Bind(map[string]any{"id": playbookID}).Execute()
if err != nil {
return fmt.Errorf("edastore: delete playbook dispatches: %w", err)
}
return nil
}
// ledgerWrite turns the unique-index refusal into ErrAlreadyDispatched and
// leaves every other failure alone. Treating all errors as "already recorded"
// would turn an at-most-once ledger into a silent at-least-once dispatcher.
func ledgerWrite(err error, what string) error {
switch {
case err == nil:
return nil
case isUniqueViolation(err):
return ErrAlreadyDispatched
default:
return fmt.Errorf("%s: %w", what, err)
}
}
// isUniqueViolation reports whether err is the unique-index refusal.
// PocketBase surfaces it as a validation failure rather than a typed error,
// so the message is what there is to match on.
func isUniqueViolation(err error) bool {
text := strings.ToLower(err.Error())
return strings.Contains(text, "unique") || strings.Contains(text, "already exists")
}
Loaded 100 of 152 files, more files were not shown because too many files have changed in this diff. Show more