From 72be0049edfd0acb49850c4a806a2ea692edb847 Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Thu, 24 Sep 2026 14:46:44 +1000 Subject: [PATCH] docs(bindings): bindings apply per event type with a filter; mappings count matches --- docs/architecture/bindings.md | 23 ++++++++++++++--------- docs/data/generated/dev-docs.json | 2 +- 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/docs/architecture/bindings.md b/docs/architecture/bindings.md index 33b759599..7ff297af9 100644 --- a/docs/architecture/bindings.md +++ b/docs/architecture/bindings.md @@ -21,7 +21,8 @@ machine** governing whether it is live. type Binding struct { ID, Name // identity Matcher // { Source string } -- the webhook path segment, e.g. "sentry" - MappingID // which Mapping resolves payload fields + MappingID // which Mapping resolves payload fields; its event type is the binding's + Filter // optional CEL over the mapping's parameters Workflow // registered workflow name to dispatch to Version // bumped on every edit Status // pending_approval | armed @@ -34,19 +35,23 @@ them from the dashboard while the daemon runs, without a restart. ### Matching -`Matcher.Source` is the only predicate today: the path segment a sender -POSTs to (e.g. a webhook hitting `.../sentry` matches bindings with -`Matcher.Source == "sentry"`). The storage column is plain `TEXT`, so adding -predicates later is non-breaking -- extend `Matcher`, don't invent a second -matcher shape. +A binding applies to a capture when the capture arrived on `Matcher.Source`, +was identified as the event type its mapping belongs to, and passes its +`Filter` (`binding.CompileFilter`, checked against the mapping's parameters on +save). Any number of bindings may share a source; each dispatches a capture at +most once, claimed in `binding_dispatches` keyed by (binding, capture) before +its task is enqueued. An event signed with any armed binding's secret on the +source is authenticated. ### Mapping `mapping.Resolve(fields, payload)` walks a captured JSON payload against a list of typed `Field`s (name, JSON path, expected type), returning resolved -values plus a `[]Failure` for anything that didn't parse or type-match. This -is schema-by-example in practice: a mapping is authored by pointing at a -real captured payload in the dashboard, not by guessing a schema in advance. +values plus a `[]Failure` for anything that didn't parse or type-match. A +mapping belongs to an event type, and `mapping.CheckSchema` refuses a field +whose path or type the type's inferred schema lacks. Each capture a mapping +resolves at dispatch is counted once in `mapping_matches`, read back as +`MatchCount` and `LastMatchedAt`. ### Versioning diff --git a/docs/data/generated/dev-docs.json b/docs/data/generated/dev-docs.json index 4736e3ce6..7f78a04cd 100644 --- a/docs/data/generated/dev-docs.json +++ b/docs/data/generated/dev-docs.json @@ -22,7 +22,7 @@ "url": "/dev/architecture/bindings/", "title": "Playbook bindings: model and threat model", "section": "Architecture", - "body": "# Playbook bindings: model and threat model\n\n**Status:** Approved foundation (documents shipped code: `archie-core-t2db.1`\nthrough `.5`, `.7`, `.9`-`.15`)\n**Date:** 2026-09-03\n\nThis document covers the binding model and the intake threat model portion\nof `archie-core-t2db.6`. It does not cover the task-lifecycle provenance rule\n(`messaging-and-work-intake.md`'s \"Intake provenance\") or the operator\nwalkthrough -- that walkthrough remains an open piece of the same ticket.\n\n## What a binding is\n\nA `binding` (`internal/domain/binding.Binding`) ties four things together:\na **matcher** (which captured events it applies to), a **mapping**\n(`internal/domain/mapping.Mapping`, how fields are pulled out of a payload),\na **workflow** (which registered workflow runs on a match), and a **state\nmachine** governing whether it is live.\n\n```go\ntype Binding struct {\n ID, Name // identity\n Matcher // { Source string } -- the webhook path segment, e.g. \"sentry\"\n MappingID // which Mapping resolves payload fields\n Workflow // registered workflow name to dispatch to\n Version // bumped on every edit\n Status // pending_approval | armed\n Secret // HMAC-SHA256 shared secret, encrypted at rest, never returned by GET\n}\n```\n\nBindings live in the store (SQLite), not `config.toml` -- operators author\nthem from the dashboard while the daemon runs, without a restart.\n\n### Matching\n\n`Matcher.Source` is the only predicate today: the path segment a sender\nPOSTs to (e.g. a webhook hitting `.../sentry` matches bindings with\n`Matcher.Source == \"sentry\"`). The storage column is plain `TEXT`, so adding\npredicates later is non-breaking -- extend `Matcher`, don't invent a second\nmatcher shape.\n\n### Mapping\n\n`mapping.Resolve(fields, payload)` walks a captured JSON payload against a\nlist of typed `Field`s (name, JSON path, expected type), returning resolved\nvalues plus a `[]Failure` for anything that didn't parse or type-match. This\nis schema-by-example in practice: a mapping is authored by pointing at a\nreal captured payload in the dashboard, not by guessing a schema in advance.\n\n### Versioning\n\n`Version` increments on every `UpdateBinding` call. This exists so an\neditor's later change can never silently rewrite the historical provenance\nof a task that already fired -- a dispatched task's binding reference is\npinned to the version that was armed when it fired, not \"whatever the\nbinding currently says.\"\n\n### State machine\n\n```\ncreate ---\u003e pending_approval ---\u003e armed\n ^ |\n +----------------+\n (any edit to an armed binding drops it back to pending_approval)\n```\n\nA binding is created `pending_approval`, so the operator who creates one can\napprove it immediately. Bindings stored as `draft` by an earlier version are\nmoved to `pending_approval` when the EDA store opens.\n\nModeled directly on Telegram's existing `dangerousAction`/`pendingApproval`\nflow (`internal/channels/telegram/dangerous.go`, `approval.go`) -- not a new\napproval mechanism invented for this feature. **Only `armed` evaluates\nagainst incoming events** (`binding.Matches`, `internal/domain/binding/\nbinding.go`); a `pending_approval` binding is inert. An edit to\nan already-armed binding cannot bypass approval by silent mutation -- it\ndrops back to `pending_approval` and requires an explicit `Approve` call to\nre-arm.\n\n## The threat model\n\nThis is a public, unauthenticated-by-default intake surface whose output is\nan agent with a repo, a worktree, and tool access. The design brief\n(`archie-core-t2db.5`, `docs/prds/webhook-intake-security.md`) named five\nnon-negotiables; each is verified below against the actual shipped code,\nnot the design intent.\n\n### 1. Per-source authentication\n\nReuses `internal/channels/webhook`'s existing HMAC-SHA256 scheme --\ncapture accepts unsigned events (visible in the dashboard, marked\nunauthenticated), but **only authenticated events can ever match a\nbinding**. `binding.Matches`' doc comment states this plainly: \"an\nunauthenticated event can never trigger a binding, no matter how well it\nwould otherwise match.\" Enforced structurally at two points, not just\nconvention:\n\n- `captured_events.authenticated` is a real column (`internal/store/\n captures.go`), set at capture time.\n- `dispatchBindings` (`internal/daemon/daemon.go:479`) walks **authenticated\n captures whose source has at least one armed binding** -- an\n unauthenticated capture is never even considered for dispatch.\n\n### 2. Human approval before a binding goes live\n\nCovered by the state machine above. Nothing self-arms: `pending_approval -\u003e\narmed` requires an explicit `Approve` call, and any edit to an armed\nbinding reverts it.\n\n### 3. Rate and volume limits per source\n\nCapture writes are bounded independently of the approval gate, so a flood\nof unauthenticated traffic cannot exhaust disk or drown the inspector:\n`InsertCapture` (`internal/store/captures.go`) prunes by both **retention\nwindow** (`WHERE received_at \u003c cutoff`) and **max event count** in the same\ninsert transaction.\n\n### 4. Blast radius bounded by the same code-level controls every task gets\n\nA playbook-originated task is dispatched into the **same** gate, diff-cap,\nsandboxed-container, worktree-isolation pipeline every task goes through\nregardless of origin. Verified precisely, not assumed: `dispatchOneBinding`\ncalls `EnqueueBindingTask` (`internal/store/store.go`), which wraps\n`EnqueueChatTask` -- the same direct-to-`tasks`-table enqueue path\nchat-spawned tasks already use, then stamps `binding_id`/`binding_version`\nfor provenance in a second statement. This is a **different** enqueue\nmechanism from forge-issue polling's `workintake.TaskEnvelope`/NATS path\n(`docs/prds/event-sources-and-reactions.md`) -- binding and forge-issue\ntasks arrive by different producers, but both become an ordinary\n`store.Task` row that `ClaimNext` picks up identically, so both get the\nsame gate/worktree pipeline downstream regardless of which producer created\nthe row. Payload content is\nordinary untrusted prompt input; the sender's text is never treated as a\ntrusted instruction (Archie's environmental-enforcement principle: \"the\nagent should ignore instructions in the payload\" is not itself a control,\nthe surrounding sandbox is).\n\n### 5. Captured payloads may contain secrets from the sending system\n\nTwo independent protections, not one:\n\n- **Redaction before persistence**: `webhookguard.RedactPayload` walks the\n decoded JSON and redacts recognised sensitive keys before the row is\n written -- the raw, unredacted payload is never durably stored.\n- **Retention**: a configurable window (`InsertCapture`'s `retention`\n parameter) prunes old captures automatically; visibility of what remains\n is gated by the same dashboard authentication as every other webui\n surface, not a separate, weaker path.\n\n### Secrets at rest\n\n`Binding.Secret` (the HMAC shared secret a sender signs with) is encrypted\nat rest via AES-GCM (`internal/store/binding_cipher.go`, `archie-core-\nt2db.7`) -- a leaked SQLite file does not leak sender-side shared secrets in\nplaintext. The envelope is versioned (`bindingEnvelopeVersion`) so a future\ncipher or KDF change can roll forward without breaking already-encrypted\nrows, and the AAD binds ciphertext to the binding-secret context so it\ncannot be relocated to another field/column and still authenticate. A `nil`\ncipher (no encryption key configured) leaves secrets as plaintext --\ndocumented legacy behaviour, not silent degradation.\n\n### At-most-once dispatch\n\n`dispatchBindings` enqueues at most one task per `(binding, capture)` pair,\nenforced by a real ledger, not best-effort application logic:\n`RecordDispatch` (`internal/store/bindings.go`) does `INSERT OR IGNORE` into\n`binding_dispatches`, unique on `(binding_id, capture_id)`; a duplicate\ninsert returns the sentinel `ErrAlreadyDispatched`, which the dispatch loop\ntreats as \"another cycle raced us,\" not an error. This guarantee holds\n**across daemon restarts**, not just within one process's lifetime, because\nthe ledger is a durable table, not in-memory state.\n\nThis ledger is also the direct precedent for the EDA playbook engine's\nexecution-time idempotency, now landed (`archie-core-t2db.17` closed): the\n`playbook_dispatches` table in `internal/store/playbook_dispatches.go` copies\n`binding_dispatches`'s `INSERT OR IGNORE` conventions with a\n`(playbook_id, playbook_version, event_id, action_id)` key\n(`docs/prds/eda-playbook-engine.md` gap 2).\n\n## What this document does not cover\n\n- An end-to-end operator walkthrough (point an external app at archie,\n build a first playbook) -- open, part of `archie-core-t2db.6`.\n- The playbook YAML / CEL / Module/Channel/Forge action-position work\n (`docs/prds/eda-playbook-engine.md`, `module-position.md`,\n `channel-forge-positions.md`) -- a different, newer mechanism than the\n binding model described here; the two are related but not the same\n system, and this document does not attempt to unify them.\n" + "body": "# Playbook bindings: model and threat model\n\n**Status:** Approved foundation (documents shipped code: `archie-core-t2db.1`\nthrough `.5`, `.7`, `.9`-`.15`)\n**Date:** 2026-09-03\n\nThis document covers the binding model and the intake threat model portion\nof `archie-core-t2db.6`. It does not cover the task-lifecycle provenance rule\n(`messaging-and-work-intake.md`'s \"Intake provenance\") or the operator\nwalkthrough -- that walkthrough remains an open piece of the same ticket.\n\n## What a binding is\n\nA `binding` (`internal/domain/binding.Binding`) ties four things together:\na **matcher** (which captured events it applies to), a **mapping**\n(`internal/domain/mapping.Mapping`, how fields are pulled out of a payload),\na **workflow** (which registered workflow runs on a match), and a **state\nmachine** governing whether it is live.\n\n```go\ntype Binding struct {\n ID, Name // identity\n Matcher // { Source string } -- the webhook path segment, e.g. \"sentry\"\n MappingID // which Mapping resolves payload fields; its event type is the binding's\n Filter // optional CEL over the mapping's parameters\n Workflow // registered workflow name to dispatch to\n Version // bumped on every edit\n Status // pending_approval | armed\n Secret // HMAC-SHA256 shared secret, encrypted at rest, never returned by GET\n}\n```\n\nBindings live in the store (SQLite), not `config.toml` -- operators author\nthem from the dashboard while the daemon runs, without a restart.\n\n### Matching\n\nA binding applies to a capture when the capture arrived on `Matcher.Source`,\nwas identified as the event type its mapping belongs to, and passes its\n`Filter` (`binding.CompileFilter`, checked against the mapping's parameters on\nsave). Any number of bindings may share a source; each dispatches a capture at\nmost once, claimed in `binding_dispatches` keyed by (binding, capture) before\nits task is enqueued. An event signed with any armed binding's secret on the\nsource is authenticated.\n\n### Mapping\n\n`mapping.Resolve(fields, payload)` walks a captured JSON payload against a\nlist of typed `Field`s (name, JSON path, expected type), returning resolved\nvalues plus a `[]Failure` for anything that didn't parse or type-match. A\nmapping belongs to an event type, and `mapping.CheckSchema` refuses a field\nwhose path or type the type's inferred schema lacks. Each capture a mapping\nresolves at dispatch is counted once in `mapping_matches`, read back as\n`MatchCount` and `LastMatchedAt`.\n\n### Versioning\n\n`Version` increments on every `UpdateBinding` call. This exists so an\neditor's later change can never silently rewrite the historical provenance\nof a task that already fired -- a dispatched task's binding reference is\npinned to the version that was armed when it fired, not \"whatever the\nbinding currently says.\"\n\n### State machine\n\n```\ncreate ---\u003e pending_approval ---\u003e armed\n ^ |\n +----------------+\n (any edit to an armed binding drops it back to pending_approval)\n```\n\nA binding is created `pending_approval`, so the operator who creates one can\napprove it immediately. Bindings stored as `draft` by an earlier version are\nmoved to `pending_approval` when the EDA store opens.\n\nModeled directly on Telegram's existing `dangerousAction`/`pendingApproval`\nflow (`internal/channels/telegram/dangerous.go`, `approval.go`) -- not a new\napproval mechanism invented for this feature. **Only `armed` evaluates\nagainst incoming events** (`binding.Matches`, `internal/domain/binding/\nbinding.go`); a `pending_approval` binding is inert. An edit to\nan already-armed binding cannot bypass approval by silent mutation -- it\ndrops back to `pending_approval` and requires an explicit `Approve` call to\nre-arm.\n\n## The threat model\n\nThis is a public, unauthenticated-by-default intake surface whose output is\nan agent with a repo, a worktree, and tool access. The design brief\n(`archie-core-t2db.5`, `docs/prds/webhook-intake-security.md`) named five\nnon-negotiables; each is verified below against the actual shipped code,\nnot the design intent.\n\n### 1. Per-source authentication\n\nReuses `internal/channels/webhook`'s existing HMAC-SHA256 scheme --\ncapture accepts unsigned events (visible in the dashboard, marked\nunauthenticated), but **only authenticated events can ever match a\nbinding**. `binding.Matches`' doc comment states this plainly: \"an\nunauthenticated event can never trigger a binding, no matter how well it\nwould otherwise match.\" Enforced structurally at two points, not just\nconvention:\n\n- `captured_events.authenticated` is a real column (`internal/store/\n captures.go`), set at capture time.\n- `dispatchBindings` (`internal/daemon/daemon.go:479`) walks **authenticated\n captures whose source has at least one armed binding** -- an\n unauthenticated capture is never even considered for dispatch.\n\n### 2. Human approval before a binding goes live\n\nCovered by the state machine above. Nothing self-arms: `pending_approval -\u003e\narmed` requires an explicit `Approve` call, and any edit to an armed\nbinding reverts it.\n\n### 3. Rate and volume limits per source\n\nCapture writes are bounded independently of the approval gate, so a flood\nof unauthenticated traffic cannot exhaust disk or drown the inspector:\n`InsertCapture` (`internal/store/captures.go`) prunes by both **retention\nwindow** (`WHERE received_at \u003c cutoff`) and **max event count** in the same\ninsert transaction.\n\n### 4. Blast radius bounded by the same code-level controls every task gets\n\nA playbook-originated task is dispatched into the **same** gate, diff-cap,\nsandboxed-container, worktree-isolation pipeline every task goes through\nregardless of origin. Verified precisely, not assumed: `dispatchOneBinding`\ncalls `EnqueueBindingTask` (`internal/store/store.go`), which wraps\n`EnqueueChatTask` -- the same direct-to-`tasks`-table enqueue path\nchat-spawned tasks already use, then stamps `binding_id`/`binding_version`\nfor provenance in a second statement. This is a **different** enqueue\nmechanism from forge-issue polling's `workintake.TaskEnvelope`/NATS path\n(`docs/prds/event-sources-and-reactions.md`) -- binding and forge-issue\ntasks arrive by different producers, but both become an ordinary\n`store.Task` row that `ClaimNext` picks up identically, so both get the\nsame gate/worktree pipeline downstream regardless of which producer created\nthe row. Payload content is\nordinary untrusted prompt input; the sender's text is never treated as a\ntrusted instruction (Archie's environmental-enforcement principle: \"the\nagent should ignore instructions in the payload\" is not itself a control,\nthe surrounding sandbox is).\n\n### 5. Captured payloads may contain secrets from the sending system\n\nTwo independent protections, not one:\n\n- **Redaction before persistence**: `webhookguard.RedactPayload` walks the\n decoded JSON and redacts recognised sensitive keys before the row is\n written -- the raw, unredacted payload is never durably stored.\n- **Retention**: a configurable window (`InsertCapture`'s `retention`\n parameter) prunes old captures automatically; visibility of what remains\n is gated by the same dashboard authentication as every other webui\n surface, not a separate, weaker path.\n\n### Secrets at rest\n\n`Binding.Secret` (the HMAC shared secret a sender signs with) is encrypted\nat rest via AES-GCM (`internal/store/binding_cipher.go`, `archie-core-\nt2db.7`) -- a leaked SQLite file does not leak sender-side shared secrets in\nplaintext. The envelope is versioned (`bindingEnvelopeVersion`) so a future\ncipher or KDF change can roll forward without breaking already-encrypted\nrows, and the AAD binds ciphertext to the binding-secret context so it\ncannot be relocated to another field/column and still authenticate. A `nil`\ncipher (no encryption key configured) leaves secrets as plaintext --\ndocumented legacy behaviour, not silent degradation.\n\n### At-most-once dispatch\n\n`dispatchBindings` enqueues at most one task per `(binding, capture)` pair,\nenforced by a real ledger, not best-effort application logic:\n`RecordDispatch` (`internal/store/bindings.go`) does `INSERT OR IGNORE` into\n`binding_dispatches`, unique on `(binding_id, capture_id)`; a duplicate\ninsert returns the sentinel `ErrAlreadyDispatched`, which the dispatch loop\ntreats as \"another cycle raced us,\" not an error. This guarantee holds\n**across daemon restarts**, not just within one process's lifetime, because\nthe ledger is a durable table, not in-memory state.\n\nThis ledger is also the direct precedent for the EDA playbook engine's\nexecution-time idempotency, now landed (`archie-core-t2db.17` closed): the\n`playbook_dispatches` table in `internal/store/playbook_dispatches.go` copies\n`binding_dispatches`'s `INSERT OR IGNORE` conventions with a\n`(playbook_id, playbook_version, event_id, action_id)` key\n(`docs/prds/eda-playbook-engine.md` gap 2).\n\n## What this document does not cover\n\n- An end-to-end operator walkthrough (point an external app at archie,\n build a first playbook) -- open, part of `archie-core-t2db.6`.\n- The playbook YAML / CEL / Module/Channel/Forge action-position work\n (`docs/prds/eda-playbook-engine.md`, `module-position.md`,\n `channel-forge-positions.md`) -- a different, newer mechanism than the\n binding model described here; the two are related but not the same\n system, and this document does not attempt to unify them.\n" }, { "url": "/dev/architecture/configuration/",