From d046ad6859cb142dd85f89b705212d462155cc2c Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Sat, 22 Aug 2026 00:54:00 +1000 Subject: [PATCH] docs(prds): decide webhook intake security ahead of the code it constrains archie-core-t2db.5 -- decides the five non-negotiables its description named before capture (t2db.1) or the intake family (7d5u.3) exist to build against it: per-source HMAC auth reusing internal/channels/webhook's existing scheme, a draft/pending_approval/armed binding state machine modeled on telegram's dangerous-command approval flow, per-source rate limiting (precedent: pairing.MaxPendingPerPlatform) that rejects rather than silently drops, blast-radius bounding via the same gate/diff-cap/sandbox every task already gets rather than a new relaxation, and 7-day retention with best-effort redaction reusing gateway/stream.go's sensitiveParameterKey heuristic. The bead stays open: its code-level acceptance criteria constrain beads that haven't landed yet and cannot be satisfied by a document alone. --- .beads/issues.jsonl | 2 +- docs/prds/webhook-intake-security.md | 101 +++++++++++++++++++++++++++ 2 files changed, 102 insertions(+), 1 deletion(-) create mode 100644 docs/prds/webhook-intake-security.md diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 90df216ab..f81aeb0f6 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -1,4 +1,4 @@ -{"_type":"issue","id":"archie-core-t2db.5","title":"Security: an unauthenticated endpoint must not be able to run an agent","description":"P0 within this epic because the feature's own pitch is the threat model: a public intake surface whose output is an agent with tool access and a repo. Get this wrong and a webhook URL is remote code execution with extra steps.\n\nNon-negotiables to design:\n\n- Per-source authentication. A source registers with a secret (HMAC, as internal/channels/webhook already does per route) before its events can trigger anything. Capture-only for unknown senders is acceptable; triggering is not.\n- Human approval before a binding goes live. An operator authoring a binding is a deliberate act; an event arriving is not. Nothing should self-arm.\n- Rate and volume limits per source, so capture cannot be used to exhaust disk or drown the inspector.\n- Payload content is untrusted input flowing into an agent prompt. Treat prompt injection as expected, not exceptional -- the sender controls that text, and the agent has tools. Bound what a playbook-triggered workflow may do.\n- Captured payloads may contain secrets from the sending system. Decide retention, redaction and who can view them; internal/memory/scrubber may already have applicable prior art.\n\nArchie's own principle applies: environmental enforcement over prompt rules. 'The agent should ignore instructions in the payload' is not a control.","acceptance_criteria":"An event from an unauthenticated source can be captured but cannot trigger a workflow; a new binding requires explicit human approval before it arms; per-source rate limits exist; the blast radius of a playbook-triggered workflow is bounded by code rather than by prompt text; retention and visibility of captured payloads is decided and documented.","status":"open","priority":0,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:46Z","created_by":"Sam Catlow","updated_at":"2026-08-21T13:45:46Z","dependencies":[{"issue_id":"archie-core-t2db.5","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:45Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} +{"_type":"issue","id":"archie-core-t2db.5","title":"Security: an unauthenticated endpoint must not be able to run an agent","description":"P0 within this epic because the feature's own pitch is the threat model: a public intake surface whose output is an agent with tool access and a repo. Get this wrong and a webhook URL is remote code execution with extra steps.\n\nNon-negotiables to design:\n\n- Per-source authentication. A source registers with a secret (HMAC, as internal/channels/webhook already does per route) before its events can trigger anything. Capture-only for unknown senders is acceptable; triggering is not.\n- Human approval before a binding goes live. An operator authoring a binding is a deliberate act; an event arriving is not. Nothing should self-arm.\n- Rate and volume limits per source, so capture cannot be used to exhaust disk or drown the inspector.\n- Payload content is untrusted input flowing into an agent prompt. Treat prompt injection as expected, not exceptional -- the sender controls that text, and the agent has tools. Bound what a playbook-triggered workflow may do.\n- Captured payloads may contain secrets from the sending system. Decide retention, redaction and who can view them; internal/memory/scrubber may already have applicable prior art.\n\nArchie's own principle applies: environmental enforcement over prompt rules. 'The agent should ignore instructions in the payload' is not a control.","acceptance_criteria":"An event from an unauthenticated source can be captured but cannot trigger a workflow; a new binding requires explicit human approval before it arms; per-source rate limits exist; the blast radius of a playbook-triggered workflow is bounded by code rather than by prompt text; retention and visibility of captured payloads is decided and documented.","notes":"2026-08-22: security design decided and documented in docs/prds/webhook-intake-security.md, covering all five non-negotiables from this bead's description:\n1. Per-source auth reuses internal/channels/webhook's existing HMAC-SHA256 scheme; capture accepts unsigned events (visible, marked unauthenticated), only authenticated events can ever match a binding.\n2. Bindings gain a draft -\u003e pending_approval -\u003e armed state machine; any edit to an armed binding drops it back to pending_approval so approval can't be bypassed by editing. Modeled on telegram's existing dangerousAction/pendingApproval flow (dangerous.go/approval.go).\n3. Per-source token-bucket rate limiting (precedent: pairing/store.go's MaxPendingPerPlatform); limit-exceeded is an observable 429, not a silent drop.\n4. Blast radius bounded by the SAME code-level controls every task already gets -- gate, diff_cap_lines, sandboxed container, worktree isolation -- no new relaxation for playbook-triggered work, and payload text is ordinary untrusted prompt input, never a trusted instruction.\n5. 7-day configurable retention on captured payloads; best-effort redaction reusing gateway/stream.go's sensitiveParameterKey heuristic before persisting; visibility gated by the same dashboard auth as every other webui surface.\n\nThis bead's code-level acceptance criteria (capture-can't-trigger, rate limits exist, blast-radius bounded, approval gate) cannot be implemented yet: they constrain t2db.1 (capture), 7d5u.3 (intake family) and t2db.4 (binding), none of which have landed. This document is the contract those beads build against -- do not build any of them without satisfying it. Leaving this bead in_progress rather than closing it; it closes when t2db.1/7d5u.3/t2db.4 land and demonstrably satisfy points 1-5, not before.","status":"in_progress","priority":0,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:46Z","created_by":"Sam Catlow","updated_at":"2026-08-21T14:53:33Z","started_at":"2026-08-21T14:51:26Z","dependencies":[{"issue_id":"archie-core-t2db.5","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:45Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"archie-core-t2db.4","title":"Playbook binding: matcher + mapping + workflow, editable at runtime","description":"The object that ties it together. A binding is: a matcher (which inbound events does this apply to), a mapping (which fields the workflow receives), and a target agentic workflow.\n\nRuntime-editable is the whole point -- creating or changing a binding must not require a redeploy or a daemon restart. That has real consequences the design must face:\n\n- Bindings are persisted state, not config. They live in the store, not config.toml, because operators create them from the UI while the daemon runs.\n- They need versioning. A binding that starts a task should record WHICH version ran, or a later edit makes historical task provenance a lie.\n- Matcher ambiguity needs a defined resolution: two bindings matching one event either both run, or the more specific wins, or it is an error. Pick one and say so.\n- A binding referencing a workflow that no longer exists must fail visibly at bind time, not at 3am when the event arrives.","acceptance_criteria":"A binding can be created, edited and disabled from the UI while the daemon runs; a task records the binding version that started it; overlapping matchers resolve by a defined and documented rule; a binding naming a missing workflow is rejected when saved.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:45Z","created_by":"Sam Catlow","updated_at":"2026-08-21T13:45:45Z","dependencies":[{"issue_id":"archie-core-t2db.4","depends_on_id":"archie-core-t2db.5","type":"blocks","created_at":"2026-08-21T23:45:53Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-t2db.4","depends_on_id":"archie-core-t2db.3","type":"blocks","created_at":"2026-08-21T23:45:53Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-t2db.4","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:45Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":2,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"archie-core-t2db.1","title":"Capture inbound events archie does not yet understand","description":"The precondition for everything else. Today an unrecognised inbound request is rejected; for schema-by-example it must be RECORDED, because you cannot map a payload you have never seen.\n\nAccept, store and retain inbound events with no binding: raw body, headers, source, arrival time, and the identity that sent it. Retention must be bounded -- an open capture endpoint is an unbounded disk write, and this is exactly the kind of surface an attacker fills.\n\nStrong argument for NATS JetStream as the store rather than the in-process events.Bus: durable, replayable, retention-bounded natively, already deployed. Note the constraint though -- internal/infrastructure/eventbus/nats/doc.go states 'when no NATS URL is configured the daemon uses its local SQLite claim flow instead; nothing here is required for single-process operation'. NATS is currently OPTIONAL. Decide deliberately whether capture requires it (making this feature unavailable in single-process deployments) or degrades to SQLite, and record which -- do not let it become a hard dependency by accident.","acceptance_criteria":"An unbound webhook POST is captured and visible rather than rejected; capture is retention-bounded with a documented limit; the single-process (no NATS) behaviour is a deliberate, recorded decision; capture cannot exhaust disk.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:18Z","created_by":"Sam Catlow","updated_at":"2026-08-21T13:45:18Z","dependencies":[{"issue_id":"archie-core-t2db.1","depends_on_id":"archie-core-7d5u.4","type":"blocks","created_at":"2026-08-21T23:45:54Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-t2db.1","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:17Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0} {"_type":"issue","id":"archie-core-t2db.2","title":"Event inspector: see real inbound payloads in the dashboard","description":"The operator-facing half of capture. A live view of recent inbound events: source, arrival time, and the actual payload, pretty-printed and explorable.\n\nThis is the screen that makes the feature feel like magic: you hit 'send test webhook' in the other app, alt-tab to archie, and the event is sitting there. Latency from arrival to visible matters more than it looks -- it is the difference between a design loop and a guessing game.\n\nShow unbound events prominently. An event nothing is listening to is not noise here, it is the queue of integrations waiting to be built.","acceptance_criteria":"Recent inbound events are listed with source, time and full payload; unbound events are distinguishable from bound ones; a newly arrived event appears without a manual refresh; payloads are explorable rather than a wall of text.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:18Z","created_by":"Sam Catlow","updated_at":"2026-08-21T13:45:18Z","dependencies":[{"issue_id":"archie-core-t2db.2","depends_on_id":"archie-core-t2db.1","type":"blocks","created_at":"2026-08-21T23:45:53Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-t2db.2","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:17Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} diff --git a/docs/prds/webhook-intake-security.md b/docs/prds/webhook-intake-security.md new file mode 100644 index 000000000..346a35f52 --- /dev/null +++ b/docs/prds/webhook-intake-security.md @@ -0,0 +1,101 @@ +# Webhook intake security -- decision + +**Status:** Decided, not yet implemented +**Date:** 2026-08-22 +**Beads issue:** `archie-core-t2db.5`, blocks `archie-core-t2db.4` + +## Why this is decided ahead of the code it constrains + +`archie-core-t2db` (no-code playbooks) and `archie-core-7d5u` (typed event +sources) both land a public intake surface whose eventual output is an agent +with tool access and a repo. Neither capture (`t2db.1`) nor the intake family +(`7d5u.3`) has shipped yet, so this decision exists to be a contract the two +epics build against, not a retrofit once the surface exists. A webhook URL +that can start an agent is remote code execution with extra steps if any of +the five points below is missing on day one. + +## 1. Per-source authentication + +Reuse the per-route HMAC-SHA256 scheme already in +`internal/channels/webhook/webhook.go` (`validateHMAC`), unchanged: a source +registers with a shared secret; the signature is checked against the raw +body. Do not invent a second scheme for intake. + +**Capture and triggering split at the signature, not at a separate flag.** An +event with a missing or invalid signature is still captured (visible in the +inspector, marked unauthenticated) so an operator can diagnose a +misconfigured sender -- but an unauthenticated event can never be matched +against a binding. Authentication is what promotes an event from "something +arrived" to "something that can run a workflow." + +## 2. Human approval before a binding arms + +A binding has three states: `draft -> pending_approval -> armed`. Only +`armed` bindings are evaluated against incoming events. Creating or editing a +binding's matcher, mapping, or target workflow always sets it to +`pending_approval`, including edits to an already-armed binding -- an edit is +not silently re-armed, or the approval gate is decorative. Approval requires +an authenticated dashboard session; there is no unattended approval path. + +Model: `internal/channels/telegram`'s dangerous-command approval flow +(`dangerousAction` / `pendingApproval` in `dangerous.go` / `approval.go`) is +the existing precedent for "a human decision gates code from running" in this +codebase -- follow its shape rather than designing a new approval primitive. + +## 3. Rate and volume limits, per source + +A token-bucket limiter keyed by source ID, not global -- one noisy or hostile +sender must not exhaust another source's capture budget. Precedent for +per-key volume capping: `internal/pairing/store.go`'s `MaxPendingPerPlatform`. + +On limit exceeded, the request is rejected (HTTP 429), not silently dropped. +A silently dropped event is indistinguishable from a sender that never fired +-- the same invisible-failure shape the update-verification work +(`archie-core-522`) exists to prevent. The rejection itself is observable in +the inspector as a distinct outcome from "not received." + +## 4. Blast radius bounded by code, not by prompt wording + +A playbook-triggered task runs through exactly the same enforcement as every +other task: the `[[repos.gate]]` quality gate, `diff_cap_lines`, the +sandboxed per-task container (`internal/container`), worktree isolation, and +"the model never runs git." No new relaxation is introduced for +playbook-triggered work, and none is needed -- the existing constraints are +the control. + +Mapped payload fields are treated as ordinary user-controlled text inside an +agent prompt, exactly like an inbound chat message today, never as trusted +instructions. "The agent should ignore instructions in the payload" is not a +control and must not appear as one anywhere in this feature -- per +`ARCHITECTURE.md`'s environmental-enforcement-over-prompt-rules principle, +the boundary is the gate and the sandbox, not a sentence in the system +prompt. + +## 5. Retention and redaction of captured payloads + +**Retention:** captured events (bound or not) are purged after 7 days, +configurable per deployment. Unbound events exist to be mapped against; nothing +justifies keeping them indefinitely once that window has passed, and an +unbounded capture store is the same disk-exhaustion risk item 3 addresses at +the request-rate level. + +**Redaction:** before a captured payload is persisted, apply the same +key-name heuristic `internal/gateway/stream.go`'s `sensitiveParameterKey` +already uses for tool-call parameters (matches `token`, `secret`, `password`, +`passwd`, `api_key`, `apikey`, `authorization`, `credential`, `cookie`, +`private_key` against normalized JSON keys) and replace matching values with +the same `[redacted]` marker. This is best-effort, not a guarantee -- a +sender that names a secret field something the heuristic misses will still +have it captured. Document that limitation where captured payloads are +displayed, rather than implying the redaction is complete. + +**Visibility:** captured payloads, redacted or not, are visible only to an +authenticated dashboard operator -- the same authorization boundary as every +other webui surface, not a separate or lower bar. + +## What this does not decide + +The intake family's registration interface (`7d5u.3`), the capture storage +mechanism (`7d5u.2`, embedded NATS), and the binding data model's persistence +and versioning (`t2db.4`) are separate decisions. This document constrains +all three: whatever they build must satisfy points 1-5 above.