docs(prds): decide webhook intake security ahead of the code it constrains
archie-core-t2db.5 -- decides the five non-negotiables its description named before capture (t2db.1) or the intake family (7d5u.3) exist to build against it: per-source HMAC auth reusing internal/channels/webhook's existing scheme, a draft/pending_approval/armed binding state machine modeled on telegram's dangerous-command approval flow, per-source rate limiting (precedent: pairing.MaxPendingPerPlatform) that rejects rather than silently drops, blast-radius bounding via the same gate/diff-cap/sandbox every task already gets rather than a new relaxation, and 7-day retention with best-effort redaction reusing gateway/stream.go's sensitiveParameterKey heuristic. The bead stays open: its code-level acceptance criteria constrain beads that haven't landed yet and cannot be satisfied by a document alone.
This commit is contained in:
1 parent
e8b053b95a
commit
d046ad6859
2 files changed
+102
-1
No files matched your search
+1
-1
@@ -1,4 +1,4 @@
|
||||
{"_type":"issue","id":"archie-core-t2db.5","title":"Security: an unauthenticated endpoint must not be able to run an agent","description":"P0 within this epic because the feature's own pitch is the threat model: a public intake surface whose output is an agent with tool access and a repo. Get this wrong and a webhook URL is remote code execution with extra steps.\n\nNon-negotiables to design:\n\n- Per-source authentication. A source registers with a secret (HMAC, as internal/channels/webhook already does per route) before its events can trigger anything. Capture-only for unknown senders is acceptable; triggering is not.\n- Human approval before a binding goes live. An operator authoring a binding is a deliberate act; an event arriving is not. Nothing should self-arm.\n- Rate and volume limits per source, so capture cannot be used to exhaust disk or drown the inspector.\n- Payload content is untrusted input flowing into an agent prompt. Treat prompt injection as expected, not exceptional -- the sender controls that text, and the agent has tools. Bound what a playbook-triggered workflow may do.\n- Captured payloads may contain secrets from the sending system. Decide retention, redaction and who can view them; internal/memory/scrubber may already have applicable prior art.\n\nArchie's own principle applies: environmental enforcement over prompt rules. 'The agent should ignore instructions in the payload' is not a control.","acceptance_criteria":"An event from an unauthenticated source can be captured but cannot trigger a workflow; a new binding requires explicit human approval before it arms; per-source rate limits exist; the blast radius of a playbook-triggered workflow is bounded by code rather than by prompt text; retention and visibility of captured payloads is decided and documented.","status":"open","priority":0,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:46Z","created_by":"Sam Catlow","updated_at":"2026-08-21T13:45:46Z","dependencies":[{"issue_id":"archie-core-t2db.5","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:45Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-t2db.5","title":"Security: an unauthenticated endpoint must not be able to run an agent","description":"P0 within this epic because the feature's own pitch is the threat model: a public intake surface whose output is an agent with tool access and a repo. Get this wrong and a webhook URL is remote code execution with extra steps.\n\nNon-negotiables to design:\n\n- Per-source authentication. A source registers with a secret (HMAC, as internal/channels/webhook already does per route) before its events can trigger anything. Capture-only for unknown senders is acceptable; triggering is not.\n- Human approval before a binding goes live. An operator authoring a binding is a deliberate act; an event arriving is not. Nothing should self-arm.\n- Rate and volume limits per source, so capture cannot be used to exhaust disk or drown the inspector.\n- Payload content is untrusted input flowing into an agent prompt. Treat prompt injection as expected, not exceptional -- the sender controls that text, and the agent has tools. Bound what a playbook-triggered workflow may do.\n- Captured payloads may contain secrets from the sending system. Decide retention, redaction and who can view them; internal/memory/scrubber may already have applicable prior art.\n\nArchie's own principle applies: environmental enforcement over prompt rules. 'The agent should ignore instructions in the payload' is not a control.","acceptance_criteria":"An event from an unauthenticated source can be captured but cannot trigger a workflow; a new binding requires explicit human approval before it arms; per-source rate limits exist; the blast radius of a playbook-triggered workflow is bounded by code rather than by prompt text; retention and visibility of captured payloads is decided and documented.","notes":"2026-08-22: security design decided and documented in docs/prds/webhook-intake-security.md, covering all five non-negotiables from this bead's description:\n1. Per-source auth reuses internal/channels/webhook's existing HMAC-SHA256 scheme; capture accepts unsigned events (visible, marked unauthenticated), only authenticated events can ever match a binding.\n2. Bindings gain a draft -\u003e pending_approval -\u003e armed state machine; any edit to an armed binding drops it back to pending_approval so approval can't be bypassed by editing. Modeled on telegram's existing dangerousAction/pendingApproval flow (dangerous.go/approval.go).\n3. Per-source token-bucket rate limiting (precedent: pairing/store.go's MaxPendingPerPlatform); limit-exceeded is an observable 429, not a silent drop.\n4. Blast radius bounded by the SAME code-level controls every task already gets -- gate, diff_cap_lines, sandboxed container, worktree isolation -- no new relaxation for playbook-triggered work, and payload text is ordinary untrusted prompt input, never a trusted instruction.\n5. 7-day configurable retention on captured payloads; best-effort redaction reusing gateway/stream.go's sensitiveParameterKey heuristic before persisting; visibility gated by the same dashboard auth as every other webui surface.\n\nThis bead's code-level acceptance criteria (capture-can't-trigger, rate limits exist, blast-radius bounded, approval gate) cannot be implemented yet: they constrain t2db.1 (capture), 7d5u.3 (intake family) and t2db.4 (binding), none of which have landed. This document is the contract those beads build against -- do not build any of them without satisfying it. Leaving this bead in_progress rather than closing it; it closes when t2db.1/7d5u.3/t2db.4 land and demonstrably satisfy points 1-5, not before.","status":"in_progress","priority":0,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:46Z","created_by":"Sam Catlow","updated_at":"2026-08-21T14:53:33Z","started_at":"2026-08-21T14:51:26Z","dependencies":[{"issue_id":"archie-core-t2db.5","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:45Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-t2db.4","title":"Playbook binding: matcher + mapping + workflow, editable at runtime","description":"The object that ties it together. A binding is: a matcher (which inbound events does this apply to), a mapping (which fields the workflow receives), and a target agentic workflow.\n\nRuntime-editable is the whole point -- creating or changing a binding must not require a redeploy or a daemon restart. That has real consequences the design must face:\n\n- Bindings are persisted state, not config. They live in the store, not config.toml, because operators create them from the UI while the daemon runs.\n- They need versioning. A binding that starts a task should record WHICH version ran, or a later edit makes historical task provenance a lie.\n- Matcher ambiguity needs a defined resolution: two bindings matching one event either both run, or the more specific wins, or it is an error. Pick one and say so.\n- A binding referencing a workflow that no longer exists must fail visibly at bind time, not at 3am when the event arrives.","acceptance_criteria":"A binding can be created, edited and disabled from the UI while the daemon runs; a task records the binding version that started it; overlapping matchers resolve by a defined and documented rule; a binding naming a missing workflow is rejected when saved.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:45Z","created_by":"Sam Catlow","updated_at":"2026-08-21T13:45:45Z","dependencies":[{"issue_id":"archie-core-t2db.4","depends_on_id":"archie-core-t2db.5","type":"blocks","created_at":"2026-08-21T23:45:53Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-t2db.4","depends_on_id":"archie-core-t2db.3","type":"blocks","created_at":"2026-08-21T23:45:53Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-t2db.4","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:45Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":2,"dependent_count":1,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-t2db.1","title":"Capture inbound events archie does not yet understand","description":"The precondition for everything else. Today an unrecognised inbound request is rejected; for schema-by-example it must be RECORDED, because you cannot map a payload you have never seen.\n\nAccept, store and retain inbound events with no binding: raw body, headers, source, arrival time, and the identity that sent it. Retention must be bounded -- an open capture endpoint is an unbounded disk write, and this is exactly the kind of surface an attacker fills.\n\nStrong argument for NATS JetStream as the store rather than the in-process events.Bus: durable, replayable, retention-bounded natively, already deployed. Note the constraint though -- internal/infrastructure/eventbus/nats/doc.go states 'when no NATS URL is configured the daemon uses its local SQLite claim flow instead; nothing here is required for single-process operation'. NATS is currently OPTIONAL. Decide deliberately whether capture requires it (making this feature unavailable in single-process deployments) or degrades to SQLite, and record which -- do not let it become a hard dependency by accident.","acceptance_criteria":"An unbound webhook POST is captured and visible rather than rejected; capture is retention-bounded with a documented limit; the single-process (no NATS) behaviour is a deliberate, recorded decision; capture cannot exhaust disk.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:18Z","created_by":"Sam Catlow","updated_at":"2026-08-21T13:45:18Z","dependencies":[{"issue_id":"archie-core-t2db.1","depends_on_id":"archie-core-7d5u.4","type":"blocks","created_at":"2026-08-21T23:45:54Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-t2db.1","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:17Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-t2db.2","title":"Event inspector: see real inbound payloads in the dashboard","description":"The operator-facing half of capture. A live view of recent inbound events: source, arrival time, and the actual payload, pretty-printed and explorable.\n\nThis is the screen that makes the feature feel like magic: you hit 'send test webhook' in the other app, alt-tab to archie, and the event is sitting there. Latency from arrival to visible matters more than it looks -- it is the difference between a design loop and a guessing game.\n\nShow unbound events prominently. An event nothing is listening to is not noise here, it is the queue of integrations waiting to be built.","acceptance_criteria":"Recent inbound events are listed with source, time and full payload; unbound events are distinguishable from bound ones; a newly arrived event appears without a manual refresh; payloads are explorable rather than a wall of text.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:18Z","created_by":"Sam Catlow","updated_at":"2026-08-21T13:45:18Z","dependencies":[{"issue_id":"archie-core-t2db.2","depends_on_id":"archie-core-t2db.1","type":"blocks","created_at":"2026-08-21T23:45:53Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-t2db.2","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:17Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# Webhook intake security -- decision
|
||||
|
||||
**Status:** Decided, not yet implemented
|
||||
**Date:** 2026-08-22
|
||||
**Beads issue:** `archie-core-t2db.5`, blocks `archie-core-t2db.4`
|
||||
|
||||
## Why this is decided ahead of the code it constrains
|
||||
|
||||
`archie-core-t2db` (no-code playbooks) and `archie-core-7d5u` (typed event
|
||||
sources) both land a public intake surface whose eventual output is an agent
|
||||
with tool access and a repo. Neither capture (`t2db.1`) nor the intake family
|
||||
(`7d5u.3`) has shipped yet, so this decision exists to be a contract the two
|
||||
epics build against, not a retrofit once the surface exists. A webhook URL
|
||||
that can start an agent is remote code execution with extra steps if any of
|
||||
the five points below is missing on day one.
|
||||
|
||||
## 1. Per-source authentication
|
||||
|
||||
Reuse the per-route HMAC-SHA256 scheme already in
|
||||
`internal/channels/webhook/webhook.go` (`validateHMAC`), unchanged: a source
|
||||
registers with a shared secret; the signature is checked against the raw
|
||||
body. Do not invent a second scheme for intake.
|
||||
|
||||
**Capture and triggering split at the signature, not at a separate flag.** An
|
||||
event with a missing or invalid signature is still captured (visible in the
|
||||
inspector, marked unauthenticated) so an operator can diagnose a
|
||||
misconfigured sender -- but an unauthenticated event can never be matched
|
||||
against a binding. Authentication is what promotes an event from "something
|
||||
arrived" to "something that can run a workflow."
|
||||
|
||||
## 2. Human approval before a binding arms
|
||||
|
||||
A binding has three states: `draft -> pending_approval -> armed`. Only
|
||||
`armed` bindings are evaluated against incoming events. Creating or editing a
|
||||
binding's matcher, mapping, or target workflow always sets it to
|
||||
`pending_approval`, including edits to an already-armed binding -- an edit is
|
||||
not silently re-armed, or the approval gate is decorative. Approval requires
|
||||
an authenticated dashboard session; there is no unattended approval path.
|
||||
|
||||
Model: `internal/channels/telegram`'s dangerous-command approval flow
|
||||
(`dangerousAction` / `pendingApproval` in `dangerous.go` / `approval.go`) is
|
||||
the existing precedent for "a human decision gates code from running" in this
|
||||
codebase -- follow its shape rather than designing a new approval primitive.
|
||||
|
||||
## 3. Rate and volume limits, per source
|
||||
|
||||
A token-bucket limiter keyed by source ID, not global -- one noisy or hostile
|
||||
sender must not exhaust another source's capture budget. Precedent for
|
||||
per-key volume capping: `internal/pairing/store.go`'s `MaxPendingPerPlatform`.
|
||||
|
||||
On limit exceeded, the request is rejected (HTTP 429), not silently dropped.
|
||||
A silently dropped event is indistinguishable from a sender that never fired
|
||||
-- the same invisible-failure shape the update-verification work
|
||||
(`archie-core-522`) exists to prevent. The rejection itself is observable in
|
||||
the inspector as a distinct outcome from "not received."
|
||||
|
||||
## 4. Blast radius bounded by code, not by prompt wording
|
||||
|
||||
A playbook-triggered task runs through exactly the same enforcement as every
|
||||
other task: the `[[repos.gate]]` quality gate, `diff_cap_lines`, the
|
||||
sandboxed per-task container (`internal/container`), worktree isolation, and
|
||||
"the model never runs git." No new relaxation is introduced for
|
||||
playbook-triggered work, and none is needed -- the existing constraints are
|
||||
the control.
|
||||
|
||||
Mapped payload fields are treated as ordinary user-controlled text inside an
|
||||
agent prompt, exactly like an inbound chat message today, never as trusted
|
||||
instructions. "The agent should ignore instructions in the payload" is not a
|
||||
control and must not appear as one anywhere in this feature -- per
|
||||
`ARCHITECTURE.md`'s environmental-enforcement-over-prompt-rules principle,
|
||||
the boundary is the gate and the sandbox, not a sentence in the system
|
||||
prompt.
|
||||
|
||||
## 5. Retention and redaction of captured payloads
|
||||
|
||||
**Retention:** captured events (bound or not) are purged after 7 days,
|
||||
configurable per deployment. Unbound events exist to be mapped against; nothing
|
||||
justifies keeping them indefinitely once that window has passed, and an
|
||||
unbounded capture store is the same disk-exhaustion risk item 3 addresses at
|
||||
the request-rate level.
|
||||
|
||||
**Redaction:** before a captured payload is persisted, apply the same
|
||||
key-name heuristic `internal/gateway/stream.go`'s `sensitiveParameterKey`
|
||||
already uses for tool-call parameters (matches `token`, `secret`, `password`,
|
||||
`passwd`, `api_key`, `apikey`, `authorization`, `credential`, `cookie`,
|
||||
`private_key` against normalized JSON keys) and replace matching values with
|
||||
the same `[redacted]` marker. This is best-effort, not a guarantee -- a
|
||||
sender that names a secret field something the heuristic misses will still
|
||||
have it captured. Document that limitation where captured payloads are
|
||||
displayed, rather than implying the redaction is complete.
|
||||
|
||||
**Visibility:** captured payloads, redacted or not, are visible only to an
|
||||
authenticated dashboard operator -- the same authorization boundary as every
|
||||
other webui surface, not a separate or lower bar.
|
||||
|
||||
## What this does not decide
|
||||
|
||||
The intake family's registration interface (`7d5u.3`), the capture storage
|
||||
mechanism (`7d5u.2`, embedded NATS), and the binding data model's persistence
|
||||
and versioning (`t2db.4`) are separate decisions. This document constrains
|
||||
all three: whatever they build must satisfy points 1-5 above.
|
||||
Reference in new issue
Block a user