Webhook platform: generic inbound adapter with HMAC + filters + templates #807
Notifications
Due Date
No due date set.
Blocks
#96 Shared webhook mechanics: HMAC verification, per-source rate limiting, payload redaction
sam/archie
#814 WebhookAdapter: declarative filters
sam/archie
#817 WebhookAdapter: deliver_only mode
sam/archie
#818 WebhookAdapter: idempotency cache
sam/archie
#815 WebhookAdapter: script transforms
sam/archie
Reference: sam/archie#807
Reference in new issue
Block a user
This is not a widening bead. Forge webhook intake never migrated onto the identity/surface configuration, and that is the defect.
Evidence: IdentityConfig.Forge is of type Forge (internal/config/config.go:573) - the same struct that carries Intake, WebhookSecret and WebhookAddr (:204-219) - so the extensible surface can already EXPRESS per-identity webhook intake: [identities.forge].intake, .webhook_secret and .webhook_addr all parse. Nothing reads them. Every read is the root block: validate.go:257,262,265,268; defaults.go:278-279; bootstrap.go:1330,1337,1340,1344. Per-identity intake is therefore accepted, silently ignored, and never validated, because it never reaches validateForgeIntake.
Consequence: the only honoured intake config is the legacy root [forge], which is exactly the configuration the multi-identity surface replaces - and because setupForgeWebhook returns early when identities is set (bootstrap.go:1333), the multi-forge profile cannot receive webhooks at all.
The work: move intake onto the identity surface - per-identity receiver and secret, per-identity event routing, and identity-keyed dedup - so identities is the single way to configure a forge, intake included. That subsumes the silent-downgrade defect (archie-core-yfcn) and is the precondition for a Gitea decode path, which is otherwise dead code.
Do NOT add a second intake mechanism: extend internal/forge/webhook/receiver.go to be per-forge and per-identity. Keep the chat-side gateway internal/channels/webhook separate - it serves LLM turns with no envelope or dedup, and unifying would push that model onto work intake and break the poll/webhook dedup invariant (AGENTS.md:240-244).
Notes
docs/prds/hermes-gap-analysis.md (referenced above) was intentionally deleted in commit
65aa5b4('chore: cleanup', 2026-07-28) after it had already spawned this and ~14 sibling beads issues from its 60 identified gaps — it is not a broken link to restore. This issue's description is the surviving record of its slice of that analysis.Foundation exists: internal/channels/webhook/webhook.go (264 lines, tested in webhook_test.go) already implements the HTTP server, single-scheme HMAC validation (GitHub-style X-Hub-Signature-256/X-Signature-256 with sha256= prefix), single-field dot-notation extraction (extractText), and always-route-to-agent dispatch. Still missing, per sub-issue: GitLab/V2 HMAC schemes (275/276), declarative filters (282), script transforms (283), multi-variable {field} interpolation vs current single-path extraction (284), an actual deliver_only skip-agent branch -- current code always calls router.Route (285), idempotency cache (286), MS Graph validationToken handshake (287). Extend this file/package, don't create a parallel one.
Maintainer 2026-09-22: UPDATE the bead. Evidence: GitHub webhook intake with HMAC verification exists (internal/forge/webhook/receiver.go) - provider one and its signature scheme. Not built: the generic multi-provider platform, declarative all/any/not filters with dot-notation field resolution, and script transforms. Note AGENTS.md deliberately keeps intake narrow (it refuses to start when identities is configured), so widening it is a design decision, not just a port.
Acceptance check 2026-09-22 (read-only, file:line): GitHub intake SHIPPED forge/webhook/receiver.go:143 ServeHTTP + :163 X-Hub-Signature-256, tests receiver_test.go; chat gateway SHIPPED channels/webhook/webhook.go:38 RouteConfig + :128-197 handler, wired at archiemessaging/compose.go:116-124 as exactly ONE route, tests webhook_test.go. ABSENT: GitLab/Svix/V2/V1 schemes (only webhookguard/hmac.go:15, single scheme, no timestamp binding), declarative filters, script transforms. Stubs: extractText webhook.go:246 (single path); deliver_only webhook.go:52 only chooses HTTP reply vs 202 and always calls client.Route at :163; no idempotency cache (only forge-level PublishUnique on owner/repo/number). Design constraint enforced at bootstrap.go:1332-1335 and ARCHITECTURE.md:224-233.
Gitea gap found 2026-09-22 (read-only, file:line): Gitea is a supported forge (internal/forge/gitea.go; forge.go:19 case "gitea") and the deployment profile deployments/multi-forge-github-gitea.toml targets it, but the webhook intake understands only GitHub signatures - forge/webhook/receiver.go:163 reads X-Hub-Signature-256 and channels/webhook/webhook.go:145-147 falls back to X-Signature-256. No X-Gitea-Signature (or X-Gogs-Signature) scheme exists anywhere, and internal/forge/webhook has no Gitea receiver, so archie can poll Gitea but cannot receive its webhooks. The providers this bead asks for instead are GitLab, Svix/AgentMail, a generic V2 and legacy V1 - none of which is a forge this deployment runs. Scope decision for the maintainer: lead with Gitea intake (small, wanted, forge client already present; Gitea sends X-Gitea-Signature as HMAC-SHA256 hex plus X-Gitea-Event), optionally GitLab token in the same pass; defer declarative filters; and hold per-route script transforms until the ze7y trust decision, since that is an execution surface over attacker-influenced input.
FULL EVALUATION 2026-09-22 (read-only, file:line). There are THREE inbound webhook surfaces, not two. (A) Forge intake internal/forge/webhook/receiver.go: bare-path POST /, X-Hub-Signature-256 verified at :164 BEFORE github.ParseWebHook at :170, only *github.IssuesEvent decoded (:180), mapped to workintake.TaskEnvelope with IdempotencyKey owner/repo/number, published via PublishUnique (daemon.go:953-966). (B) Chat inbound gateway internal/channels/webhook/webhook.go: per-route POST, auth OPTIONAL and only when route.Secret is set (:144), dot-path text extraction, routes to ChatContract as an LLM turn - no envelope, no idempotency, no dispatch predicate. (C) Capture intake internal/infrastructure/captureintake/receiver.go: POST /webhooks/capture/{source} on the webui bypass mux (server.go:305), unauthenticated by design with per-source HMAC only when exactly one armed binding, persists CapturedEvent. internal/webhookguard is a shared library, not a receiver.
WHAT IS ACTUALLY LIVE: (A) is opt-in via Forge.Intake, and single-forge-github.toml does not set it - the default is "poll" (config.go:198), so the shipped GitHub profile does NOT run the receiver at all. multi-forge-github-gitea.toml uses identities, and setupForgeWebhook returns early at bootstrap.go:1333 regardless of intake, so a Gitea deployment receives NO webhooks and every identity polls. (B) is live only when Chat.WebhookAddr is set, which neither deployment sets. (C) runs wherever the archieui process runs.
CONCLUSION: a Gitea decode path (option 1, ~2-4 days) is DEAD CODE for the multi-forge profile, because the multi-identity guard refuses to start before the receiver is reached. Gitea webhooks therefore require multi-identity routing (option 3, ~2-4 weeks) and Gitea decode together; Gitea decode alone pays off only on a separate single-forge type="gitea" deployment. Options in increasing cost: (1) Gitea-native decode in (A) ~2-4 days; (2) forge-neutral decode abstraction ~1-2 weeks; (3) multi-identity per-identity routing ~2-4 weeks; (4) full capture-to-binding-to-dispatch (half-built already in (C)) weeks, and a different epic. Keep (A) and (B) apart: unifying would push (B) no-dedup/no-dispatch model onto work intake and break poll/webhook dedup, which is a stated invariant.
Root cause recorded 2026-09-22 (maintainer observation, confirmed): the intake knobs live on the shared Forge struct so the extensible surface accepts them, but every code path reads the root [forge] block. This is the same failure class as archie-core-xrux (overlay dropped a map field) and the parallel_tool_calls projection omission fixed in a2a26c8d - a config field that parses and has no effect. Related bead: archie-core-yfcn (silent downgrade to polling).
RIGHT-SIZED 2026-09-22, correcting the earlier estimate. The evaluation called the full migration ~2-4 weeks. That was an unaudited guess and it is wrong; I repeated it without sizing it. What the tree holds: internal/forge/webhook/receiver.go is 278 lines and Receiver is already parameterised by (secret, trigger, label, botUser, publish, log) - New(...) takes exactly the per-identity values - so per-identity intake is N receivers or one listener routing per identity, not new machinery. IdentityConfig already carries Forge, Dispatch, Repos and PollInterval (config.go:504-505, 544, 570). The dedup key is already repo-scoped: archie:// (envelope.go:105-107), so identity-keyed dedup is unnecessary because different identities repos cannot collide. The guard at bootstrap.go:1333 exists because one receiver cannot carry per-identity trigger/label/bot-user, and its comment already says extending it is a separate feature. Real work: construct N receivers (or route by path/secret), remove the guard, and add a Gitea decode (X-Gitea-Signature and X-Gitea-Event headers plus a Gitea-shaped payload decode, since taskFromEvent is typed *github.IssuesEvent). The open DESIGN question is the listener shape - one listener per identity on distinct addresses, or one listener with per-identity routes and secrets. That is what needs a decision, not weeks of implementation.
Close reason
Closed
Imported from beads
archie-core-1786637497036-275-8838629e· GitHub #296 · type feature · priority P2 · created 2026-08-08 · closed 2026-10-01