Shared webhook mechanics: HMAC verification, per-source rate limiting, payload redaction #96

Closed
opened 2026-10-27 13:34:56 +00:00 by sam · 0 comments
Owner

NARROWED 2026-08-22 from this bead's prior scope, which bundled capture-storage and an approval state machine into a single 'shared mechanics' package. On reflection those are not genuinely shared yet:

  • Capture STORAGE is t2db.1's job -- it depends on the NATS-vs-SQLite decision 7d5u.2 owns, and building a storage engine here would either duplicate or contradict that. This package may be USED by t2db.1's capture path, but does not own persistence.
  • The draft/pending_approval/armed state machine belongs to the entity that has that state -- t2db.4's binding -- not a generic package. There is exactly one consumer (bindings) today; per 'extract into a shared primitive on the second real consumer, never in anticipation of one,' building it here now would be speculative.

What remains, and is genuinely storage-agnostic and reusable by any inbound HTTP source regardless of what consumes it:

  1. HMAC-SHA256 verification -- generalize internal/channels/webhook/webhook.go's existing unexported validateHMAC into an exported function here, and have that package call into it instead of keeping its own copy. One implementation of signature checking, not two that can drift.
  2. Per-source rate limiting -- a token bucket keyed by source ID (not global), so one noisy sender cannot exhaust another's budget. Limit-exceeded is an observable rejection, not a silent drop, per t2db.5's decision.
  3. Payload redaction -- generalize internal/gateway/stream.go's unexported sensitiveParameterKey/redactToolParameters heuristic (matches token/secret/password/passwd/api_key/apikey/authorization/credential/cookie/private_key against normalized JSON keys) into a reusable, storage-agnostic transform any capture path can apply before persisting.

Acceptance criteria

webhookguard.VerifyHMAC replaces internal/channels/webhook's private copy with no behaviour change (its existing tests still pass); a per-source rate limiter rejects over-budget requests observably; a redaction function matches the same key heuristic already proven in gateway/stream.go; the package has zero imports of any domain, infrastructure or app package.

Notes

--notes=2026-08-22: merged to main as f3d3d32 (PR #534, squash of aa23ac9+46401ab+ce4f945).

Close reason

Built internal/webhookguard: VerifyHMAC (behavior-preserving extraction from internal/channels/webhook, which now calls it instead of keeping a private copy), RateLimiter (per-source token bucket, independent budgets, observable rejection), RedactPayload (key-name heuristic matching gateway/stream.go's marker list, but -- deliberately unlike that function -- redacts only values under sensitive keys and leaves ordinary strings/numbers/bools untouched, since a captured payload exists to be inspected for field mapping in t2db.3).

All four acceptance criteria met and independently confirmed by a fresh adversarial review with zero surviving findings: HMAC extraction is byte-for-byte behavior-preserving (diffed against the deleted function); rate limiter hand-traced correct on two scenarios plus race-clean under -race; redaction's divergence from gateway/stream.go verified by reading both functions side by side; zero non-stdlib imports confirmed by grep. The reviewer also mutation-tested both the redaction and rate-limit guards (disabled each in turn) and confirmed the test suite catches both, then reverted and re-verified go test -race and git status were clean.

task check exit 0, golangci-lint 0 issues (one forcetypeassert finding was caught and fixed properly during build, not suppressed), gofumpt clean. All 10 pre-existing internal/channels/webhook tests pass unmodified; the one now-redundant unit test was removed with a comment pointing to where its coverage relocated.


Imported from beads archie-core-7d5u.3 · GitHub #546 · type feature · priority P1 · created 2026-08-21 · closed 2026-08-21

NARROWED 2026-08-22 from this bead's prior scope, which bundled capture-storage and an approval state machine into a single 'shared mechanics' package. On reflection those are not genuinely shared yet: - Capture STORAGE is t2db.1's job -- it depends on the NATS-vs-SQLite decision 7d5u.2 owns, and building a storage engine here would either duplicate or contradict that. This package may be USED by t2db.1's capture path, but does not own persistence. - The draft/pending_approval/armed state machine belongs to the entity that has that state -- t2db.4's binding -- not a generic package. There is exactly one consumer (bindings) today; per 'extract into a shared primitive on the second real consumer, never in anticipation of one,' building it here now would be speculative. What remains, and is genuinely storage-agnostic and reusable by any inbound HTTP source regardless of what consumes it: 1. HMAC-SHA256 verification -- generalize internal/channels/webhook/webhook.go's existing unexported validateHMAC into an exported function here, and have that package call into it instead of keeping its own copy. One implementation of signature checking, not two that can drift. 2. Per-source rate limiting -- a token bucket keyed by source ID (not global), so one noisy sender cannot exhaust another's budget. Limit-exceeded is an observable rejection, not a silent drop, per t2db.5's decision. 3. Payload redaction -- generalize internal/gateway/stream.go's unexported sensitiveParameterKey/redactToolParameters heuristic (matches token/secret/password/passwd/api_key/apikey/authorization/credential/cookie/private_key against normalized JSON keys) into a reusable, storage-agnostic transform any capture path can apply before persisting. ### Acceptance criteria webhookguard.VerifyHMAC replaces internal/channels/webhook's private copy with no behaviour change (its existing tests still pass); a per-source rate limiter rejects over-budget requests observably; a redaction function matches the same key heuristic already proven in gateway/stream.go; the package has zero imports of any domain, infrastructure or app package. ### Notes --notes=2026-08-22: merged to main as f3d3d32 (PR #534, squash of aa23ac9+46401ab+ce4f945). ### Close reason Built internal/webhookguard: VerifyHMAC (behavior-preserving extraction from internal/channels/webhook, which now calls it instead of keeping a private copy), RateLimiter (per-source token bucket, independent budgets, observable rejection), RedactPayload (key-name heuristic matching gateway/stream.go's marker list, but -- deliberately unlike that function -- redacts only values under sensitive keys and leaves ordinary strings/numbers/bools untouched, since a captured payload exists to be inspected for field mapping in t2db.3). All four acceptance criteria met and independently confirmed by a fresh adversarial review with zero surviving findings: HMAC extraction is byte-for-byte behavior-preserving (diffed against the deleted function); rate limiter hand-traced correct on two scenarios plus race-clean under -race; redaction's divergence from gateway/stream.go verified by reading both functions side by side; zero non-stdlib imports confirmed by grep. The reviewer also mutation-tested both the redaction and rate-limit guards (disabled each in turn) and confirmed the test suite catches both, then reverted and re-verified go test -race and git status were clean. task check exit 0, golangci-lint 0 issues (one forcetypeassert finding was caught and fixed properly during build, not suppressed), gofumpt clean. All 10 pre-existing internal/channels/webhook tests pass unmodified; the one now-redundant unit test was removed with a comment pointing to where its coverage relocated. --- _Imported from beads `archie-core-7d5u.3` · GitHub #546 · type feature · priority P1 · created 2026-08-21 · closed 2026-08-21_
sam added the highfeature labels 2026-10-27 13:34:56 +00:00
sam self-assigned this 2026-10-27 13:34:56 +00:00
sam closed this issue 2026-10-27 13:34:56 +00:00
sam added a new dependency 2026-10-27 13:41:42 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: sam/archie#96