Commit Graph
6 Commits
Author SHA1 Message Date
sam 89c74d7097 refactor(archied): move webhook capture intake to the process that owns work intake
The dashboard's POST /webhooks/capture/{source} handler becomes
internal/infrastructure/captureintake.Receiver, mounted by the daemon on
the same bypass mux through webui.Server.CaptureIntake; the dashboard
keeps only the GET /api/captures read and the mapping preview's scan
window. The UI process now mounts no intake route at all instead of one
that answers 503 (archie-core-8cda.5.4).
2026-09-10 00:35:14 +10:00
sam 2b102b04a2 feat(state-store): generalize storerpc as the State Store gRPC contract (archie-core-8cda.4.2) (#770)
* feat(state-store): generalize storerpc as the State Store gRPC contract

Implements archie-core-8cda.4.2 against docs/prds/state-store-contract.md
rev. 2c: generate protobuf/gRPC client and server adapters for every
ratified store contract (proto/state/v1/state.proto, 40 RPCs, one
StateStoreService), add internal/infrastructure/staterpc with a bufconn
conformance suite exercising both local and remote adapters, serve the
service in-process (bridge address + mandatory per-task bearer token when
agent containers exist, loopback-only otherwise), and flip archie-agent's
workflow.Store to a long-lived gRPC client when the daemon injects
STATE_STORE_URL/STATE_STORE_TOKEN (falling back to the legacy NATS
storerpc path otherwise -- one agent process is never both).

Also drops the *sql.Tx parameter from store.BindingDispatcher.RecordDispatch
(a required wire-blocker fix: a transaction cannot cross a gRPC boundary,
and production always passed nil).

fix(store): drop RecordDispatch's unusable *sql.Tx parameter
feat(config): add services.state contract-adapter selector

* fix(state-store): apply no-mistakes review decisions

- Drop StreamTokenInterceptor (unused; the State Store service has no
  streaming RPCs and nothing calls it).
- Make StateStoreTokens.Validate compare tokens in constant time
  (crypto/subtle) per the auto-fix hardening.ing finding.
- Amend config.example.toml / config.Services.State doc text so it frames
  [services.state].target as the ratified surface consumed by the upcoming
  standalone archie-state-store binary (.4.3), not active today.
2026-09-07 03:01:33 +10:00
sam 543c10ac9a feat(webui): playbook bindings -- matcher + mapping + workflow, HMAC-gated
Implements archie-core-t2db.4: the runtime-editable binding entity that
turns a captured webhook into an archie task, plus the per-source HMAC
verification at handleCapture that makes t2db.5's 'unauthenticated event
cannot trigger a binding' invariant structurally enforceable.

Captures, mappings and bindings are now separate concerns on the same
single *store.Store; the new BindingStore / BindingDispatcher /
BindingTaskCreator split keeps each interface under the interfacebloat
limit (8 methods).

State machine: draft -> pending_approval -> armed; any edit on an armed
binding drops it back to pending_approval, enforced in SQL via
status='pending_approval' on UPDATE so the rule cannot be bypassed by a
direct status field in the request body. Approve is the only sanctioned
transition into armed (WHERE status='pending_approval' guard).

Overlap: two bindings may not share a source in any status; enforced at
InsertBinding and UpdateBinding (transaction-local SELECT). Approve has
a separate TOCTOU guard so a race between Insert and Approve cannot
// bypass the check.

HMAC at handleCapture: reads X-Hub-Signature-256 (then X-Signature-256)
// fallback, verifies via webhookguard.VerifyHMAC against the binding's
// per-source secret, marks CapturedEvent.Authenticated=true on match.
// Captures with no armed binding, missing signature, or wrong signature
all persist as authenticated=false -- the dispatch loop's
binding.Matcher.Matches gate (requires authenticated) is what stops
them from spawning a task. Two armed bindings for one source returns
409 at capture time as a belt-and-braces TOCTOU guard.

Dispatch loop: added Daemon.dispatchBindings, called from both Cycle
and maintainAndDrain between poll and drain. Walks
captured_events where authenticated=1 AND source IN armed bindings
AND id NOT IN binding_dispatches, runs the binding's matcher, resolves
the mapping, enqueues a chat-style task with binding_id /
// binding_version provenance stamped in the same transaction as the
// row insert. The binding_dispatches ledger (composite PK on
(binding_id, capture_id)) is the at-most-once guarantee across cycles
// and daemon restarts.

Tasks schema migration: binding_id INTEGER NOT NULL DEFAULT 0 +
binding_version INTEGER NOT NULL DEFAULT 0 (taskSchemaVersion bumped
1 -> 2). Existing tasks keep 0 / 0.

Out of scope (documented follow-ups):
- secret encryption at rest (plaintext today)
- full dashboard HTML for the binding editor (CRUD API only this bead)
- per-binding rate limits (existing per-remote-address limit applies)
- binding.Owner / binding.Repo fields (dispatch falls back to the
  single configured repo; multi-repo deployments log and skip)
2026-09-02 23:09:25 +10:00
sam 52b37333f4 fix(webui): stop the capture live-event notification duplicating payloads into the unbounded events table
Adversarial review of the just-shipped event inspector (3771492) found that
handleCapture's s.emit() call embedded the full (up to CaptureMaxBodyBytes,
default 256KiB) redacted body and headers into the events table's Data
column on every capture. That table has no retention or row-count prune
anywhere in the codebase -- unlike captured_events, whose three disk-bound
mechanics (size cap, rate limit, prune-on-write) this feature exists to
provide. An unauthenticated, rate-limited-only sender could grow the events
table without bound even though captured_events itself stayed correctly
capped, silently defeating docs/prds/event-capture-storage.md's stated
"max_events * max_body_bytes" worst-case guarantee.

Fix: the live "capture" SSE event now carries only {id, source} -- enough
for the dashboard to know something arrived -- and the event inspector
treats it purely as an invalidation signal, refetching GET /api/captures
for the authoritative row data instead of merging partial event fields into
its local list. This also fixes two smaller issues the same review found:
the previous approach's frontend dedup-by-id logic depended on a field
(Data["id"]) no test actually asserted, and received_at differed in
precision between the live-pushed and REST-listed copies of the same row --
both are moot now that the frontend has one source of truth for row data.
2026-08-22 05:16:49 +10:00
sam 37714928fc feat(webui): event inspector -- see captured payloads in the dashboard
Adds GET /api/captures (token-gated, matching the memory/skills
enabled-false-not-an-error convention when Server.Captures is nil) and a
new dashboard page at /captures listing recent inbound captures, newest
first, with per-row expand to explore the redacted payload and headers
pretty-printed.

Live updates reuse the existing operator-activity event pipeline
(Server.emit -> Events.Publish -> the /events SSE stream every other page
already subscribes to) rather than polling or inventing new push
plumbing: handleCapture now emits a Kind "capture" event with the same
field shape the list endpoint returns, so a captured event appears in the
inspector without a manual refresh.

Frontend follows the tasks/task-row.js precedent: row rendering is split
into ui/src/captures/capture-row.js so it can be unit tested without the
page's fetch/SSE wiring.

"Unbound" is shown for every row today since no binding data model exists
yet (t2db.4) -- the column is wired now so that bead only needs to change
what it renders, not add it.
2026-08-22 05:10:27 +10:00
sam 101c133239 feat(capture): store unbound webhook events, retention-bounded
Adds POST /webhooks/capture/{source}: an unauthenticated endpoint (mounted
alongside /healthz, bypassing dashboard token auth by design) that persists
any inbound webhook archie has no binding for yet, so it can be inspected
and mapped later (t2db.2/t2db.3).

docs/prds/event-capture-storage.md decides SQLite over NATS JetStream for
storage: NATS is not unconditionally available until 7d5u.2 ships embedded
support, and capture's single-writer/single-reader access pattern doesn't
need JetStream's fan-out/replay machinery anyway.

internal/store/captures.go adds a captured_events table with prune-on-write
(retention age + row-count cap, both configurable). The HTTP handler applies
webhookguard redaction before persisting, and enforces two more independent
disk-bound mechanics before that: a per-request body-size cap and a
per-remote-address rate limit. New [capture] config section with documented
defaults; all fields optional, capture is on by default.

Adversarial review (manual, after the usual fresh-subagent pass hit a
session API limit mid-run) caught and fixed three real defects before this
landed, each with a regression test confirmed to fail against the pre-fix
code:
- the body-size cap silently no-op'd when CaptureMaxBodyBytes was left
  unset, the one bound that limits a single payload's size rather than row
  count or write rate
- the rate limiter was keyed by the request's own source path segment --
  unregistered and attacker-chosen by design for this endpoint, violating
  webhookguard.RateLimiter's documented bounded-key-space assumption and
  letting a sender bypass the limit by rotating the segment; rekeyed to
  remote address
- the retention prune compared time.RFC3339Nano strings, whose trimmed
  variable-width fractional seconds break lexicographic ordering within the
  same second; switched to fixed-width RFC3339

Closes archie-core-t2db.1.
2026-08-22 04:51:25 +10:00