feat(forge): observable webhook receiver + dedup regression proof; decide adversarial review design

archie-core-7d5u.5 (idempotency and poll/webhook parity): the dedup
mechanism (TaskEnvelope.IdempotencyKey + PublishUnique + JetStream
Nats-Msg-Id dedup) and the shared dispatch predicate were already fully
wired by 7d5u.4. What was missing: proof the dedup actually holds, and
observability for a wedged receiver.

- internal/infrastructure/eventbus/nats/dedup_test.go: regression test
  proving PublishUnique suppresses a duplicate publish on the same
  WorkQueuePolicy stream ARCHIE_TASKS uses in production.
- internal/forge/webhook/receiver.go: Receiver.Status() tracks
  authenticated deliveries separately from successful publishes; a GET
  to the receiver's listen address (GitHub only ever POSTs) now returns
  it as JSON, so a wedged/non-delivering receiver is observable instead
  of indistinguishable from an idle repo.

Poll/webhook parity (disabling webhook still polls) needed no code
change: Cycle()/Run() never reference cfg.Forge.Intake anywhere in
internal/daemon, only setupForgeWebhook gates the webhook HTTP server.

archie-core-h019.1 (design: adversarial self-review stage): one-page
decision at docs/prds/adversarial-self-review.md. Reviewer runs in a
second workspace pointed at a .git-stripped snapshot of the reviewed
commit, never the implementer's worktree, so isolation is a property of
what's mounted rather than a prompt instruction. Findings contract is
workflow.ReviewFinding, owned by internal/domain/workflow. A surviving
error-level finding sets StatusParked, not WaitingHuman -- the epic's
informal wording named two different states in the actual machine; this
resolves which.

Both changes reviewed by a fresh subagent with no memory of the
implementation, per CLAUDE.md's adversarial-verification requirement;
zero blocking findings, two minor coverage gaps closed.
This commit is contained in:
sam committed 2026-08-22 19:45:31 +10:00
1 parent ee650997c7
commit 8a799cd8cd
6 files changed
+469 -10

No files matched your search

+2
View File
@@ -31,3 +31,5 @@
{"id":"int-ca26e9b428d406303510eea34c2c8d05","kind":"field_change","created_at":"2026-08-21T19:18:55.467881536Z","actor":"Sam Catlow","issue_id":"archie-core-7d5u.2","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Embedded NATS wired into production startup: mode=embedded/external/off resolves from url, daemon owns the in-process server lifecycle, Retention parameterized for a future fan-out reaction stream. task check + lint clean; adversarial review zero surviving findings. Committed 2c7db34."}}
{"id":"int-484f5673b3d49ea1731bec29e2a84074","kind":"field_change","created_at":"2026-08-22T09:01:53.455811598Z","actor":"Sam Catlow","issue_id":"archie-core-t2db.3","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Payload field mapping: domain (internal/domain/mapping), SQLite store (internal/store/mappings.go), HTTP API (internal/webui/api_mapping.go), and dashboard editor (ui/src/mappings) shipped. PRD at docs/prds/payload-field-mapping.md. task check green; adversarial review found 4 issues (stale preview after rename, duplicate field names, hardcoded capture scan window, malformed path silently reinterpreted), all fixed and re-verified."}}
{"id":"int-0606ca827b33f964b6a5de2f5f4a1966","kind":"field_change","created_at":"2026-08-22T09:21:16.136296742Z","actor":"Sam Catlow","issue_id":"archie-core-7d5u.4","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"GitHub webhook receiver shipped: [forge] intake=poll|webhook|both, shared MatchesDispatch predicate (case-insensitive assignee, case-sensitive label) consumed by poller and receiver, HMAC-verified, publishes via shared PublishTask. Adversarial self-review found and fixed 3 real defects (boot-crashing secret-resolve hard-fail under intake=both, assignee case-sensitivity mismatch vs GitHub's own filter, silent multi-identity misdispatch) before landing. task check + lint clean. Committed 41e43dc."}}
{"id":"int-fe2b5654477a99738b27c129267c3636","kind":"field_change","created_at":"2026-08-22T09:45:14.980953288Z","actor":"Sam Catlow","issue_id":"archie-core-h019.1","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"One-page decision written at docs/prds/adversarial-self-review.md: reviewer runs in a second workspace (container or in-process) pointed at a .git-stripped snapshot of the reviewed commit, never the implementer's worktree; findings contract is workflow.ReviewFinding, owned by internal/domain/workflow, reported via a structured capture tool; a surviving error-level finding sets StatusParked (not WaitingHuman -- epic wording named two different states, this resolves which); model/effort reuse the existing Models[role] map and per-stage budget override, gated by a new opt-in Repo.ReviewEnabled so h019.4 doesn't gate every repo on an uncalibrated reviewer before h019.5 lands."}}
{"id":"int-0f63389a86a8600c181ac3a924308b2d","kind":"field_change","created_at":"2026-08-22T09:45:16.096384559Z","actor":"Sam Catlow","issue_id":"archie-core-7d5u.5","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Investigated first: the dedup mechanism (TaskEnvelope.IdempotencyKey + PublishUnique + JetStream Nats-Msg-Id dedup) and the shared dispatch predicate were already fully wired by 7d5u.4 -- no double-dispatch code change was needed. Closed the two real gaps: (1) internal/infrastructure/eventbus/nats/dedup_test.go is a new regression test proving PublishUnique actually suppresses a duplicate publish on the same WorkQueuePolicy stream ARCHIE_TASKS uses in production; (2) internal/forge/webhook/receiver.go gained Status()/GET support tracking authenticated deliveries vs successful publishes separately, so a wedged or non-delivering receiver is observable rather than indistinguishable from an idle repo. AC's poll/webhook parity clause (disabling webhook still polls) is satisfied architecturally with no code change possible to violate it -- verified Cycle()/Run() never reference cfg.Forge.Intake anywhere in internal/daemon; only setupForgeWebhook in cmd/archied/bootstrap.go gates the webhook HTTP server. Adversarial review pass (fresh subagent, -race) found zero blocking issues; closed two minor coverage gaps it flagged."}}
+3 -3
View File
@@ -5,14 +5,14 @@
{"_type":"issue","id":"archie-core-t2db.3","title":"Map payload fields by example, from a real captured event","description":"Pick a captured event, click through its payload, and bind JSON paths to named fields the workflow will receive. Schema by example, not schema by specification.\n\nThe hard part is not the happy path, it is drift. A mapping authored against one payload will eventually meet a payload where the path is missing, null, or a different type -- the sending app shipped a change nobody told you about. That must fail LOUDLY and land somewhere the operator sees, not silently pass an empty string into an agent prompt. An agent handed a blank field will confabulate around it, which turns a schema change into an agent doing confident nonsense in a repo.\n\nSupport a preview: given this mapping and this captured event, show exactly what the workflow would receive. Never ship a mapping the operator has not seen resolve against real data.","acceptance_criteria":"Fields are mappable from a captured payload without writing code; a mapping preview resolves against a real event before saving; a missing, null or type-changed path at runtime produces a visible failure rather than an empty value; the failure names the path and the event.","status":"closed","priority":1,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:18Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:05Z","started_at":"2026-08-22T08:39:40Z","closed_at":"2026-08-22T09:01:53Z","close_reason":"Payload field mapping: domain (internal/domain/mapping), SQLite store (internal/store/mappings.go), HTTP API (internal/webui/api_mapping.go), and dashboard editor (ui/src/mappings) shipped. PRD at docs/prds/payload-field-mapping.md. task check green; adversarial review found 4 issues (stale preview after rename, duplicate field names, hardcoded capture scan window, malformed path silently reinterpreted), all fixed and re-verified.","external_ref":"https://github.com/samcharles93/archie-core/issues/540","dependencies":[{"issue_id":"archie-core-t2db.3","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:18Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-t2db.3","depends_on_id":"archie-core-t2db.1","type":"blocks","created_at":"2026-08-21T23:45:53Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"archie-core-t2db","title":"[EPIC] Event capture and no-code playbooks: turn any inbound webhook into an agentic workflow","description":"The pitch, in the operator's words: point any app's outgoing webhook at archie, open archie, SEE the event that just arrived with its real payload, map the fields you care about, bind them to an agentic workflow, save. No code change, no redeploy, no restart. Adding a new integration stops being a development task.\n\nWhy this is different from n8n or Power Automate rather than a copy of them: their terminal node is a deterministic connector. Archie's is an AGENT with a repo, a worktree, tools and a gate. 'When Sentry reports a new issue in this project, open a PR that fixes it' is one binding here and an entire product elsewhere.\n\nThe design insight that makes it buildable: you cannot write a payload mapper against a payload you have not seen. Schema-by-example requires real recent events to map against, which means archie must CAPTURE events it does not yet understand -- including ones with no binding at all -- and retain them long enough to design against. That is also the strongest argument for putting inbound events on NATS JetStream rather than the lossy in-process bus: durable, replayable, retention-bounded, and already deployed. The capture requirement and the transport choice are the same decision.\n\nSequence: capture unrecognised events -\u003e inspect real payloads in the UI -\u003e map fields by example -\u003e bind matcher+mapping to a workflow -\u003e the next matching event runs it.\n\nThis is a public, unauthenticated-by-default intake surface whose output is an agent with tool access. Security is not a later hardening pass; it is a first-class child of this epic.","acceptance_criteria":"An operator can point a new external webhook at archie, see its payload in the UI without writing code, map fields, bind it to a workflow, and have the next matching event run that workflow -- with no redeploy and no daemon restart. A binding cannot go live without explicit human approval.","status":"open","priority":1,"issue_type":"epic","owner":"sam@catlow.tech","created_at":"2026-08-21T13:44:56Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:06Z","external_ref":"https://github.com/samcharles93/archie-core/issues/541","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-7d5u.4","title":"GitHub forge webhook receiver behind a feature flag","description":"The concrete autonomy win: react to an issue being assigned or labelled the moment it happens, instead of up to poll_interval (default 60s) later.\n\nVerified absent on 2026-08-21: internal/forge/ and internal/daemon/ contain zero webhook references, and no bead tracked forge-event intake before this epic. internal/channels/webhook exists but is a CHAT channel -- it routes inbound text through gateway.Router for LLM processing, a peer of Telegram and email. It is not forge intake and should not be bent into it.\n\nShip as a source in the typed intake family, off by default: [forge] intake = 'poll' (default) | 'webhook' | 'both'. Verify GitHub's HMAC signature. Reuse the label/assignee dispatch predicate the poller already uses -- do NOT reimplement it, since it just took a fix (3209a1f, refusing to poll on an empty label match) and two implementations will drift.\n\nDeployment reality from CLAUDE.md: archied may run on a host, in a container, or behind no public ingress at all. A webhook receiver that assumes a reachable public URL will not fit every profile, which is exactly why polling has to remain a first-class option rather than a legacy fallback.","acceptance_criteria":"With intake='webhook', a labelled issue starts a task without waiting for the poll interval; an invalid HMAC is rejected and logged; intake='poll' behaves exactly as today; the dispatch predicate is shared with the poller, not duplicated; the flag defaults to poll.","status":"closed","priority":1,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:26:52Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:07Z","started_at":"2026-08-22T08:43:35Z","closed_at":"2026-08-22T09:21:16Z","close_reason":"GitHub webhook receiver shipped: [forge] intake=poll|webhook|both, shared MatchesDispatch predicate (case-insensitive assignee, case-sensitive label) consumed by poller and receiver, HMAC-verified, publishes via shared PublishTask. Adversarial self-review found and fixed 3 real defects (boot-crashing secret-resolve hard-fail under intake=both, assignee case-sensitivity mismatch vs GitHub's own filter, silent multi-identity misdispatch) before landing. task check + lint clean. Committed 41e43dc.","external_ref":"https://github.com/samcharles93/archie-core/issues/542","dependencies":[{"issue_id":"archie-core-7d5u.4","depends_on_id":"archie-core-7d5u","type":"parent-child","created_at":"2026-08-21T23:26:51Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.4","depends_on_id":"archie-core-t2db.1","type":"relates-to","created_at":"2026-08-22T01:01:02Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.4","depends_on_id":"archie-core-7d5u.1","type":"blocks","created_at":"2026-08-21T23:26:58Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.4","depends_on_id":"archie-core-7d5u.3","type":"blocks","created_at":"2026-08-21T23:26:58Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":2,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"archie-core-7d5u.5","title":"Idempotency and poll/webhook parity: never double-dispatch, never lose work","description":"Webhooks deliver at-least-once, out of order, and sometimes not at all. Polling is the opposite: slow but eventually consistent. Running both (intake='both') is the honest production configuration, and it is also where the two failure modes meet.\n\nTwo things must hold. A single issue event must not start two tasks because the webhook and the next poll both saw it. And a webhook outage must not silently stop work -- if the receiver is wedged or the ingress is down, polling has to pick the work up, which is the entire argument for keeping it.\n\nNote archie-core-...-286 (WebhookAdapter: idempotency cache) already exists for the chat adapter. Decide whether that mechanism generalises to intake or whether forge dispatch needs its own key; do not build a second one by accident.\n\nThe silent-failure shape to design against is the one this codebase keeps hitting: a path that looks like it is working because nothing errors. A webhook receiver that has silently stopped delivering looks identical to a repo with no activity.","acceptance_criteria":"A duplicate delivery of the same forge event starts exactly one task; with intake='both', disabling the webhook path still results in the work being picked up by polling; a wedged or non-delivering webhook receiver is observable rather than indistinguishable from an idle repo.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:26:52Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:08Z","external_ref":"https://github.com/samcharles93/archie-core/issues/543","dependencies":[{"issue_id":"archie-core-7d5u.5","depends_on_id":"archie-core-7d5u","type":"parent-child","created_at":"2026-08-21T23:26:51Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.5","depends_on_id":"archie-core-7d5u.1","type":"blocks","created_at":"2026-08-21T23:26:58Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.5","depends_on_id":"archie-core-7d5u.4","type":"blocks","created_at":"2026-08-21T23:26:58Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.5","depends_on_id":"archie-core-1786637497328-286-9c985f28","type":"relates-to","created_at":"2026-08-21T23:27:06Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":2,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-7d5u.5","title":"Idempotency and poll/webhook parity: never double-dispatch, never lose work","description":"Webhooks deliver at-least-once, out of order, and sometimes not at all. Polling is the opposite: slow but eventually consistent. Running both (intake='both') is the honest production configuration, and it is also where the two failure modes meet.\n\nTwo things must hold. A single issue event must not start two tasks because the webhook and the next poll both saw it. And a webhook outage must not silently stop work -- if the receiver is wedged or the ingress is down, polling has to pick the work up, which is the entire argument for keeping it.\n\nNote archie-core-...-286 (WebhookAdapter: idempotency cache) already exists for the chat adapter. Decide whether that mechanism generalises to intake or whether forge dispatch needs its own key; do not build a second one by accident.\n\nThe silent-failure shape to design against is the one this codebase keeps hitting: a path that looks like it is working because nothing errors. A webhook receiver that has silently stopped delivering looks identical to a repo with no activity.","acceptance_criteria":"A duplicate delivery of the same forge event starts exactly one task; with intake='both', disabling the webhook path still results in the work being picked up by polling; a wedged or non-delivering webhook receiver is observable rather than indistinguishable from an idle repo.","status":"closed","priority":1,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:26:52Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:45:16Z","started_at":"2026-08-22T09:30:20Z","closed_at":"2026-08-22T09:45:16Z","close_reason":"Investigated first: the dedup mechanism (TaskEnvelope.IdempotencyKey + PublishUnique + JetStream Nats-Msg-Id dedup) and the shared dispatch predicate were already fully wired by 7d5u.4 -- no double-dispatch code change was needed. Closed the two real gaps: (1) internal/infrastructure/eventbus/nats/dedup_test.go is a new regression test proving PublishUnique actually suppresses a duplicate publish on the same WorkQueuePolicy stream ARCHIE_TASKS uses in production; (2) internal/forge/webhook/receiver.go gained Status()/GET support tracking authenticated deliveries vs successful publishes separately, so a wedged or non-delivering receiver is observable rather than indistinguishable from an idle repo. AC's poll/webhook parity clause (disabling webhook still polls) is satisfied architecturally with no code change possible to violate it -- verified Cycle()/Run() never reference cfg.Forge.Intake anywhere in internal/daemon; only setupForgeWebhook in cmd/archied/bootstrap.go gates the webhook HTTP server. Adversarial review pass (fresh subagent, -race) found zero blocking issues; closed two minor coverage gaps it flagged.","external_ref":"https://github.com/samcharles93/archie-core/issues/543","dependencies":[{"issue_id":"archie-core-7d5u.5","depends_on_id":"archie-core-7d5u","type":"parent-child","created_at":"2026-08-21T23:26:51Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.5","depends_on_id":"archie-core-7d5u.1","type":"blocks","created_at":"2026-08-21T23:26:58Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.5","depends_on_id":"archie-core-7d5u.4","type":"blocks","created_at":"2026-08-21T23:26:58Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.5","depends_on_id":"archie-core-1786637497328-286-9c985f28","type":"relates-to","created_at":"2026-08-21T23:27:06Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":2,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-7d5u.1","title":"Design: event sources, reaction delivery, and how extensions hook them","description":"No settled design covers this. docs/architecture/plugins-and-extensions.md settles the PLUGIN FRAMEWORK (metadata-only contract, typed capability families, owning registries) but says nothing about event sources, reaction delivery, or intake. Under the scope-discipline rule merged as PR #533, that means design first -- one page, decisive -- then a diff.\n\nDecide and record in docs/prds/:\n\n1. THE HEADLINE QUESTION, which changes everything downstream: do extensions react in-process through typed capability families (what the approved rule mandates), or out-of-process like tau's generic DispatchEvent? If out-of-process generic dispatch is genuinely wanted, docs/architecture/plugins-and-extensions.md is an APPROVED document and must be amended first -- CLAUDE.md's organisation rule is explicit that you change the doc before the rule it states. Do not quietly work around it.\n\n2. Reaction delivery. events.Bus drops on overflow by design and must stay that way for the dashboard. Decide whether reactions get a second in-process bus with at-least-once delivery, or ride the existing NATS eventbus (internal/infrastructure/eventbus), which already has durable semantics and the work-queue traps documented in CLAUDE.md.\n\n3. Whether a reaction may INFLUENCE an outcome (veto, mutate, add work) or only observe. This is the difference between a hook and a subscriber, and it determines whether ordering and failure semantics matter. If a reaction can block, decide what a slow or crashed extension does -- tau's answer was a dispatch timeout, and it still deadlocked unload once.\n\n4. The typed source/intake family: its interface, its owning registry, and how a source declares what it produces.\n\nRespect dependency direction: cmd -\u003e app -\u003e {domain, infrastructure}. A domain declares the contract; app wires implementations. Cross-cutting packages may be imported by any layer and import none.","acceptance_criteria":"A one-page decision in docs/prds/ answering all four questions, naming the packages it will touch, and stating explicitly whether plugins-and-extensions.md needs amending. It decides rather than surveys.","notes":"CORRECTION 2026-08-22 (second): decision question 2 in this bead's description asked whether reactions get a second in-process bus or ride the existing NATS eventbus. That framing is wrong -- there is no 'or'. NATS is the settled transport; the only open axis is external-server vs embedded, not NATS vs something else. Do not design a second bus. See archie-core-7d5u and archie-core-7d5u.2 for the corrected framing and the verified state of github.com/nats-io/nats-server/v2 as an already-present, test-proven, production-unwired dependency.","status":"closed","priority":1,"issue_type":"decision","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:26:26Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:09Z","started_at":"2026-08-21T14:56:21Z","closed_at":"2026-08-21T15:00:55Z","close_reason":"Decided and documented in docs/prds/event-sources-and-reactions.md, satisfying this bead's own acceptance criterion (a one-page decision naming the packages it touches and stating whether plugins-and-extensions.md needs amending -- it does not).\n\nAnswers: (1) in-process typed families, no out-of-process dispatch -- archie has no existing extension-process boundary and neither epic's stated payoff needs one. (2) Reaction delivery reuses internal/eventbus's Publisher/Consumer contract (PublishUnique + Ack/Nak already provide at-least-once with dedup), but NOT the existing nats.Client as-is: it hardcodes Retention: jetstream.WorkQueuePolicy (client.go:65) for the competing-consumer ARCHIE_TASKS stream, which silently starves a second consumer on overlapping subjects per CLAUDE.md's documented trap -- reactions need fan-out, so retention needs to become a Config field and reactions get their own stream. (3) Reactions in these two epics are producer-only (cheap wake via curator's WakeOnPrimaryInput pattern, or produce real work via the existing pollNATS-\u003epublishTask path) -- no veto/mutate-in-flight-work capability is being built here; that belongs to archie-core-h019 if it's ever needed. (4) No single generic Source interface -- forge intake reuses workintake.TaskEnvelope unchanged (its IdempotencyKey is already keyed on owner/repo/number, not delivery source, so a webhook-sourced envelope naturally dedups against a poll-discovered one for free); chat webhook stays internal/channels/webhook, unmodified; what's genuinely new and shared is a small cross-cutting package for HMAC/rate-limit/approval/capture mechanics, owned by neither domain.","external_ref":"https://github.com/samcharles93/archie-core/issues/544","dependencies":[{"issue_id":"archie-core-7d5u.1","depends_on_id":"archie-core-7d5u","type":"parent-child","created_at":"20Line truncated
{"_type":"issue","id":"archie-core-7d5u.2","title":"Reaction delivery via embedded NATS, no external server required","description":"Reactions need at-least-once delivery distinct from the lossy events.Bus (see epic description). The delivery mechanism is NATS -- that part is settled, not a design question. What's open is deployment shape.\n\nVERIFIED 2026-08-22: config.NATSConfig.URL empty currently means 'no NATS, use SQLite ClaimNext' (internal/config/config.go:573-582). That fallback was designed for task distribution working without a broker at all. It was never meant to generalise to 'archie has no NATS' as a permanent architectural stance -- the intended second deployment shape alongside an external NATS server was always NATS EMBEDDED (in-process), not a different eventbus.\n\ngithub.com/nats-io/nats-server/v2 is already a direct dependency and is proven to work in this codebase: internal/infrastructure/agenttransport/nats/transport_test.go and others already start a real embedded server via nats-server/v2/test for tests. It has just never been wired into production startup -- cmd/archied only knows 'external URL configured' vs 'fall back to SQLite'. No new library, no new bus, no architectural gap to design around: this is 'wire the dependency we already have and already test against into the daemon's own startup path.'\n\nDesign questions that remain, now correctly scoped: does an embedded NATS instance replace the SQLite fallback outright for single-process deployments, or run alongside it; what config controls embedded vs external vs disabled; does the daemon own the embedded server's lifecycle the same way it owns everything else (start/stop paired with daemon start/stop, per the shutdown-ordering rules in ARCHITECTURE.md).","acceptance_criteria":"A single-process deployment with no external NATS URL configured still gets reaction delivery via an embedded NATS instance; the daemon starts and stops the embedded server as part of its own lifecycle; no second eventbus type is introduced; internal/infrastructure/eventbus/nats is reused as the transport rather than duplicated.","notes":"2026-08-22, from docs/prds/event-sources-and-reactions.md: do not reuse the existing nats.Client's stream creation as-is. internal/infrastructure/eventbus/nats/client.go:65 hardcodes Retention: jetstream.WorkQueuePolicy -- correct for ARCHIE_TASKS (task distribution, one consumer claims each message) and wrong for reactions, which need fan-out (every registered reaction sees every matching event). Two reactions on overlapping subjects under WorkQueuePolicy hit the exact trap CLAUDE.md documents: the second consumer silently gets nothing. Config.StreamName/Config.Subjects are already composition-supplied (config.go:30-38); Retention is not parameterized and needs to become a Config field, defaulting to today's WorkQueuePolicy so ARCHIE_TASKS is unaffected, with reactions getting a second stream under a fan-out-capable policy and their own subject namespace.","status":"closed","priority":1,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:26:26Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:10Z","started_at":"2026-08-21T18:57:41Z","closed_at":"2026-08-21T19:18:55Z","close_reason":"Embedded NATS wired into production startup: mode=embedded/external/off resolves from url, daemon owns the in-process server lifecycle, Retention parameterized for a future fan-out reaction stream. task check + lint clean; adversarial review zero surviving findings. Committed 2c7db34.","external_ref":"https://github.com/samcharles93/archie-core/issues/545","dependencies":[{"issue_id":"archie-core-7d5u.2","depends_on_id":"archie-core-7d5u.1","type":"blocks","created_at":"2026-08-21T23:26:57Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.2","depends_on_id":"archie-core-7d5u","type":"parent-child","created_at":"2026-08-21T23:26:26Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-7d5u.3","title":"Shared webhook mechanics: HMAC verification, per-source rate limiting, payload redaction","description":"NARROWED 2026-08-22 from this bead's prior scope, which bundled capture-storage and an approval state machine into a single 'shared mechanics' package. On reflection those are not genuinely shared yet:\n\n- Capture STORAGE is t2db.1's job -- it depends on the NATS-vs-SQLite decision 7d5u.2 owns, and building a storage engine here would either duplicate or contradict that. This package may be USED by t2db.1's capture path, but does not own persistence.\n- The draft/pending_approval/armed state machine belongs to the entity that has that state -- t2db.4's binding -- not a generic package. There is exactly one consumer (bindings) today; per 'extract into a shared primitive on the second real consumer, never in anticipation of one,' building it here now would be speculative.\n\nWhat remains, and is genuinely storage-agnostic and reusable by any inbound HTTP source regardless of what consumes it:\n\n1. HMAC-SHA256 verification -- generalize internal/channels/webhook/webhook.go's existing unexported validateHMAC into an exported function here, and have that package call into it instead of keeping its own copy. One implementation of signature checking, not two that can drift.\n2. Per-source rate limiting -- a token bucket keyed by source ID (not global), so one noisy sender cannot exhaust another's budget. Limit-exceeded is an observable rejection, not a silent drop, per t2db.5's decision.\n3. Payload redaction -- generalize internal/gateway/stream.go's unexported sensitiveParameterKey/redactToolParameters heuristic (matches token/secret/password/passwd/api_key/apikey/authorization/credential/cookie/private_key against normalized JSON keys) into a reusable, storage-agnostic transform any capture path can apply before persisting.","acceptance_criteria":"webhookguard.VerifyHMAC replaces internal/channels/webhook's private copy with no behaviour change (its existing tests still pass); a per-source rate limiter rejects over-budget requests observably; a redaction function matches the same key heuristic already proven in gateway/stream.go; the package has zero imports of any domain, infrastructure or app package.","notes":"--notes=2026-08-22: merged to main as f3d3d32 (PR #534, squash of aa23ac9+46401ab+ce4f945).","status":"closed","priority":1,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:26:26Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:11Z","started_at":"2026-08-21T15:19:09Z","closed_at":"2026-08-21T15:26:01Z","close_reason":"Built internal/webhookguard: VerifyHMAC (behavior-preserving extraction from internal/channels/webhook, which now calls it instead of keeping a private copy), RateLimiter (per-source token bucket, independent budgets, observable rejection), RedactPayload (key-name heuristic matching gateway/stream.go's marker list, but -- deliberately unlike that function -- redacts only values under sensitive keys and leaves ordinary strings/numbers/bools untouched, since a captured payload exists to be inspected for field mapping in t2db.3).\n\nAll four acceptance criteria met and independently confirmed by a fresh adversarial review with zero surviving findings: HMAC extraction is byte-for-byte behavior-preserving (diffed against the deleted function); rate limiter hand-traced correct on two scenarios plus race-clean under -race; redaction's divergence from gateway/stream.go verified by reading both functions side by side; zero non-stdlib imports confirmed by grep. The reviewer also mutation-tested both the redaction and rate-limit guards (disabled each in turn) and confirmed the test suite catches both, then reverted and re-verified go test -race and git status were clean.\n\ntask check exit 0, golangci-lint 0 issues (one forcetypeassert finding was caught and fixed properly during build, not suppressed), gofumpt clean. All 10 pre-existing internal/channels/webhook tests pass unmodified; the one now-redundant unit test was removed with a comment pointing to where its coverage relocated.","external_ref":"https://github.com/samcharles93/archie-core/issues/546","dependencies":[{"issue_id":"archie-core-7d5u.3","depends_on_id":"archie-core-1786637497036-275-8838629e","type":"relates-to","created_at":"2026-08-21T23:27:06Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.3","depends_on_id":"archie-core-7d5u","type":"parent-child","created_at":"2026-08-21T23:26:26Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-7d5u.3","depends_on_id":"archie-core-7d5u.1","type":"blocks","created_at":"2026-08-21T23:26:58Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"archie-core-7d5u","title":"[EPIC] Event sources and typed reactions: archie as an extensible platform","description":"Goal: let archie react to things happening -- forge activity, inbound callers, schedules -- and let extensions hook those reactions, without a generic untyped event bus.\n\nTwo findings shape this, both verified in the tree on 2026-08-21.\n\n1. ARCHIE HAS TWO EVENT NEEDS AND ONE BUS. internal/events is documented as 'archied's in-process observability stream' with 'bounded per-subscriber buffers that DROP on overflow, so a stalled dashboard connection can never apply backpressure to the task engine'. Publish is fire-and-forget; subscribers get a read-only channel and cannot influence anything. Correct for the dashboard, wrong for reactions: a hook that silently misses events under load and cannot affect an outcome is worse than no hook, because it looks like it ran. Reactions need at-least-once delivery and the ability to change what happens. Do not bolt reactions onto events.Bus.\n\n2. THE OBVIOUS DESIGN IS PROHIBITED, DELIBERATELY. The tau model this is drawn from dispatches generically: DispatchEvent(ctx, eventName string, sessionID, ...) to out-of-process gRPC plugins. archie's approved foundation (docs/architecture/plugins-and-extensions.md) forbids that shape -- the generic plugin contract stays metadata-only and MUST NOT expose untyped hooks. plugin.Plugin is Name()/Version() only, enforced by internal/plugin/architecture_test.go and by the plugin engine rule in ARCHITECTURE.md. tau already carries the scar the rule avoids: TestUnload_DoesNotWaitForInFlightDispatch exists because a stuck plugin blocked plugin unload.\n\nThe reconciliation, and why this is an epic rather than a feature: separate INBOUND (callers push work in -- forge webhook, chat webhook, cron, channel) from REACTION (extensions respond to something that happened). Inbound becomes a typed source/intake capability family with an owning registry. Reactions become typed capability families owned by the domain whose semantics they affect. 'Webhooks aligned to types of callers' is exactly the typed-family model applied to intake: the instinct is right, the generic bus is what has to go.\n\nNear-term payoff: archie reacts to an issue being labelled the moment it happens, instead of up to poll_interval later.","acceptance_criteria":"A forge webhook triggers work behind a feature flag with polling still available as a backstop; inbound callers register through one typed family rather than bespoke wiring per caller; reactions have delivery guarantees distinct from the lossy observability bus; the generic plugin contract remains metadata-only and internal/plugin/architecture_test.go still passes.","notes":"CORRECTION 2026-08-22 (second): the earlier summary I gave characterized 'NATS is currently optional' as an open architectural gap for this epic to navigate around. That's wrong, and Sam corrected it directly: NATS was never meant to be optional as a technology. What's optional is running a standalone NATS SERVER -- config.NATSConfig.URL empty falls back to the existing SQLite ClaimNext flow (internal/config/config.go:573-582, internal/infrastructure/eventbus/nats/doc.go:24). The intended second deployment shape is NATS EMBEDDED (in-process, no external server to stand up), not 'no NATS, use some other eventbus'. There is no other eventbus in scope, was never meant to be one, and this epic must not introduce one.\n\nCurrent code state, verified: github.com/nats-io/nats-server/v2 (the embeddable NATS server package) is already a direct, non-indirect dependency, and is genuinely used to spin up an in-process server -- but only inside _test.go files (internal/infrastructure/agenttransport/nats/transport_test.go and others use github.com/nats-io/nats-server/v2/test to run tests against a real embedded server). It has never been wired into production as an actual deployment mode. So embedded NATS is proven to work in this codebase's own test harness, and the gap is purely that production startup only knows 'external URL configured' vs 'no NATS, fall back to SQLite' -- there is no production code path that starts nats-server in-process.\n\nThis changes 7d5u.2 (reaction delivery) materially: 'ride the existing NATS eventbus, or a second in-process bus' was the wrong framing. The actual question is whether reaction delivery requires an external NATS deployment or can run on an embedded NATS instance so single-process deployments still get it -- not whether to use NATS at all versus some other bus. If embedded NATS is wired for reaction delivery, capture (t2db.1) gets it too essentially for free, since they were flagged as needing the same underlying decision.","status":"open","priority":1,"issue_type":"epic","owner":"sam@catlow.tech","created_at":"2026-08-21T13:25:57Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:11Z","external_ref":"https://github.com/samcharles93/archie-core/issues/547","dependenLine truncated
{"_type":"issue","id":"archie-core-h019.3","title":"Reviewer runs in a context that provably never saw the implementation","description":"This is the whole feature. A reviewer that shares the implementer's context is a second opinion from the same mind, and it will rationalise the same mistakes -- which is exactly the failure mode CLAUDE.md's 'never self-certified by the implementer' rule exists to prevent.\n\nArchie already has the machinery: each task runs in an isolated container against its own worktree. Launch the reviewer the same way, seeded only with the diff and the originating issue. It must not receive the implementer's conversation, its notes, its reasoning, or its own claims about what it did.\n\nPer 'environmental enforcement over prompt rules', isolation must be a property of construction, not a sentence in the prompt. The agent cannot talk its way out of a context it was never given. Assert it in a test rather than assuming it: a test that would fail if implementer context were threaded through.","acceptance_criteria":"The reviewer is launched with only diff + issue; a test fails if implementer context reaches it; the reviewer's prompt instructs it to assume every line is wrong until proven correct; the mechanism reuses the existing container/worktree isolation rather than inventing a second one.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:54Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:12Z","external_ref":"https://github.com/samcharles93/archie-core/issues/548","dependencies":[{"issue_id":"archie-core-h019.3","depends_on_id":"archie-core-h019.1","type":"blocks","created_at":"2026-08-21T13:03:25Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.3","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:54Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"archie-core-h019.4","title":"Wire review as an implement-workflow stage: after gates, before PR open","description":"Sequence matters. The review runs after gates pass -- there is no point reviewing code that does not build or test -- and before the PR is opened, because the point is to not spend the operator's attention on a PR that is not ready.\n\nA surviving CONFIRMED finding must not open a PR. It parks the task as waiting_human with the findings attached, which is the existing lifecycle state for 'a human needs to look at this' and needs no new state. Per CLAUDE.md, a finding that survives blocks closing the work; it does not get waved through.\n\nScope to the implement workflow first. Routing to tdd/bootstrap/feasibility is a later decision -- feasibility in particular produces a document rather than a diff, so 'review the diff' may not even apply.","acceptance_criteria":"An implement task with a clean gate and no confirmed findings opens a PR as it does today; one with a surviving confirmed finding parks as waiting_human with findings recorded and opens no PR; the stage is observable in the task's running(workflow:stage) status; retry after operator action re-runs the review.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:54Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:13Z","external_ref":"https://github.com/samcharles93/archie-core/issues/549","dependencies":[{"issue_id":"archie-core-h019.4","depends_on_id":"archie-core-h019.1","type":"blocks","created_at":"2026-08-21T13:03:25Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.4","depends_on_id":"archie-core-h019.3","type":"blocks","created_at":"2026-08-21T13:03:26Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.4","depends_on_id":"archie-core-h019.2","type":"blocks","created_at":"2026-08-21T13:03:25Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.4","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:54Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":3,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"archie-core-h019.1","title":"Design: adversarial self-review stage (one-page decision in docs/prds/)","description":"No settled design exists for this in docs/architecture/, so under the scope-discipline rule merged as PR #533 the design comes first -- briefly and decisively -- and is not finished until a diff follows it.\n\nDecide and record, in one page under docs/prds/: where the reviewer runs (reuse the per-task container pool, or a separate short-lived container); what it is given (diff plus originating issue, and explicitly what it must NOT be given); the findings contract and who owns the type; what a surviving finding does to the task lifecycle; and how the reviewer's model/effort is configured.\n\nConstraints from the repo that the design must respect: dependency direction is cmd -\u003e app -\u003e {domain, infrastructure}, so the reviewer contract belongs to the workflow domain and the execution to infrastructure/app. Agent execution is a data boundary. The model never runs git. Prefer environmental enforcement over prompt rules -- 'the reviewer cannot see the implementation' must be a property of how it is launched, not an instruction in its prompt.\n\nDo not write an architecture document, migration plan or review campaign. One page that decides.","acceptance_criteria":"A one-page decision exists under docs/prds/ answering: reviewer isolation mechanism, inputs and forbidden inputs, findings contract owner, lifecycle effect of a surviving finding, and model/effort configuration. It decides rather than surveys, and cites the packages it will touch.","status":"open","priority":1,"issue_type":"decision","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:13Z","external_ref":"https://github.com/samcharles93/archie-core/issues/550","dependencies":[{"issue_id":"archie-core-h019.1","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":5,"comment_count":0}
{"_type":"issue","id":"archie-core-h019.1","title":"Design: adversarial self-review stage (one-page decision in docs/prds/)","description":"No settled design exists for this in docs/architecture/, so under the scope-discipline rule merged as PR #533 the design comes first -- briefly and decisively -- and is not finished until a diff follows it.\n\nDecide and record, in one page under docs/prds/: where the reviewer runs (reuse the per-task container pool, or a separate short-lived container); what it is given (diff plus originating issue, and explicitly what it must NOT be given); the findings contract and who owns the type; what a surviving finding does to the task lifecycle; and how the reviewer's model/effort is configured.\n\nConstraints from the repo that the design must respect: dependency direction is cmd -\u003e app -\u003e {domain, infrastructure}, so the reviewer contract belongs to the workflow domain and the execution to infrastructure/app. Agent execution is a data boundary. The model never runs git. Prefer environmental enforcement over prompt rules -- 'the reviewer cannot see the implementation' must be a property of how it is launched, not an instruction in its prompt.\n\nDo not write an architecture document, migration plan or review campaign. One page that decides.","acceptance_criteria":"A one-page decision exists under docs/prds/ answering: reviewer isolation mechanism, inputs and forbidden inputs, findings contract owner, lifecycle effect of a surviving finding, and model/effort configuration. It decides rather than surveys, and cites the packages it will touch.","status":"closed","priority":1,"issue_type":"decision","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:45:15Z","started_at":"2026-08-22T09:30:19Z","closed_at":"2026-08-22T09:45:15Z","close_reason":"One-page decision written at docs/prds/adversarial-self-review.md: reviewer runs in a second workspace (container or in-process) pointed at a .git-stripped snapshot of the reviewed commit, never the implementer's worktree; findings contract is workflow.ReviewFinding, owned by internal/domain/workflow, reported via a structured capture tool; a surviving error-level finding sets StatusParked (not WaitingHuman -- epic wording named two different states, this resolves which); model/effort reuse the existing Models[role] map and per-stage budget override, gated by a new opt-in Repo.ReviewEnabled so h019.4 doesn't gate every repo on an uncalibrated reviewer before h019.5 lands.","external_ref":"https://github.com/samcharles93/archie-core/issues/550","dependencies":[{"issue_id":"archie-core-h019.1","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":5,"comment_count":0}
{"_type":"issue","id":"archie-core-h019.2","title":"Findings contract: a structured review result the workflow can act on","description":"The reviewer's output must be data the workflow can branch on, not prose a human has to read to find out whether the PR is safe to open. Prose is what makes a review advisory; structure is what lets a finding block.\n\nDefine the finding type in the workflow domain (the domain declares the contract it needs; it never names who implements it). At minimum a finding carries: file and line, a one-sentence defect, a concrete failure scenario (inputs/state -\u003e wrong output), a verdict distinguishing CONFIRMED from PLAUSIBLE, and a disposition once acted on (fixed / skipped / no_change_needed).\n\nThe CONFIRMED/PLAUSIBLE split is load-bearing and must not be collapsed into a single boolean: only confirmed findings may block, or the feature degrades into crying wolf and the operator learns to skim past it.","acceptance_criteria":"A finding type exists in internal/domain/workflow with file, line, summary, failure scenario, verdict and disposition; it is serialisable across the agent-execution data boundary; a review result with zero findings is representable and distinct from a review that did not run.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:14Z","external_ref":"https://github.com/samcharles93/archie-core/issues/551","dependencies":[{"issue_id":"archie-core-h019.2","depends_on_id":"archie-core-h019.1","type":"blocks","created_at":"2026-08-21T13:03:24Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.2","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0}
{"_type":"issue","id":"archie-core-h019","title":"[EPIC] Adversarial self-review before archie opens a PR","description":"Archie enforces gates on every PR it opens, but it does not run the adversarial review CLAUDE.md makes mandatory for every change in this repo: a fresh reviewer that did not write the code, told to assume every line is wrong, working a defined checklist, where a surviving finding blocks the work. It is the one discipline the project requires of its contributors and exempts itself from.\n\nWhy it matters, with evidence rather than assertion: on 2026-08-20/21 the update-verification change (PR #530) passed task check at every checkpoint -- exit 0, golangci-lint 0 issues, gofumpt clean -- and three adversarial passes still found six real defects, two of them blocking. A 'dev' version placeholder would have reported healthy updates as failures, and an 'unknown' install claim was still being worded as confirmed success. The gates were green throughout and would have stayed green. This is CLAUDE.md's own recorded lesson: tests catch behaviour that breaks, not behaviour that quietly starts meaning something else.\n\nArchie is unusually well placed to do this. It already runs each task in an isolated container and worktree, so it can spawn a reviewer that genuinely never saw the implementation -- not a second opinion from the same context pretending to be fresh, which is the failure mode of every single-context tool. The isolation boundary built for safety is what makes honest self-review possible.\n\nWhat the operator gets: a PR that arrives with a findings section -- what was checked, what was found, what was fixed, what survived. A surviving finding parks the task as waiting_human instead of opening a PR that looks ready and is not. Review shifts from hunting for bugs to adjudicating findings.\n\nThe risk that kills the feature is a reviewer that cries wolf. Findings must be adversarially confirmed before they surface, and the reviewer must be able to report zero findings on properties it checked and cleared. Calibration is the feature, not the finding count.","acceptance_criteria":"An implement-workflow task runs an adversarial review after gates pass and before the PR is opened; the reviewer provably cannot see the implementer's context; a surviving confirmed finding blocks PR creation and parks the task; findings and their disposition are visible on the PR and the dashboard.","status":"open","priority":1,"issue_type":"epic","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:17Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:15Z","external_ref":"https://github.com/samcharles93/archie-core/issues/552","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-3jag","title":"Chat/task agent persona is overconfident: presents unverified claims as verified, fabricates infrastructure details","description":"The chat assistant (archie, running via the Telegram/webui gateway) repeatedly\npresents unverified or fabricated information with false confidence, and only\nretracts it when directly challenged. This is a persona/system-prompt problem,\nnot a model quality problem -- reproduced on more than one underlying model.\n\nTwo concrete instances from a live Telegram transcript (2026-08-15):\n\n1. Fabricated infrastructure: asked \"are logs going anywhere else?\", the\n assistant invented an entire second log-shipping subsystem (\"Hermes\n gateway\") with specific fake file paths\n (/home/sam/.hermes/profiles/local-agent/logs/) that do not exist anywhere\n in archie-core, and stated it as verified fact (\"I confirmed the web UI\n token and task lifecycle messages are there\"). When challenged (\"Is there\n something in the codebase which tells you you're Hermes?\"), it admitted\n the paths were invented and carried forward from unrelated, unverified\n context.\n\n2. Overclaimed verification: asked \"Why did that last task fail?\", the\n assistant answered with a bulleted list explicitly labelled \"Verified\n details\" (attempt count, iteration count, token count, exact model name).\n When pressed with \"You checked it?\", it admitted the model name and\n change count were \"overstated\" -- i.e. it labelled unverified specifics as\n verified the first time, and only corrected itself under direct\n challenge.\n\nThe common shape: confident, specific claims presented as fact/verified\nfirst; a walked-back, hedged correction only appears after the user\nexplicitly challenges it. This erodes trust in every claim the assistant\nmakes about system state, since the default posture is confident assertion\nrather than \"I have/haven't actually checked this.\"","notes":"Root cause is believed to be the persona/system-prompt layer (internal/gateway/ BuildSystemPrompt + persona templates), not model selection -- reproduced across models per the reporter. Needs a system-prompt-level fix enforcing 'never label something verified unless it was actually checked this turn' and 'never state unconfirmed system details as fact.' Sam is reviewing the persona/prompt separately; this bead tracks the finding so it isn't lost.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-15T02:25:37Z","created_by":"Sam Catlow","updated_at":"2026-08-20T12:55:18Z","started_at":"2026-08-20T12:50:34Z","closed_at":"2026-08-20T12:55:18Z","close_reason":"Fixed in fix/persona-overconfidence (fba80e3): reworded core rule 1 and added rules 8/9 to the chat agent system prompt enforcing this-turn verification before labelling claims verified/confirmed.","external_ref":"https://github.com/samcharles93/archie-core/issues/524","dependency_count":0,"dependent_count":0,"comment_count":0}
@@ -398,7 +398,7 @@
{"_type":"issue","id":"archie-core-1786637492640-109-820d4da3","title":"Verify: MessageType enum for gateway messages","description":"### Acceptance Criteria\n\nIndependent adversarial review by fresh agent. Verify: no dead code paths, no lint issues (golangci-lint run clean), tests pass (go test ./...), code is formatted (gofmt), no hardcoded values in tests. Review the implementation against the original acceptance criteria. Report findings or confirm clean.","status":"closed","priority":2,"issue_type":"task","created_at":"2026-08-08T14:35:47Z","updated_at":"2026-08-08T14:35:48Z","closed_at":"2026-08-08T14:35:49Z","external_ref":"https://github.com/samcharles93/archie-core/issues/130","source_system":"github:https://github.com/samcharles93/archie-core/issues/130:130","metadata":null,"labels":["task"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-1786637492617-108-33718282","title":"TEST Verify: MessageType enum for gateway messages","description":"### Acceptance Criteria\n\nTEST TASK, DELETE ME","status":"closed","priority":2,"issue_type":"task","created_at":"2026-08-08T14:35:43Z","updated_at":"2026-08-08T14:35:45Z","closed_at":"2026-08-08T14:35:46Z","external_ref":"https://github.com/samcharles93/archie-core/issues/129","source_system":"github:https://github.com/samcharles93/archie-core/issues/129:129","metadata":null,"labels":["task"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-1786637492590-107-92482ef1","title":"Media utilities: cache, validate, deliver images/audio/video/documents","description":"Shared media utilities for handling images/audio/video/documents across platform adapters: download from the platform, validate (MIME type, size ceiling), cache to avoid re-fetching, and hand back a normalized reference the gateway can attach to a `MessageEvent` or deliver outbound.\n\n### Context for implementers\n\nNo media file exists yet in `internal/gateway`. `MessageEvent` (`internal/gateway/messageevent.go`) is the shared event model this should plug into — check its current field set before adding a parallel media type. This is infrastructure other Gateway epic items depend on conceptually (any adapter method that sends/receives media, e.g. `archie-core-1786637493329-136-49b39c78` BasePlatformAdapter optional methods) but there is no formal `bd dep` between them yet.\n\n### Acceptance Criteria\n\nA shared utility validates MIME type and size before accepting media, caches downloaded media keyed by platform + media ID with TTL/size-bounded eviction, and exposes a normalized reference type usable from any adapter. Oversized or disallowed-MIME media is rejected with a caller-visible error, not silently dropped.\n","status":"open","priority":2,"issue_type":"task","created_at":"2026-08-08T14:35:41Z","updated_at":"2026-08-15T12:46:47Z","external_ref":"https://github.com/samcharles93/archie-core/issues/128","source_system":"github:https://github.com/samcharles93/archie-core/issues/128:128","metadata":null,"labels":["task"],"dependencies":[{"issue_id":"archie-core-1786637492590-107-92482ef1","depends_on_id":"archie-core-1786637493329-136-49b39c78","type":"related","created_at":"2026-08-15T12:46:57Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-1786637492559-106-5ea2315d","title":"Code-fence-aware message truncation","description":"Truncate outbound messages to a platform's length limit (e.g. Telegram's 4096 chars) without ever cutting inside a ``` code fence — a truncation mid-fence leaves the rest of the message rendered as broken/unclosed markdown on the client.\n\n### Context for implementers\n\nNo truncation helper exists yet in `internal/gateway`. This is distinct from the Markdown→HTML conversion trap documented in `CLAUDE.md` (\"`EscapeMarkdown` does the *inverse* of formatting\" / `go-telegram/bot`'s `SendRichMessage`) — that trap is about format conversion, this issue is about length-based cropping. Keep the two concerns separate: this helper should decide *where* to cut; rendering is a separate step.\n\n### Acceptance Criteria\n\nA message longer than the platform limit is truncated at a safe boundary: never inside an open code fence, preferring a paragraph/sentence boundary before the limit. If truncation would land inside a fence, the cut point backs up to before the fence opens (or the fence is closed and reopened on continuation, if multi-part delivery is in scope — confirm with the maintainer which behavior is wanted before implementing). Covered by a table-driven test with fenced and unfenced inputs at/over the limit.\n","status":"open","priority":2,"issue_type":"task","created_at":"2026-08-08T14:35:39Z","updated_at":"2026-08-15T12:46:47Z","external_ref":"https://github.com/samcharles93/archie-core/issues/127","source_system":"github:https://github.com/samcharles93/archie-core/issues/127:127","metadata":null,"labels":["task"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-1786637492559-106-5ea2315d","title":"Code-fence-aware message truncation","description":"Truncate outbound messages to a platform's length limit (e.g. Telegram's 4096 chars) without ever cutting inside a ``` code fence — a truncation mid-fence leaves the rest of the message rendered as broken/unclosed markdown on the client.\n\n### Context for implementers\n\nNo truncation helper exists yet in `internal/gateway`. This is distinct from the Markdown→HTML conversion trap documented in `CLAUDE.md` (\"`EscapeMarkdown` does the *inverse* of formatting\" / `go-telegram/bot`'s `SendRichMessage`) — that trap is about format conversion, this issue is about length-based cropping. Keep the two concerns separate: this helper should decide *where* to cut; rendering is a separate step.\n\n### Acceptance Criteria\n\nA message longer than the platform limit is truncated at a safe boundary: never inside an open code fence, preferring a paragraph/sentence boundary before the limit. If truncation would land inside a fence, the cut point backs up to before the fence opens (or the fence is closed and reopened on continuation, if multi-part delivery is in scope — confirm with the maintainer which behavior is wanted before implementing). Covered by a table-driven test with fenced and unfenced inputs at/over the limit.\n","status":"in_progress","priority":2,"issue_type":"task","assignee":"Sam Catlow","created_at":"2026-08-08T14:35:39Z","updated_at":"2026-08-22T09:32:18Z","started_at":"2026-08-22T09:32:18Z","external_ref":"https://github.com/samcharles93/archie-core/issues/127","source_system":"github:https://github.com/samcharles93/archie-core/issues/127:127","metadata":null,"labels":["task"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-1786637492528-105-c2b3caab","title":"TextBatchAggregator for rapid-fire text events","description":"Batch rapid-fire inbound text events from the same sender/channel into a single dispatch instead of invoking the agent loop once per fragment. Chat clients (Telegram in particular) often split a user's thought across several quick messages; dispatching each separately wastes a full agent turn per fragment and can interleave responses oddly.\n\n### Context for implementers\n\nNo aggregator exists yet in `internal/gateway`. This sits upstream of the agent-loop dispatch, after platform adapters normalize events into `MessageEvent` (`internal/gateway/messageevent.go`) but before a turn is started (see `internal/gateway/turn.go`, `turns.go`).\n\n### Acceptance Criteria\n\nText events from the same channel arriving within a short configurable window are coalesced into one aggregated event before dispatch. A single non-batched message is not delayed by more than the window. Aggregation window is configurable, not hardcoded. Covered by a test using an injected clock.\n","status":"open","priority":2,"issue_type":"task","created_at":"2026-08-08T14:35:37Z","updated_at":"2026-08-15T12:46:47Z","external_ref":"https://github.com/samcharles93/archie-core/issues/126","source_system":"github:https://github.com/samcharles93/archie-core/issues/126:126","metadata":null,"labels":["task"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-1786637492498-104-1bde76ae","title":"MessageDeduplicator with TTL-based dedup cache","description":"Prevent duplicate processing of the same platform message. At-least-once delivery (webhook retries, Telegram long-poll re-fetch after a slow ack, forge redelivery) can hand the gateway the same `MessageEvent` more than once; without a dedup layer it gets answered twice.\n\n### Context for implementers\n\n`internal/gateway/messageevent.go` already carries a platform-assigned `ID` field (`MessageEvent.ID`, see line ~59) suitable as a dedup key alongside `ChannelID`/`Platform`. There is currently no dedup file in `internal/gateway` — this is new code, not a rename. Related but distinct: `archie-core-ippu` (gateway: replay and redelivery lose tool activity) covers a different symptom — a replayed *turn* losing its tool-call history in the transcript, not a duplicate inbound message being processed twice. Fix this one first; it may reduce (but won't eliminate) triggers for the other.\n\n### Acceptance Criteria\n\nA `MessageDeduplicator` (or equivalent) keyed on platform + channel + message ID rejects a second delivery of the same message within a configurable TTL window. TTL expiry is covered by a test using an injected clock, not a real sleep. Memory use is bounded (LRU or TTL sweep), not an unbounded map.\n","status":"open","priority":2,"issue_type":"task","created_at":"2026-08-08T14:35:35Z","updated_at":"2026-08-15T12:46:47Z","external_ref":"https://github.com/samcharles93/archie-core/issues/125","source_system":"github:https://github.com/samcharles93/archie-core/issues/125:125","metadata":null,"labels":["task"],"dependencies":[{"issue_id":"archie-core-1786637492498-104-1bde76ae","depends_on_id":"archie-core-ippu","type":"related","created_at":"2026-08-15T12:46:57Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"archie-core-1786637492468-103-72e44236","title":"SessionStore interface and implementation","status":"closed","priority":2,"issue_type":"task","created_at":"2026-08-08T14:35:31Z","updated_at":"2026-08-08T14:35:33Z","closed_at":"2026-08-08T14:35:34Z","external_ref":"https://github.com/samcharles93/archie-core/issues/124","source_system":"github:https://github.com/samcharles93/archie-core/issues/124:124","metadata":null,"labels":["task"],"dependency_count":0,"dependent_count":0,"comment_count":0}
+165
View File
@@ -0,0 +1,165 @@
# Adversarial self-review stage -- decision
**Status:** Decided, not yet implemented
**Date:** 2026-08-22
**Beads issue:** `archie-core-h019.1`, blocks `h019.2/.3/.4/.5`
Answers the five questions in `h019.1`'s description against the actual
implement workflow, not designed from scratch: the gate pattern, the
container pool, and the workflow engine's `Outcome` mechanism already answer
most of this.
## 1. Where it runs: a second workspace, not a second worktree
The implement workflow's stage list
(`internal/domain/workflow/implement.go:110-185`) already has the right slot:
after `StageDiffCap()` (line 178), before `StageOpenPR()` (line 179) -- gates
have passed, the diff is final, and nothing has told the forge about this
change yet.
**The isolating property is the workspace path, not container-vs-in-process.**
Both execution modes already scope an agent's tools to a directory --
`agentexec.Runner.Run(ctx, workspace string, ...)`
(`internal/agentexec/inprocess.go:20-26`) roots tool access there in-process,
and container mode mounts only what `Pool.Acquire`
(`internal/container/pool.go:161-177`) is given. So "the reviewer cannot see
the implementation" becomes an environmental fact by construction, in both
deployment shapes, as long as the reviewer's workspace is never
`tc.WorktreeDir`.
**What the workspace is:** before running the reviewer, the app-layer code
that owns this stage exports a `.git`-free snapshot of the reviewed commit
(`git archive <commit> | tar -x`) into a fresh scratch directory next to, not
inside, the task's real worktree, then writes two files alongside it:
`diff.patch` (the unified diff already computed for `StageYaegiGate`'s
`Trees.Diff()`) and `issue.md` (the originating issue's title and body, read
from `store.Task`, nothing else). The reviewer's `agentexec.Request.Workspace`
points at this directory. Stripping `.git` is deliberate: it is what makes
commit messages, branch name, and reflog -- the actual channel a multi-commit
implementer's back-and-forth reasoning would leak through -- structurally
unreachable rather than merely undisclosed. Container mode acquires a second,
distinct container for this (`Pool.Acquire` again, different mounts); in-process
mode builds a second `agentexec.Request` with this workspace. Neither needs a
new execution mechanism.
## 2. What it's given, and what it must never be given
**Given:** the clean file-content snapshot (so the reviewer can read whole
files -- follow an import, check interface satisfaction, see a call site --
not just diff hunks, which the manual-read checklist in CLAUDE.md requires),
`diff.patch`, and `issue.md`.
**Never mounted or passed, by construction:** the implementer's
`agentexec.Result` (summary, captures, stop reason), any prior turn's tool-call
transcript, commit messages/git history/branch name (excluded by stripping
`.git`, not by instruction), daemon internal state, and any other task's data.
This is the environmental-enforcement principle applied literally: the
reviewer is never told "ignore the implementer's reasoning," because there is
no code path that hands it any.
## 3. Findings contract: `workflow.ReviewFinding`, workflow domain owns it
Not a reuse of `internal/gate.Finding` (`gate.go:22-31`), despite the similar
shape. That type belongs to the Yaegi gate contract (`.archie/gate.go`,
repo-owner-authored checks); coupling the review stage's blocking contract to
gate's evolution would violate "a domain declares the interface it needs, it
never names who implements it." `h019.2` implements, in
`internal/domain/workflow`:
```go
type ReviewFinding struct {
Level string // "error" (blocks) | "warn" (advisory)
Category string // dead-code | unchecked-error | hardcoded-value |
// interface-satisfaction | nil-risk | goroutine-leak |
// race | other -- CLAUDE.md's own checklist categories
File string
Line int
Message string
}
func ReviewBlocking(findings []ReviewFinding) bool // mirrors gate.Blocking
```
The domain owns the type and the pure blocking decision; turning the
reviewer agent's raw output into `[]ReviewFinding` is app/infrastructure code
(`h019.3`/`h019.4`), not domain code -- same direction as gate's Yaegi
evaluator producing `[]gate.Finding` for the domain to consume.
**Findings are reported through a structured capture tool**, the same
`agentexec.CaptureTool` pattern `feasibility.go`'s `decideCaptureTools`
already uses, not free text the caller regexes out of a final message. This
is the findings contract's own application of environmental enforcement:
the reviewer cannot end its turn without calling a typed tool, rather than
being asked to format its answer as JSON and trusted to comply.
## 4. Lifecycle effect of a surviving finding: `StatusParked`, not `WaitingHuman`
The epic's acceptance criteria say a surviving finding "parks the task as
`waiting_human`," which names two different states in the actual machine
(`internal/taskstate/taskstate.go:44-59`). This decides between them:
**`store.StatusParked`**, following `StageDiffCap`'s existing precedent
(`steps.go:91-94`) exactly -- a stage-computed policy stop with a `Detail`
explaining why, before PR open. `Parked`'s available actions, retry and
abandon, are the right shape here: fix the flagged code and retry, or
abandon/override if a finding is a false positive. `WaitingHuman`'s
approve/reject actions have no obvious referent -- there is no PR yet to
approve or reject.
A `Level == "error"` finding that survives sets
`tc.Outcome = Outcome{Status: store.StatusParked, Detail: <rendered findings>}`
and the stage returns `nil`; `StageOpenPR` never runs. `Level == "warn"`
findings, and the zero-findings case, leave `tc.Outcome` unset and the stage
returns `nil`, so `StageOpenPR` runs next. Attaching warn-level findings to
the task record for PR-body/dashboard display is `h019.6`'s job, not decided
here.
## 5. Model, effort, and whether a repo even runs this yet
**Model:** a new `"reviewer"` key in the existing `Config.Models` map (already
a generic `map[string]string`, no schema change) with the same single-level
fallback precedent `agent.go:68-73` already uses for planner->builder:
`Models["reviewer"]`, falling back to `Models["builder"]` if unset. Operators
who want a genuinely independent second opinion configure a distinct model;
nothing forces one, because CLAUDE.md's project scope is "smallest change
that works," not a mandated model-diversity policy.
**Effort/budget:** reuse the per-stage budget override `AgentStage` already
supports (`agent.go:35`), not a new config surface -- a repo wanting a
cheaper or more thorough reviewer sets `MaxSteps`/`MaxTokens` the same way
the plan stage already can.
**Opt-in, not automatic:** `h019.5` (calibration -- confirm findings
adversarially before they surface) is a separate, lower-priority bead than
`h019.4` (wiring). Shipping the stage unconditionally the moment `h019.4`
lands would gate every existing repo's PR flow on an uncalibrated reviewer
before calibration exists, which is exactly the "reviewer that cries wolf"
risk the epic names as the thing that kills this feature. `h019.4` adds
`Repo.ReviewEnabled bool` (default `false`) to `internal/config`; an operator
opts a repo in once they trust it, the same shape as any other per-repo gate
setting.
## Packages this touches
- `internal/domain/workflow` (`h019.2`, `h019.4`): `ReviewFinding`,
`ReviewBlocking`, and a `ReviewStage` inserted between `StageDiffCap` and
`StageOpenPR` in `implement.go`. Contract only; no execution mechanism.
- `internal/worktree` or a sibling package (`h019.3`): the clean-snapshot
export (`git archive`, no `.git`) building the reviewer's isolated
workspace.
- `internal/agentexec` / `internal/container` (`h019.3`): no new mechanism --
a second `Request`/`Pool.Acquire` scoped to the snapshot directory reuses
the existing `Runner` interface.
- `internal/config` (`h019.4`): `Models["reviewer"]` (no type change) and
`Repo.ReviewEnabled bool`.
- `internal/store`/`internal/taskstate`: unchanged; reuses `StatusParked`.
- Surfacing findings on the PR body and dashboard is `h019.6`, out of scope
here.
## What this does not decide
The exact review checklist prompt content, how many findings can survive
before `MaxSteps` is exhausted, and the adversarial-confirmation mechanism
that keeps a finding from surfacing on a single reviewer's say-so are
`h019.5`. The PR-body/dashboard rendering of findings is `h019.6`. This
document decides isolation, inputs, the findings contract's owner and shape,
the lifecycle effect, and model/effort/opt-in configuration only.
+94 -7
View File
@@ -12,13 +12,33 @@
// GitHub's webhook must be configured with "Content type: application/json"
// (the UI default). The alternate application/x-www-form-urlencoded delivery
// is not decoded here and is rejected as a bad payload.
//
// Idempotency is not this package's job: it decodes into the same
// TaskEnvelope and calls the same publish path the poller uses, so
// PublishUnique's dedup (keyed on TaskEnvelope.IdempotencyKey) covers both
// sources for free -- see archie-core-7d5u.5. What this package does own is
// making a wedged or non-delivering receiver observable: a GET to the
// listen address (GitHub only ever POSTs) returns Status, tracking
// authenticated deliveries and successful publishes separately so an
// operator can tell "receiver is up but nothing is arriving" apart from
// "repo has no issue activity."
//
// The GET/Status route is deliberately unauthenticated, on the same
// internet-facing listener GitHub POSTs signed payloads to: it reveals
// counters and timestamps only, never repo data, issue content, or the
// webhook secret, so treating it as a public liveness probe (the same
// tradeoff an unauthenticated /healthz makes) is an acceptable exchange for
// not requiring a second credential just to check if the receiver is alive.
package webhook
import (
"context"
"encoding/json"
"io"
"log/slog"
"net/http"
"sync"
"time"
"github.com/google/go-github/v78/github"
@@ -45,6 +65,12 @@ type Receiver struct {
botUser string
publish PublishFunc
log *slog.Logger
mu sync.Mutex
startedAt time.Time
lastReceivedAt time.Time
deliveries uint64
publishes uint64
}
// New returns an unstarted Receiver.
@@ -53,17 +79,73 @@ func New(secret, trigger, label, botUser string, publish PublishFunc, log *slog.
log = slog.New(slog.DiscardHandler)
}
return &Receiver{
secret: secret,
trigger: trigger,
label: label,
botUser: botUser,
publish: publish,
log: log.With("component", "forge-webhook"),
secret: secret,
trigger: trigger,
label: label,
botUser: botUser,
publish: publish,
log: log.With("component", "forge-webhook"),
startedAt: time.Now(),
}
}
// ServeHTTP handles one forge webhook delivery.
// Status reports the receiver's delivery activity so an operator can tell a
// wedged or non-delivering receiver apart from a repo with no issue
// activity (archie-core-7d5u.5): Deliveries counts every authenticated,
// parseable webhook -- proof GitHub reached this process at all -- while
// Publishes counts only what the dispatch predicate turned into work.
// LastReceivedAt is nil until the first authenticated delivery arrives.
type Status struct {
StartedAt time.Time `json:"started_at"`
LastReceivedAt *time.Time `json:"last_received_at"`
Deliveries uint64 `json:"deliveries"`
Publishes uint64 `json:"publishes"`
}
// Status returns a snapshot of the receiver's activity counters.
func (r *Receiver) Status() Status {
r.mu.Lock()
defer r.mu.Unlock()
s := Status{
StartedAt: r.startedAt,
Deliveries: r.deliveries,
Publishes: r.publishes,
}
if !r.lastReceivedAt.IsZero() {
last := r.lastReceivedAt
s.LastReceivedAt = &last
}
return s
}
// recordDelivery marks one authenticated, parseable webhook as received.
// Called only after HMAC verification and payload parsing succeed, so an
// unauthenticated request (anyone spamming the endpoint with a bad
// signature) can never inflate the "receiver is alive" signal.
func (r *Receiver) recordDelivery() {
r.mu.Lock()
defer r.mu.Unlock()
r.deliveries++
r.lastReceivedAt = time.Now()
}
// recordPublish marks one delivery as having produced a task.
func (r *Receiver) recordPublish() {
r.mu.Lock()
defer r.mu.Unlock()
r.publishes++
}
// ServeHTTP handles one forge webhook delivery. GitHub only ever POSTs here,
// so any GET is answered with the receiver's Status instead -- an operator
// (or a monitor) can curl the same address the webhook is configured against
// to check liveness, with no separate health-check surface to wire up.
func (r *Receiver) ServeHTTP(w http.ResponseWriter, req *http.Request) {
if req.Method == http.MethodGet {
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(r.Status())
return
}
if req.Method != http.MethodPost {
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
@@ -91,6 +173,9 @@ func (r *Receiver) ServeHTTP(w http.ResponseWriter, req *http.Request) {
http.Error(w, "bad payload", http.StatusBadRequest)
return
}
// Authenticated and parseable: this is a genuine GitHub delivery,
// independent of whether it turns out to be dispatch-eligible.
r.recordDelivery()
issueEvent, ok := event.(*github.IssuesEvent)
if !ok {
@@ -113,6 +198,8 @@ func (r *Receiver) ServeHTTP(w http.ResponseWriter, req *http.Request) {
http.Error(w, "publish failed", http.StatusInternalServerError)
return
}
r.recordPublish()
r.log.Info("published", "task", task.Ref())
w.WriteHeader(http.StatusAccepted)
}
+139
View File
@@ -6,6 +6,7 @@ import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"slices"
@@ -247,3 +248,141 @@ func TestPublishedEnvelopeCarriesIssueIdentityAndKind(t *testing.T) {
t.Errorf("idempotency key = %q, want archie:owner/repo/42", task.IdempotencyKey())
}
}
// TestReceiverStatusTracksAuthenticatedDeliveries pins the observability this
// receiver needs for archie-core-7d5u.5: a wedged or non-delivering receiver
// must not be indistinguishable from an idle repo. Deliveries counts every
// authenticated, parseable delivery (proof GitHub is reaching the process at
// all) independent of whether that delivery matched the dispatch predicate;
// Publishes counts only what actually became work. An unauthenticated
// request must not inflate either counter -- that would let anyone spamming
// the endpoint with a bad signature manufacture a false "receiver is alive
// and busy" signal.
func TestReceiverStatusTracksAuthenticatedDeliveries(t *testing.T) {
r := New(testSecret, "label", testLabel, testBot, nil, nil)
before := r.Status()
if before.Deliveries != 0 || before.Publishes != 0 {
t.Fatalf("initial status = %+v, want zero counters", before)
}
if before.LastReceivedAt != nil {
t.Fatalf("initial LastReceivedAt = %v, want nil", before.LastReceivedAt)
}
// Invalid signature: not a real GitHub delivery, must not count.
rejected := issueEvent(t, "labeled", func(ev *github.IssuesEvent) {
ev.Issue.Labels = []*github.Label{{Name: new(testLabel)}}
})
rejectedPayload, err := json.Marshal(rejected)
if err != nil {
t.Fatalf("marshal event: %v", err)
}
serve(t, r, signedRequest(t, "wrong-secret", "issues", rejectedPayload))
afterRejected := r.Status()
if afterRejected.Deliveries != 0 {
t.Fatalf("Deliveries after unauthenticated request = %d, want 0", afterRejected.Deliveries)
}
// Authenticated but dispatch-ineligible: counts as a delivery (GitHub
// really did reach the receiver), not as a publish.
ineligible := issueEvent(t, "unlabeled", func(ev *github.IssuesEvent) {
ev.Issue.Labels = []*github.Label{{Name: new(testLabel)}}
})
ineligiblePayload, err := json.Marshal(ineligible)
if err != nil {
t.Fatalf("marshal event: %v", err)
}
serve(t, r, signedRequest(t, testSecret, "issues", ineligiblePayload))
afterIneligible := r.Status()
if afterIneligible.Deliveries != 1 {
t.Fatalf("Deliveries after ineligible authenticated request = %d, want 1", afterIneligible.Deliveries)
}
if afterIneligible.Publishes != 0 {
t.Fatalf("Publishes after ineligible authenticated request = %d, want 0", afterIneligible.Publishes)
}
if afterIneligible.LastReceivedAt == nil {
t.Fatal("LastReceivedAt after authenticated request = nil, want set")
}
// Authenticated and dispatch-eligible: counts as both.
matched := issueEvent(t, "labeled", func(ev *github.IssuesEvent) {
ev.Issue.Labels = []*github.Label{{Name: new(testLabel)}}
})
matchedPayload, err := json.Marshal(matched)
if err != nil {
t.Fatalf("marshal event: %v", err)
}
serve(t, r, signedRequest(t, testSecret, "issues", matchedPayload))
final := r.Status()
if final.Deliveries != 2 {
t.Fatalf("Deliveries = %d, want 2", final.Deliveries)
}
if final.Publishes != 1 {
t.Fatalf("Publishes = %d, want 1", final.Publishes)
}
}
// TestReceiverStatusDoesNotCountFailedPublish pins that a Publisher error
// still counts the delivery (GitHub really did reach the receiver, and the
// payload really was dispatch-eligible) but must not count as a publish --
// otherwise Status would report work that never actually reached the queue.
func TestReceiverStatusDoesNotCountFailedPublish(t *testing.T) {
r := New(testSecret, "label", testLabel, testBot, nil, nil)
r.publish = func(context.Context, workintake.TaskEnvelope) error {
return errors.New("publish unavailable")
}
ev := issueEvent(t, "labeled", func(ev *github.IssuesEvent) {
ev.Issue.Labels = []*github.Label{{Name: new(testLabel)}}
})
payload, err := json.Marshal(ev)
if err != nil {
t.Fatalf("marshal event: %v", err)
}
rec := httptest.NewRecorder()
r.ServeHTTP(rec, signedRequest(t, testSecret, "issues", payload))
if rec.Code != http.StatusInternalServerError {
t.Fatalf("status = %d, want 500", rec.Code)
}
got := r.Status()
if got.Deliveries != 1 {
t.Errorf("Deliveries = %d, want 1", got.Deliveries)
}
if got.Publishes != 0 {
t.Errorf("Publishes = %d, want 0 (publish failed)", got.Publishes)
}
}
// TestReceiverServesStatusOnGET proves the counters are reachable without a
// separate health-check subsystem: any GET is answered with the JSON status
// body, since GitHub only ever POSTs to this receiver.
func TestReceiverServesStatusOnGET(t *testing.T) {
r := New(testSecret, "label", testLabel, testBot, nil, nil)
ev := issueEvent(t, "labeled", func(ev *github.IssuesEvent) {
ev.Issue.Labels = []*github.Label{{Name: new(testLabel)}}
})
payload, err := json.Marshal(ev)
if err != nil {
t.Fatalf("marshal event: %v", err)
}
serve(t, r, signedRequest(t, testSecret, "issues", payload))
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
rec := httptest.NewRecorder()
r.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("GET status = %d, want 200", rec.Code)
}
var got Status
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatalf("decode status body: %v (%s)", err, rec.Body.String())
}
if got.Deliveries != 1 || got.Publishes != 1 {
t.Errorf("decoded status = %+v, want Deliveries=1 Publishes=1", got)
}
if got.LastReceivedAt == nil {
t.Error("decoded status LastReceivedAt = nil, want set")
}
}
@@ -0,0 +1,66 @@
package nats
import (
"errors"
"testing"
"github.com/nats-io/nats-server/v2/server"
natstest "github.com/nats-io/nats-server/v2/test"
"github.com/samcharles93/archie-core/internal/eventbus"
)
// TestPublishUniqueSuppressesDuplicateKeyOnWorkQueueStream is the regression
// proof behind archie-core-7d5u.5 (idempotency and poll/webhook parity): a
// forge issue discovered independently by the poller and the webhook
// receiver produces two TaskEnvelopes with the identical
// workintake.TaskEnvelope.IdempotencyKey(), published to the SAME
// WorkQueuePolicy stream ARCHIE_TASKS uses in production (Config.Retention
// left unset, per TestWithDefaultsRetention). A second PublishUnique call
// with the same key inside the dedup window must be silently suppressed by
// the server, so a single consumer only ever observes one message -- never
// two tasks for one issue.
func TestPublishUniqueSuppressesDuplicateKeyOnWorkQueueStream(t *testing.T) {
srv := natstest.RunServer(&server.Options{Port: -1, JetStream: true, StoreDir: t.TempDir()})
t.Cleanup(srv.Shutdown)
c, err := Connect(t.Context(), Config{
URL: srv.ClientURL(),
StreamName: "TASKS_DEDUP_TEST",
Subjects: []string{"tasks.dedup.test.>"},
FilterSubject: "tasks.dedup.test.>",
// Retention left unset: defaults to WorkQueuePolicy, the same policy
// ARCHIE_TASKS uses in production for task distribution.
}, discardLogger())
if err != nil {
t.Fatalf("Connect = %v", err)
}
t.Cleanup(c.Close)
const key = "archie:owner/repo/42"
// Poll-discovered publish, then webhook-discovered publish of the same
// issue moments later -- both carry the identical idempotency key.
if err := c.PublishUnique(t.Context(), "tasks.dedup.test.issue", key, []byte("poll")); err != nil {
t.Fatalf("PublishUnique (poll) = %v", err)
}
if err := c.PublishUnique(t.Context(), "tasks.dedup.test.issue", key, []byte("webhook")); err != nil {
t.Fatalf("PublishUnique (webhook) = %v", err)
}
msg, err := c.Fetch(t.Context())
if err != nil {
t.Fatalf("Fetch (first) = %v", err)
}
if err := msg.Ack(); err != nil {
t.Fatalf("Ack = %v", err)
}
// A second delivery for the same issue would be a double-dispatched
// task. There must be none queued.
if _, err := c.Fetch(t.Context()); err == nil {
t.Fatal("Fetch (second) succeeded, want eventbus.ErrNoMessage -- duplicate was not suppressed")
} else if !errors.Is(err, eventbus.ErrNoMessage) {
t.Fatalf("Fetch (second) = %v, want eventbus.ErrNoMessage", err)
}
}