From e561264692483733a7bdc3cbb0dfcc4feee705fa Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Thu, 3 Sep 2026 04:21:17 +1000 Subject: [PATCH] chore(beads): add binding_dispatches ledger precedent to t2db.17 (idempotency) --- .beads/issues.jsonl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index 18ffc9c34..fc680f919 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -14,7 +14,7 @@ {"_type":"issue","id":"archie-core-1ysw","title":"Repair internal/worktree correctness and trust-boundary defects","description":"Track the concrete worktree-package defects proven by the 2026-08-22 evidence-gated review. Child bugs cover RPC authority/path containment, repository-key isolation, retry/base/diff correctness, push lifecycle, terminal cleanup, cancellation, and branch naming. This epic groups implementation only; each child has its own observable acceptance criteria.","acceptance_criteria":"All child bugs are closed with red-green regression tests, scoped lint passes, and the repository quality gate is clean.","status":"closed","priority":0,"issue_type":"epic","owner":"sam@catlow.tech","created_at":"2026-08-22T11:01:18Z","created_by":"Sam Catlow","updated_at":"2026-08-24T08:16:00Z","closed_at":"2026-08-24T08:16:00Z","close_reason":"All 9 children complete. The P0/P1 trust-boundary and correctness defects (.1 RPC task binding, .2 colliding directory keys, .3 untracked cleanup, .4 fail-closed base resolution, .5 three-dot merge-base semantics) were already fixed on main and reconciled 2026-08-24. The remaining four (.6 push result, .7 no-change worktree leak, .8 commit cancellation, .9 branch naming contract) were implemented this session, merged as 474c31d and its two merge commits, task check green, with an independent adversarial review reporting zero findings.","external_ref":"https://github.com/samcharles93/archie-core/issues/590","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"archie-core-t2db.5","title":"Security: an unauthenticated endpoint must not be able to run an agent","description":"P0 within this epic because the feature's own pitch is the threat model: a public intake surface whose output is an agent with tool access and a repo. Get this wrong and a webhook URL is remote code execution with extra steps.\n\nNon-negotiables to design:\n\n- Per-source authentication. A source registers with a secret (HMAC, as internal/channels/webhook already does per route) before its events can trigger anything. Capture-only for unknown senders is acceptable; triggering is not.\n- Human approval before a binding goes live. An operator authoring a binding is a deliberate act; an event arriving is not. Nothing should self-arm.\n- Rate and volume limits per source, so capture cannot be used to exhaust disk or drown the inspector.\n- Payload content is untrusted input flowing into an agent prompt. Treat prompt injection as expected, not exceptional -- the sender controls that text, and the agent has tools. Bound what a playbook-triggered workflow may do.\n- Captured payloads may contain secrets from the sending system. Decide retention, redaction and who can view them; internal/memory/scrubber may already have applicable prior art.\n\nArchie's own principle applies: environmental enforcement over prompt rules. 'The agent should ignore instructions in the payload' is not a control.","acceptance_criteria":"An event from an unauthenticated source can be captured but cannot trigger a workflow; a new binding requires explicit human approval before it arms; per-source rate limits exist; the blast radius of a playbook-triggered workflow is bounded by code rather than by prompt text; retention and visibility of captured payloads is decided and documented.","notes":"2026-08-22: security design decided and documented in docs/prds/webhook-intake-security.md, covering all five non-negotiables from this bead's description:\n1. Per-source auth reuses internal/channels/webhook's existing HMAC-SHA256 scheme; capture accepts unsigned events (visible, marked unauthenticated), only authenticated events can ever match a binding.\n2. Bindings gain a draft -\u003e pending_approval -\u003e armed state machine; any edit to an armed binding drops it back to pending_approval so approval can't be bypassed by editing. Modeled on telegram's existing dangerousAction/pendingApproval flow (dangerous.go/approval.go).\n3. Per-source token-bucket rate limiting (precedent: pairing/store.go's MaxPendingPerPlatform); limit-exceeded is an observable 429, not a silent drop.\n4. Blast radius bounded by the SAME code-level controls every task already gets -- gate, diff_cap_lines, sandboxed container, worktree isolation -- no new relaxation for playbook-triggered work, and payload text is ordinary untrusted prompt input, never a trusted instruction.\n5. 7-day configurable retention on captured payloads; best-effort redaction reusing gateway/stream.go's sensitiveParameterKey heuristic before persisting; visibility gated by the same dashboard auth as every other webui surface.\n\nThis bead's code-level acceptance criteria (capture-can't-trigger, rate limits exist, blast-radius bounded, approval gate) cannot be implemented yet: they constrain t2db.1 (capture), 7d5u.3 (intake family) and t2db.4 (binding), none of which have landed. This document is the contract those beads build against -- do not build any of them without satisfying it. Leaving this bead in_progress rather than closing it; it closes when t2db.1/7d5u.3/t2db.4 land and demonstrably satisfy points 1-5, not before.","status":"closed","priority":0,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:45:46Z","created_by":"Sam Catlow","updated_at":"2026-09-02T13:09:39Z","started_at":"2026-08-21T14:51:26Z","closed_at":"2026-09-02T13:09:39Z","close_reason":"Closed","external_ref":"https://github.com/samcharles93/archie-core/issues/536","dependencies":[{"issue_id":"archie-core-t2db.5","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-08-21T23:45:45Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"archie-core-t2db.18","title":"eda: resolve mid-playbook action failure semantics (execution-time gap 1)","description":"Design-investigation ticket, same shape as the idempotency one above -- produces a design-doc resolution, not implementation. Gates multi-action playbooks (t2db.15 is deliberately single-action to avoid needing this). docs/prds/eda-playbook-engine.md's 'Execution-time gap 1' has no default yet and explicitly says a default must be a stated decision, not something nobody chose.\n\nGrounded by a crew investigation pass (2026-09-03, not committed -- content folded in here): read event-sources-and-reactions.md's producer-only/no-veto/no-mutation language, existing internal/domain/workflow stage-failure conventions (does a failing stage stop/wrap/resume?), and the load-collision report shape (drop+log+report to caller) from t2db.9-.12 as the precedent for how a run outcome should be reported.","design":"Decide and write the resolution into eda-playbook-engine.md:\n1. On action N of M failing: stop-on-first-failure (matches this doc's existing anti-clever-recovery bias) vs continue-to-independent-later-actions vs rollback. State which, and why the alternatives were rejected -- don't just restate the doc's existing hedge.\n2. What 'rollback' means (likely: nothing, since a posted forge comment isn't revocable) -- state this explicitly rather than leaving it implied.\n3. Producer-only interaction: decide whether a playbook reacting to its own action's failure by running compensating actions is in scope (arguably still 'producing new work' per event-sources-and-reactions.md, not veto/mutation of in-flight work) -- state the boundary explicitly against that doc's rule.\n4. Condition-failure vs action-failure: J3 (CEL condition eval error -\u003e false, skip, log, continue) is already decided for conditions specifically. State cleanly how this differs from an actual action failure, and whether J3 needs revisiting once this gap has a per-action-control answer.","acceptance_criteria":"eda-playbook-engine.md's Execution-time gap 1 section is marked resolved with a stated default (not a list of candidates) and explicit reasoning against event-sources-and-reactions.md's producer-only rule. No code changes required for this ticket to close.","status":"open","priority":1,"issue_type":"task","owner":"sam@catlow.tech","created_at":"2026-09-02T18:12:15Z","created_by":"Sam Catlow","updated_at":"2026-09-02T18:12:15Z","dependencies":[{"issue_id":"archie-core-t2db.18","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-09-02T18:12:15Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"archie-core-t2db.17","title":"eda: resolve idempotency keying for Module/Channel/Forge actions (execution-time gap 2)","description":"Design-investigation ticket (like the CEL question, archie-core-t2db resolution in eda-playbook-engine.md) -- produces a design-doc resolution, NOT implementation. This is the sharper of the two unresolved execution-time gaps named in docs/prds/eda-playbook-engine.md's 'Execution-time gaps' section: workflow-kind actions inherit idempotency for free via TaskEnvelope; channel/forge/module actions with real side effects have no keying scheme, so a redelivered trigger event (internal/eventbus is at-least-once PublishUnique) would double-fire a comment, a message, or a billed image generation.\n\nGrounded by a crew investigation pass (2026-09-03, further-investigation.md, not committed -- content folded into this ticket): read internal/eventbus's PublishUnique key computation and existing dedup surface, TaskEnvelope's idempotency key minting/consumption path, event-sources-and-reactions.md's decided contract (extend it, don't contradict it), and the CEL context surface in internal/domain/eda/expr (event/actions fields available for key derivation).","design":"Investigate and decide, then write the resolution into eda-playbook-engine.md (mark gap 2 resolved, same shape as the CEL open-question-1 resolution):\n1. Granularity: one idempotency key per trigger-event+playbook (whole run dedups) vs per-action key derived from the run key. The doc's stated bias is the former unless a specific action needs finer dedup -- turn that bias into a stated decision.\n2. Where the key lives: does a playbook run get its own envelope/context carrying the key, and how does it reach the target position's invocation (Module.Invoke, a future Channel/Forge call)?\n3. What 'dedup' does on redelivery: is the whole run skipped, or just the already-fired action? What gets recorded (a run ledger -- reuse the existing event/capture store from t2db.1-.4 rather than inventing a parallel one; investigate whether it already fits this need before designing a new one).\n4. Key derivation: a CEL-reachable expression over event fields, or a stable hash of event-identity+playbook-id -- pick one and justify it against the expr package's actual context surface.\n\nThis is investigation and a design-doc resolution only. Do not implement Channel/Forge/side-effecting Module kinds against this decision in the same ticket -- those are follow-up tickets once this lands.","acceptance_criteria":"eda-playbook-engine.md's Execution-time gap 2 section is marked resolved with a stated decision (not a survey of options) covering granularity, key storage/lookup, redelivery behavior, and key derivation. Consistent with event-sources-and-reactions.md's existing idempotency contract -- explicitly states how it extends rather than contradicts it. No code changes required for this ticket to close.","status":"open","priority":1,"issue_type":"task","owner":"sam@catlow.tech","created_at":"2026-09-02T18:12:13Z","created_by":"Sam Catlow","updated_at":"2026-09-02T18:12:13Z","dependencies":[{"issue_id":"archie-core-t2db.17","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-09-02T18:12:13Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} +{"_type":"issue","id":"archie-core-t2db.17","title":"eda: resolve idempotency keying for Module/Channel/Forge actions (execution-time gap 2)","description":"Design-investigation ticket (like the CEL question, archie-core-t2db resolution in eda-playbook-engine.md) -- produces a design-doc resolution, NOT implementation. This is the sharper of the two unresolved execution-time gaps named in docs/prds/eda-playbook-engine.md's 'Execution-time gaps' section: workflow-kind actions inherit idempotency for free via TaskEnvelope; channel/forge/module actions with real side effects have no keying scheme, so a redelivered trigger event (internal/eventbus is at-least-once PublishUnique) would double-fire a comment, a message, or a billed image generation.\n\nGrounded by a crew investigation pass (2026-09-03, further-investigation.md, not committed -- content folded into this ticket): read internal/eventbus's PublishUnique key computation and existing dedup surface, TaskEnvelope's idempotency key minting/consumption path, event-sources-and-reactions.md's decided contract (extend it, don't contradict it), and the CEL context surface in internal/domain/eda/expr (event/actions fields available for key derivation).","design":"Design-decide and write into eda-playbook-engine.md:\n1. Granularity: one idempotency key per trigger-event + playbook (whole run dedups) vs per-action key derived from the run key. The doc's stated bias is the former unless a specific action needs finer dedup -- turn that bias into a stated decision.\n2. Where the key lives: does a playbook run get its own envelope/context carrying the key, and how does it reach the target position's invocation (Module.Invoke, a future Channel/Forge call)?\n3. What 'dedup' does on redelivery: is the whole run skipped, or just the already-fired action? What gets recorded?\n\nSTRONG REUSABLE PRECEDENT FOUND IN THIS CODEBASE (verified directly, 2026-09-03) -- do not design a new ledger without reading this first: internal/store/bindings.go already implements exactly this shape for playbook bindings. binding_dispatches table, unique on (binding_id, capture_id), written via RecordDispatch's 'INSERT OR IGNORE' -- a duplicate insert returns the sentinel ErrAlreadyDispatched, which dispatchBindings (internal/daemon/daemon.go:479) treats as a normal 'another cycle raced us' outcome, not an error. This is a proven, tested, at-most-once dispatch guarantee across cycles AND daemon restarts, for the exact same kind of problem (don't re-fire a binding for a capture it already dispatched). The playbook-run idempotency key almost certainly should reuse this exact pattern: a new ledger table (or a generalization of binding_dispatches) keyed on (playbook_id, event_id) or (playbook_id, trigger_event_identity), written with the same INSERT OR IGNORE + sentinel-error convention. Read internal/store/bindings.go:29-61,351-378 and internal/daemon/daemon.go:479-514 before designing anything -- this may reduce gap 2 to 'apply the existing pattern to a new key shape' rather than new design.\n4. Key derivation: a CEL-reachable expression over event fields, or a stable hash of event-identity+playbook-id -- pick one and justify it against the expr package's actual context surface, and against how capture identity is derived today (check what 'capture_id' actually is -- likely a stable hash or row id from the same capture pipeline a playbook trigger would also read).","acceptance_criteria":"eda-playbook-engine.md's Execution-time gap 2 section is marked resolved with a stated decision (not a survey of options) covering granularity, key storage/lookup, redelivery behavior, and key derivation. Consistent with event-sources-and-reactions.md's existing idempotency contract -- explicitly states how it extends rather than contradicts it. No code changes required for this ticket to close.","status":"open","priority":1,"issue_type":"task","owner":"sam@catlow.tech","created_at":"2026-09-02T18:12:13Z","created_by":"Sam Catlow","updated_at":"2026-09-02T18:21:12Z","dependencies":[{"issue_id":"archie-core-t2db.17","depends_on_id":"archie-core-t2db","type":"parent-child","created_at":"2026-09-02T18:12:13Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"archie-core-e7bo","title":"fix stale wfextract go:generate directive; audit pluginextract/secretextract drift","description":"Discovered 2026-09-03 while scoping archie-core-t2db.13 (Module position). Two separate, pre-existing issues, unrelated to the playbook engine work -- do not fix as a drive-by inside that ticket.\n\n1. internal/domain/workflow/wfextract/wfextract.go's //go:generate directive still targets 'github.com/samcharles93/archie-core/internal/workflow', a path that has not existed since the domain migration (7e896c9, 2026-08-07 - the package moved to internal/domain/workflow). The committed generated file was hand-patched at some point to import the new path, but the directive itself was never fixed. Net effect: 'go generate ./...' has been broken repo-wide (at least for wfextract) since that migration, silently, for anyone who hasn't tried running it.\n\n2. Running 'go generate ./...' anyway (to see what breaks) also produced real, substantive diffs in internal/plugin/pluginextract and internal/secret/secretextract's generated symbol tables -- new wrapper types (Host, Module) appearing that are NOT in the currently-committed, in-production symbol tables. This means the Yaegi plugin and secret-engine extension surfaces have been running against stale, incomplete generated symbol tables for an unknown period: interpreted plugin/secret-engine code may be unable to reference types it should have access to. This is a latent correctness gap with real blast radius on two extension surfaces, not cosmetic drift.","design":"Two-part fix, do not conflate:\n1. Fix wfextract's directive to point at internal/domain/workflow (matching what the generated file already imports). Verify 'go generate ./internal/domain/workflow/wfextract/...' now succeeds and produces output identical (or intentionally different, documented why) to what's committed.\n2. For pluginextract/secretextract: do NOT blindly regenerate-and-commit. First determine why they're stale (when did plugin.go/secret's Engine gain the types the fresh generation adds -- git log on internal/plugin/plugin.go and internal/secret's engine files vs. when the generated files were last touched) and whether any existing test exercises a Yaegi plugin/secret-engine referencing the missing types (if none does, that's itself a coverage gap worth noting). Regenerate, gofumpt, and add/confirm a test that would have caught the drift, before committing the regenerated symbol tables -- this is a correctness-sensitive change to two live extension surfaces and needs real verification, not a mechanical re-run.","acceptance_criteria":"'go generate ./...' completes without error across the whole repo. wfextract's directive matches its actual package path. pluginextract/secretextract regeneration (if done) is backed by a test demonstrating the previously-missing types are now reachable from interpreted code, and task check passes. No behavior change to existing plugins/secret engines that were already working (regression-tested, not assumed).","status":"closed","priority":1,"issue_type":"bug","owner":"sam@catlow.tech","created_at":"2026-09-02T17:17:42Z","created_by":"Sam Catlow","updated_at":"2026-09-02T17:30:34Z","closed_at":"2026-09-02T17:30:34Z","close_reason":"Fixed and gate-verified as e3a20f6 (pushed to main). wfextract directive now targets internal/domain/workflow; 'go generate ./...' completes repo-wide. All four extract surfaces regenerated; nil-guards re-applied on interface wrappers (documented IMPORTANT note restated in wfextract package doc). Previously-missing symbols proven reachable from interpreted code: wfextract (Forger/Reviewer/Trees/routing loaders) and pluginextract (Host/Module/Manifest/NewHost/HostAPIVersion) via new reachability tests; secretextract exports were current (nil-guard drift only). Red-then-green verified: fresh regen wipes guards -\u003e guard tests panic -\u003e re-apply -\u003e pass. task check clean (113/113 UI, exit 0).","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"archie-core-95dj","title":"StageCommitPush discards baseline-fix commits when build makes no changes","description":"Confirmed via carina production logs (task 6, chat-spawned synthetic test task, 2026-09-02). StageBaselineGate (internal/domain/workflow/implement.go:37) found the repo's gate red at the base commit, spent 405,004 tokens/14 iterations genuinely fixing two broken pre-existing tests, and committed the fix locally via tc.Trees.CommitAll (implement.go:104) -- verified passing. plan and build stages then correctly found nothing else to do for the (synthetic, content-free) task. StageCommitPush -\u003e closeNoChangesIssue (steps.go:56) decides 'no changes' purely from tc.BuildNoChanges, which the build AgentStage sets from its own len(res.Changes)==0 (implement.go:176) -- it has zero awareness that StageBaselineGate already committed real changes two stages earlier. Never pushed, never opened a PR; the entire worktree containing a real, gate-verified fix for genuinely broken tests was discarded when the task closed as 'merged: completed -- no changes required'.\n\nCost: 669,421 tokens burned to fix a real bug that then evaporated. This recurs: any task that hits the same red baseline pays the identical tax again since the fix never lands. Evidence a prior incident already happened: an orphaned 'fix: baseline gate repair' commit (e49f3cd) sits on a stale, unmerged branch (fix/441-archied-repos-entries-cant-span-more-than-one-forge) from an earlier occurrence.","design":"Fix direction: either (a) StageBaselineGate sets a flag /flips tc.BuildNoChanges appropriately when it commits, or (b) StageCommitPush's no-changes branch checks the worktree's actual diff against the base branch (tc.Trees, similar to StageDiffCap's ChangedLines) instead of trusting tc.BuildNoChanges alone. Prefer (a): cheaper, and keeps the diff-against-base check (which already exists for StageDiffCap) from being computed twice per task.","acceptance_criteria":"A task whose baseline-fix stage committed real changes, but whose plan/build stages found nothing further to do, still pushes and opens a PR carrying the baseline-fix commit -- it must never be silently discarded. tc.BuildNoChanges (or StageCommitPush's decision) accounts for what StageBaselineGate already committed, not just the build stage's own diff. Regression test reproduces the exact scenario (baseline-fix commits, build stage reports zero changes) and proves a PR opens. task check passes.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-09-02T11:20:48Z","created_by":"Sam Catlow","updated_at":"2026-09-02T14:56:02Z","started_at":"2026-09-02T11:20:51Z","closed_at":"2026-09-02T11:25:53Z","close_reason":"Fixed via a new TaskContext.BaselineFixed flag: StageBaselineGate sets it when its own CommitAll reports a real committed change (implement.go:104-111). The build stage's OnResult only sets BuildNoChanges when its own diff is empty AND BaselineFixed is false (implement.go:177), so a baseline-only fix can no longer be masked as 'no changes'. StageCommitPush itself now calls CommitAll directly and pushes whenever either a new commit was made OR BaselineFixed is true (steps.go), instead of erroring on 'nothing to commit' when the real change already sits on the branch from the baseline-fix commit. Two new regression tests (TestStageCommitPushPushesBaselineFixEvenWithNothingNewToCommit, TestStageBaselineGateSetsBaselineFixedWhenItCommits) plus a guard test that the old empty-tree error still fires without a baseline fix. task check clean.","external_ref":"https://github.com/samcharles93/archie-core/issues/682","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"archie-core-s81k","title":"Preserve bound slog attributes in persistent log sinks","description":"PR #650 scopes daemon, Web UI, and curator loggers with slog.Logger.With. internal/logging.FeedHandler.WithAttrs retains those bound attributes for the live feed but does not pass them to the wrapped handler, while Handle forwards the bare slog.Record. As a result, component is present in live Web UI entries but absent from stderr and file JSON; after restart, historical component discovery and filtering cannot see those records. Fix bound-attribute propagation without breaking slog group semantics or duplicating attributes.","acceptance_criteria":"Bound attributes appear in both Feed entries and the wrapped JSON sink; component values survive file persistence and are discoverable and filterable through historical /api/logs reads; regression tests exercise logger.With for both live and persistent paths; task check passes.","status":"closed","priority":1,"issue_type":"bug","owner":"sam@catlow.tech","created_at":"2026-09-02T09:37:55Z","created_by":"Sam Catlow","updated_at":"2026-09-02T09:48:54Z","closed_at":"2026-09-02T09:43:01Z","close_reason":"Fixed: FeedHandler.WithAttrs/WithGroup now propagate to the wrapped handler, matching SystemLogHandler's existing pattern. Added regression test covering both feed and persistent sink.","external_ref":"https://github.com/samcharles93/archie-core/issues/655","dependency_count":0,"dependent_count":0,"comment_count":0}