eda: resolve idempotency keying for Module/Channel/Forge actions (execution-time gap 2) #440

Closed
opened 2026-10-27 13:36:20 +00:00 by sam · 0 comments
Owner

Design-investigation ticket (like the CEL question, archie-core-t2db resolution in eda-playbook-engine.md) -- produces a design-doc resolution, NOT implementation. This is the sharper of the two unresolved execution-time gaps named in docs/prds/eda-playbook-engine.md's 'Execution-time gaps' section: workflow-kind actions inherit idempotency for free via TaskEnvelope; channel/forge/module actions with real side effects have no keying scheme, so a redelivered trigger event (internal/eventbus is at-least-once PublishUnique) would double-fire a comment, a message, or a billed image generation.

Grounded by a crew investigation pass (2026-09-03, further-investigation.md, not committed -- content folded into this ticket): read internal/eventbus's PublishUnique key computation and existing dedup surface, TaskEnvelope's idempotency key minting/consumption path, event-sources-and-reactions.md's decided contract (extend it, don't contradict it), and the CEL context surface in internal/domain/eda/expr (event/actions fields available for key derivation).

Design

Design-decide and write into eda-playbook-engine.md:

  1. Granularity: one idempotency key per trigger-event + playbook (whole run dedups) vs per-action key derived from the run key. The doc's stated bias is the former unless a specific action needs finer dedup -- turn that bias into a stated decision.
  2. Where the key lives: does a playbook run get its own envelope/context carrying the key, and how does it reach the target position's invocation (Module.Invoke, a future Channel/Forge call)?
  3. What 'dedup' does on redelivery: is the whole run skipped, or just the already-fired action? What gets recorded?

STRONG REUSABLE PRECEDENT FOUND IN THIS CODEBASE (verified directly, 2026-09-03) -- do not design a new ledger without reading this first: internal/store/bindings.go already implements exactly this shape for playbook bindings. binding_dispatches table, unique on (binding_id, capture_id), written via RecordDispatch's 'INSERT OR IGNORE' -- a duplicate insert returns the sentinel ErrAlreadyDispatched, which dispatchBindings (internal/daemon/daemon.go:479) treats as a normal 'another cycle raced us' outcome, not an error. This is a proven, tested, at-most-once dispatch guarantee across cycles AND daemon restarts, for the exact same kind of problem (don't re-fire a binding for a capture it already dispatched). The playbook-run idempotency key almost certainly should reuse this exact pattern: a new ledger table (or a generalization of binding_dispatches) keyed on (playbook_id, event_id) or (playbook_id, trigger_event_identity), written with the same INSERT OR IGNORE + sentinel-error convention. Read internal/store/bindings.go:29-61,351-378 and internal/daemon/daemon.go:479-514 before designing anything -- this may reduce gap 2 to 'apply the existing pattern to a new key shape' rather than new design.
4. Key derivation: a CEL-reachable expression over event fields, or a stable hash of event-identity+playbook-id -- pick one and justify it against the expr package's actual context surface, and against how capture identity is derived today (check what 'capture_id' actually is -- likely a stable hash or row id from the same capture pipeline a playbook trigger would also read).

Acceptance criteria

eda-playbook-engine.md's Execution-time gap 2 section is marked resolved with a stated decision (not a survey of options) covering granularity, key storage/lookup, redelivery behavior, and key derivation. Consistent with event-sources-and-reactions.md's existing idempotency contract -- explicitly states how it extends rather than contradicts it. No code changes required for this ticket to close.

Notes

Proposed resolution drafted 2026-09-03 in docs/prds/eda-playbook-engine.md, commit 99eaa7a -- reuses internal/store/bindings.go's binding_dispatches ledger shape. Explicitly marked 'awaiting Sam's sign-off,' not settled. Surfaced two concrete prerequisites while drafting: t2db.15's Playbook struct needs an ID/Version (doesn't have one), and DispatchInput needs a task identity threaded through (doesn't have one either) -- both stated honestly as blockers for implementation, not glossed over.

Close reason

Closed


Imported from beads archie-core-t2db.17 · GitHub #726 · type task · priority P1 · created 2026-09-02 · closed 2026-09-05

Design-investigation ticket (like the CEL question, archie-core-t2db resolution in eda-playbook-engine.md) -- produces a design-doc resolution, NOT implementation. This is the sharper of the two unresolved execution-time gaps named in docs/prds/eda-playbook-engine.md's 'Execution-time gaps' section: workflow-kind actions inherit idempotency for free via TaskEnvelope; channel/forge/module actions with real side effects have no keying scheme, so a redelivered trigger event (internal/eventbus is at-least-once PublishUnique) would double-fire a comment, a message, or a billed image generation. Grounded by a crew investigation pass (2026-09-03, further-investigation.md, not committed -- content folded into this ticket): read internal/eventbus's PublishUnique key computation and existing dedup surface, TaskEnvelope's idempotency key minting/consumption path, event-sources-and-reactions.md's decided contract (extend it, don't contradict it), and the CEL context surface in internal/domain/eda/expr (event/actions fields available for key derivation). ### Design Design-decide and write into eda-playbook-engine.md: 1. Granularity: one idempotency key per trigger-event + playbook (whole run dedups) vs per-action key derived from the run key. The doc's stated bias is the former unless a specific action needs finer dedup -- turn that bias into a stated decision. 2. Where the key lives: does a playbook run get its own envelope/context carrying the key, and how does it reach the target position's invocation (Module.Invoke, a future Channel/Forge call)? 3. What 'dedup' does on redelivery: is the whole run skipped, or just the already-fired action? What gets recorded? STRONG REUSABLE PRECEDENT FOUND IN THIS CODEBASE (verified directly, 2026-09-03) -- do not design a new ledger without reading this first: internal/store/bindings.go already implements exactly this shape for playbook bindings. binding_dispatches table, unique on (binding_id, capture_id), written via RecordDispatch's 'INSERT OR IGNORE' -- a duplicate insert returns the sentinel ErrAlreadyDispatched, which dispatchBindings (internal/daemon/daemon.go:479) treats as a normal 'another cycle raced us' outcome, not an error. This is a proven, tested, at-most-once dispatch guarantee across cycles AND daemon restarts, for the exact same kind of problem (don't re-fire a binding for a capture it already dispatched). The playbook-run idempotency key almost certainly should reuse this exact pattern: a new ledger table (or a generalization of binding_dispatches) keyed on (playbook_id, event_id) or (playbook_id, trigger_event_identity), written with the same INSERT OR IGNORE + sentinel-error convention. Read internal/store/bindings.go:29-61,351-378 and internal/daemon/daemon.go:479-514 before designing anything -- this may reduce gap 2 to 'apply the existing pattern to a new key shape' rather than new design. 4. Key derivation: a CEL-reachable expression over event fields, or a stable hash of event-identity+playbook-id -- pick one and justify it against the expr package's actual context surface, and against how capture identity is derived today (check what 'capture_id' actually is -- likely a stable hash or row id from the same capture pipeline a playbook trigger would also read). ### Acceptance criteria eda-playbook-engine.md's Execution-time gap 2 section is marked resolved with a stated decision (not a survey of options) covering granularity, key storage/lookup, redelivery behavior, and key derivation. Consistent with event-sources-and-reactions.md's existing idempotency contract -- explicitly states how it extends rather than contradicts it. No code changes required for this ticket to close. ### Notes Proposed resolution drafted 2026-09-03 in docs/prds/eda-playbook-engine.md, commit 99eaa7a -- reuses internal/store/bindings.go's binding_dispatches ledger shape. Explicitly marked 'awaiting Sam's sign-off,' not settled. Surfaced two concrete prerequisites while drafting: t2db.15's Playbook struct needs an ID/Version (doesn't have one), and DispatchInput needs a task identity threaded through (doesn't have one either) -- both stated honestly as blockers for implementation, not glossed over. ### Close reason Closed --- _Imported from beads `archie-core-t2db.17` · GitHub #726 · type task · priority P1 · created 2026-09-02 · closed 2026-09-05_
sam added the hightask labels 2026-10-27 13:36:20 +00:00
sam self-assigned this 2026-10-27 13:36:20 +00:00
sam closed this issue 2026-10-27 13:36:20 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: sam/archie#440