diff --git a/docs/data/generated/dev-docs.json b/docs/data/generated/dev-docs.json index 91b5b11a1..c25863456 100644 --- a/docs/data/generated/dev-docs.json +++ b/docs/data/generated/dev-docs.json @@ -298,7 +298,7 @@ "url": "/dev/prds/multi-action-playbooks/", "title": "Multi-action playbooks and typed action results", "section": "PRDs", - "body": "# Multi-action playbooks and typed action results\n\n**Status:** Approved\n**Date:** 2026-09-22\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-nx30` (unblocks `archie-core-t2db.27` and `archie-core-t2db.31`)\n\n## Why this exists\n\n`eda-playbook-engine.md`'s hard boundary rejects any playbook that is not\nexactly one `workflow` action, so the parent document's data-flow, Module and\nChannel/Forge decisions have no loadable shape to attach to. The boundary's own\nstated condition is now met: both execution-time gaps are resolved\n(`eda-playbook-engine.md`, \"Execution-time gaps\"; the `playbook_dispatches`\nledger). This document designs the relaxation.\n\n## A playbook is one of two shapes\n\n- A **workflow playbook** is exactly one `workflow` action. Unchanged: it names\n a workflow definition and is routed by the daemon's definition pin.\n- An **action playbook** is one or more `module` actions in order. Each action\n has an optional `id`, a `kind` naming a registered Module kind, `args`, and an\n optional `when`.\n\nOne playbook does not mix the two shapes.\n\n```yaml\ntrigger: { kind: bug }\nactions:\n - id: build\n position: module\n kind: log\n args: { message: '\"build started\"' }\n - id: done\n position: module\n kind: log\n args: { message: '\"done: \" + string(actions.build.result.written)' }\n when: actions.build.result.written == true\n```\n\nThe first Module kind is the side-effect-free `log` kind\n(`module-position.md`, \"Recommended first slice\"). A Module kind with an\nexternal side effect, and the Channel and Forge positions, are out of scope.\n\n## A result type is the kind's generated Result struct\n\nEach Module kind owns a hand-written `Args` and `Result` struct\n(`internal/domain/eda/module/log` is the worked example); `ModuleRegistry.Invoke`\nreturns the result as a flat `map[string]any`. `ModuleRegistry.DecodeResult`\n(`internal/domain/eda/module`) is the single site that converts that map into\nthe kind's `Result` struct before a later expression reads it, so the value the\nchecker validates and the value the run reads have one definition.\n\n`actions.\u003cid\u003e` is typed by the kind of the action that declares `\u003cid\u003e`:\n`{ result: \u003cKindResult\u003e }`. `event` stays `dyn` (J4,\n`playbook-expression-syntax.md`).\n\n## Reject-at-load, preserved\n\nRelaxing the boundary must not make a bad reference resolvable. The load fails,\nnaming the playbook and the offending action, for:\n\n- an `id` missing where a later action references it, duplicated, or outside\n its shape's grammar: a workflow action id keeps the shared\n stable-identifier grammar (J1, already enforced), while a module action id\n must be a lowercase CEL field name writable as `actions.\u003cid\u003e` (so a CEL\n keyword such as `in`, `true`, `false`, or `null` is rejected at load);\n- an `actions.\u003cid\u003e` read of an id no earlier action declares;\n- an `actions` read that is not statically resolvable to an id (any map index,\n literal or dynamic, `in`, a comprehension): the typed `actions` root is an\n object type, so only field selection `actions.\u003cid\u003e` resolves and the\n `actions[\"id\"]` spelling is not accepted;\n- a `result` field the referenced kind's `Result` does not define;\n- a reference to an earlier action that produces no result;\n- an unknown `position`, an unknown `kind`, or a `module` action with no `kind`;\n- a field belonging to the other shape (`kind` on a `workflow` action, or\n `workflow` on a `module` action);\n- an arg KEY the kind's `Args` schema does not define. Arg VALUE type-checking\n is tracked as `archie-core-t2db` debt, not shipped here.\n\nDeclaring the kind's `Result` schema to CEL at load makes a result-field typo a\nload failure rather than a dispatch failure. Matching arg keys against the\nkind's `Args` schema makes an arg-key typo a load failure; arg value\ntype-checking is tracked as `archie-core-t2db` debt.\n\n## The environment is per playbook\n\nResult typing depends on the ids a playbook declares and the kind each id runs,\nso the CEL environment is built from one playbook's action list rather than once\nper `Store`, and `expr.NewEnv` takes that list: each declared id becomes a field\nof the `actions` object typed by its kind's `Result`.\n\n## Running an action playbook is out of scope\n\nThis document makes an action playbook load and its references type-check. It\ndoes not invoke a Module kind. Invocation, the `playbook_dispatches` gate, and\nstop-on-first-failure across actions are `archie-core-t2db.31`.\n\n## Decisions\n\n**D1 — dispatch before `t2db.31`.** Settled: an action playbook loads and\nvalidates but is not routed; the definition-pin dispatch reports no match for it,\nand the daemon logs a visible warning naming it.\n\n**D2 — mixing positions.** Settled: one shape per playbook. A `workflow` action\nyields a task, not a result, so it cannot feed `actions.\u003cid\u003e.result`.\n\n**D3 — the CEL typing mechanism.** Settled: a per-playbook environment whose\n`actions` root is an object type with one field per declared id, each field typed\nby the kind's `Result` struct. `internal/domain/eda/expr` implements it with the\ngeneric `actionResultProvider`.\n\n**D4 — `args` against the kind's `Args` schema.** Settled: arg KEY checking ships\nnow. Arg value type-checking is tracked as `archie-core-t2db` debt.\n\n**D5 — where the result map becomes the `Result` struct.** Settled:\n`ModuleRegistry` stays schema-agnostic. `ModuleRegistry.DecodeResult`\n(`internal/domain/eda/module`) is the conversion site.\n\n## Verification\n\n- A two-action `module` playbook with a valid `actions.\u003cid\u003e.result.\u003cfield\u003e` read\n loads; the same read of a field the kind's `Result` does not define fails the\n load naming the field; a read of an id no earlier action declares fails; a\n dynamically indexed `actions` read fails.\n- An arg KEY the kind's `Args` schema does not define fails the load. Arg VALUE\n type-checking is tracked as `archie-core-t2db` debt.\n- The one-action `workflow` playbook shape, its dispatch, and its existing\n load-time checks are unchanged.\n" + "body": "# Multi-action playbooks and typed action results\n\n**Status:** Approved\n**Date:** 2026-09-22\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-nx30` (unblocks `archie-core-t2db.27` and `archie-core-t2db.31`)\n\n## Why this exists\n\n`eda-playbook-engine.md`'s hard boundary rejects any playbook that is not\nexactly one `workflow` action, so the parent document's data-flow, Module and\nChannel/Forge decisions have no loadable shape to attach to. The boundary's own\nstated condition is now met: both execution-time gaps are resolved\n(`eda-playbook-engine.md`, \"Execution-time gaps\"; the `playbook_dispatches`\nledger). This document designs the relaxation.\n\n## A playbook is one of two shapes\n\n- A **workflow playbook** is exactly one `workflow` action. Unchanged: it names\n a workflow definition and is routed by the daemon's definition pin.\n- An **action playbook** is one or more `module` actions in order. Each action\n has an optional `id`, a `kind` naming a registered Module kind, `args`, and an\n optional `when`.\n\nOne playbook does not mix the two shapes.\n\n```yaml\ntrigger: { kind: bug }\nactions:\n - id: build\n position: module\n kind: log\n args: { message: '\"build started\"' }\n - id: done\n position: module\n kind: log\n args: { message: '\"done: \" + string(actions.build.result.written)' }\n when: actions.build.result.written == true\n```\n\nThe first Module kind is the side-effect-free `log` kind\n(`module-position.md`, \"Recommended first slice\"). A Module kind with an\nexternal side effect, and the Channel and Forge positions, are out of scope.\n\n## A result type is the kind's generated Result struct\n\nEach Module kind owns a hand-written `Args` and `Result` struct\n(`internal/domain/eda/module/log` is the worked example); `ModuleRegistry.Invoke`\nreturns the result as a flat `map[string]any`. `ModuleRegistry.DecodeResult`\n(`internal/domain/eda/module`) is the single site that converts that map into\nthe kind's `Result` struct before a later expression reads it, so the value the\nchecker validates and the value the run reads have one definition.\n\n`actions.\u003cid\u003e` is typed by the kind of the action that declares `\u003cid\u003e`:\n`{ result: \u003cKindResult\u003e }`. `event` stays `dyn` (J4,\n`playbook-expression-syntax.md`).\n\n## Reject-at-load, preserved\n\nRelaxing the boundary must not make a bad reference resolvable. The load fails,\nnaming the playbook and the offending action, for:\n\n- an `id` missing where a later action references it, duplicated, or outside\n its shape's grammar: a workflow action id keeps the shared\n stable-identifier grammar (J1, already enforced), while a module action id\n must be a lowercase CEL field name writable as `actions.\u003cid\u003e` (so a CEL\n keyword such as `in`, `true`, `false`, or `null` is rejected at load);\n- an `actions.\u003cid\u003e` read of an id no earlier action declares;\n- an `actions` read that is not statically resolvable to an id (any map index,\n literal or dynamic, `in`, a comprehension): the typed `actions` root is an\n object type, so only field selection `actions.\u003cid\u003e` resolves and the\n `actions[\"id\"]` spelling is not accepted;\n- a `result` field the referenced kind's `Result` does not define;\n- a reference to an earlier action that produces no result;\n- an unknown `position`, an unknown `kind`, or a `module` action with no `kind`;\n- a field belonging to the other shape (`kind` on a `workflow` action, or\n `workflow` on a `module` action);\n- an arg KEY the kind's `Args` schema does not define. Arg VALUE type-checking\n is tracked as `archie-core-t2db` debt, not shipped here.\n\nDeclaring the kind's `Result` schema to CEL at load makes a result-field typo a\nload failure rather than a dispatch failure. Matching arg keys against the\nkind's `Args` schema makes an arg-key typo a load failure; arg value\ntype-checking is tracked as `archie-core-t2db` debt.\n\n## The environment is per playbook\n\nResult typing depends on the ids a playbook declares and the kind each id runs,\nso the CEL environment is built from one playbook's action list rather than once\nper `Store`, and `expr.NewEnv` takes that list: each declared id becomes a field\nof the `actions` object typed by its kind's `Result`.\n\n## Running an action playbook is out of scope\n\nThis document makes an action playbook load and its references type-check.\nRunning one is designed in `docs/prds/action-playbook-run.md`.\n\n## Decisions\n\n**D1 — dispatch before `t2db.31`.** Superseded by\n`docs/prds/action-playbook-run.md`. Was: an action playbook loads and\nvalidates but is not routed; the definition-pin dispatch reports no match for it,\nand the daemon logs a visible warning naming it.\n\n**D2 — mixing positions.** Settled: one shape per playbook. A `workflow` action\nyields a task, not a result, so it cannot feed `actions.\u003cid\u003e.result`.\n\n**D3 — the CEL typing mechanism.** Settled: a per-playbook environment whose\n`actions` root is an object type with one field per declared id, each field typed\nby the kind's `Result` struct. `internal/domain/eda/expr` implements it with the\ngeneric `actionResultProvider`.\n\n**D4 — `args` against the kind's `Args` schema.** Settled: arg KEY checking ships\nnow. Arg value type-checking is tracked as `archie-core-t2db` debt.\n\n**D5 — where the result map becomes the `Result` struct.** Settled:\n`ModuleRegistry` stays schema-agnostic. `ModuleRegistry.DecodeResult`\n(`internal/domain/eda/module`) is the conversion site.\n\n## Verification\n\n- A two-action `module` playbook with a valid `actions.\u003cid\u003e.result.\u003cfield\u003e` read\n loads; the same read of a field the kind's `Result` does not define fails the\n load naming the field; a read of an id no earlier action declares fails; a\n dynamically indexed `actions` read fails.\n- An arg KEY the kind's `Args` schema does not define fails the load. Arg VALUE\n type-checking is tracked as `archie-core-t2db` debt.\n- The one-action `workflow` playbook shape, its dispatch, and its existing\n load-time checks are unchanged.\n" }, { "url": "/dev/prds/news-and-releases/", diff --git a/docs/prds/action-playbook-run.md b/docs/prds/action-playbook-run.md new file mode 100644 index 000000000..e09de4d29 --- /dev/null +++ b/docs/prds/action-playbook-run.md @@ -0,0 +1,83 @@ +# Running action playbooks + +**Status:** Approved +**Date:** 2026-09-23 +**Parent:** `docs/prds/multi-action-playbooks.md`, epic `archie-core-t2db` +**Beads issue:** `archie-core-t2db.31` + +## Why this exists + +An action playbook loads and type-checks but never runs: `Store.Dispatch` +skips it and the daemon logs a warning at boot (`multi-action-playbooks.md`, +D1). This document defines when it runs, what each action does, and how the +`playbook_dispatches` ledger gates it. + +## When a playbook runs + +The daemon runs action playbooks at the point it pins a task's workflow +definition (`Daemon.resolveWorkflowID`), using the same `DispatchInput` the +workflow playbooks use. The definition is pinned once per task, so the run +happens once per task. + +- Every action playbook whose trigger matches runs, in load order. A workflow + playbook stops at the first match because it chooses one workflow; action + playbooks choose nothing, so no match suppresses another. +- A run never changes routing. The task is pinned by the workflow playbooks + and bindings exactly as when no action playbook exists. +- An action or playbook failure never parks or fails the task. It is logged + at warning level with the playbook, the action and the task. +- A task that already names a workflow (the approval requeue) is skipped, as + it is for workflow playbooks. +- Actions run synchronously in the pin path, bounded by the daemon's context. + The only Module kind, `log`, returns immediately. A slow kind must bring its + own timeout. + +## What one action does + +Actions run in order against one evaluation context whose `actions` map +starts empty. + +1. Evaluate `when`. False or an evaluation error skips the action (J3). A + skipped action records nothing and produces no result. +2. Evaluate every `args` value. An evaluation error stops the playbook. +3. Call `playbook.InvokeOnce` with the Module invoke as its function. The + ledger row is written first. A duplicate row skips the invoke. +4. An invoke error stops the playbook. The row stays, so the action is not + retried: the ledger promises at most once, not at least once. +5. On success, `ModuleRegistry.DecodeResult` converts the result, and it is + stored as `actions..result` when the action declares an id. + +The event id in the ledger key is `DispatchInput.TaskID`, the issue's +idempotency key (`owner/repo/number`). An action therefore runs at most once +per issue for each playbook version. Editing a playbook changes its version +and lets it run again for issues it already handled. + +A skipped duplicate has no result. A later action that reads it sees a +missing field: its `when` evaluates false, or its args fail and the playbook +stops. This only happens when the same issue is pinned twice under one +playbook version. + +## Wiring + +- `storecontract.PlaybookDispatcher` is lifted onto the daemon from the State + Store client in `internal/app/archied/bootstrap.go`, the same way + `BindingDispatcher` is. +- The daemon's `Playbooks` dependency gains the run method. The Module + registry is passed to the playbook store at load, so the daemon needs no + new dependency for it. +- With no `PlaybookDispatcher` wired, action playbooks do not run, and the + boot warning names each one. They never run without the ledger. +- The boot warning that action playbooks do not execute is removed when the + ledger is wired. + +## Verification + +- A matching two-action `log` playbook runs both actions in order, and the + second reads the first's result. +- Pinning the same task twice under one playbook version invokes each action + once. +- A `when` that evaluates false skips that action and the run continues. +- An invoke error stops the playbook. The task is still pinned to the same + workflow it would get with no action playbook. +- A non-matching trigger runs nothing. +- With no ledger wired, nothing runs and the warning names the playbook. diff --git a/docs/prds/multi-action-playbooks.md b/docs/prds/multi-action-playbooks.md index 14ad05424..64ff4e01e 100644 --- a/docs/prds/multi-action-playbooks.md +++ b/docs/prds/multi-action-playbooks.md @@ -92,13 +92,13 @@ of the `actions` object typed by its kind's `Result`. ## Running an action playbook is out of scope -This document makes an action playbook load and its references type-check. It -does not invoke a Module kind. Invocation, the `playbook_dispatches` gate, and -stop-on-first-failure across actions are `archie-core-t2db.31`. +This document makes an action playbook load and its references type-check. +Running one is designed in `docs/prds/action-playbook-run.md`. ## Decisions -**D1 — dispatch before `t2db.31`.** Settled: an action playbook loads and +**D1 — dispatch before `t2db.31`.** Superseded by +`docs/prds/action-playbook-run.md`. Was: an action playbook loads and validates but is not routed; the definition-pin dispatch reports no match for it, and the daemon logs a visible warning naming it. diff --git a/internal/app/archied/bootstrap.go b/internal/app/archied/bootstrap.go index 69a19b3cc..733a17be1 100644 --- a/internal/app/archied/bootstrap.go +++ b/internal/app/archied/bootstrap.go @@ -774,9 +774,8 @@ func (b *boot) loadModules(cfg config.Config, log *slog.Logger) error { // rule -- any malformed playbook, mixed/unsupported action shape, unknown // module kind, when compile failure, or args key failure aborts startup, // matching the routing-file load pattern (not degrade-and-skip). A nonexistent -// dir is an empty store. Action playbooks load and type-check but have no run -// path yet (t2db.31), so each one is logged as a visible warning here rather -// than routed by the definition pin. +// dir is an empty store. Action playbooks run through the daemon once +// buildDaemon wires the dispatch ledger (docs/prds/action-playbook-run.md). func (b *boot) loadEDAPlaybooks(cfg config.Config, log *slog.Logger) error { var err error b.playbooks, err = playbook.Load(cfg.EDAPlaybookDir, b.modules) @@ -784,11 +783,6 @@ func (b *boot) loadEDAPlaybooks(cfg config.Config, log *slog.Logger) error { log.Error("eda playbook load failed", "dir", cfg.EDAPlaybookDir, "err", err) return err } - for _, pb := range b.playbooks.Playbooks { - if pb.IsActionPlaybook() { - log.Warn("eda action playbook loaded: action playbooks do not execute yet", "playbook", pb.ID) - } - } log.Info("eda playbooks loaded", "dir", cfg.EDAPlaybookDir, "playbooks", len(b.playbooks.Playbooks)) return nil } @@ -1326,9 +1320,25 @@ func (b *boot) buildDaemon() { if btc, ok := b.stateStore.(storecontract.BindingTaskCreator); ok { b.d.BindingTaskCreator = btc } + b.d.PlaybookLedger = playbookLedger(b.stateStore, b.playbooks, b.log) b.setupForgeWebhook() } +// playbookLedger returns source's dispatch ledger, or nil with a warning +// naming each action playbook that therefore will not run: an action playbook +// never fires without its at-most-once gate. +func playbookLedger(source any, playbooks *playbook.Store, log *slog.Logger) storecontract.PlaybookDispatcher { + if pd, ok := source.(storecontract.PlaybookDispatcher); ok { + return pd + } + for _, pb := range playbooks.Playbooks { + if pb.IsActionPlaybook() { + log.Warn("eda action playbook will not run: no dispatch ledger is wired", "playbook", pb.ID) + } + } + return nil +} + // setupForgeWebhook starts the forge webhook receiver when intake is "webhook" // or "both". It decodes GitHub issue events into task envelopes and publishes // them through the same path the poller uses, so a labelled or assigned issue diff --git a/internal/app/archied/eda_playbooks_test.go b/internal/app/archied/eda_playbooks_test.go index 12ebb810f..660924351 100644 --- a/internal/app/archied/eda_playbooks_test.go +++ b/internal/app/archied/eda_playbooks_test.go @@ -2,6 +2,7 @@ package archied import ( "bytes" + "context" "log/slog" "os" "path/filepath" @@ -10,71 +11,65 @@ import ( "github.com/samcharles93/archie-core/internal/config" "github.com/samcharles93/archie-core/internal/domain/eda/module" + "github.com/samcharles93/archie-core/internal/domain/eda/playbook" ) -// TestLoadEDAPlaybooksWarnsOnActionPlaybook pins the D1 visible warning: an -// action playbook loads and validates but has no run path yet, so the daemon -// logs a warning (not debug) naming it and stating plainly that action -// playbooks do not execute yet -- rather than silently loading a playbook that -// can never be routed. -func TestLoadEDAPlaybooksWarnsOnActionPlaybook(t *testing.T) { +type stubPlaybookLedger struct{} + +func (stubPlaybookLedger) RecordPlaybookDispatch(context.Context, string, string, string, string) error { + return nil +} + +func (stubPlaybookLedger) DeletePlaybookDispatches(context.Context, string) error { return nil } + +func loadBootPlaybooks(t *testing.T, document string) *playbook.Store { + t.Helper() dir := t.TempDir() - if err := os.WriteFile(filepath.Join(dir, "pb.yaml"), []byte(`trigger: + if err := os.WriteFile(filepath.Join(dir, "pb.yaml"), []byte(document), 0o600); err != nil { + t.Fatal(err) + } + b := &boot{cfg: config.Config{EDAPlaybookDir: dir}, modules: module.New()} + if err := b.loadEDAPlaybooks(b.cfg, slog.New(slog.DiscardHandler)); err != nil { + t.Fatalf("loadEDAPlaybooks: %v", err) + } + return b.playbooks +} + +const actionPlaybookDoc = `trigger: kind: bug actions: - position: module kind: log args: message: '"build started"' -`), 0o600); err != nil { - t.Fatal(err) - } +` - var buf bytes.Buffer - log := slog.New(slog.NewTextHandler(&buf, nil)) - b := &boot{cfg: config.Config{EDAPlaybookDir: dir}, modules: module.New()} - if err := b.loadEDAPlaybooks(b.cfg, log); err != nil { - t.Fatalf("loadEDAPlaybooks: %v", err) +// An action playbook never runs without the dispatch ledger, and the +// operator is told which ones are inert rather than finding out by silence. +func TestPlaybookLedger(t *testing.T) { + tests := []struct { + name string + source any + document string + wantLedger bool + wantWarn bool + }{ + {name: "ledger wired", source: stubPlaybookLedger{}, document: actionPlaybookDoc, wantLedger: true}, + {name: "no ledger warns per action playbook", source: struct{}{}, document: actionPlaybookDoc, wantWarn: true}, + {name: "no ledger, workflow playbook only", source: struct{}{}, document: "trigger:\n kind: bug\nactions:\n - position: workflow\n workflow: tdd\n"}, } - - out := buf.String() - if !strings.Contains(out, "pb.yaml") { - t.Errorf("warning output does not name the action playbook: %q", out) - } - if !strings.Contains(out, "do not execute yet") { - t.Errorf("warning output does not state action playbooks do not execute yet: %q", out) - } - if strings.Contains(out, "level=DEBUG") { - t.Errorf("warning output was logged at debug, want a visible warning: %q", out) - } -} - -// TestLoadEDAPlaybooksNoWarningOnWorkflowPlaybook is the inverse of the D1 -// warning: a workflow playbook uses the same two-shape predicate but must not -// be logged as a non-routing action playbook. -func TestLoadEDAPlaybooksNoWarningOnWorkflowPlaybook(t *testing.T) { - dir := t.TempDir() - if err := os.WriteFile(filepath.Join(dir, "pb.yaml"), []byte(`trigger: - kind: bug -actions: - - position: workflow - workflow: tdd -`), 0o600); err != nil { - t.Fatal(err) - } - - var buf bytes.Buffer - log := slog.New(slog.NewTextHandler(&buf, nil)) - b := &boot{cfg: config.Config{EDAPlaybookDir: dir}, modules: module.New()} - if err := b.loadEDAPlaybooks(b.cfg, log); err != nil { - t.Fatalf("loadEDAPlaybooks: %v", err) - } - - out := buf.String() - if strings.Contains(out, "do not execute yet") { - t.Errorf("workflow playbook was warned as an action playbook: %q", out) - } - if strings.Contains(out, "level=WARN") { - t.Errorf("workflow playbook produced a warning: %q", out) + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + var buf bytes.Buffer + got := playbookLedger(tt.source, loadBootPlaybooks(t, tt.document), slog.New(slog.NewTextHandler(&buf, nil))) + if (got != nil) != tt.wantLedger { + t.Fatalf("ledger = %v, want wired=%v", got, tt.wantLedger) + } + out := buf.String() + warned := strings.Contains(out, "level=WARN") && strings.Contains(out, "pb.yaml") && strings.Contains(out, "will not run") + if warned != tt.wantWarn { + t.Fatalf("warning output = %q, want warned=%v", out, tt.wantWarn) + } + }) } } diff --git a/internal/daemon/daemon.go b/internal/daemon/daemon.go index 72174a8ff..7823f027b 100644 --- a/internal/daemon/daemon.go +++ b/internal/daemon/daemon.go @@ -220,7 +220,11 @@ type Daemon struct { // and routing is exactly the binding behaviour. Playbooks interface { Dispatch(playbook.DispatchInput) (playbook.Decision, bool) + Run(context.Context, storecontract.PlaybookDispatcher, *slog.Logger, playbook.DispatchInput) error } + // PlaybookLedger is the at-most-once gate action playbooks run through. + // Nil means action playbooks do not run: they never fire unguarded. + PlaybookLedger storecontract.PlaybookDispatcher // WorkflowDefinitions supplies the active database definitions. A task is // pinned once before dispatch; retries reuse the task's stored YAML. @@ -1552,6 +1556,7 @@ func (d *Daemon) pinWorkflowDefinition(ctx context.Context, task *workflow.Task) } return nil } + d.runActionPlaybooks(ctx, task) if d.WorkflowDefinitions == nil { return d.pinWorkflowFromCollection(ctx, task, workflow.ShippedDefinitions(), 0) } @@ -1562,6 +1567,20 @@ func (d *Daemon) pinWorkflowDefinition(ctx context.Context, task *workflow.Task) return d.pinWorkflowFromCollection(ctx, task, collection, version) } +// runActionPlaybooks runs the action playbooks matching a task that is about +// to be pinned, which happens once per task (docs/prds/action-playbook-run.md). +// A run never affects routing and its failure never fails the task. A task +// that already names a workflow is the approval requeue and is skipped, as +// it is for workflow playbooks. +func (d *Daemon) runActionPlaybooks(ctx context.Context, task *workflow.Task) { + if d.Playbooks == nil || d.PlaybookLedger == nil || task.Workflow != "" { + return + } + if err := d.Playbooks.Run(ctx, d.PlaybookLedger, d.Log, playbookInput(task)); err != nil { + d.Log.Warn("action playbook run failed", "task", task.ID, "err", err) + } +} + // resolveWorkflowID picks the definition id to pin. A playbook whose trigger // matches the task decides, ahead of the kind/label bindings; anything else // falls through to workflow.ResolveWorkflowID unchanged. diff --git a/internal/daemon/playbook_run_test.go b/internal/daemon/playbook_run_test.go new file mode 100644 index 000000000..7b7b069b4 --- /dev/null +++ b/internal/daemon/playbook_run_test.go @@ -0,0 +1,80 @@ +package daemon + +import ( + "context" + "errors" + "log/slog" + "testing" + + "github.com/samcharles93/archie-core/internal/domain/eda/playbook" + "github.com/samcharles93/archie-core/internal/domain/storecontract" + "github.com/samcharles93/archie-core/internal/store" +) + +// runRecorder is a playbook set that matches no workflow playbook and records +// each action-playbook run, failing it when err is set. +type runRecorder struct { + runs []playbook.DispatchInput + err error +} + +func (r *runRecorder) Dispatch(playbook.DispatchInput) (playbook.Decision, bool) { + return playbook.Decision{}, false +} + +func (r *runRecorder) Run(_ context.Context, _ storecontract.PlaybookDispatcher, _ *slog.Logger, input playbook.DispatchInput) error { + r.runs = append(r.runs, input) + return r.err +} + +type nopPlaybookLedger struct{} + +func (nopPlaybookLedger) RecordPlaybookDispatch(context.Context, string, string, string, string) error { + return nil +} + +func (nopPlaybookLedger) DeletePlaybookDispatches(context.Context, string) error { return nil } + +func TestPinWorkflowDefinitionRunsActionPlaybooks(t *testing.T) { + tests := []struct { + name string + ledger storecontract.PlaybookDispatcher + workflow string + runErr error + wantRuns int + }{ + {name: "runs once with a ledger", ledger: nopPlaybookLedger{}, wantRuns: 1}, + {name: "a failed run leaves routing alone", ledger: nopPlaybookLedger{}, runErr: errors.New("boom"), wantRuns: 1}, + {name: "never runs without a ledger", wantRuns: 0}, + {name: "skips a task that already names a workflow", ledger: nopPlaybookLedger{}, workflow: "tdd", wantRuns: 0}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + resources := store.OpenTest(t) + defer resources.Close() + task := forgeTask(t, resources, "bug") + task.Workflow = tt.workflow + recorder := &runRecorder{err: tt.runErr} + d := &Daemon{ + Store: resources, + WorkflowDefinitions: routableDefinitions(), + Log: slog.New(slog.DiscardHandler), + Playbooks: recorder, + PlaybookLedger: tt.ledger, + } + if err := d.pinWorkflowDefinition(t.Context(), task); err != nil { + t.Fatal(err) + } + // A second pin reuses the stored definition and must not run again. + if err := d.pinWorkflowDefinition(t.Context(), task); err != nil { + t.Fatal(err) + } + if len(recorder.runs) != tt.wantRuns { + t.Fatalf("runs = %d, want %d", len(recorder.runs), tt.wantRuns) + } + if task.Workflow != "tdd" { + t.Fatalf("pinned workflow = %q, want tdd (the kind binding, untouched by the run)", task.Workflow) + } + }) + } +} diff --git a/internal/domain/eda/playbook/playbook.go b/internal/domain/eda/playbook/playbook.go index dfe2559e1..edb531e83 100644 --- a/internal/domain/eda/playbook/playbook.go +++ b/internal/domain/eda/playbook/playbook.go @@ -7,8 +7,8 @@ // - a workflow playbook is exactly one `workflow` action, unchanged from // the original boundary, and is routed by the daemon's definition pin; // - an action playbook is one or more `module` actions in order, each with -// a registered `kind`, `args`, and an optional `when`/`id`. It loads and -// type-checks now but has no run path yet (t2db.31). +// a registered `kind`, `args`, and an optional `when`/`id`. Store.Run +// executes it (docs/prds/action-playbook-run.md). // // The two shapes never mix in one playbook. This is an ADDITIONAL routing // source alongside the flat kind/label binding files (t2db.9/.10/.11): the @@ -18,6 +18,7 @@ package playbook import ( + "context" "crypto/sha256" "fmt" "os" @@ -43,9 +44,19 @@ type KindSchemas interface { KindSchema(kind string) (args, result reflect.Type, ok bool) } +// Modules is the module source an action playbook loads and runs against: +// the kinds' schemas, the invoke, and the result decode. *module.ModuleRegistry +// satisfies it. +type Modules interface { + KindSchemas + Invoke(ctx context.Context, kind string, args map[string]any) (map[string]any, error) + DecodeResult(kind string, raw map[string]any) (any, error) +} + // Store is the loaded set of playbooks, validated and compiled at load time. type Store struct { Playbooks []*Playbook + modules Modules } // Playbook is one trigger+actions document. @@ -190,10 +201,10 @@ func validateModuleActionIDs(path string, actions []rawAction) error { // compile error, an args key error -- fails the whole load: the reject-at-load // philosophy of the parent design doc. A missing directory is an empty store // (matching the flat binding loaders' convention). -func Load(dir string, schemas KindSchemas) (*Store, error) { +func Load(dir string, modules Modules) (*Store, error) { entries, err := os.ReadDir(dir) if os.IsNotExist(err) { - return &Store{}, nil + return &Store{modules: modules}, nil } if err != nil { return nil, fmt.Errorf("read playbook dir %s: %w", dir, err) @@ -210,10 +221,10 @@ func Load(dir string, schemas KindSchemas) (*Store, error) { } sort.Strings(names) - store := &Store{} + store := &Store{modules: modules} for _, name := range names { path := filepath.Join(dir, name) - pb, err := loadOne(dir, path, schemas) + pb, err := loadOne(dir, path, modules) if err != nil { return nil, err } @@ -549,8 +560,8 @@ type DispatchInput struct { // IsActionPlaybook reports whether pb is an action playbook (one or more // module actions), as opposed to a workflow playbook (exactly one workflow // action). It is the single two-shape predicate shared by Dispatch (which -// skips action playbooks) and the daemon's D1 load warning (which logs them), -// so the two can never disagree. +// routes only workflow playbooks) and Run (which executes only action +// playbooks), so the two can never disagree. func (pb *Playbook) IsActionPlaybook() bool { return len(pb.Actions) != 1 || pb.Actions[0].Position != "workflow" } @@ -604,9 +615,8 @@ type Decision struct { // Dispatch returns the workflow name the first matching workflow playbook // selects for the input, and whether any playbook matched. Action playbooks -// are loaded and validated but not routed (multi-action-playbooks.md, D1), so -// this considers workflow playbooks only: an action playbook cannot hijack the -// definition pin before its run path exists (t2db.31). No match means trigger +// choose no workflow, so they are skipped here and executed by Run. No match +// means trigger // mismatch or a when condition evaluating false, and the caller keeps its own // routing. // @@ -635,16 +645,8 @@ func (s *Store) Dispatch(input DispatchInput) (Decision, bool) { continue } a := pb.Actions[0] - if a.When != nil { - val, err := a.env.Eval(a.When, ctx) - if err != nil { - // J3: evaluation error -> false (skip), caller logs. - continue - } - b, ok := val.(bool) - if !ok || !b { - continue - } + if !a.whenHolds(ctx) { + continue } return Decision{PlaybookID: pb.ID, Version: pb.Version, Workflow: a.Workflow, ActionID: a.ID, ActionPosition: 1}, true } @@ -662,9 +664,8 @@ func evalContext(input DispatchInput) expr.Context { } // EvalArgs evaluates a compiled action's args against the dispatch context, -// returning the resulting name->value map. No shipped position consumes args -// yet; the consumer arrives with the first side-effecting position (t2db.31). -// An action declaring no args evaluates to an empty map, nil-program entries +// returning the resulting name->value map. Run evaluates args the same way +// against a context that also carries earlier actions' results. An action declaring no args evaluates to an empty map, nil-program entries // are skipped, and the first evaluation error is returned. Nil-receiver-safe // for the pre-load composition phase. // @@ -673,10 +674,29 @@ func evalContext(input DispatchInput) expr.Context { // meaningful substitute and is returned to the caller to abort the dispatch. // Ratifying that rule against a real consumer is tracked by archie-core-1h05. func (s *Store) EvalArgs(a Action, input DispatchInput) (map[string]any, error) { - if s == nil || len(a.Args) == 0 { + if s == nil { return map[string]any{}, nil } - ctx := evalContext(input) + return a.evalArgs(evalContext(input)) +} + +// whenHolds reports whether the action's `when` is absent or evaluates to +// true. An evaluation error is false (J3: skip, caller logs). +func (a Action) whenHolds(ctx expr.Context) bool { + if a.When == nil { + return true + } + val, err := a.env.Eval(a.When, ctx) + if err != nil { + return false + } + b, ok := val.(bool) + return ok && b +} + +// evalArgs evaluates the action's args against ctx in key order, returning +// the first evaluation error. +func (a Action) evalArgs(ctx expr.Context) (map[string]any, error) { out := make(map[string]any, len(a.Args)) keys := make([]string, 0, len(a.Args)) for key := range a.Args { diff --git a/internal/domain/eda/playbook/playbook_test.go b/internal/domain/eda/playbook/playbook_test.go index ebe01b0b0..9e2afc19c 100644 --- a/internal/domain/eda/playbook/playbook_test.go +++ b/internal/domain/eda/playbook/playbook_test.go @@ -30,7 +30,7 @@ func writeFile(t *testing.T, dir, name, content string) string { // tests: *module.ModuleRegistry satisfies playbook.KindSchemas. It consults // the built-in kind registry, so no module file needs to be installed for the // schema lookup to work. -func testSchemas(t *testing.T) KindSchemas { +func testSchemas(t *testing.T) Modules { t.Helper() return module.New() } diff --git a/internal/domain/eda/playbook/run.go b/internal/domain/eda/playbook/run.go new file mode 100644 index 000000000..7fe2dcf7e --- /dev/null +++ b/internal/domain/eda/playbook/run.go @@ -0,0 +1,72 @@ +package playbook + +import ( + "context" + "errors" + "fmt" + "log/slog" + "strconv" + + "github.com/samcharles93/archie-core/internal/domain/eda/expr" + "github.com/samcharles93/archie-core/internal/domain/storecontract" +) + +// Run executes every action playbook whose trigger matches input, in load +// order (docs/prds/action-playbook-run.md). Each action goes through +// InvokeOnce, so it fires at most once per (playbook, version, event, +// action). A failure stops only its own playbook; the rest still run, and +// the failures are returned joined, each naming its playbook and action. +func (s *Store) Run(ctx context.Context, ledger storecontract.PlaybookDispatcher, log *slog.Logger, input DispatchInput) error { + if s == nil { + return nil + } + var errs []error + for _, pb := range s.Playbooks { + if !pb.IsActionPlaybook() || !pb.Match(input) { + continue + } + if err := s.runPlaybook(ctx, ledger, log, pb, input); err != nil { + errs = append(errs, fmt.Errorf("playbook %s: %w", pb.ID, err)) + } + } + return errors.Join(errs...) +} + +// runPlaybook runs one playbook's actions in order against one evaluation +// context, so a later action reads an earlier one's result. +func (s *Store) runPlaybook(ctx context.Context, ledger storecontract.PlaybookDispatcher, log *slog.Logger, pb *Playbook, input DispatchInput) error { + evalCtx := expr.Context{Event: input.Event, Actions: map[string]map[string]any{}} + for i, a := range pb.Actions { + label := a.ID + if label == "" { + label = "#" + strconv.Itoa(i+1) + } + if !a.whenHolds(evalCtx) { + continue + } + args, err := a.evalArgs(evalCtx) + if err != nil { + return fmt.Errorf("action %s: %w", label, err) + } + var raw map[string]any + decision := Decision{PlaybookID: pb.ID, Version: pb.Version, ActionID: a.ID, ActionPosition: i + 1} + invoked := false + err = InvokeOnce(ctx, ledger, log, decision, input, func(ctx context.Context) error { + invoked = true + raw, err = s.modules.Invoke(ctx, a.Kind, args) + return err + }) + if err != nil { + return fmt.Errorf("action %s: %w", label, err) + } + if !invoked || a.ID == "" { + continue + } + result, err := s.modules.DecodeResult(a.Kind, raw) + if err != nil { + return fmt.Errorf("action %s: %w", label, err) + } + evalCtx.Actions[a.ID] = map[string]any{"result": result} + } + return nil +} diff --git a/internal/domain/eda/playbook/run_test.go b/internal/domain/eda/playbook/run_test.go new file mode 100644 index 000000000..8d599dfc0 --- /dev/null +++ b/internal/domain/eda/playbook/run_test.go @@ -0,0 +1,143 @@ +package playbook + +import ( + "context" + "errors" + "log/slog" + "slices" + "strings" + "testing" + + "github.com/samcharles93/archie-core/internal/domain/eda/module" +) + +// recordingModules is the real kind-schema source with a fake invoker: it +// records each log message and fails any message listed in fail. +type recordingModules struct { + *module.ModuleRegistry + messages []string + fail map[string]bool +} + +func (m *recordingModules) Invoke(_ context.Context, kind string, args map[string]any) (map[string]any, error) { + msg, _ := args["message"].(string) + m.messages = append(m.messages, msg) + if m.fail[msg] { + return nil, errors.New("invoke failed") + } + return map[string]any{"written": true, "level": "info"}, nil +} + +func loadRunStore(t *testing.T, mods *recordingModules, document string) *Store { + t.Helper() + dir := t.TempDir() + writeFile(t, dir, "pb.yaml", document) + store, err := Load(dir, mods) + if err != nil { + t.Fatalf("Load: %v", err) + } + return store +} + +const twoLogActions = ` +trigger: + kind: bug +actions: + - id: first + position: module + kind: log + args: { message: '"first"' } + - id: second + position: module + kind: log + args: { message: '"second saw " + string(actions.first.result.written)' } +` + +func bugInput() DispatchInput { + return DispatchInput{Labels: []string{"bug"}, Kind: "bug", TaskID: "acme/widget#7", Event: map[string]any{}} +} + +func TestRunExecutesActionsInOrderAndFeedsResults(t *testing.T) { + mods := &recordingModules{ModuleRegistry: module.New()} + store := loadRunStore(t, mods, twoLogActions) + + if err := store.Run(t.Context(), &fakePlaybookLedger{}, slog.New(slog.DiscardHandler), bugInput()); err != nil { + t.Fatalf("Run: %v", err) + } + if want := []string{"first", "second saw true"}; !slices.Equal(mods.messages, want) { + t.Fatalf("invoked messages = %q, want %q", mods.messages, want) + } +} + +func TestRunInvokesEachActionOncePerEvent(t *testing.T) { + mods := &recordingModules{ModuleRegistry: module.New()} + store := loadRunStore(t, mods, twoLogActions) + ledger := &fakePlaybookLedger{} + log := slog.New(slog.DiscardHandler) + + if err := store.Run(t.Context(), ledger, log, bugInput()); err != nil { + t.Fatalf("Run: %v", err) + } + // The repeat skips "first" as a duplicate, so it has no result and + // "second" cannot evaluate its args: the playbook stops there + // (action-playbook-run.md). Neither action is invoked again. + _ = store.Run(t.Context(), ledger, log, bugInput()) + if len(mods.messages) != 2 { + t.Fatalf("invoked messages = %q, want each action once", mods.messages) + } +} + +func TestRunSkipsAnActionWhoseWhenIsFalse(t *testing.T) { + mods := &recordingModules{ModuleRegistry: module.New()} + store := loadRunStore(t, mods, ` +trigger: + kind: bug +actions: + - position: module + kind: log + when: 'event.kind == "feature"' + args: { message: '"skipped"' } + - position: module + kind: log + args: { message: '"ran"' } +`) + if err := store.Run(t.Context(), &fakePlaybookLedger{}, slog.New(slog.DiscardHandler), DispatchInput{ + Labels: []string{"bug"}, Kind: "bug", TaskID: "acme/widget#7", Event: map[string]any{"kind": "bug"}, + }); err != nil { + t.Fatalf("Run: %v", err) + } + if want := []string{"ran"}; !slices.Equal(mods.messages, want) { + t.Fatalf("invoked messages = %q, want %q", mods.messages, want) + } +} + +func TestRunStopsThePlaybookOnAnInvokeError(t *testing.T) { + mods := &recordingModules{ModuleRegistry: module.New(), fail: map[string]bool{"first": true}} + store := loadRunStore(t, mods, twoLogActions) + + err := store.Run(t.Context(), &fakePlaybookLedger{}, slog.New(slog.DiscardHandler), bugInput()) + if err == nil || !strings.Contains(err.Error(), "pb.yaml") || !strings.Contains(err.Error(), "first") { + t.Fatalf("Run error = %v, want one naming the playbook and the action", err) + } + if want := []string{"first"}; !slices.Equal(mods.messages, want) { + t.Fatalf("invoked messages = %q, want the run stopped at %q", mods.messages, want) + } +} + +func TestRunIgnoresNonMatchingAndWorkflowPlaybooks(t *testing.T) { + mods := &recordingModules{ModuleRegistry: module.New()} + store := loadRunStore(t, mods, twoLogActions) + input := bugInput() + input.Labels, input.Kind = []string{"feature"}, "feature" + + if err := store.Run(t.Context(), &fakePlaybookLedger{}, slog.New(slog.DiscardHandler), input); err != nil { + t.Fatalf("Run: %v", err) + } + workflowStore := loadRunStore(t, mods, "trigger:\n kind: bug\nactions:\n - position: workflow\n workflow: tdd\n") + if err := workflowStore.Run(t.Context(), &fakePlaybookLedger{}, slog.New(slog.DiscardHandler), bugInput()); err != nil { + t.Fatalf("Run: %v", err) + } + if len(mods.messages) != 0 { + t.Fatalf("invoked messages = %q, want none", mods.messages) + } +}