docs(eda): ratify args evaluation-failure semantics (J6)
An args value that cannot be evaluated aborts its playbook run: the action and every later action in that playbook do not run, other playbooks matching the event still run, and the task is unaffected. J3's skip-and-continue rule stays specific to `when`, because a predicate has a meaningful false and data does not. Ratified now that the action runner is the production consumer, and locked by a call-site test that fails if the rule is weakened to a skip. Drop the stale "unbuilt decisions" header line, since J1, J2 and the Result schema declarations have shipped.
This commit is contained in:
4 files changed
+46
-6
No files matched your search
@@ -406,7 +406,7 @@
|
||||
"url": "/dev/prds/playbook-expression-syntax/",
|
||||
"title": "Playbook data-flow and condition syntax",
|
||||
"section": "PRDs",
|
||||
"body": "# Playbook data-flow and condition syntax\n\n**Status:** Approved\n**Date:** 2026-09-03\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-t2db.14` (expression environment).\nUnbuilt decisions: `archie-core-t2db.25` (J1 action ids),\n`archie-core-t2db.26` (J2 args as CEL),\n`archie-core-t2db.27` (Result schemas declared to CEL).\n\n**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression\nmechanism covers both an action's `when` condition and its `args` values.\n\n## The requirement, stated directly\n\nA playbook action is gated and parametrised by runtime data: (a) a condition\ndeciding whether the action runs; (b) a later action's `args` values referencing\nan earlier action's `Result` and the triggering event's fields. Both are **CEL\nexpressions**, evaluated against one evaluation context. There is no separate\ninterpolation syntax: CEL reads nested maps and fields natively, so `args`\nvalues that need data are written as CEL, and only plain literal args remain\nplain YAML scalars.\n\nBoth halves are read-only: expressions may read context values and compute a\nresult from them, never mutate, never call host functions, never perform I/O.\nCEL enforces this structurally (no side effects, linear evaluation when macros\nare bounded, cost-limited).\n\n```yaml\nactions:\n - id: build\n position: module\n kind: log\n args:\n message: '\"build finished\"' # literal\n when: 'event.label == \"bugfix\"' # gate on event field\n - position: module\n kind: log\n args:\n message: '\"done: \" + string(actions.build.result.written)'\n when: \"actions.build.result.written == true\"\n```\n\n## Trust boundary of the expression\n\nA playbook YAML is **operator-installed, in-process, daemon-privileged** -- the\nsame tier as `ModuleDir`/`PluginDir`/`SecretEngineDir` (module-position.md's\ntrust-boundary section): loaded from a configured directory at startup, never\nfrom a webhook body or task worktree. The schema-by-example flow uses live\nevents to _design against_; the saved playbook is operator-authored file content\nin an operator-configured location.\n\nTwo consequences:\n\n1. The expression string is **trusted**. Operator-authored config, not attacker\n input; no sandboxing against a hostile expression string is required. If a\n future playbook source is less trusted (auto-generated bindings from\n unauthenticated captures), that is the playbook-file trust tier moving -- the\n loader's acceptance check is the right place, not the evaluator.\n2. The **data the expression reads is NOT trusted** (decoded webhook body, prior\n results through the same pipeline). Robustness against hostile data is\n required: no panic on wrong-typed/missing fields, no unbounded cost on\n deep/cyclic maps, bounded evaluation time. CEL provides this structurally\n (cost limits, linear non-Turing-complete evaluation, error values instead of\n panics -- verified in the vetting record below).\n\n## The gate precedent is deliberately NOT copied\n\n`internal/gate/gateeval` evaluates arbitrary interpreted Go (`.archie/gate.go`)\nas the repo's conditional-logic precedent. That is the wrong precedent for\nplaybook conditions because this document's premise is: **YAML is pure\norchestration data; Yaegi is where imperative logic lives** (Problem section;\nmodule-position.md's \"no generic hook\" section). Yaegi snippets in YAML would\nmove imperative logic into the data layer -- the inversion this design exists to\nprevent -- and make playbooks un-lintable.\n\nThe line: conditions are **declarative predicates over runtime data, not\nprograms**. CEL is non-Turing-complete and side-effect-free, so it cannot become\na program; a condition that needs real logic gets a Module kind (Yaegi),\nreferenced by the action, not a stronger expression body.\n\n## Plugin-engine-rule interaction\n\nThe plugin engine rule forbids a generic `Module` interface with an `any`\npayload; it exists to keep static implementation contracts typed. Evaluating CEL\nagainst runtime `map[string]any` is **not the same problem**: the rule concerns\n_implementation contracts_ (static operations a capability family exposes), not\n_runtime data values_ (legitimately dynamic -- an event payload's shape is\nunknown until it arrives). The Module contract stays fully typed (generated\n`Args`/`Result` schemas); CEL is a read-only data reader under the playbook\nengine family, not a new capability contract. Stated once.\n\n## Minimum viable evaluation context (what expressions see)\n\nOne flat context, namespaced from the playbook run, built at dispatch time:\n\n| Name | Type | Source Line truncated
|
||||
"body": "# Playbook data-flow and condition syntax\n\n**Status:** Approved\n**Date:** 2026-09-03\n**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`\n**Beads issue:** `archie-core-t2db.14` (expression environment).\n\n**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression\nmechanism covers both an action's `when` condition and its `args` values.\n\n## The requirement, stated directly\n\nA playbook action is gated and parametrised by runtime data: (a) a condition\ndeciding whether the action runs; (b) a later action's `args` values referencing\nan earlier action's `Result` and the triggering event's fields. Both are **CEL\nexpressions**, evaluated against one evaluation context. There is no separate\ninterpolation syntax: CEL reads nested maps and fields natively, so `args`\nvalues that need data are written as CEL, and only plain literal args remain\nplain YAML scalars.\n\nBoth halves are read-only: expressions may read context values and compute a\nresult from them, never mutate, never call host functions, never perform I/O.\nCEL enforces this structurally (no side effects, linear evaluation when macros\nare bounded, cost-limited).\n\n```yaml\nactions:\n - id: build\n position: module\n kind: log\n args:\n message: '\"build finished\"' # literal\n when: 'event.label == \"bugfix\"' # gate on event field\n - position: module\n kind: log\n args:\n message: '\"done: \" + string(actions.build.result.written)'\n when: \"actions.build.result.written == true\"\n```\n\n## Trust boundary of the expression\n\nA playbook YAML is **operator-installed, in-process, daemon-privileged** -- the\nsame tier as `ModuleDir`/`PluginDir`/`SecretEngineDir` (module-position.md's\ntrust-boundary section): loaded from a configured directory at startup, never\nfrom a webhook body or task worktree. The schema-by-example flow uses live\nevents to _design against_; the saved playbook is operator-authored file content\nin an operator-configured location.\n\nTwo consequences:\n\n1. The expression string is **trusted**. Operator-authored config, not attacker\n input; no sandboxing against a hostile expression string is required. If a\n future playbook source is less trusted (auto-generated bindings from\n unauthenticated captures), that is the playbook-file trust tier moving -- the\n loader's acceptance check is the right place, not the evaluator.\n2. The **data the expression reads is NOT trusted** (decoded webhook body, prior\n results through the same pipeline). Robustness against hostile data is\n required: no panic on wrong-typed/missing fields, no unbounded cost on\n deep/cyclic maps, bounded evaluation time. CEL provides this structurally\n (cost limits, linear non-Turing-complete evaluation, error values instead of\n panics -- verified in the vetting record below).\n\n## The gate precedent is deliberately NOT copied\n\n`internal/gate/gateeval` evaluates arbitrary interpreted Go (`.archie/gate.go`)\nas the repo's conditional-logic precedent. That is the wrong precedent for\nplaybook conditions because this document's premise is: **YAML is pure\norchestration data; Yaegi is where imperative logic lives** (Problem section;\nmodule-position.md's \"no generic hook\" section). Yaegi snippets in YAML would\nmove imperative logic into the data layer -- the inversion this design exists to\nprevent -- and make playbooks un-lintable.\n\nThe line: conditions are **declarative predicates over runtime data, not\nprograms**. CEL is non-Turing-complete and side-effect-free, so it cannot become\na program; a condition that needs real logic gets a Module kind (Yaegi),\nreferenced by the action, not a stronger expression body.\n\n## Plugin-engine-rule interaction\n\nThe plugin engine rule forbids a generic `Module` interface with an `any`\npayload; it exists to keep static implementation contracts typed. Evaluating CEL\nagainst runtime `map[string]any` is **not the same problem**: the rule concerns\n_implementation contracts_ (static operations a capability family exposes), not\n_runtime data values_ (legitimately dynamic -- an event payload's shape is\nunknown until it arrives). The Module contract stays fully typed (generated\n`Args`/`Result` schemas); CEL is a read-only data reader under the playbook\nengine family, not a new capability contract. Stated once.\n\n## Minimum viable evaluation context (what expressions see)\n\nOne flat context, namespaced from the playbook run, built at dispatch time:\n\n| Name | Type | Source |\n| --------- | ------------------------ | ---------------------------------------------------------------------------------Line truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/pr-review-agent/",
|
||||
|
||||
@@ -4,9 +4,6 @@
|
||||
**Date:** 2026-09-03
|
||||
**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`
|
||||
**Beads issue:** `archie-core-t2db.14` (expression environment).
|
||||
Unbuilt decisions: `archie-core-t2db.25` (J1 action ids),
|
||||
`archie-core-t2db.26` (J2 args as CEL),
|
||||
`archie-core-t2db.27` (Result schemas declared to CEL).
|
||||
|
||||
**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression
|
||||
mechanism covers both an action's `when` condition and its `args` values.
|
||||
@@ -222,6 +219,14 @@ cost of CEL is measured and bounded.
|
||||
applied to every expression; the linter and daemon use the same value so they
|
||||
agree. Alternative considered: exact number is a constant to tune at first
|
||||
use.
|
||||
- **J6 -- `args` failure semantics.** An `args` value that cannot be evaluated
|
||||
aborts its playbook run: the action and every later action in that playbook do
|
||||
not run, because data has no false value to fall back on and an action run
|
||||
without its arguments is a silent one. Other playbooks matching the same event
|
||||
still run, and the task is unaffected: a playbook run never changes routing.
|
||||
The daemon log reports the failure, naming the task, the playbook and the
|
||||
action; no task event is written. Alternative considered: mirroring J3,
|
||||
rejected because a predicate has a meaningful false and data does not.
|
||||
|
||||
## Interaction with the execution-time gaps
|
||||
|
||||
|
||||
@@ -735,8 +735,8 @@ func evalContext(input DispatchInput) expr.Context {
|
||||
//
|
||||
// Asymmetry with `when`: `when` is a predicate, so an evaluation error is
|
||||
// false (J3: skip + log); `args` is data, so an evaluation error has no
|
||||
// meaningful substitute and is returned to the caller to abort the dispatch.
|
||||
// Ratifying that rule against a real consumer is tracked by archie-core-1h05.
|
||||
// meaningful substitute and is returned to the caller to abort the playbook run
|
||||
// (J6, docs/prds/playbook-expression-syntax.md).
|
||||
func (s *Store) EvalArgs(a Action, input DispatchInput) (map[string]any, error) {
|
||||
if s == nil {
|
||||
return map[string]any{}, nil
|
||||
|
||||
@@ -124,6 +124,41 @@ func TestRunStopsThePlaybookOnAnInvokeError(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunAbortsThePlaybookOnAnArgsEvaluationFailure pins J6
|
||||
// (docs/prds/playbook-expression-syntax.md): an args value that cannot be
|
||||
// evaluated stops its playbook before the action runs, unlike a `when` that
|
||||
// errors, which skips the action and continues (J3). The error names the
|
||||
// playbook and the action whose args failed.
|
||||
func TestRunAbortsThePlaybookOnAnArgsEvaluationFailure(t *testing.T) {
|
||||
mods := &recordingModules{ModuleRegistry: module.New()}
|
||||
store := loadRunStore(t, mods, `
|
||||
trigger:
|
||||
kind: bug
|
||||
actions:
|
||||
- id: first
|
||||
position: module
|
||||
kind: log
|
||||
args: { message: 'event.absent' }
|
||||
- id: second
|
||||
position: module
|
||||
kind: log
|
||||
args: { message: '"second"' }
|
||||
`)
|
||||
|
||||
err := store.Run(t.Context(), &fakePlaybookLedger{}, slog.New(slog.DiscardHandler), bugInput())
|
||||
if err == nil {
|
||||
t.Fatal("Run(args evaluation failure) = nil, want the playbook aborted")
|
||||
}
|
||||
for _, want := range []string{"pb.yaml", "first", "evaluate args"} {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("Run error = %q, want it to contain %q", err.Error(), want)
|
||||
}
|
||||
}
|
||||
if len(mods.messages) != 0 {
|
||||
t.Fatalf("invoked messages = %q, want none: the playbook aborts before the first action runs", mods.messages)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunIgnoresNonMatchingAndWorkflowPlaybooks(t *testing.T) {
|
||||
mods := &recordingModules{ModuleRegistry: module.New()}
|
||||
store := loadRunStore(t, mods, twoLogActions)
|
||||
|
||||
Reference in new issue
Block a user