Files
archie/docs/prds/playbook-expression-syntax.md
T
sam cd3d684b49 docs(eda): point the expression PRD at its unbuilt decisions
Three of its decisions have no implementation: J1 action ids, J2 args as CEL,
and generated Result schemas declared to CEL for field-level result typing.
That is why it stays Approved rather than Finalised, so the header names the
beads that close the gap.
2026-09-22 01:57:00 +10:00

274 lines
14 KiB
Markdown

# Playbook data-flow and condition syntax
**Status:** Approved
**Date:** 2026-09-03
**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`
**Beads issue:** `archie-core-t2db.14` (expression environment).
Unbuilt decisions: `archie-core-t2db.25` (J1 action ids),
`archie-core-t2db.26` (J2 args as CEL),
`archie-core-t2db.27` (Result schemas declared to CEL).
**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression
mechanism covers both an action's `when` condition and its `args` values.
## The requirement, stated directly
A playbook action is gated and parametrised by runtime data: (a) a
condition deciding whether the action runs; (b) a later action's `args`
values referencing an earlier action's `Result` and the triggering
event's fields. Both are **CEL expressions**, evaluated against one
evaluation context. There is no separate interpolation syntax: CEL
reads nested maps and fields natively, so `args` values that need data
are written as CEL, and only plain literal args remain plain YAML
scalars.
Both halves are read-only: expressions may read context values and
compute a result from them, never mutate, never call host functions,
never perform I/O. CEL enforces this structurally (no side effects,
linear evaluation when macros are bounded, cost-limited).
```yaml
actions:
- position: module
kind: log
args:
message: '"build finished"' # literal
when: 'event.label == "bugfix"' # gate on event field
- position: module
kind: notify
args:
message: '"priority " + string(event.priority)'
when: 'actions.notify.result.delivered == true'
```
## Trust boundary of the expression
A playbook YAML is **operator-installed, in-process, daemon-privileged**
-- the same tier as `ModuleDir`/`PluginDir`/`SecretEngineDir`
(module-position.md's trust-boundary section): loaded from a configured
directory at startup, never from a webhook body or task worktree. The
schema-by-example flow uses live events to *design against*; the saved
playbook is operator-authored file content in an operator-configured
location.
Two consequences:
1. The expression string is **trusted**. Operator-authored config, not
attacker input; no sandboxing against a hostile expression string is
required. If a future playbook source is less trusted (auto-generated
bindings from unauthenticated captures), that is the playbook-file
trust tier moving -- the loader's acceptance check is the right
place, not the evaluator.
2. The **data the expression reads is NOT trusted** (decoded webhook
body, prior results through the same pipeline). Robustness against
hostile data is required: no panic on wrong-typed/missing fields,
no unbounded cost on deep/cyclic maps, bounded evaluation time. CEL
provides this structurally (cost limits, linear non-Turing-complete
evaluation, error values instead of panics -- verified in the
vetting record below).
## The gate precedent is deliberately NOT copied
`internal/gate/gateeval` evaluates arbitrary interpreted Go
(`.archie/gate.go`) as the repo's conditional-logic precedent. That is
the wrong precedent for playbook conditions because this document's
premise is: **YAML is pure orchestration data; Yaegi is where
imperative logic lives** (Problem section; module-position.md's
"no generic hook" section). Yaegi snippets in YAML would move
imperative logic into the data layer -- the inversion this design
exists to prevent -- and make playbooks un-lintable.
The line: conditions are **declarative predicates over runtime data,
not programs**. CEL is non-Turing-complete and side-effect-free, so it
cannot become a program; a condition that needs real logic gets a
Module kind (Yaegi), referenced by the action, not a stronger
expression body.
## Plugin-engine-rule interaction
The plugin engine rule forbids a generic `Module` interface with an
`any` payload; it exists to keep static implementation contracts
typed. Evaluating CEL against runtime `map[string]any` is **not the
same problem**: the rule concerns *implementation contracts* (static
operations a capability family exposes), not *runtime data values*
(legitimately dynamic -- an event payload's shape is unknown until it
arrives). The Module contract stays fully typed (generated
`Args`/`Result` schemas); CEL is a read-only data reader under the
playbook engine family, not a new capability contract. Stated once.
## Minimum viable evaluation context (what expressions see)
One flat context, namespaced from the playbook run, built at dispatch
time:
| Name | Type | Source |
|---|---|---|
| `event` | `map(string, dyn)` | The triggering event's decoded payload (webhook body / forge issue / schedule tick). Field access via `event.<field>`, map access via `event["field"]`, presence via `has(event.<field>)`. |
| `actions` | `map(string, dyn)` | Previous actions' results, keyed by the action's `id` as declared in the playbook (`actions.<id>.result.<field>`), so a later action reads an earlier one's `Result` map regardless of position. The current action and later actions are not present. |
Expressions may also read literal-only state (numbers, strings,
booleans) directly. CEL's `has()` macro covers the missing-key case
(`has(event.labels)` -> bool), which replaces any bespoke `has`/
`contains` operator.
`actions.<id>.result` is the `map[string]any` `Module.Invoke` already
returns (`archie-core-t2db.13`), so no result-shape change is needed for data flow;
channel/forge kinds' results are declared to the same shape.
Judgement call J1: `action` id is a required field on every action
that later actions reference; ids are validated at load (unique,
stable-identifier-shaped), and a `when`/`args` expression referencing
an unknown `actions.<id>` is a **load failure**, not a runtime miss --
the same reject-at-load rule as collisions (see below).
## Load-time validation (consistent with the collision rule)
The collision-handling rule is: schema defines the accepted message;
anything outside it is rejected at load, logged plainly, reported to
the caller -- never deferred to runtime. Expressions are validated the
same way, at playbook load:
- **Syntax**: every `when` and every CEL-typed `args` value is parsed
and checked by CEL's checker when the playbook is loaded. A syntax
or type error is a reported load failure (the playbook is dropped and
the error goes to the caller), not a runtime surprise.
- **Context conformance**: with the context declared to CEL
(`event` as `map(string, dyn)`, `actions` as `map(string, dyn)`),
unknown top-level identifiers are rejected at compile time.
Field-level typing inside `dyn` values is deferred to runtime by
CEL's design (verified: `data.items.lenght` with `data` dyn compiles
clean and errors at eval) -- so the loader declares as much type as
the schema provides (the action kinds' generated `Result` structs
become CEL type declarations at load, giving field-level rejection
for result reads), and `event` stays `dyn` because its shape is
unknown by design (schema-by-example).
- **Names**: unknown `actions.<id>` references and unknown roots are
compile-time/lint-visible errors (J1).
This means the lint tool and the daemon share one checker
-- same single-source argument as the playbook loaders.
## Candidate comparison
| Candidate | Maintenance | Safety surface | Cost (measured) | Verdict |
|---|---|---|---|---|
| **CEL (cel.dev/cel-go)** | Actively maintained (google/cel, monthly releases) | Non-Turing-complete, linear eval, `CostLimit`/`CostTracking`/`ParserRecursionLimit`, no side effects, error values not panics | **+13.0MB** (2.34MB -> 15.25MB) | **Chosen** |
| `expr-lang/expr` v1.17.8 | Actively maintained | Side-effect-free, `DisableAllBuiltins`, depth limits (CVE fixed w/ tests) | +3.9MB (2.34MB -> 6.28MB) | Not chosen: two safety CVEs this cycle; smaller ecosystem; no protobuf/type-model fit with future schema-gen |
| `gval` v1.2.3 | Maintained, slower | No builtin-restriction surface; extras (ternary/`??`) not wanted | +3.0MB (2 transitive deps) | Not chosen |
| `govaluate` | **ARCHIVED** (author archived 2024) | n/a | n/a | Not chosen: dead |
| `starlark-go` | Maintained (Bazel) | Turing-complete with step limits | large (full language) | Not chosen: whole-language surface |
## Vetting record
Source reviewed at HEAD:
- **Repo hygiene**: clean. No binaries/media in tree (the only
non-source artifacts are expected generated protobuf files); 16-line
`.gitignore`.
- **Hard-boundary correctness**: the place correctness is hard for an
expression evaluator is bounding evaluation and surviving hostile
input. Dedicated regression tests exist and pass at HEAD:
`TestCostLimit`, `TestCostTracking*`, `TestParserRecursionLimit`,
and a large conformance suite. Verified by execution: evaluation
against a 2000-deep nested map under `CostLimit` completes in ~10
microseconds and returns an error value for a missing key -- no
panic, no unbounded walk.
- **Dependency footprint**: antlr (parser), protobuf + genproto (CEL's
type model), `cel.dev/expr` (CEL spec types), yaml.v3. All
mainstream, canonical, vendored widely by Google-adjacent projects.
No runtime-impure third-party code; the REPL-only deps (readline,
tview, tcell) are not in the library import graph.
- **Security record**: no advisories on record for `cel.dev/cel-go`
(GitHub Advisory Database).
- **Module path**: migrated to `cel.dev/cel-go` (v0.32.0 is current);
the old `github.com/google/cel-go` path is the pre-migration identity.
Pin `cel.dev/cel-go`.
- **Binary size cost**: +13.0MB is the largest of the candidates
(measured: baseline 2.34MB -> 15.25MB with CEL). Accepted because:
the daemon already embeds a large Yaegi runtime; CEL's type model is
the load-bearing win (compile-time rejection of result-path typos,
matching the reject-at-load rule); and the alternative bespoke
grammar would re-implement CEL's checker to get the same guarantee.
Flagged for sign-off: if 13MB is unacceptable for a specific
deployment target, the fallback is a bespoke grammar with a
re-implemented subset of this checker -- a strictly worse trade.
## Why bespoke was not sufficient
A bespoke grammar for the shipped cases (path read + comparison + bool
composition) is small, but the load-time-validation requirement makes
the comparison: the reject-at-load rule requires *field-level* static
checking of result paths against generated `Result` schemas, which is
CEL's type-checker's job. Re-implementing that on a hand-rolled
grammar means re-implementing a type system -- more code than the
grammar itself, with worse diagnostics, no conformance suite, and no
community surface for future expression needs (even declarative ones).
The bespoke option only wins on binary size, and the size cost of CEL
is measured and bounded.
## Judgment calls (flagged for sign-off)
- **J1 -- `id` required on referenced actions, unknown id = load
failure.** Matches the reject-at-load rule. Alternative considered: an alternative
is positional `actions[0]` access, which breaks on reorder; named is
the operator-friendlier shape.
- **J2 -- every `args` value is evaluated as a CEL expression.** No
split between "literal args" and "expression args": a plain string
value is a CEL string literal, a number is a CEL number literal, and
any value that reads context data is written in CEL. One mechanism,
no interpolation marker, no second syntax to learn. Alternative considered: cEL
string literals must be quoted inside YAML (e.g. `message:
'"build finished"'`), which is slightly noisy for mostly-literal
args; accepted for uniformity -- the linter catches quoting mistakes
at load.
- **J3 -- condition failure semantics.** A condition that errors at
runtime (missing/wrong-typed field) evaluates to **false**: the
action is skipped, logged, run continues. This is the simplest
candidate consistent with gap 1's bias against clever recovery;
gap 1 (`archie-core-t2db.18`) commits
stop-on-first-failure, so per-action control is not coming and J3
stands. Alternative considered: cEL separates
false from error; we treat evaluation error as false + log.
- **J4 -- `event` stays `dyn` (schema-by-example).** Load-time we
cannot know event field types; declaring them typed would reject
valid plays. Result reads get field-level checking via generated
schema declarations. Alternative considered: when schema-by-example ships, tighten
event field declarations from captured samples.
- **J5 -- cost limit default.** A `CostLimit` at load time (e.g.
100,000) is applied to every expression; the linter and daemon use
the same value so they agree. Alternative considered: exact number is a constant to
tune at first use.
## Interaction with the execution-time gaps
This resolution does not resolve the execution-time gaps (both are
resolved separately: gap 1 by `archie-core-t2db.18`, gap 2 by the
gap-2 decision). Conditions make gap 1
concrete (condition-failure vs. action-failure semantics, J3) and
CEL's read-only result access makes gap 2's keying derivation
readable (an idempotency key expression can read event fields) -- both
flagged as the gaps' first real exercise. Gap 2's exercise is
subsequently solved by that decision: the
answer is a fixed structural derivation, and idempotency keys are
deliberately NOT CEL expressions (see the "No CEL in the key"
passage).
## Implementation placement (for the follow-up slice)
- New `internal/domain/eda/expr` package: builds the CEL environment
(context declarations, generated `Result` schema types as CEL type
declarations, cost limit), compiles/validates playbook expressions
at load, evaluates them at dispatch. Domain layer; no infrastructure
imports; owned by the playbook engine family. One new dependency:
`cel.dev/cel-go` (pinned), `go.mod` otherwise unchanged.
- Consumption: the playbook event coordinator (open question 4's
loader) evaluates `when` before dispatch and
evaluates `args` values before `Module.Invoke`; the lint tool
(`archie-core-t2db.12`) shares the same compile/validate path for author-time
diagnostics.
- Schema integration: action kinds' generated `Result`/`Args` structs
(`archie-core-t2db.13`'s `logextract` pattern) are declared to CEL at load so
`actions.<id>.result.<field>` type-checks; `args` expressions
type-check against the target kind's `Args` schema before the
playbook is accepted.