Files
archie/docs/prds/playbook-expression-syntax.md
T
sam 459bbcd5fc
deploy / gate (push) Failing after 5m32s
deploy / build-and-push (push) Skipped
deploy / dist-zip (push) Skipped
docs(prds): reflow playbook-expression-syntax
2026-09-25 02:00:16 +10:00

253 lines
17 KiB
Markdown

# Playbook data-flow and condition syntax
**Status:** Approved
**Date:** 2026-09-03
**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`
**Beads issue:** `archie-core-t2db.14` (expression environment).
Unbuilt decisions: `archie-core-t2db.25` (J1 action ids),
`archie-core-t2db.26` (J2 args as CEL),
`archie-core-t2db.27` (Result schemas declared to CEL).
**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression
mechanism covers both an action's `when` condition and its `args` values.
## The requirement, stated directly
A playbook action is gated and parametrised by runtime data: (a) a condition
deciding whether the action runs; (b) a later action's `args` values referencing
an earlier action's `Result` and the triggering event's fields. Both are **CEL
expressions**, evaluated against one evaluation context. There is no separate
interpolation syntax: CEL reads nested maps and fields natively, so `args`
values that need data are written as CEL, and only plain literal args remain
plain YAML scalars.
Both halves are read-only: expressions may read context values and compute a
result from them, never mutate, never call host functions, never perform I/O.
CEL enforces this structurally (no side effects, linear evaluation when macros
are bounded, cost-limited).
```yaml
actions:
- id: build
position: module
kind: log
args:
message: '"build finished"' # literal
when: 'event.label == "bugfix"' # gate on event field
- position: module
kind: log
args:
message: '"done: " + string(actions.build.result.written)'
when: "actions.build.result.written == true"
```
## Trust boundary of the expression
A playbook YAML is **operator-installed, in-process, daemon-privileged** -- the
same tier as `ModuleDir`/`PluginDir`/`SecretEngineDir` (module-position.md's
trust-boundary section): loaded from a configured directory at startup, never
from a webhook body or task worktree. The schema-by-example flow uses live
events to _design against_; the saved playbook is operator-authored file content
in an operator-configured location.
Two consequences:
1. The expression string is **trusted**. Operator-authored config, not attacker
input; no sandboxing against a hostile expression string is required. If a
future playbook source is less trusted (auto-generated bindings from
unauthenticated captures), that is the playbook-file trust tier moving -- the
loader's acceptance check is the right place, not the evaluator.
2. The **data the expression reads is NOT trusted** (decoded webhook body, prior
results through the same pipeline). Robustness against hostile data is
required: no panic on wrong-typed/missing fields, no unbounded cost on
deep/cyclic maps, bounded evaluation time. CEL provides this structurally
(cost limits, linear non-Turing-complete evaluation, error values instead of
panics -- verified in the vetting record below).
## The gate precedent is deliberately NOT copied
`internal/gate/gateeval` evaluates arbitrary interpreted Go (`.archie/gate.go`)
as the repo's conditional-logic precedent. That is the wrong precedent for
playbook conditions because this document's premise is: **YAML is pure
orchestration data; Yaegi is where imperative logic lives** (Problem section;
module-position.md's "no generic hook" section). Yaegi snippets in YAML would
move imperative logic into the data layer -- the inversion this design exists to
prevent -- and make playbooks un-lintable.
The line: conditions are **declarative predicates over runtime data, not
programs**. CEL is non-Turing-complete and side-effect-free, so it cannot become
a program; a condition that needs real logic gets a Module kind (Yaegi),
referenced by the action, not a stronger expression body.
## Plugin-engine-rule interaction
The plugin engine rule forbids a generic `Module` interface with an `any`
payload; it exists to keep static implementation contracts typed. Evaluating CEL
against runtime `map[string]any` is **not the same problem**: the rule concerns
_implementation contracts_ (static operations a capability family exposes), not
_runtime data values_ (legitimately dynamic -- an event payload's shape is
unknown until it arrives). The Module contract stays fully typed (generated
`Args`/`Result` schemas); CEL is a read-only data reader under the playbook
engine family, not a new capability contract. Stated once.
## Minimum viable evaluation context (what expressions see)
One flat context, namespaced from the playbook run, built at dispatch time:
| Name | Type | Source |
| --------- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `event` | `map(string, dyn)` | The triggering event's decoded payload (webhook body / forge issue / schedule tick). Field access via `event.<field>`, map access via `event["field"]`, presence via `has(event.<field>)`. |
| `actions` | per-playbook object type | Previous actions' results, one field per declared action `id` (`actions.<id>`), each field typed `{ result: <KindResult> }` so a later action reads an earlier one's `Result` struct (`actions.<id>.result.<field>`) with field-level checking. The current action and later actions are not present. |
Expressions may also read literal-only state (numbers, strings, booleans)
directly. CEL's `has()` macro covers the missing-key case (`has(event.labels)`
-> bool), which replaces any bespoke `has`/ `contains` operator.
`actions.<id>.result` is the kind's hand-written `Result` struct:
`ModuleRegistry.DecodeResult` converts `Module.Invoke`'s flat `map[string]any`
(`archie-core-t2db.13`) back into that struct before a later expression reads
it, so the checker and the run read the same typed value.
Judgement call J1: `action` id is a required field on every action that later
actions reference; ids are validated at load (unique, stable-identifier-shaped),
and a `when`/`args` expression referencing an unknown `actions.<id>` is a **load
failure**, not a runtime miss -- the same reject-at-load rule as collisions (see
below).
## Load-time validation (consistent with the collision rule)
The collision-handling rule is: schema defines the accepted message; anything
outside it is rejected at load, logged plainly, reported to the caller -- never
deferred to runtime. Expressions are validated the same way, at playbook load:
- **Syntax**: every `when` and every CEL-typed `args` value is parsed and
checked by CEL's checker when the playbook is loaded. A syntax or type error
is a reported load failure (the playbook is dropped and the error goes to the
caller), not a runtime surprise.
- **Context conformance**: with the context declared to CEL (`event` as
`map(string, dyn)`, `actions` as a per-playbook object type with one typed
field per declared id), unknown top-level identifiers are rejected at compile
time. Field-level typing inside `dyn` values is deferred to runtime by CEL's
design (verified: `data.items.lenght` with `data` dyn compiles clean and
errors at eval) -- so the loader declares as much type as the schema provides
(the action kinds' generated `Result` structs become CEL type declarations at
load, giving field-level rejection for result reads), and `event` stays `dyn`
because its shape is unknown by design (schema-by-example).
- **Names**: unknown `actions.<id>` references and unknown roots are
compile-time/lint-visible errors (J1).
This means the lint tool and the daemon share one checker -- same single-source
argument as the playbook loaders.
## Candidate comparison
| Candidate | Maintenance | Safety surface | Cost (measured) | Verdict |
| ------------------------ | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| **CEL (cel.dev/cel-go)** | Actively maintained (google/cel, monthly releases) | Non-Turing-complete, linear eval, `CostLimit`/`CostTracking`/`ParserRecursionLimit`, no side effects, error values not panics | **+13.0MB** (2.34MB -> 15.25MB) | **Chosen** |
| `expr-lang/expr` v1.17.8 | Actively maintained | Side-effect-free, `DisableAllBuiltins`, depth limits (CVE fixed w/ tests) | +3.9MB (2.34MB -> 6.28MB) | Not chosen: two safety CVEs this cycle; smaller ecosystem; no protobuf/type-model fit with future schema-gen |
| `gval` v1.2.3 | Maintained, slower | No builtin-restriction surface; extras (ternary/`??`) not wanted | +3.0MB (2 transitive deps) | Not chosen |
| `govaluate` | **ARCHIVED** (author archived 2024) | n/a | n/a | Not chosen: dead |
| `starlark-go` | Maintained (Bazel) | Turing-complete with step limits | large (full language) | Not chosen: whole-language surface |
## Vetting record
Source reviewed at HEAD:
- **Repo hygiene**: clean. No binaries/media in tree (the only non-source
artifacts are expected generated protobuf files); 16-line `.gitignore`.
- **Hard-boundary correctness**: the place correctness is hard for an expression
evaluator is bounding evaluation and surviving hostile input. Dedicated
regression tests exist and pass at HEAD: `TestCostLimit`, `TestCostTracking*`,
`TestParserRecursionLimit`, and a large conformance suite. Verified by
execution: evaluation against a 2000-deep nested map under `CostLimit`
completes in ~10 microseconds and returns an error value for a missing key --
no panic, no unbounded walk.
- **Dependency footprint**: antlr (parser), protobuf + genproto (CEL's type
model), `cel.dev/expr` (CEL spec types), yaml.v3. All mainstream, canonical,
vendored widely by Google-adjacent projects. No runtime-impure third-party
code; the REPL-only deps (readline, tview, tcell) are not in the library
import graph.
- **Security record**: no advisories on record for `cel.dev/cel-go` (GitHub
Advisory Database).
- **Module path**: migrated to `cel.dev/cel-go` (v0.32.0 is current); the old
`cel.dev/cel-go/cel` path is the pre-migration identity. Pin `cel.dev/cel-go`.
- **Binary size cost**: +13.0MB is the largest of the candidates (measured:
baseline 2.34MB -> 15.25MB with CEL). Accepted because: the daemon already
embeds a large Yaegi runtime; CEL's type model is the load-bearing win
(compile-time rejection of result-path typos, matching the reject-at-load
rule); and the alternative bespoke grammar would re-implement CEL's checker to
get the same guarantee. Flagged for sign-off: if 13MB is unacceptable for a
specific deployment target, the fallback is a bespoke grammar with a
re-implemented subset of this checker -- a strictly worse trade.
## Why bespoke was not sufficient
A bespoke grammar for the shipped cases (path read + comparison + bool
composition) is small, but the load-time-validation requirement makes the
comparison: the reject-at-load rule requires _field-level_ static checking of
result paths against generated `Result` schemas, which is CEL's type-checker's
job. Re-implementing that on a hand-rolled grammar means re-implementing a type
system -- more code than the grammar itself, with worse diagnostics, no
conformance suite, and no community surface for future expression needs (even
declarative ones). The bespoke option only wins on binary size, and the size
cost of CEL is measured and bounded.
## Judgement calls (flagged for sign-off)
- **J1 -- `id` required on referenced actions, unknown id = load failure.**
Matches the reject-at-load rule. Alternative considered: an alternative is
positional `actions[0]` access, which breaks on reorder; named is the
operator-friendlier shape.
- **J2 -- every `args` value is evaluated as a CEL expression.** No split
between "literal args" and "expression args": a plain string value is a CEL
string literal, a number is a CEL number literal, and any value that reads
context data is written in CEL. One mechanism, no interpolation marker, no
second syntax to learn. Alternative considered: cEL string literals must be
quoted inside YAML (e.g. `message: '"build finished"'`), which is slightly
noisy for mostly-literal args; accepted for uniformity -- the linter catches
quoting mistakes at load.
- **J3 -- condition failure semantics.** A condition that errors at runtime
(missing/wrong-typed field) evaluates to **false**: the action is skipped,
logged, run continues. This is the simplest candidate consistent with gap 1's
bias against clever recovery; gap 1 (`archie-core-t2db.18`) commits
stop-on-first-failure, so per-action control is not coming and J3 stands.
Alternative considered: cEL separates false from error; we treat evaluation
error as false + log.
- **J4 -- `event` stays `dyn` (schema-by-example).** Load-time we cannot know
event field types; declaring them typed would reject valid plays. Result reads
get field-level checking via generated schema declarations. Alternative
considered: when schema-by-example ships, tighten event field declarations
from captured samples.
- **J5 -- cost limit default.** A `CostLimit` at load time (e.g. 100,000) is
applied to every expression; the linter and daemon use the same value so they
agree. Alternative considered: exact number is a constant to tune at first
use.
## Interaction with the execution-time gaps
This resolution does not resolve the execution-time gaps (both are resolved
separately: gap 1 by `archie-core-t2db.18`, gap 2 by the gap-2 decision).
Conditions make gap 1 concrete (condition-failure vs. action-failure semantics,
J3) and CEL's read-only result access makes gap 2's keying derivation readable
(an idempotency key expression can read event fields) -- both flagged as the
gaps' first real exercise. Gap 2's exercise is subsequently solved by that
decision: the answer is a fixed structural derivation, and idempotency keys are
deliberately NOT CEL expressions (see the "No CEL in the key" passage).
## Implementation placement (for the follow-up slice)
- New `internal/domain/eda/expr` package: builds the CEL environment (context
declarations, generated `Result` schema types as CEL type declarations, cost
limit), compiles/validates playbook expressions at load, evaluates them at
dispatch. Domain layer; no infrastructure imports; owned by the playbook
engine family. One new dependency: `cel.dev/cel-go` (pinned), `go.mod`
otherwise unchanged.
- Consumption: the playbook event coordinator (open question 4's loader)
evaluates `when` before dispatch and evaluates `args` values before
`Module.Invoke`; the lint tool (`archie-core-t2db.12`) shares the same
compile/validate path for author-time diagnostics.
- Schema integration: action kinds' generated `Result`/`Args` structs
(`archie-core-t2db.13`'s `logextract` pattern) are declared to CEL at load so
`actions.<id>.result.<field>` type-checks; `args` expressions type-check
against the target kind's `Args` schema before the playbook is accepted.