277 lines
14 KiB
Markdown
277 lines
14 KiB
Markdown
# Playbook data-flow and condition syntax
|
|
|
|
**Status:** Approved
|
|
**Date:** 2026-09-03
|
|
**Parent:** `docs/prds/eda-playbook-engine.md`, epic `archie-core-t2db`
|
|
**Beads issue:** `archie-core-t2db.14` (expression environment).
|
|
Unbuilt decisions: `archie-core-t2db.25` (J1 action ids),
|
|
`archie-core-t2db.26` (J2 args as CEL),
|
|
`archie-core-t2db.27` (Result schemas declared to CEL).
|
|
|
|
**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned. One expression
|
|
mechanism covers both an action's `when` condition and its `args` values.
|
|
|
|
## The requirement, stated directly
|
|
|
|
A playbook action is gated and parametrised by runtime data: (a) a
|
|
condition deciding whether the action runs; (b) a later action's `args`
|
|
values referencing an earlier action's `Result` and the triggering
|
|
event's fields. Both are **CEL expressions**, evaluated against one
|
|
evaluation context. There is no separate interpolation syntax: CEL
|
|
reads nested maps and fields natively, so `args` values that need data
|
|
are written as CEL, and only plain literal args remain plain YAML
|
|
scalars.
|
|
|
|
Both halves are read-only: expressions may read context values and
|
|
compute a result from them, never mutate, never call host functions,
|
|
never perform I/O. CEL enforces this structurally (no side effects,
|
|
linear evaluation when macros are bounded, cost-limited).
|
|
|
|
```yaml
|
|
actions:
|
|
- id: build
|
|
position: module
|
|
kind: log
|
|
args:
|
|
message: '"build finished"' # literal
|
|
when: 'event.label == "bugfix"' # gate on event field
|
|
- position: module
|
|
kind: log
|
|
args:
|
|
message: '"done: " + string(actions.build.result.written)'
|
|
when: 'actions.build.result.written == true'
|
|
```
|
|
|
|
## Trust boundary of the expression
|
|
|
|
A playbook YAML is **operator-installed, in-process, daemon-privileged**
|
|
-- the same tier as `ModuleDir`/`PluginDir`/`SecretEngineDir`
|
|
(module-position.md's trust-boundary section): loaded from a configured
|
|
directory at startup, never from a webhook body or task worktree. The
|
|
schema-by-example flow uses live events to *design against*; the saved
|
|
playbook is operator-authored file content in an operator-configured
|
|
location.
|
|
|
|
Two consequences:
|
|
|
|
1. The expression string is **trusted**. Operator-authored config, not
|
|
attacker input; no sandboxing against a hostile expression string is
|
|
required. If a future playbook source is less trusted (auto-generated
|
|
bindings from unauthenticated captures), that is the playbook-file
|
|
trust tier moving -- the loader's acceptance check is the right
|
|
place, not the evaluator.
|
|
2. The **data the expression reads is NOT trusted** (decoded webhook
|
|
body, prior results through the same pipeline). Robustness against
|
|
hostile data is required: no panic on wrong-typed/missing fields,
|
|
no unbounded cost on deep/cyclic maps, bounded evaluation time. CEL
|
|
provides this structurally (cost limits, linear non-Turing-complete
|
|
evaluation, error values instead of panics -- verified in the
|
|
vetting record below).
|
|
|
|
## The gate precedent is deliberately NOT copied
|
|
|
|
`internal/gate/gateeval` evaluates arbitrary interpreted Go
|
|
(`.archie/gate.go`) as the repo's conditional-logic precedent. That is
|
|
the wrong precedent for playbook conditions because this document's
|
|
premise is: **YAML is pure orchestration data; Yaegi is where
|
|
imperative logic lives** (Problem section; module-position.md's
|
|
"no generic hook" section). Yaegi snippets in YAML would move
|
|
imperative logic into the data layer -- the inversion this design
|
|
exists to prevent -- and make playbooks un-lintable.
|
|
|
|
The line: conditions are **declarative predicates over runtime data,
|
|
not programs**. CEL is non-Turing-complete and side-effect-free, so it
|
|
cannot become a program; a condition that needs real logic gets a
|
|
Module kind (Yaegi), referenced by the action, not a stronger
|
|
expression body.
|
|
|
|
## Plugin-engine-rule interaction
|
|
|
|
The plugin engine rule forbids a generic `Module` interface with an
|
|
`any` payload; it exists to keep static implementation contracts
|
|
typed. Evaluating CEL against runtime `map[string]any` is **not the
|
|
same problem**: the rule concerns *implementation contracts* (static
|
|
operations a capability family exposes), not *runtime data values*
|
|
(legitimately dynamic -- an event payload's shape is unknown until it
|
|
arrives). The Module contract stays fully typed (generated
|
|
`Args`/`Result` schemas); CEL is a read-only data reader under the
|
|
playbook engine family, not a new capability contract. Stated once.
|
|
|
|
## Minimum viable evaluation context (what expressions see)
|
|
|
|
One flat context, namespaced from the playbook run, built at dispatch
|
|
time:
|
|
|
|
| Name | Type | Source |
|
|
|---|---|---|
|
|
| `event` | `map(string, dyn)` | The triggering event's decoded payload (webhook body / forge issue / schedule tick). Field access via `event.<field>`, map access via `event["field"]`, presence via `has(event.<field>)`. |
|
|
| `actions` | per-playbook object type | Previous actions' results, one field per declared action `id` (`actions.<id>`), each field typed `{ result: <KindResult> }` so a later action reads an earlier one's `Result` struct (`actions.<id>.result.<field>`) with field-level checking. The current action and later actions are not present. |
|
|
|
|
Expressions may also read literal-only state (numbers, strings,
|
|
booleans) directly. CEL's `has()` macro covers the missing-key case
|
|
(`has(event.labels)` -> bool), which replaces any bespoke `has`/
|
|
`contains` operator.
|
|
|
|
`actions.<id>.result` is the kind's hand-written `Result` struct:
|
|
`ModuleRegistry.DecodeResult` converts `Module.Invoke`'s flat
|
|
`map[string]any` (`archie-core-t2db.13`) back into that struct before a later
|
|
expression reads it, so the checker and the run read the same typed value.
|
|
|
|
Judgement call J1: `action` id is a required field on every action
|
|
that later actions reference; ids are validated at load (unique,
|
|
stable-identifier-shaped), and a `when`/`args` expression referencing
|
|
an unknown `actions.<id>` is a **load failure**, not a runtime miss --
|
|
the same reject-at-load rule as collisions (see below).
|
|
|
|
## Load-time validation (consistent with the collision rule)
|
|
|
|
The collision-handling rule is: schema defines the accepted message;
|
|
anything outside it is rejected at load, logged plainly, reported to
|
|
the caller -- never deferred to runtime. Expressions are validated the
|
|
same way, at playbook load:
|
|
|
|
- **Syntax**: every `when` and every CEL-typed `args` value is parsed
|
|
and checked by CEL's checker when the playbook is loaded. A syntax
|
|
or type error is a reported load failure (the playbook is dropped and
|
|
the error goes to the caller), not a runtime surprise.
|
|
- **Context conformance**: with the context declared to CEL
|
|
(`event` as `map(string, dyn)`, `actions` as a per-playbook object type
|
|
with one typed field per declared id),
|
|
unknown top-level identifiers are rejected at compile time.
|
|
Field-level typing inside `dyn` values is deferred to runtime by
|
|
CEL's design (verified: `data.items.lenght` with `data` dyn compiles
|
|
clean and errors at eval) -- so the loader declares as much type as
|
|
the schema provides (the action kinds' generated `Result` structs
|
|
become CEL type declarations at load, giving field-level rejection
|
|
for result reads), and `event` stays `dyn` because its shape is
|
|
unknown by design (schema-by-example).
|
|
- **Names**: unknown `actions.<id>` references and unknown roots are
|
|
compile-time/lint-visible errors (J1).
|
|
|
|
This means the lint tool and the daemon share one checker
|
|
-- same single-source argument as the playbook loaders.
|
|
|
|
## Candidate comparison
|
|
|
|
| Candidate | Maintenance | Safety surface | Cost (measured) | Verdict |
|
|
|---|---|---|---|---|
|
|
| **CEL (cel.dev/cel-go)** | Actively maintained (google/cel, monthly releases) | Non-Turing-complete, linear eval, `CostLimit`/`CostTracking`/`ParserRecursionLimit`, no side effects, error values not panics | **+13.0MB** (2.34MB -> 15.25MB) | **Chosen** |
|
|
| `expr-lang/expr` v1.17.8 | Actively maintained | Side-effect-free, `DisableAllBuiltins`, depth limits (CVE fixed w/ tests) | +3.9MB (2.34MB -> 6.28MB) | Not chosen: two safety CVEs this cycle; smaller ecosystem; no protobuf/type-model fit with future schema-gen |
|
|
| `gval` v1.2.3 | Maintained, slower | No builtin-restriction surface; extras (ternary/`??`) not wanted | +3.0MB (2 transitive deps) | Not chosen |
|
|
| `govaluate` | **ARCHIVED** (author archived 2024) | n/a | n/a | Not chosen: dead |
|
|
| `starlark-go` | Maintained (Bazel) | Turing-complete with step limits | large (full language) | Not chosen: whole-language surface |
|
|
|
|
## Vetting record
|
|
|
|
Source reviewed at HEAD:
|
|
|
|
- **Repo hygiene**: clean. No binaries/media in tree (the only
|
|
non-source artifacts are expected generated protobuf files); 16-line
|
|
`.gitignore`.
|
|
- **Hard-boundary correctness**: the place correctness is hard for an
|
|
expression evaluator is bounding evaluation and surviving hostile
|
|
input. Dedicated regression tests exist and pass at HEAD:
|
|
`TestCostLimit`, `TestCostTracking*`, `TestParserRecursionLimit`,
|
|
and a large conformance suite. Verified by execution: evaluation
|
|
against a 2000-deep nested map under `CostLimit` completes in ~10
|
|
microseconds and returns an error value for a missing key -- no
|
|
panic, no unbounded walk.
|
|
- **Dependency footprint**: antlr (parser), protobuf + genproto (CEL's
|
|
type model), `cel.dev/expr` (CEL spec types), yaml.v3. All
|
|
mainstream, canonical, vendored widely by Google-adjacent projects.
|
|
No runtime-impure third-party code; the REPL-only deps (readline,
|
|
tview, tcell) are not in the library import graph.
|
|
- **Security record**: no advisories on record for `cel.dev/cel-go`
|
|
(GitHub Advisory Database).
|
|
- **Module path**: migrated to `cel.dev/cel-go` (v0.32.0 is current);
|
|
the old `github.com/google/cel-go` path is the pre-migration identity.
|
|
Pin `cel.dev/cel-go`.
|
|
- **Binary size cost**: +13.0MB is the largest of the candidates
|
|
(measured: baseline 2.34MB -> 15.25MB with CEL). Accepted because:
|
|
the daemon already embeds a large Yaegi runtime; CEL's type model is
|
|
the load-bearing win (compile-time rejection of result-path typos,
|
|
matching the reject-at-load rule); and the alternative bespoke
|
|
grammar would re-implement CEL's checker to get the same guarantee.
|
|
Flagged for sign-off: if 13MB is unacceptable for a specific
|
|
deployment target, the fallback is a bespoke grammar with a
|
|
re-implemented subset of this checker -- a strictly worse trade.
|
|
|
|
## Why bespoke was not sufficient
|
|
|
|
A bespoke grammar for the shipped cases (path read + comparison + bool
|
|
composition) is small, but the load-time-validation requirement makes
|
|
the comparison: the reject-at-load rule requires *field-level* static
|
|
checking of result paths against generated `Result` schemas, which is
|
|
CEL's type-checker's job. Re-implementing that on a hand-rolled
|
|
grammar means re-implementing a type system -- more code than the
|
|
grammar itself, with worse diagnostics, no conformance suite, and no
|
|
community surface for future expression needs (even declarative ones).
|
|
The bespoke option only wins on binary size, and the size cost of CEL
|
|
is measured and bounded.
|
|
|
|
## Judgment calls (flagged for sign-off)
|
|
|
|
- **J1 -- `id` required on referenced actions, unknown id = load
|
|
failure.** Matches the reject-at-load rule. Alternative considered: an alternative
|
|
is positional `actions[0]` access, which breaks on reorder; named is
|
|
the operator-friendlier shape.
|
|
- **J2 -- every `args` value is evaluated as a CEL expression.** No
|
|
split between "literal args" and "expression args": a plain string
|
|
value is a CEL string literal, a number is a CEL number literal, and
|
|
any value that reads context data is written in CEL. One mechanism,
|
|
no interpolation marker, no second syntax to learn. Alternative considered: cEL
|
|
string literals must be quoted inside YAML (e.g. `message:
|
|
'"build finished"'`), which is slightly noisy for mostly-literal
|
|
args; accepted for uniformity -- the linter catches quoting mistakes
|
|
at load.
|
|
- **J3 -- condition failure semantics.** A condition that errors at
|
|
runtime (missing/wrong-typed field) evaluates to **false**: the
|
|
action is skipped, logged, run continues. This is the simplest
|
|
candidate consistent with gap 1's bias against clever recovery;
|
|
gap 1 (`archie-core-t2db.18`) commits
|
|
stop-on-first-failure, so per-action control is not coming and J3
|
|
stands. Alternative considered: cEL separates
|
|
false from error; we treat evaluation error as false + log.
|
|
- **J4 -- `event` stays `dyn` (schema-by-example).** Load-time we
|
|
cannot know event field types; declaring them typed would reject
|
|
valid plays. Result reads get field-level checking via generated
|
|
schema declarations. Alternative considered: when schema-by-example ships, tighten
|
|
event field declarations from captured samples.
|
|
- **J5 -- cost limit default.** A `CostLimit` at load time (e.g.
|
|
100,000) is applied to every expression; the linter and daemon use
|
|
the same value so they agree. Alternative considered: exact number is a constant to
|
|
tune at first use.
|
|
|
|
## Interaction with the execution-time gaps
|
|
|
|
This resolution does not resolve the execution-time gaps (both are
|
|
resolved separately: gap 1 by `archie-core-t2db.18`, gap 2 by the
|
|
gap-2 decision). Conditions make gap 1
|
|
concrete (condition-failure vs. action-failure semantics, J3) and
|
|
CEL's read-only result access makes gap 2's keying derivation
|
|
readable (an idempotency key expression can read event fields) -- both
|
|
flagged as the gaps' first real exercise. Gap 2's exercise is
|
|
subsequently solved by that decision: the
|
|
answer is a fixed structural derivation, and idempotency keys are
|
|
deliberately NOT CEL expressions (see the "No CEL in the key"
|
|
passage).
|
|
|
|
## Implementation placement (for the follow-up slice)
|
|
|
|
- New `internal/domain/eda/expr` package: builds the CEL environment
|
|
(context declarations, generated `Result` schema types as CEL type
|
|
declarations, cost limit), compiles/validates playbook expressions
|
|
at load, evaluates them at dispatch. Domain layer; no infrastructure
|
|
imports; owned by the playbook engine family. One new dependency:
|
|
`cel.dev/cel-go` (pinned), `go.mod` otherwise unchanged.
|
|
- Consumption: the playbook event coordinator (open question 4's
|
|
loader) evaluates `when` before dispatch and
|
|
evaluates `args` values before `Module.Invoke`; the lint tool
|
|
(`archie-core-t2db.12`) shares the same compile/validate path for author-time
|
|
diagnostics.
|
|
- Schema integration: action kinds' generated `Result`/`Args` structs
|
|
(`archie-core-t2db.13`'s `logextract` pattern) are declared to CEL at load so
|
|
`actions.<id>.result.<field>` type-checks; `args` expressions
|
|
type-check against the target kind's `Args` schema before the
|
|
playbook is accepted.
|