docs(prds): resolve data-flow/condition syntax as CEL (open question 1)

CEL (cel.dev/cel-go, pinned v0.32.0) is the single expression mechanism
covering both an action's `when` condition and its `args` values,
evaluated against a read-only context (`event`, `actions.<id>.result`).
Rejects the custom-gate precedent (Yaegi-in-YAML) as inverting the
design's own "YAML is data, Yaegi is logic" premise. Expressions
compile/type-check at playbook load, consistent with the existing
collision-handling reject-at-load rule; generated Module Result schemas
become CEL type declarations so result-path typos reject at load too.

Vetted against expr-lang/expr, gval, govaluate (archived), starlark-go:
CEL wins on structural safety (non-Turing-complete, CostLimit-bounded,
error values not panics, zero security advisories on record) despite
the largest binary cost (+13MB) of the candidates -- accepted because
the alternative is re-implementing CEL's type-checker on a bespoke
grammar for the same guarantee.

Sign-off (Sam, 2026-09-03): all judgment calls (J1-J5) accepted as
proposed, including J2's per-value CEL-literal quoting cost and the
+13MB binary size trade.

Cross-references in module-position.md updated to point at this
resolution instead of citing open question 1 as unresolved.
This commit is contained in:
sam committed 2026-09-03 03:55:24 +10:00
1 parent eea0544ee6
commit 7282afc9f5
2 files changed
+278 -9

No files matched your search

+268
View File
@@ -273,6 +273,274 @@ before Channel/Forge actions or multi-action playbooks ship.
Go expression evaluator (there may be one already in the codebase worth
checking, e.g. anything backing gate conditions), or a bespoke minimal
grammar? Not decided here.
**Status 2026-09-03: resolved.** CEL (cel.dev/cel-go) is the single
mechanism for both halves. The resolution below replaces this entry
and is not an open question. Judgment calls are flagged inline.
## Data-flow and condition syntax -- decision (resolves open question 1)
**Status:** Draft, awaiting sign-off (not yet in `docs/architecture/`)
**Date:** 2026-09-03
**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned; one
expression mechanism covers both an action's `when` condition and its
`args` values.
### The requirement, stated directly
A playbook action is gated and parametrised by runtime data: (a) a
condition deciding whether the action runs; (b) a later action's `args`
values referencing an earlier action's `Result` and the triggering
event's fields. Both are **CEL expressions**, evaluated against one
evaluation context. There is no separate interpolation syntax: CEL
reads nested maps and fields natively, so `args` values that need data
are written as CEL, and only plain literal args remain plain YAML
scalars.
Both halves are read-only: expressions may read context values and
compute a result from them, never mutate, never call host functions,
never perform I/O. CEL enforces this structurally (no side effects,
linear evaluation when macros are bounded, cost-limited).
```yaml
actions:
- position: module
kind: log
args:
message: '"build finished"' # literal
when: 'event.label == "bugfix"' # gate on event field
- position: module
kind: notify
args:
message: '"priority " + string(event.priority)'
when: 'actions.notify.result.delivered == true'
```
### Trust boundary of the expression
A playbook YAML is **operator-installed, in-process, daemon-privileged**
-- the same tier as `ModuleDir`/`PluginDir`/`SecretEngineDir`
(module-position.md's trust-boundary section): loaded from a configured
directory at startup, never from a webhook body or task worktree. The
schema-by-example flow uses live events to *design against*; the saved
playbook is operator-authored file content in an operator-configured
location.
Two consequences:
1. The expression string is **trusted**. Operator-authored config, not
attacker input; no sandboxing against a hostile expression string is
required. If a future playbook source is less trusted (auto-generated
bindings from unauthenticated captures), that is the playbook-file
trust tier moving -- the loader's acceptance check is the right
place, not the evaluator.
2. The **data the expression reads is NOT trusted** (decoded webhook
body, prior results through the same pipeline). Robustness against
hostile data is required: no panic on wrong-typed/missing fields,
no unbounded cost on deep/cyclic maps, bounded evaluation time. CEL
provides this structurally (cost limits, linear non-Turing-complete
evaluation, error values instead of panics -- verified in the
vetting record below).
### The gate precedent is deliberately NOT copied
`internal/gate/gateeval` evaluates arbitrary interpreted Go
(`.archie/gate.go`) as the repo's conditional-logic precedent. That is
the wrong precedent for playbook conditions because this document's
premise is: **YAML is pure orchestration data; Yaegi is where
imperative logic lives** (Problem section; module-position.md's
"no generic hook" section). Yaegi snippets in YAML would move
imperative logic into the data layer -- the inversion this design
exists to prevent -- and make playbooks un-lintable.
The line: conditions are **declarative predicates over runtime data,
not programs**. CEL is non-Turing-complete and side-effect-free, so it
cannot become a program; a condition that needs real logic gets a
Module kind (Yaegi), referenced by the action, not a stronger
expression body.
### Plugin-engine-rule interaction: confirmed non-issue (stated once)
The plugin engine rule forbids a generic `Module` interface with an
`any` payload; it exists to keep static implementation contracts
typed. Evaluating CEL against runtime `map[string]any` is **not the
same problem**: the rule concerns *implementation contracts* (static
operations a capability family exposes), not *runtime data values*
(legitimately dynamic -- an event payload's shape is unknown until it
arrives). The Module contract stays fully typed (generated
`Args`/`Result` schemas); CEL is a read-only data reader under the
playbook engine family, not a new capability contract. Stated once.
### Minimum viable evaluation context (what expressions see)
One flat context, namespaced from the playbook run, built at dispatch
time:
| Name | Type | Source |
|---|---|---|
| `event` | `map(string, dyn)` | The triggering event's decoded payload (webhook body / forge issue / schedule tick). Field access via `event.<field>`, map access via `event["field"]`, presence via `has(event.<field>)`. |
| `actions` | `map(string, dyn)` | Previous actions' results, keyed by the action's `id` as declared in the playbook (`actions.<id>.result.<field>`), so a later action reads an earlier one's `Result` map regardless of position. The current action and later actions are not present. |
Expressions may also read literal-only state (numbers, strings,
booleans) directly. CEL's `has()` macro covers the missing-key case
(`has(event.labels)` -> bool), which replaces any bespoke `has`/
`contains` operator.
`actions.<id>.result` is the `map[string]any` `Module.Invoke` already
returns (t2db.13), so no result-shape change is needed for data flow;
channel/forge kinds' results are declared to the same shape.
Judgement call J1: `action` id is a required field on every action
that later actions reference; ids are validated at load (unique,
stable-identifier-shaped), and a `when`/`args` expression referencing
an unknown `actions.<id>` is a **load failure**, not a runtime miss --
the same reject-at-load rule as collisions (see below).
### Load-time validation (consistent with the collision rule)
The collision-handling rule is: schema defines the accepted message;
anything outside it is rejected at load, logged plainly, reported to
the caller -- never deferred to runtime. Expressions are validated the
same way, at playbook load:
- **Syntax**: every `when` and every CEL-typed `args` value is parsed
and checked by CEL's checker when the playbook is loaded. A syntax
or type error is a reported load failure (the playbook is dropped and
the error goes to the caller), not a runtime surprise.
- **Context conformance**: with the context declared to CEL
(`event` as `map(string, dyn)`, `actions` as `map(string, dyn)`),
unknown top-level identifiers are rejected at compile time.
Field-level typing inside `dyn` values is deferred to runtime by
CEL's design (verified: `data.items.lenght` with `data` dyn compiles
clean and errors at eval) -- so the loader declares as much type as
the schema provides (the action kinds' generated `Result` structs
become CEL type declarations at load, giving field-level rejection
for result reads), and `event` stays `dyn` because its shape is
unknown by design (schema-by-example).
- **Names**: unknown `actions.<id>` references and unknown roots are
compile-time/lint-visible errors (J1).
This means the lint tool (t2db.12) and the daemon share one checker
-- same single-source argument as the playbook loaders.
### Candidate comparison (surveyed 2026-09-03, pkg.go.dev + GitHub;
govaluate and gval checked against actual maintenance state, not
training memory)
| Candidate | Maintenance | Safety surface | Cost (measured) | Verdict |
|---|---|---|---|---|
| **CEL (cel.dev/cel-go)** | Actively maintained (google/cel, monthly releases) | Non-Turing-complete, linear eval, `CostLimit`/`CostTracking`/`ParserRecursionLimit`, no side effects, error values not panics | **+13.0MB** (2.34MB -> 15.25MB) | **Chosen** |
| `expr-lang/expr` v1.17.8 | Actively maintained | Side-effect-free, `DisableAllBuiltins`, depth limits (CVE fixed w/ tests) | +3.9MB (2.34MB -> 6.28MB) | Not chosen: two safety CVEs this cycle; smaller ecosystem; no protobuf/type-model fit with future schema-gen |
| `gval` v1.2.3 | Maintained, slower | No builtin-restriction surface; extras (ternary/`??`) not wanted | +3.0MB (2 transitive deps) | Not chosen |
| `govaluate` | **ARCHIVED** (author archived 2024) | n/a | n/a | Not chosen: dead |
| `starlark-go` | Maintained (Bazel) | Turing-complete with step limits | large (full language) | Not chosen: whole-language surface |
### Vetting record (vetting-dependencies skill, 2026-09-03)
Source reviewed at HEAD, not docs:
- **Repo hygiene**: clean. No binaries/media in tree (the only
non-source artifacts are expected generated protobuf files); 16-line
`.gitignore`.
- **Hard-boundary correctness**: the place correctness is hard for an
expression evaluator is bounding evaluation and surviving hostile
input. Dedicated regression tests exist and pass at HEAD:
`TestCostLimit`, `TestCostTracking*`, `TestParserRecursionLimit`,
and a large conformance suite. Verified by execution: evaluation
against a 2000-deep nested map under `CostLimit` completes in ~10
microseconds and returns an error value for a missing key -- no
panic, no unbounded walk.
- **Dependency footprint**: antlr (parser), protobuf + genproto (CEL's
type model), `cel.dev/expr` (CEL spec types), yaml.v3. All
mainstream, canonical, vendored widely by Google-adjacent projects.
No runtime-impure third-party code; the REPL-only deps (readline,
tview, tcell) are not in the library import graph.
- **Security record**: no advisories on record for `cel.dev/cel-go`
(GitHub Advisory Database, queried 2026-09-03).
- **Module path**: migrated to `cel.dev/cel-go` (v0.32.0 is current);
the old `github.com/google/cel-go` path is the pre-migration identity.
Pin `cel.dev/cel-go`.
- **Binary size cost**: +13.0MB is the largest of the candidates
(measured: baseline 2.34MB -> 15.25MB with CEL). Accepted because:
the daemon already embeds a large Yaegi runtime; CEL's type model is
the load-bearing win (compile-time rejection of result-path typos,
matching the reject-at-load rule); and the alternative bespoke
grammar would re-implement CEL's checker to get the same guarantee.
Flagged for sign-off: if 13MB is unacceptable for a specific
deployment target, the fallback is a bespoke grammar with a
re-implemented subset of this checker -- a strictly worse trade.
### Why bespoke was not sufficient
A bespoke grammar for the shipped cases (path read + comparison + bool
composition) is small, but the load-time-validation requirement makes
the comparison: the reject-at-load rule requires *field-level* static
checking of result paths against generated `Result` schemas, which is
CEL's type-checker's job. Re-implementing that on a hand-rolled
grammar means re-implementing a type system -- more code than the
grammar itself, with worse diagnostics, no conformance suite, and no
community surface for future expression needs (even declarative ones).
The bespoke option only wins on binary size, and the size cost of CEL
is measured and bounded.
### Judgment calls (flagged for sign-off)
- **J1 -- `id` required on referenced actions, unknown id = load
failure.** Matches the reject-at-load rule. { Call: an alternative
is positional `actions[0]` access, which breaks on reorder; named is
the operator-friendlier shape. }
- **J2 -- every `args` value is evaluated as a CEL expression.** No
split between "literal args" and "expression args": a plain string
value is a CEL string literal, a number is a CEL number literal, and
any value that reads context data is written in CEL. One mechanism,
no interpolation marker, no second syntax to learn. { Call: CEL
string literals must be quoted inside YAML (e.g. `message:
'"build finished"'`), which is slightly noisy for mostly-literal
args; accepted for uniformity -- the linter catches quoting mistakes
at load. }
- **J3 -- condition failure semantics.** A condition that errors at
runtime (missing/wrong-typed field) evaluates to **false**: the
action is skipped, logged, run continues. This is the simplest
candidate consistent with gap 1's bias against clever recovery;
revisit if gap 1 wants per-action control. { Call: CEL separates
false from error; we treat evaluation error as false + log. }
- **J4 -- `event` stays `dyn` (schema-by-example).** Load-time we
cannot know event field types; declaring them typed would reject
valid plays. Result reads get field-level checking via generated
schema declarations. { Call: when schema-by-example ships, tighten
event field declarations from captured samples. }
- **J5 -- cost limit default.** A `CostLimit` at load time (e.g.
100,000) is applied to every expression; the linter and daemon use
the same value so they agree. { Call: exact number is a constant to
tune at first use. }
### Interaction with the unresolved execution-time gaps
This resolution does not resolve gaps 1-2. Conditions make gap 1
concrete (condition-failure vs. action-failure semantics, J3) and
CEL's read-only result access makes gap 2's keying derivation
readable (an idempotency key expression can read event fields) -- both
flagged as the gaps' first real exercise, not solved here.
### Implementation placement (for the follow-up slice)
- New `internal/domain/eda/expr` package: builds the CEL environment
(context declarations, generated `Result` schema types as CEL type
declarations, cost limit), compiles/validates playbook expressions
at load, evaluates them at dispatch. Domain layer; no infrastructure
imports; owned by the playbook engine family. One new dependency:
`cel.dev/cel-go` (pinned), `go.mod` otherwise unchanged.
- Consumption: the playbook event coordinator (open question 4's
loader, not yet built) evaluates `when` before dispatch and
evaluates `args` values before `Module.Invoke`; the lint tool
(t2db.12) shares the same compile/validate path for author-time
diagnostics.
- Schema integration: action kinds' generated `Result`/`Args` structs
(t2db.13's `logextract` pattern) are declared to CEL at load so
`actions.<id>.result.<field>` type-checks; `args` expressions
type-check against the target kind's `Args` schema before the
playbook is accepted.
2. **Playbook directory config field.** Where this lives in
`internal/config` / `configuration.Document`, and whether it's one
directory or a list (mirroring `SkillsDir`'s shape) -- follow existing
+10 -9
View File
@@ -193,11 +193,11 @@ wired in bootstrap.go with log-and-abort on load failure.
Playbook YAML action dispatch is **deliberately out of this slice**: a
minimal 'run this one module action, no chaining' dispatch still requires
naming how `args` values reach the action (parent doc open question 1
-- data-flow/interpolation syntax) and what the result feeds. Both are
unresolved in the parent doc; claiming them here would half-answer a
question the design explicitly defers. Dispatch is a follow-up ticket
that lands once Q1 is answered, and it will consume this registry as-is.
the parent doc's data-flow mechanism (now resolved: CEL expressions
over an evaluation context -- see eda-playbook-engine.md's resolved open
question 1) and what the result feeds. Dispatch is a follow-up ticket
that lands once the coordinator exists, and it will consume this
registry as-is.
## Open questions
@@ -213,7 +213,8 @@ that lands once Q1 is answered, and it will consume this registry as-is.
subtypes) should be confirmed against `organisation.md`'s file-layout
rule before the first slice starts.
3. **Result surfacing.** A Module's `Result` needs to reach later actions
in the same playbook once data-flow (parent doc's open question 1) is
designed -- this doc does not attempt that, but the `map[string]any`
result shape above should stay compatible with whatever that
mechanism turns out to be.
in the same playbook once data-flow is designed -- resolved in the
parent doc as CEL expressions over `actions.<id>.result.<field>` (see
eda-playbook-engine.md's resolved open question 1); the `map[string]any`
result shape above is what the evaluation context exposes, so it stays
compatible by construction.