docs(prds): resolve data-flow/condition syntax as CEL (open question 1)
CEL (cel.dev/cel-go, pinned v0.32.0) is the single expression mechanism covering both an action's `when` condition and its `args` values, evaluated against a read-only context (`event`, `actions.<id>.result`). Rejects the custom-gate precedent (Yaegi-in-YAML) as inverting the design's own "YAML is data, Yaegi is logic" premise. Expressions compile/type-check at playbook load, consistent with the existing collision-handling reject-at-load rule; generated Module Result schemas become CEL type declarations so result-path typos reject at load too. Vetted against expr-lang/expr, gval, govaluate (archived), starlark-go: CEL wins on structural safety (non-Turing-complete, CostLimit-bounded, error values not panics, zero security advisories on record) despite the largest binary cost (+13MB) of the candidates -- accepted because the alternative is re-implementing CEL's type-checker on a bespoke grammar for the same guarantee. Sign-off (Sam, 2026-09-03): all judgment calls (J1-J5) accepted as proposed, including J2's per-value CEL-literal quoting cost and the +13MB binary size trade. Cross-references in module-position.md updated to point at this resolution instead of citing open question 1 as unresolved.
This commit is contained in:
1 parent
eea0544ee6
commit
7282afc9f5
2 files changed
+278
-9
No files matched your search
@@ -273,6 +273,274 @@ before Channel/Forge actions or multi-action playbooks ship.
|
||||
Go expression evaluator (there may be one already in the codebase worth
|
||||
checking, e.g. anything backing gate conditions), or a bespoke minimal
|
||||
grammar? Not decided here.
|
||||
|
||||
**Status 2026-09-03: resolved.** CEL (cel.dev/cel-go) is the single
|
||||
mechanism for both halves. The resolution below replaces this entry
|
||||
and is not an open question. Judgment calls are flagged inline.
|
||||
|
||||
## Data-flow and condition syntax -- decision (resolves open question 1)
|
||||
|
||||
**Status:** Draft, awaiting sign-off (not yet in `docs/architecture/`)
|
||||
**Date:** 2026-09-03
|
||||
**Decision:** CEL via `cel.dev/cel-go` (v0.32.0), pinned; one
|
||||
expression mechanism covers both an action's `when` condition and its
|
||||
`args` values.
|
||||
|
||||
### The requirement, stated directly
|
||||
|
||||
A playbook action is gated and parametrised by runtime data: (a) a
|
||||
condition deciding whether the action runs; (b) a later action's `args`
|
||||
values referencing an earlier action's `Result` and the triggering
|
||||
event's fields. Both are **CEL expressions**, evaluated against one
|
||||
evaluation context. There is no separate interpolation syntax: CEL
|
||||
reads nested maps and fields natively, so `args` values that need data
|
||||
are written as CEL, and only plain literal args remain plain YAML
|
||||
scalars.
|
||||
|
||||
Both halves are read-only: expressions may read context values and
|
||||
compute a result from them, never mutate, never call host functions,
|
||||
never perform I/O. CEL enforces this structurally (no side effects,
|
||||
linear evaluation when macros are bounded, cost-limited).
|
||||
|
||||
```yaml
|
||||
actions:
|
||||
- position: module
|
||||
kind: log
|
||||
args:
|
||||
message: '"build finished"' # literal
|
||||
when: 'event.label == "bugfix"' # gate on event field
|
||||
- position: module
|
||||
kind: notify
|
||||
args:
|
||||
message: '"priority " + string(event.priority)'
|
||||
when: 'actions.notify.result.delivered == true'
|
||||
```
|
||||
|
||||
### Trust boundary of the expression
|
||||
|
||||
A playbook YAML is **operator-installed, in-process, daemon-privileged**
|
||||
-- the same tier as `ModuleDir`/`PluginDir`/`SecretEngineDir`
|
||||
(module-position.md's trust-boundary section): loaded from a configured
|
||||
directory at startup, never from a webhook body or task worktree. The
|
||||
schema-by-example flow uses live events to *design against*; the saved
|
||||
playbook is operator-authored file content in an operator-configured
|
||||
location.
|
||||
|
||||
Two consequences:
|
||||
|
||||
1. The expression string is **trusted**. Operator-authored config, not
|
||||
attacker input; no sandboxing against a hostile expression string is
|
||||
required. If a future playbook source is less trusted (auto-generated
|
||||
bindings from unauthenticated captures), that is the playbook-file
|
||||
trust tier moving -- the loader's acceptance check is the right
|
||||
place, not the evaluator.
|
||||
2. The **data the expression reads is NOT trusted** (decoded webhook
|
||||
body, prior results through the same pipeline). Robustness against
|
||||
hostile data is required: no panic on wrong-typed/missing fields,
|
||||
no unbounded cost on deep/cyclic maps, bounded evaluation time. CEL
|
||||
provides this structurally (cost limits, linear non-Turing-complete
|
||||
evaluation, error values instead of panics -- verified in the
|
||||
vetting record below).
|
||||
|
||||
### The gate precedent is deliberately NOT copied
|
||||
|
||||
`internal/gate/gateeval` evaluates arbitrary interpreted Go
|
||||
(`.archie/gate.go`) as the repo's conditional-logic precedent. That is
|
||||
the wrong precedent for playbook conditions because this document's
|
||||
premise is: **YAML is pure orchestration data; Yaegi is where
|
||||
imperative logic lives** (Problem section; module-position.md's
|
||||
"no generic hook" section). Yaegi snippets in YAML would move
|
||||
imperative logic into the data layer -- the inversion this design
|
||||
exists to prevent -- and make playbooks un-lintable.
|
||||
|
||||
The line: conditions are **declarative predicates over runtime data,
|
||||
not programs**. CEL is non-Turing-complete and side-effect-free, so it
|
||||
cannot become a program; a condition that needs real logic gets a
|
||||
Module kind (Yaegi), referenced by the action, not a stronger
|
||||
expression body.
|
||||
|
||||
### Plugin-engine-rule interaction: confirmed non-issue (stated once)
|
||||
|
||||
The plugin engine rule forbids a generic `Module` interface with an
|
||||
`any` payload; it exists to keep static implementation contracts
|
||||
typed. Evaluating CEL against runtime `map[string]any` is **not the
|
||||
same problem**: the rule concerns *implementation contracts* (static
|
||||
operations a capability family exposes), not *runtime data values*
|
||||
(legitimately dynamic -- an event payload's shape is unknown until it
|
||||
arrives). The Module contract stays fully typed (generated
|
||||
`Args`/`Result` schemas); CEL is a read-only data reader under the
|
||||
playbook engine family, not a new capability contract. Stated once.
|
||||
|
||||
### Minimum viable evaluation context (what expressions see)
|
||||
|
||||
One flat context, namespaced from the playbook run, built at dispatch
|
||||
time:
|
||||
|
||||
| Name | Type | Source |
|
||||
|---|---|---|
|
||||
| `event` | `map(string, dyn)` | The triggering event's decoded payload (webhook body / forge issue / schedule tick). Field access via `event.<field>`, map access via `event["field"]`, presence via `has(event.<field>)`. |
|
||||
| `actions` | `map(string, dyn)` | Previous actions' results, keyed by the action's `id` as declared in the playbook (`actions.<id>.result.<field>`), so a later action reads an earlier one's `Result` map regardless of position. The current action and later actions are not present. |
|
||||
|
||||
Expressions may also read literal-only state (numbers, strings,
|
||||
booleans) directly. CEL's `has()` macro covers the missing-key case
|
||||
(`has(event.labels)` -> bool), which replaces any bespoke `has`/
|
||||
`contains` operator.
|
||||
|
||||
`actions.<id>.result` is the `map[string]any` `Module.Invoke` already
|
||||
returns (t2db.13), so no result-shape change is needed for data flow;
|
||||
channel/forge kinds' results are declared to the same shape.
|
||||
|
||||
Judgement call J1: `action` id is a required field on every action
|
||||
that later actions reference; ids are validated at load (unique,
|
||||
stable-identifier-shaped), and a `when`/`args` expression referencing
|
||||
an unknown `actions.<id>` is a **load failure**, not a runtime miss --
|
||||
the same reject-at-load rule as collisions (see below).
|
||||
|
||||
### Load-time validation (consistent with the collision rule)
|
||||
|
||||
The collision-handling rule is: schema defines the accepted message;
|
||||
anything outside it is rejected at load, logged plainly, reported to
|
||||
the caller -- never deferred to runtime. Expressions are validated the
|
||||
same way, at playbook load:
|
||||
|
||||
- **Syntax**: every `when` and every CEL-typed `args` value is parsed
|
||||
and checked by CEL's checker when the playbook is loaded. A syntax
|
||||
or type error is a reported load failure (the playbook is dropped and
|
||||
the error goes to the caller), not a runtime surprise.
|
||||
- **Context conformance**: with the context declared to CEL
|
||||
(`event` as `map(string, dyn)`, `actions` as `map(string, dyn)`),
|
||||
unknown top-level identifiers are rejected at compile time.
|
||||
Field-level typing inside `dyn` values is deferred to runtime by
|
||||
CEL's design (verified: `data.items.lenght` with `data` dyn compiles
|
||||
clean and errors at eval) -- so the loader declares as much type as
|
||||
the schema provides (the action kinds' generated `Result` structs
|
||||
become CEL type declarations at load, giving field-level rejection
|
||||
for result reads), and `event` stays `dyn` because its shape is
|
||||
unknown by design (schema-by-example).
|
||||
- **Names**: unknown `actions.<id>` references and unknown roots are
|
||||
compile-time/lint-visible errors (J1).
|
||||
|
||||
This means the lint tool (t2db.12) and the daemon share one checker
|
||||
-- same single-source argument as the playbook loaders.
|
||||
|
||||
### Candidate comparison (surveyed 2026-09-03, pkg.go.dev + GitHub;
|
||||
govaluate and gval checked against actual maintenance state, not
|
||||
training memory)
|
||||
|
||||
| Candidate | Maintenance | Safety surface | Cost (measured) | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| **CEL (cel.dev/cel-go)** | Actively maintained (google/cel, monthly releases) | Non-Turing-complete, linear eval, `CostLimit`/`CostTracking`/`ParserRecursionLimit`, no side effects, error values not panics | **+13.0MB** (2.34MB -> 15.25MB) | **Chosen** |
|
||||
| `expr-lang/expr` v1.17.8 | Actively maintained | Side-effect-free, `DisableAllBuiltins`, depth limits (CVE fixed w/ tests) | +3.9MB (2.34MB -> 6.28MB) | Not chosen: two safety CVEs this cycle; smaller ecosystem; no protobuf/type-model fit with future schema-gen |
|
||||
| `gval` v1.2.3 | Maintained, slower | No builtin-restriction surface; extras (ternary/`??`) not wanted | +3.0MB (2 transitive deps) | Not chosen |
|
||||
| `govaluate` | **ARCHIVED** (author archived 2024) | n/a | n/a | Not chosen: dead |
|
||||
| `starlark-go` | Maintained (Bazel) | Turing-complete with step limits | large (full language) | Not chosen: whole-language surface |
|
||||
|
||||
### Vetting record (vetting-dependencies skill, 2026-09-03)
|
||||
|
||||
Source reviewed at HEAD, not docs:
|
||||
|
||||
- **Repo hygiene**: clean. No binaries/media in tree (the only
|
||||
non-source artifacts are expected generated protobuf files); 16-line
|
||||
`.gitignore`.
|
||||
- **Hard-boundary correctness**: the place correctness is hard for an
|
||||
expression evaluator is bounding evaluation and surviving hostile
|
||||
input. Dedicated regression tests exist and pass at HEAD:
|
||||
`TestCostLimit`, `TestCostTracking*`, `TestParserRecursionLimit`,
|
||||
and a large conformance suite. Verified by execution: evaluation
|
||||
against a 2000-deep nested map under `CostLimit` completes in ~10
|
||||
microseconds and returns an error value for a missing key -- no
|
||||
panic, no unbounded walk.
|
||||
- **Dependency footprint**: antlr (parser), protobuf + genproto (CEL's
|
||||
type model), `cel.dev/expr` (CEL spec types), yaml.v3. All
|
||||
mainstream, canonical, vendored widely by Google-adjacent projects.
|
||||
No runtime-impure third-party code; the REPL-only deps (readline,
|
||||
tview, tcell) are not in the library import graph.
|
||||
- **Security record**: no advisories on record for `cel.dev/cel-go`
|
||||
(GitHub Advisory Database, queried 2026-09-03).
|
||||
- **Module path**: migrated to `cel.dev/cel-go` (v0.32.0 is current);
|
||||
the old `github.com/google/cel-go` path is the pre-migration identity.
|
||||
Pin `cel.dev/cel-go`.
|
||||
- **Binary size cost**: +13.0MB is the largest of the candidates
|
||||
(measured: baseline 2.34MB -> 15.25MB with CEL). Accepted because:
|
||||
the daemon already embeds a large Yaegi runtime; CEL's type model is
|
||||
the load-bearing win (compile-time rejection of result-path typos,
|
||||
matching the reject-at-load rule); and the alternative bespoke
|
||||
grammar would re-implement CEL's checker to get the same guarantee.
|
||||
Flagged for sign-off: if 13MB is unacceptable for a specific
|
||||
deployment target, the fallback is a bespoke grammar with a
|
||||
re-implemented subset of this checker -- a strictly worse trade.
|
||||
|
||||
### Why bespoke was not sufficient
|
||||
|
||||
A bespoke grammar for the shipped cases (path read + comparison + bool
|
||||
composition) is small, but the load-time-validation requirement makes
|
||||
the comparison: the reject-at-load rule requires *field-level* static
|
||||
checking of result paths against generated `Result` schemas, which is
|
||||
CEL's type-checker's job. Re-implementing that on a hand-rolled
|
||||
grammar means re-implementing a type system -- more code than the
|
||||
grammar itself, with worse diagnostics, no conformance suite, and no
|
||||
community surface for future expression needs (even declarative ones).
|
||||
The bespoke option only wins on binary size, and the size cost of CEL
|
||||
is measured and bounded.
|
||||
|
||||
### Judgment calls (flagged for sign-off)
|
||||
|
||||
- **J1 -- `id` required on referenced actions, unknown id = load
|
||||
failure.** Matches the reject-at-load rule. { Call: an alternative
|
||||
is positional `actions[0]` access, which breaks on reorder; named is
|
||||
the operator-friendlier shape. }
|
||||
- **J2 -- every `args` value is evaluated as a CEL expression.** No
|
||||
split between "literal args" and "expression args": a plain string
|
||||
value is a CEL string literal, a number is a CEL number literal, and
|
||||
any value that reads context data is written in CEL. One mechanism,
|
||||
no interpolation marker, no second syntax to learn. { Call: CEL
|
||||
string literals must be quoted inside YAML (e.g. `message:
|
||||
'"build finished"'`), which is slightly noisy for mostly-literal
|
||||
args; accepted for uniformity -- the linter catches quoting mistakes
|
||||
at load. }
|
||||
- **J3 -- condition failure semantics.** A condition that errors at
|
||||
runtime (missing/wrong-typed field) evaluates to **false**: the
|
||||
action is skipped, logged, run continues. This is the simplest
|
||||
candidate consistent with gap 1's bias against clever recovery;
|
||||
revisit if gap 1 wants per-action control. { Call: CEL separates
|
||||
false from error; we treat evaluation error as false + log. }
|
||||
- **J4 -- `event` stays `dyn` (schema-by-example).** Load-time we
|
||||
cannot know event field types; declaring them typed would reject
|
||||
valid plays. Result reads get field-level checking via generated
|
||||
schema declarations. { Call: when schema-by-example ships, tighten
|
||||
event field declarations from captured samples. }
|
||||
- **J5 -- cost limit default.** A `CostLimit` at load time (e.g.
|
||||
100,000) is applied to every expression; the linter and daemon use
|
||||
the same value so they agree. { Call: exact number is a constant to
|
||||
tune at first use. }
|
||||
|
||||
### Interaction with the unresolved execution-time gaps
|
||||
|
||||
This resolution does not resolve gaps 1-2. Conditions make gap 1
|
||||
concrete (condition-failure vs. action-failure semantics, J3) and
|
||||
CEL's read-only result access makes gap 2's keying derivation
|
||||
readable (an idempotency key expression can read event fields) -- both
|
||||
flagged as the gaps' first real exercise, not solved here.
|
||||
|
||||
### Implementation placement (for the follow-up slice)
|
||||
|
||||
- New `internal/domain/eda/expr` package: builds the CEL environment
|
||||
(context declarations, generated `Result` schema types as CEL type
|
||||
declarations, cost limit), compiles/validates playbook expressions
|
||||
at load, evaluates them at dispatch. Domain layer; no infrastructure
|
||||
imports; owned by the playbook engine family. One new dependency:
|
||||
`cel.dev/cel-go` (pinned), `go.mod` otherwise unchanged.
|
||||
- Consumption: the playbook event coordinator (open question 4's
|
||||
loader, not yet built) evaluates `when` before dispatch and
|
||||
evaluates `args` values before `Module.Invoke`; the lint tool
|
||||
(t2db.12) shares the same compile/validate path for author-time
|
||||
diagnostics.
|
||||
- Schema integration: action kinds' generated `Result`/`Args` structs
|
||||
(t2db.13's `logextract` pattern) are declared to CEL at load so
|
||||
`actions.<id>.result.<field>` type-checks; `args` expressions
|
||||
type-check against the target kind's `Args` schema before the
|
||||
playbook is accepted.
|
||||
|
||||
2. **Playbook directory config field.** Where this lives in
|
||||
`internal/config` / `configuration.Document`, and whether it's one
|
||||
directory or a list (mirroring `SkillsDir`'s shape) -- follow existing
|
||||
|
||||
@@ -193,11 +193,11 @@ wired in bootstrap.go with log-and-abort on load failure.
|
||||
|
||||
Playbook YAML action dispatch is **deliberately out of this slice**: a
|
||||
minimal 'run this one module action, no chaining' dispatch still requires
|
||||
naming how `args` values reach the action (parent doc open question 1
|
||||
-- data-flow/interpolation syntax) and what the result feeds. Both are
|
||||
unresolved in the parent doc; claiming them here would half-answer a
|
||||
question the design explicitly defers. Dispatch is a follow-up ticket
|
||||
that lands once Q1 is answered, and it will consume this registry as-is.
|
||||
the parent doc's data-flow mechanism (now resolved: CEL expressions
|
||||
over an evaluation context -- see eda-playbook-engine.md's resolved open
|
||||
question 1) and what the result feeds. Dispatch is a follow-up ticket
|
||||
that lands once the coordinator exists, and it will consume this
|
||||
registry as-is.
|
||||
|
||||
## Open questions
|
||||
|
||||
@@ -213,7 +213,8 @@ that lands once Q1 is answered, and it will consume this registry as-is.
|
||||
subtypes) should be confirmed against `organisation.md`'s file-layout
|
||||
rule before the first slice starts.
|
||||
3. **Result surfacing.** A Module's `Result` needs to reach later actions
|
||||
in the same playbook once data-flow (parent doc's open question 1) is
|
||||
designed -- this doc does not attempt that, but the `map[string]any`
|
||||
result shape above should stay compatible with whatever that
|
||||
mechanism turns out to be.
|
||||
in the same playbook once data-flow is designed -- resolved in the
|
||||
parent doc as CEL expressions over `actions.<id>.result.<field>` (see
|
||||
eda-playbook-engine.md's resolved open question 1); the `map[string]any`
|
||||
result shape above is what the evaluation context exposes, so it stays
|
||||
compatible by construction.
|
||||
Reference in new issue
Block a user