feat(workflow): routing and playbook findings carry the key's file:line
This commit is contained in:
6 files changed
+138
-47
No files matched your search
@@ -8,11 +8,8 @@
|
||||
// modes are entrypoints into the same validation source in
|
||||
// internal/domain/workflow.
|
||||
//
|
||||
// Known limitation (as of this slice): findings are file-granular, not
|
||||
// line-granular. The loader decodes playbooks with yaml.Unmarshal into a
|
||||
// plain map, which discards line numbers; a compiler-style file:line
|
||||
// diagnostic needs a yaml.Node decoding upgrade, which is out of scope
|
||||
// here and tracked separately.
|
||||
// A finding about one binding key leads with the key's file:line; a file
|
||||
// that does not parse is reported by path with the YAML parser's message.
|
||||
package main
|
||||
|
||||
import (
|
||||
|
||||
@@ -226,7 +226,7 @@
|
||||
"url": "/dev/prds/eda-playbook-engine/",
|
||||
"title": "EDA playbook engine -- design",
|
||||
"section": "PRDs",
|
||||
"body": "# EDA playbook engine -- design\n\n**Status:** Draft\n**Date:** 2026-09-03\n**Beads epic:** `archie-core-t2db`\n\n## Problem\n\n\"Which workflow runs for which trigger\" is hardcoded Go: a literal\nlabel vocabulary (`workintake.labelKinds`) and a literal label-\u003eworkflow map\n(`workflow/routing.go`), extended only by adding a Go case or shipping a\nskill whose `metadata.archie.workflow` overrides one *named* built-in\nworkflow. Extending the system to a new trigger, a new action, or a new\nevent source means a feature branch, a code change in a hardcoded\npath, and new Go tests -- for every addition. That is the barrier this\ndocument exists to remove.\n\nThe goal: triggers, the actions they run, and the\nconditions that connect them should be **drop-in and load**, not **drop-in\nand touch application code**. Yaegi-evaluated Go is the unit of imperative\n\"do a thing\" logic; YAML playbooks are pure orchestration data -- no logic,\njust \"on this event, with this data, call these actions, in this order,\nunder these conditions.\" The daemon's event coordinator doesn't know what a\n`bugfix` or `image-gen` action does, it resolves the name against a\nregistry and pipes typed data through.\n\nThis is explicitly **not** scoped to code-generation workflows. A playbook\naction can be any registered capability -- run a codegen workflow, send a\nmessage on a channel, open something on a forge, generate an image -- so\nlong as it's implemented behind one of the typed engine families below.\n\n## Relationship to existing decisions\n\n- `docs/prds/event-sources-and-reactions.md` ruled\n that reactions in the two epics it covers are **producer-only** (no veto,\n no mutation of in-flight work) and explicitly rejected a single generic\n `Source` interface as \"the untyped-hook shape the plugin engine rule\n exists to prevent.\" **This document extends that scope**: playbooks can\n dispatch to multiple typed action kinds, not just \"produce a\n `TaskEnvelope`.\" It does not relitigate producer-only-ness (still no\n veto/mutation of in-flight work here) and it does not introduce one\n generic interface -- see \"Typed positions, not one generic hook\" below,\n which is how it stays compliant with the plugin engine rule instead of\n reopening that document's #4.\n- `docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`\n governs every extension point\n introduced here. Every position below is a typed domain contract with an\n owning registry, not a generic callback map. This is a hard constraint,\n confirmed against the existing rule text before writing this doc.\n- Yaegi is already a proven in-process extension mechanism\n (`internal/plugin`, `internal/domain/workflow/wfeval/yaegi.go`,\n `internal/gate/gateeval/yaegi.go`, `internal/skill/plugin.go`) -- this\n design generalizes its existing role, it does not introduce a new\n execution mechanism.\n- `internal/domain/workflow/skillbuild.BuildRegistry` already composes a\n `workflow.Registry` from a skills catalog, plugin-defined overriding\n built-in. The playbook loader described here supersedes label routing\n specifically; it does not replace `skillbuild`'s workflow-composition role,\n it becomes an additional input the daemon composes at startup.\n- Webhook event intake is a separate design. This\n document defines the router each webhook-sourced event should be able to\n reach; it does not redesign webhook receipt itself.\n\n## Shape: four logic positions, each a typed engine family\n\nEvery entrypoint kind (mechanism, action, loop, schedule, route, trigger) is itself a schema emitter: defining a new kind produces the\ngenerated contract downstream tooling consumes. The kinds slot into a fixed\nhierarchy of logic positions:\n\n```\nWorkflow -\u003e Module -\u003e Channel (telegram/whatsapp/email/...)\n -\u003e Forge (GitHub/Gitea/GitLab/...)\n```\n\nEach position is its own typed engine family per the plugin engine rule:\nidentity + capability-specific operations, an owning `Registry`/`Manager`,\nexplicit lifecycle where the implementation owns resources, narrow host\naccess. Concretely:\n\n- **Workflow** -- already exists (`internal/domain/workflow`). A playbook\n action of kind `workflow` invokes a registered `workflow.Registry` entry\n by name, unchanged from the existing execution path.\n- **Module** -- the new general-purpose action position. A `Module` is a\n Yaegi-evaluated implementation of a generated interface (args in, typed\n result out) for one declared action kind (e.g. `image-gen`, `notify`,\n `open-pr-comment`). This is the position that makes actions \"not\n restricted to code-generation.\" Full design (schema-per-kind, the\n registry shape, the trust boundary, the recommended first slice):\n `docs/prds/module-position.md`.\n- **Channel** -- already exists as `channels.Channel` (telegram, email,\n webhook). A playbook action of kind `channel` sends through Line truncated
|
||||
"body": "# EDA playbook engine -- design\n\n**Status:** Draft\n**Date:** 2026-09-03\n**Beads epic:** `archie-core-t2db`\n\n## Problem\n\n\"Which workflow runs for which trigger\" is hardcoded Go: a literal\nlabel vocabulary (`workintake.labelKinds`) and a literal label-\u003eworkflow map\n(`workflow/routing.go`), extended only by adding a Go case or shipping a\nskill whose `metadata.archie.workflow` overrides one *named* built-in\nworkflow. Extending the system to a new trigger, a new action, or a new\nevent source means a feature branch, a code change in a hardcoded\npath, and new Go tests -- for every addition. That is the barrier this\ndocument exists to remove.\n\nThe goal: triggers, the actions they run, and the\nconditions that connect them should be **drop-in and load**, not **drop-in\nand touch application code**. Yaegi-evaluated Go is the unit of imperative\n\"do a thing\" logic; YAML playbooks are pure orchestration data -- no logic,\njust \"on this event, with this data, call these actions, in this order,\nunder these conditions.\" The daemon's event coordinator doesn't know what a\n`bugfix` or `image-gen` action does, it resolves the name against a\nregistry and pipes typed data through.\n\nThis is explicitly **not** scoped to code-generation workflows. A playbook\naction can be any registered capability -- run a codegen workflow, send a\nmessage on a channel, open something on a forge, generate an image -- so\nlong as it's implemented behind one of the typed engine families below.\n\n## Relationship to existing decisions\n\n- `docs/prds/event-sources-and-reactions.md` ruled\n that reactions in the two epics it covers are **producer-only** (no veto,\n no mutation of in-flight work) and explicitly rejected a single generic\n `Source` interface as \"the untyped-hook shape the plugin engine rule\n exists to prevent.\" **This document extends that scope**: playbooks can\n dispatch to multiple typed action kinds, not just \"produce a\n `TaskEnvelope`.\" It does not relitigate producer-only-ness (still no\n veto/mutation of in-flight work here) and it does not introduce one\n generic interface -- see \"Typed positions, not one generic hook\" below,\n which is how it stays compliant with the plugin engine rule instead of\n reopening that document's #4.\n- `docs/architecture/plugins-and-extensions.md#plugin-engine-rule-strict`\n governs every extension point\n introduced here. Every position below is a typed domain contract with an\n owning registry, not a generic callback map. This is a hard constraint,\n confirmed against the existing rule text before writing this doc.\n- Yaegi is already a proven in-process extension mechanism\n (`internal/plugin`, `internal/domain/workflow/wfeval/yaegi.go`,\n `internal/gate/gateeval/yaegi.go`, `internal/skill/plugin.go`) -- this\n design generalizes its existing role, it does not introduce a new\n execution mechanism.\n- `internal/domain/workflow/skillbuild.BuildRegistry` already composes a\n `workflow.Registry` from a skills catalog, plugin-defined overriding\n built-in. The playbook loader described here supersedes label routing\n specifically; it does not replace `skillbuild`'s workflow-composition role,\n it becomes an additional input the daemon composes at startup.\n- Webhook event intake is a separate design. This\n document defines the router each webhook-sourced event should be able to\n reach; it does not redesign webhook receipt itself.\n\n## Shape: four logic positions, each a typed engine family\n\nEvery entrypoint kind (mechanism, action, loop, schedule, route, trigger) is itself a schema emitter: defining a new kind produces the\ngenerated contract downstream tooling consumes. The kinds slot into a fixed\nhierarchy of logic positions:\n\n```\nWorkflow -\u003e Module -\u003e Channel (telegram/whatsapp/email/...)\n -\u003e Forge (GitHub/Gitea/GitLab/...)\n```\n\nEach position is its own typed engine family per the plugin engine rule:\nidentity + capability-specific operations, an owning `Registry`/`Manager`,\nexplicit lifecycle where the implementation owns resources, narrow host\naccess. Concretely:\n\n- **Workflow** -- already exists (`internal/domain/workflow`). A playbook\n action of kind `workflow` invokes a registered `workflow.Registry` entry\n by name, unchanged from the existing execution path.\n- **Module** -- the new general-purpose action position. A `Module` is a\n Yaegi-evaluated implementation of a generated interface (args in, typed\n result out) for one declared action kind (e.g. `image-gen`, `notify`,\n `open-pr-comment`). This is the position that makes actions \"not\n restricted to code-generation.\" Full design (schema-per-kind, the\n registry shape, the trust boundary, the recommended first slice):\n `docs/prds/module-position.md`.\n- **Channel** -- already exists as `channels.Channel` (telegram, email,\n webhook). A playbook action of kind `channel` sends through Line truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/embedded-nats/",
|
||||
|
||||
@@ -167,10 +167,8 @@ a case that should just be visible and fixed by a human. Instead:
|
||||
playbook directories against the exact loaders the daemon uses
|
||||
(`LoadPlaybookDirs`/`LoadKindWorkflowsYAML`/`LoadLabelWorkflowsYAML`),
|
||||
exiting non-zero on any collision / malformed file / invalid binding.
|
||||
Findings are file-granular, not line-granular: the loader decodes with
|
||||
`yaml.Unmarshal` into a plain map, which discards line numbers. A
|
||||
compiler-style file:line diagnostic needs a `yaml.Node` decoding
|
||||
upgrade, tracked separately -- the linter agrees with runtime
|
||||
A finding about one binding key leads with the key's `file:line`, read
|
||||
from the `yaml.Node` the loader decodes; the linter agrees with runtime
|
||||
validation by construction, which is the load-bearing property.
|
||||
Discoverable via `task lint:playbooks` or direct `go run
|
||||
./cmd/archie-playbooks lint -dir ...`. The LSP/serve mode is a later
|
||||
|
||||
@@ -98,11 +98,10 @@ func TestLintReportsLoadErrorAsFinding(t *testing.T) {
|
||||
}
|
||||
joined := strings.Join(result.Findings, "\n")
|
||||
// Both "bug" and "security" collide across the two files; the loader
|
||||
// sorts a file's keys before checking them (routing.go's
|
||||
// loadPlaybookFile), so "bug" -- alphabetically first -- is always the
|
||||
// one reported, deterministically.
|
||||
if !strings.Contains(joined, "bug") {
|
||||
t.Fatalf("Lint() findings = %q, want the colliding key named", joined)
|
||||
// checks a file's keys in document order, so the first line of b.yaml
|
||||
// is always the one reported, deterministically.
|
||||
if want := filepath.Join(dir, "b.yaml") + `:1: label "security"`; !strings.Contains(joined, want) {
|
||||
t.Fatalf("Lint() findings = %q, want %q", joined, want)
|
||||
}
|
||||
// Fail-fast like the daemon: one definition failure is reported (the
|
||||
// loader returns the first), not an enumeration of every collision.
|
||||
|
||||
@@ -66,17 +66,17 @@ func LoadKindWorkflowsYAML(path string) (KindWorkflows, error) {
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read kind-workflows file %s: %w", path, err)
|
||||
}
|
||||
raw := map[string]string{}
|
||||
if err := yaml.Unmarshal(data, &raw); err != nil {
|
||||
bindings, err := decodeBindings(data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse kind-workflows file %s: %w", path, err)
|
||||
}
|
||||
kw := make(KindWorkflows, len(raw))
|
||||
for k, name := range raw {
|
||||
kind := workintake.Kind(k)
|
||||
kw := make(KindWorkflows, len(bindings))
|
||||
for _, b := range bindings {
|
||||
kind := workintake.Kind(b.key)
|
||||
if err := kind.Validate(); err != nil {
|
||||
return nil, fmt.Errorf("kind-workflows file %s: %w", path, err)
|
||||
return nil, fmt.Errorf("kind-workflows file %s:%d: %w", path, b.line, err)
|
||||
}
|
||||
kw[kind] = name
|
||||
kw[kind] = b.value
|
||||
}
|
||||
return kw, nil
|
||||
}
|
||||
@@ -157,22 +157,12 @@ func loadPlaybookFile(path string, kw KindWorkflows, lw LabelWorkflows) error {
|
||||
if err != nil {
|
||||
return fmt.Errorf("read playbook file %s: %w", path, err)
|
||||
}
|
||||
raw := map[string]string{}
|
||||
if err := yaml.Unmarshal(data, &raw); err != nil {
|
||||
bindings, err := decodeBindings(data)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse playbook file %s: %w", path, err)
|
||||
}
|
||||
// Sorted, not range order: LoadPlaybookDirs' doc comment promises a
|
||||
// reproducible error when a file declares more than one colliding key,
|
||||
// which a raw map range (Go's iteration order is randomized) cannot
|
||||
// keep -- the same file could report a different "first" collision on
|
||||
// every run otherwise.
|
||||
keys := make([]string, 0, len(raw))
|
||||
for key := range raw {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
for _, key := range keys {
|
||||
if err := bindPlaybookKey(path, key, raw[key], kw, lw); err != nil {
|
||||
for _, b := range bindings {
|
||||
if err := bindPlaybookKey(fmt.Sprintf("%s:%d", path, b.line), b.key, b.value, kw, lw); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
@@ -182,18 +172,19 @@ func loadPlaybookFile(path string, kw KindWorkflows, lw LabelWorkflows) error {
|
||||
// bindPlaybookKey classifies one playbook binding key as a closed-vocabulary
|
||||
// kind or an arbitrary label, then merges it into kw/lw -- applying the same
|
||||
// collision and empty-value rules LoadPlaybookDirs' doc comment describes.
|
||||
func bindPlaybookKey(path, key, value string, kw KindWorkflows, lw LabelWorkflows) error {
|
||||
// pos is the key's file:line, which every finding leads with.
|
||||
func bindPlaybookKey(pos, key, value string, kw KindWorkflows, lw LabelWorkflows) error {
|
||||
trimmed := strings.TrimSpace(key)
|
||||
if trimmed == "" {
|
||||
return fmt.Errorf("playbook file %s: empty binding key", path)
|
||||
return fmt.Errorf("playbook file %s: empty binding key", pos)
|
||||
}
|
||||
if value == "" {
|
||||
return fmt.Errorf("playbook file %s: key %q has no workflow name", path, trimmed)
|
||||
return fmt.Errorf("playbook file %s: key %q has no workflow name", pos, trimmed)
|
||||
}
|
||||
if err := workintake.Kind(trimmed).Validate(); err == nil {
|
||||
// It is a kind binding: the closed Kind vocabulary accepts it.
|
||||
if existing, ok := kw[workintake.Kind(trimmed)]; ok {
|
||||
return fmt.Errorf("playbook dirs: kind %q bound in two sources (%q and %q)", trimmed, existing, value)
|
||||
return fmt.Errorf("playbook file %s: kind %q bound in two sources (%q and %q)", pos, trimmed, existing, value)
|
||||
}
|
||||
kw[workintake.Kind(trimmed)] = value
|
||||
return nil
|
||||
@@ -201,10 +192,10 @@ func bindPlaybookKey(path, key, value string, kw KindWorkflows, lw LabelWorkflow
|
||||
// Otherwise it is an arbitrary-label binding. Reject kind-owned labels
|
||||
// and empty labels with the same rules the single-file loader applies.
|
||||
if _, ok := defaultKindWorkflows[workintake.Kind(trimmed)]; ok {
|
||||
return fmt.Errorf("playbook file %s: label %q is already owned by the kind routing layer", path, trimmed)
|
||||
return fmt.Errorf("playbook file %s: label %q is already owned by the kind routing layer", pos, trimmed)
|
||||
}
|
||||
if existing, ok := lw[trimmed]; ok {
|
||||
return fmt.Errorf("playbook dirs: label %q bound in two sources (%q and %q)", trimmed, existing, value)
|
||||
return fmt.Errorf("playbook file %s: label %q bound in two sources (%q and %q)", pos, trimmed, existing, value)
|
||||
}
|
||||
lw[trimmed] = value
|
||||
return nil
|
||||
@@ -357,13 +348,15 @@ func LoadLabelWorkflowsYAML(path string) (LabelWorkflows, error) {
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read label-workflows file %s: %w", path, err)
|
||||
}
|
||||
raw := map[string]string{}
|
||||
if err := yaml.Unmarshal(data, &raw); err != nil {
|
||||
bindings, err := decodeBindings(data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse label-workflows file %s: %w", path, err)
|
||||
}
|
||||
lw := make(LabelWorkflows, len(raw))
|
||||
for label, name := range raw {
|
||||
trimmed := strings.TrimSpace(label)
|
||||
lw := make(LabelWorkflows, len(bindings))
|
||||
for _, b := range bindings {
|
||||
path := fmt.Sprintf("%s:%d", path, b.line)
|
||||
name := b.value
|
||||
trimmed := strings.TrimSpace(b.key)
|
||||
if trimmed == "" {
|
||||
return nil, fmt.Errorf("label-workflows file %s: empty label is not a valid binding", path)
|
||||
}
|
||||
@@ -380,3 +373,38 @@ func LoadLabelWorkflowsYAML(path string) (LabelWorkflows, error) {
|
||||
}
|
||||
return lw, nil
|
||||
}
|
||||
|
||||
// binding is one key of a routing file with the line it was declared on.
|
||||
type binding struct {
|
||||
key, value string
|
||||
line int
|
||||
}
|
||||
|
||||
// decodeBindings decodes a routing file's top-level string map in document
|
||||
// order, so the first finding is the same on every run and each can name
|
||||
// its key's line. Values decode exactly as they did into map[string]string,
|
||||
// including yaml.v3's duplicate-key refusal.
|
||||
func decodeBindings(data []byte) ([]binding, error) {
|
||||
var doc yaml.Node
|
||||
if err := yaml.Unmarshal(data, &doc); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(doc.Content) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
root := doc.Content[0]
|
||||
values := map[string]string{}
|
||||
if err := root.Decode(&values); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
bindings := make([]binding, 0, len(values))
|
||||
for i := 0; i+1 < len(root.Content); i += 2 {
|
||||
keyNode := root.Content[i]
|
||||
var key string
|
||||
if err := keyNode.Decode(&key); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
bindings = append(bindings, binding{key: key, value: values[key], line: keyNode.Line})
|
||||
}
|
||||
return bindings, nil
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package workflow
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A key-level finding names the file and the line of the offending key, so
|
||||
// the lint tool reports it compiler-style instead of leaving the author to
|
||||
// search the file.
|
||||
func TestBindingFindingsCarryTheKeyLine(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
content string
|
||||
load func(path string) error
|
||||
line string
|
||||
}{
|
||||
{
|
||||
name: "playbook dir, empty workflow name",
|
||||
content: "# routes\nsecurity: security-review\ndocs: \"\"\n",
|
||||
load: func(path string) error {
|
||||
_, _, err := LoadPlaybookDirs([]string{filepath.Dir(path)})
|
||||
return err
|
||||
},
|
||||
line: ":3:",
|
||||
},
|
||||
{
|
||||
name: "playbook dir, kind with no workflow name",
|
||||
content: "security: security-review\n\n\nbug: \"\"\n",
|
||||
load: func(path string) error {
|
||||
_, _, err := LoadPlaybookDirs([]string{filepath.Dir(path)})
|
||||
return err
|
||||
},
|
||||
line: ":4:",
|
||||
},
|
||||
{
|
||||
name: "label file, kind-owned label",
|
||||
content: "security: security-review\nfeature: other\n",
|
||||
load: func(path string) error {
|
||||
_, err := LoadLabelWorkflowsYAML(path)
|
||||
return err
|
||||
},
|
||||
line: ":2:",
|
||||
},
|
||||
{
|
||||
name: "kind file, unknown kind",
|
||||
content: "bug: tdd\n\nnot-a-kind: x\n",
|
||||
load: func(path string) error {
|
||||
_, err := LoadKindWorkflowsYAML(path)
|
||||
return err
|
||||
},
|
||||
line: ":3:",
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "routes.yaml")
|
||||
writeFile(t, path, tt.content)
|
||||
err := tt.load(path)
|
||||
if err == nil {
|
||||
t.Fatal("load error = nil, want a finding")
|
||||
}
|
||||
if want := path + tt.line; !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("error = %q, want it to name %q", err, want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user