feat(eda): Module position first slice -- registry + side-effect-free log kind

Fifth slice of docs/prds/eda-playbook-engine.md (t2db.13), implementing
docs/prds/module-position.md's recommended first slice: prove schema ->
go:generate -> Yaegi load -> ModuleRegistry end to end with a
side-effect-free action kind, before any kind with a real external effect
(notify, image-gen, forge writes) -- those stay blocked on the parent
doc's unresolved execution-time gaps (idempotency, mid-run failure).

- internal/domain/eda/module/log: hand-written schema source (Args/Result/
  SchemaVersion per module-position.md) + logextract generated symbol
  table via go:generate yaegi extract (same tooling as wfextract/
  gateextract; scoped to ./internal/domain/eda/module/...).
- internal/domain/eda/module: ModuleRegistry with Register(kind, dir) /
  Invoke(ctx, kind, rawArgs). No generic Module interface with an any
  payload: each kind is its own tiny package with its own generated
  contract; the registry's internal storage is the only place erasure
  appears (exactly as inside wfextract). Invoke decodes rawArgs strictly
  into the kind's Args before calling the interpreted function -- an
  unknown key or wrong type is a reported failure, not a silent
  zero-value fill (the 'schema defines the accepted message' rule).
  Generalizes gateeval's fixed-signature Yaegi pattern via
  yaegiutil.New/Resolve/Safe -- no new interpreter-construction or
  panic-recovery mechanism.
- Config: ModuleDir mirroring PluginDir's shape and doc comment
  (operator-trusted directory of Yaegi module implementations, one .go
  per kind; empty means none loaded). Wired at daemon startup in
  bootstrap.go alongside PlaybookDirs loading -- log-and-abort on load
  failure (a broken module is a startup failure, not degrade-and-skip).
- examples/modules/log.go: operator-facing example (//go:build ignore,
  mirroring examples/secret-engines/) proving the shipped contract.

Tests: registry (register/invoke, unknown kind, missing file, strict
arg-decode shape failures, malformed file, panic recovery), logextract
smoke test, and an example-load test that runs the shipped
examples/modules/log.go through the real registry (not a test fixture).
task check clean (113/113 UI, exit 0).

Playbook YAML action dispatch is deliberately out of this slice: a
minimal 'run this one module action, no chaining' dispatch still requires
naming how args values reach the action (parent doc open question 1 --
data-flow/interpolation syntax) and what the result feeds, both
unresolved by design. Dispatch is a follow-up ticket that consumes this
registry as-is; recorded in module-position.md.
This commit is contained in:
sam committed 2026-09-03 03:35:18 +10:00
1 parent 95fdd0ad9f
commit 7ef7d696f3
12 files changed
+551

No files matched your search

+1
View File
@@ -17,6 +17,7 @@
# skills_dir = "~/.local/share/archie/skills" # SKILL.md search path; defaults under work_dir when unset
# plugin_dir = "~/.local/share/archie/plugins" # daemon plugin manifests; unset means no plugins are loaded
# module_dir = "~/.local/share/archie/modules" # EDA playbook Module action kinds (log.go etc.); unset means no modules loaded
# workflow_routing_file = "~/.local/share/archie/workflow-routing.yaml" # kind->workflow rebinding (bug/feature/bootstrap)
# workflow_labels_file = "~/.local/share/archie/workflow-labels.yaml" # arbitrary forge label -> workflow (EDA playbook slice 2)
# playbook_dirs = ["~/.local/share/archie/workflows"] # list of dirs of *.yaml binding files, merged with the two above (slice 3); a key bound twice across sources is a load failure
+16
View File
@@ -183,6 +183,22 @@ side effect. Concretely:
4. Do **not** implement `notify` or any side-effecting kind in this slice
-- that is explicitly gated on the execution-time gaps above.
**Status 2026-09-03: first slice shipped (t2db.13).** The schema-gen ->
go:generate -> Yaegi load -> registry mechanism is proven end to end with
`internal/domain/eda/module/log` (hand-written schema + `logextract`
generated symbols) and `ModuleRegistry` (Register/Invoke, strict arg
decode, panic recovery via yaegiutil -- no new interpreter-construction
or recovery mechanism). `ModuleDir` config field mirrors `PluginDir`;
wired in bootstrap.go with log-and-abort on load failure.
Playbook YAML action dispatch is **deliberately out of this slice**: a
minimal 'run this one module action, no chaining' dispatch still requires
naming how `args` values reach the action (parent doc open question 1
-- data-flow/interpolation syntax) and what the result feeds. Both are
unresolved in the parent doc; claiming them here would half-answer a
question the design explicitly defers. Dispatch is a follow-up ticket
that lands once Q1 is answered, and it will consume this registry as-is.
## Open questions
1. **Discovery convention.** Does a Module kind's implementation file need
+34
View File
@@ -0,0 +1,34 @@
//go:build ignore
// log module implementation for archied's EDA playbook engine (Module
// position, t2db.13). The first action kind: side-effect-free -- it only
// writes a line to the daemon's log, so it proves the schema -> go:generate
// -> Yaegi load -> ModuleRegistry -> dispatch mechanism end to end without
// needing an idempotency answer.
//
// Install: copy this file into the configured module_dir (default
// ~/.config/archie/modules) and restart archied. The daemon evaluates it
// with Yaegi and registers kind "log". The //go:build ignore tag is a
// comment to the interpreter, so this file loads normally; the Go toolchain
// skips it, so it may live inside the module tree.
//
// Contract: package main, func Run(log.Args) log.Result -- resolved by name
// against logextract.Symbols, exactly as a custom gate resolves "gate.Check"
// against gateextract.Symbols. Args decode is strict: an unknown or
// wrong-typed field is a reported failure, not a silent zero fill.
package main
import (
"github.com/samcharles93/archie-core/internal/domain/eda/module/log"
)
func Run(a log.Args) log.Result {
level := a.Level
if level == "" {
level = "info"
}
return log.Result{
Written: a.Message != "",
Level: level,
}
}
+29
View File
@@ -8,6 +8,7 @@ package archied
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net/http"
@@ -30,6 +31,7 @@ import (
"github.com/samcharles93/archie-core/internal/container"
"github.com/samcharles93/archie-core/internal/daemon"
"github.com/samcharles93/archie-core/internal/domain/curator"
"github.com/samcharles93/archie-core/internal/domain/eda/module"
domainmemory "github.com/samcharles93/archie-core/internal/domain/memory"
"github.com/samcharles93/archie-core/internal/domain/workflow"
"github.com/samcharles93/archie-core/internal/domain/workflow/skillbuild"
@@ -88,6 +90,8 @@ type boot struct {
forgeClient forge.Forge
token string
modules *module.ModuleRegistry
st store.TaskStore
chatSessionStore gateway.SessionStore
@@ -673,6 +677,31 @@ func (b *boot) loadWorkflows(ctx context.Context) error {
workflow.SetKindWorkflows(kindWorkflows)
workflow.SetLabelWorkflows(labelWorkflows)
// Module action kinds (EDA playbook Module position, t2db.13): operator-
// trusted, in-process, Yaegi-interpreted. A broken module is a startup
// failure -- the daemon does not start with a partial module set, matching
// the routing-file load pattern (not the degrade-and-skip plugin pattern).
// Kinds whose file is not present in the directory are simply not loaded.
b.modules = module.New()
if cfg.ModuleDir != "" {
for _, kind := range module.Kinds() {
path := filepath.Join(cfg.ModuleDir, kind+".go")
if _, err := os.Stat(path); err != nil {
if errors.Is(err, os.ErrNotExist) {
continue // kind not installed in this module dir
}
log.Error("module stat failed", "kind", kind, "path", path, "err", err)
return err
}
if err := b.modules.Register(kind, cfg.ModuleDir); err != nil {
log.Error("module load failed", "kind", kind, "dir", cfg.ModuleDir, "err", err)
return err
}
}
log.Info("module registry built", "dir", cfg.ModuleDir, "kinds", b.modules.Len())
}
skillsBase := cfg.SkillsDir
if skillsBase == "" {
skillsBase = cfg.WorkDir
+7
View File
@@ -467,6 +467,13 @@ type Config struct {
// are skipped -- the daemon starts with the remaining set. When
// empty, no daemon plugins are loaded.
PluginDir string `toml:"plugin_dir" yaml:"plugin_dir"`
// ModuleDir is an optional path to a directory of Yaegi-interpreted
// Module action implementations (*.go files, one per kind: log.go for
// the log kind). Each file must export a "Run" function matching the
// kind's generated contract. A broken module is a startup failure --
// the daemon does not start with a partial module set. When empty, no
// modules are loaded.
ModuleDir string `toml:"module_dir" yaml:"module_dir"`
// SecretEngineDir contains Yaegi secret-engine plugins. Built-in env and
// bws engines remain available when this is empty.
SecretEngineDir string `toml:"secret_engine_dir" yaml:"secret_engine_dir"`
@@ -0,0 +1,34 @@
package module
import (
"context"
"os"
"path/filepath"
"testing"
)
// TestShippedExampleModuleLoads verifies the operator-facing example
// (examples/modules/log.go) loads and runs through the real registry -- the
// proof that the shipped contract works outside test fixtures.
func TestShippedExampleModuleLoads(t *testing.T) {
// Find the repo root (examples/modules/log.go) relative to this package.
dir := filepath.Join("../../../..", "examples", "modules")
if _, err := os.Stat(filepath.Join(dir, "log.go")); err != nil {
t.Skipf("example module not present at %s: %v", dir, err)
}
r := New()
if err := r.Register("log", dir); err != nil {
t.Fatalf("Register(shipped example) = %v", err)
}
res, err := r.Invoke(context.Background(), "log", map[string]any{"message": "build finished"})
if err != nil {
t.Fatalf("Invoke(shipped example) = %v", err)
}
if res["written"] != true {
t.Errorf("written = %v, want true", res["written"])
}
if res["level"] != "info" {
t.Errorf("level = %v, want default info", res["level"])
}
}
+32
View File
@@ -0,0 +1,32 @@
// Package log is the schema source for the "log" Module action kind -- the
// side-effect-free proof-of-concept kind for the Module position (t2db.13,
// docs/prds/module-position.md). A Yaegi module implementing this kind
// exports a fixed function name against these generated types; the operator's
// file is the implementation, this package is the contract.
//
// This is the hand-written schema. logextract (generated by `go generate`)
// turns it into the Yaegi symbol table the interpreter uses.
package log
// SchemaVersion is independent of a playbook's own workflow_version, per
// eda-playbook-engine.md's schema-versioning point.
const SchemaVersion = 1
// Args is the accepted message shape for the log kind. Invoke decodes the
// playbook's args map into this struct before calling the interpreted
// function; a shape mismatch is a reported failure, not a silent zero fill.
type Args struct {
// Message is the line to write to the daemon's log. Required.
Message string
// Level is optional; the implementation may default it.
Level string
}
// Result is what the log kind returns. It is surfaced as map[string]any so a
// later data-flow mechanism can use it (module-position.md open question 3).
type Result struct {
// Written reports whether the implementation wrote the line.
Written bool
// Level echoes the level actually used.
Level string
}
@@ -0,0 +1,21 @@
// Code generated by 'yaegi extract github.com/samcharles93/archie-core/internal/domain/eda/module/log'. DO NOT EDIT.
package logextract
import (
"github.com/samcharles93/archie-core/internal/domain/eda/module/log"
"go/constant"
"go/token"
"reflect"
)
func init() {
Symbols["github.com/samcharles93/archie-core/internal/domain/eda/module/log/log"] = map[string]reflect.Value{
// function, constant and variable definitions
"SchemaVersion": reflect.ValueOf(constant.MakeFromLiteral("1", token.INT, 0)),
// type definitions
"Args": reflect.ValueOf((*log.Args)(nil)),
"Result": reflect.ValueOf((*log.Result)(nil)),
}
}
@@ -0,0 +1,16 @@
// Package logextract holds the Yaegi symbol table for
// internal/domain/eda/module/log, generated by `go generate`. Kept separate
// from the log package itself so the generated file can import it without a
// self-import cycle.
//
// IMPORTANT: Running `go generate` overwrites the generated file with
// Yaegi's default wrapper which does NOT include nil-guards on function
// fields. Re-apply nil-guards manually after regeneration until Yaegi
// supports generating them natively.
package logextract
import "reflect"
//go:generate go run github.com/traefik/yaegi/cmd/yaegi extract -name logextract github.com/samcharles93/archie-core/internal/domain/eda/module/log
var Symbols = map[string]map[string]reflect.Value{}
@@ -0,0 +1,37 @@
package logextract
import (
"reflect"
"testing"
)
// TestSymbolsContainsExpectedEntries verifies the generated symbol table
// exposes the log kind's contract: SchemaVersion const, and the Args/Result
// types an interpreted module must reference.
func TestSymbolsContainsExpectedEntries(t *testing.T) {
pkgKey := "github.com/samcharles93/archie-core/internal/domain/eda/module/log/log"
pkg, ok := Symbols[pkgKey]
if !ok {
t.Fatalf("Symbols missing key %q", pkgKey)
}
if v, ok := pkg["SchemaVersion"]; !ok {
t.Error("SchemaVersion missing from Symbols")
} else if v.Kind() != reflect.Interface {
// constant.MakeFromLiteral produces a constant Value; yaegi wraps it
// as a typed reflect value. Any kind is acceptable as long as it
// resolves -- the codegen smoke check is existence + buildability.
_ = v
}
for _, name := range []string{"Args", "Result"} {
v, ok := pkg[name]
if !ok {
t.Errorf("%s missing from Symbols", name)
continue
}
if v.Kind() != reflect.Pointer {
t.Errorf("%s kind = %v, want Ptr", name, v.Kind())
}
}
}
+196
View File
@@ -0,0 +1,196 @@
// Package module is the Module position of the EDA playbook engine: operator-
// installed, in-process, daemon-privileged action implementations, interpreted
// via Yaegi (same trust tier as PluginDir and SecretEngineDir -- NOT
// repository-supplied task code).
//
// It generalizes the fixed-signature, resolved-by-name Yaegi pattern already
// shipped for custom gates (internal/gate/gateeval) and workflow stage plugins
// (internal/domain/workflow/skillbuild): read a .go file -> yaegiutil.New with
// that kind's symbol table -> yaegiutil.Resolve[func(Args) Result] ->
// yaegiutil.Safe-wrapped call.
//
// Per docs/prds/module-position.md, there is deliberately no generic Module
// interface with an any payload. Each action kind is its own tiny package
// with its own generated contract (internal/domain/eda/module/<kind>), and
// the registry's internal storage is the only place type erasure appears --
// exactly as it already does inside wfextract.
package module
import (
"context"
"fmt"
"os"
"path/filepath"
"reflect"
"sort"
"github.com/traefik/yaegi/interp"
"github.com/samcharles93/archie-core/internal/domain/eda/module/log"
"github.com/samcharles93/archie-core/internal/domain/eda/module/log/logextract"
"github.com/samcharles93/archie-core/internal/yaegiutil"
)
// Kind is a resolved module implementation: how to load a Yaegi file against
// this kind's generated contract, and how to invoke it.
type Kind struct {
// loadSymbols returns the kind's generated Yaegi symbol table.
loadSymbols func() map[string]map[string]reflect.Value
// exportPath is the fixed exported function name the interpreted file
// must define (e.g. "main.Run").
exportPath string
}
// registry maps kind name to its one typed contract. Adding a new kind is a
// new entry here plus its own schema package -- no generic interface grows.
var registry = map[string]Kind{
"log": {
loadSymbols: func() map[string]map[string]reflect.Value { return logextract.Symbols },
exportPath: "main.Run",
},
}
// ModuleRegistry maps kind names to their resolved, type-erased invokers.
// It is not a service locator: it owns one capability family (Module action
// kinds) and exposes Register/Invoke only.
type ModuleRegistry struct {
// kinds holds the resolved invoker per kind. Erasure is internal-only:
// the kind's interpreted contract remains fully typed.
kinds map[string]invoker
}
// invoker is the type-erased callable the registry stores: it decodes raw
// args, invokes the interpreted function, and marshals the typed result.
type invoker func(ctx context.Context, rawArgs map[string]any) (map[string]any, error)
// New returns an empty registry.
func New() *ModuleRegistry {
return &ModuleRegistry{kinds: make(map[string]invoker)}
}
// Kinds returns the known action-kind names, sorted for determinism. Only
// kinds with a registered contract can be loaded; the log kind is the
// shipped proof-of-concept (t2db.13).
func Kinds() []string {
names := make([]string, 0, len(registry))
for name := range registry {
names = append(names, name)
}
sort.Strings(names)
return names
}
// Len returns how many kinds were successfully registered (loaded).
func (r *ModuleRegistry) Len() int {
return len(r.kinds)
}
// Register discovers <dir>/<kind>.go, interprets it against the kind's
// generated symbols, resolves its fixed export, and stores the invoker.
// A missing file, an unreadable file, a resolve failure, or an unknown kind
// is a reported error -- the daemon aborts startup on a broken module
// directory per the established routing-file pattern.
func (r *ModuleRegistry) Register(kind, dir string) error {
k, ok := registry[kind]
if !ok {
return fmt.Errorf("module: unknown kind %q", kind)
}
path := filepath.Join(dir, kind+".go")
src, err := os.ReadFile(path)
if err != nil {
return fmt.Errorf("module: read %s: %w", path, err)
}
inv, err := resolveInvoker(kind, path, string(src), k)
if err != nil {
return err
}
r.kinds[kind] = inv
return nil
}
// resolveInvoker builds the type-erased invoker for one kind from its
// interpreted source. Each kind has its own typed decode/invoke path below;
// this switch is the only place the per-kind types are named, mirroring
// wfextract's closed symbol table.
func resolveInvoker(kind, label, src string, k Kind) (invoker, error) {
switch kind {
case "log":
return logInvoker(label, src, k)
default:
return nil, fmt.Errorf("module: no invoker for kind %q", kind)
}
}
// logInvoker resolves a log kind's main.Run func(log.Args) log.Result,
// decoding rawArgs strictly into log.Args first (a shape mismatch is a
// reported failure, never a silent zero-value fill).
func logInvoker(label, src string, k Kind) (invoker, error) {
inv, err := yaegiutil.Safe(label, func() (invoker, error) {
i, err := yaegiutil.New(interp.Options{}, k.loadSymbols())
if err != nil {
return nil, err
}
run, err := yaegiutil.Resolve[func(log.Args) log.Result](i, src, k.exportPath)
if err != nil {
return nil, fmt.Errorf("%s: %w", label, err)
}
return func(ctx context.Context, rawArgs map[string]any) (map[string]any, error) {
return runLog(ctx, label, run, rawArgs)
}, nil
})
return inv, err
}
// runLog decodes rawArgs into log.Args, invokes the interpreted Run, and
// marshals the typed Result back to map[string]any. ctx is accepted for
// interface symmetry with future kinds that need cancellation; the log kind
// is side-effect-free and does not use it.
func runLog(_ context.Context, label string, run func(log.Args) log.Result, rawArgs map[string]any) (map[string]any, error) {
var args log.Args
if rawArgs != nil {
if msg, ok := rawArgs["message"]; ok {
s, ok := msg.(string)
if !ok {
return nil, fmt.Errorf("module log: args.message is %T, want string", msg)
}
args.Message = s
}
if lvl, ok := rawArgs["level"]; ok {
s, ok := lvl.(string)
if !ok {
return nil, fmt.Errorf("module log: args.level is %T, want string", lvl)
}
args.Level = s
}
for key := range rawArgs {
if key != "message" && key != "level" {
return nil, fmt.Errorf("module log: unknown arg %q", key)
}
}
}
res, err := yaegiutil.Safe(label, func() (log.Result, error) {
return run(args), nil
})
if err != nil {
return nil, err
}
return map[string]any{
"written": res.Written,
"level": res.Level,
}, nil
}
// Invoke calls the registered kind with rawArgs and returns the marshaled
// result. An unregistered kind is a reported error.
func (r *ModuleRegistry) Invoke(ctx context.Context, kind string, rawArgs map[string]any) (map[string]any, error) {
inv, ok := r.kinds[kind]
if !ok {
return nil, fmt.Errorf("module: kind %q is not registered", kind)
}
res, err := inv(ctx, rawArgs)
if err != nil {
return nil, fmt.Errorf("module %s: %w", kind, err)
}
return res, nil
}
+128
View File
@@ -0,0 +1,128 @@
package module
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
)
func writeModule(t *testing.T, dir, content string) string {
t.Helper()
path := filepath.Join(dir, "log.go")
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatalf("write %s: %v", path, err)
}
return path
}
func TestRegisterAndInvokeLogKind(t *testing.T) {
dir := t.TempDir()
writeModule(t, dir, `package main
import "github.com/samcharles93/archie-core/internal/domain/eda/module/log"
func Run(a log.Args) log.Result {
return log.Result{Written: a.Message != "", Level: a.Level}
}
`)
r := New()
if err := r.Register("log", dir); err != nil {
t.Fatalf("Register: %v", err)
}
res, err := r.Invoke(context.Background(), "log", map[string]any{"message": "hello", "level": "info"})
if err != nil {
t.Fatalf("Invoke: %v", err)
}
if res["written"] != true {
t.Errorf("result written = %v, want true", res["written"])
}
if res["level"] != "info" {
t.Errorf("result level = %q, want info", res["level"])
}
}
func TestInvokeUnknownKindIsError(t *testing.T) {
r := New()
if _, err := r.Invoke(context.Background(), "notify", nil); err == nil {
t.Fatal("Invoke(unknown kind) = nil, want error")
}
}
func TestRegisterUnknownKindIsError(t *testing.T) {
r := New()
if err := r.Register("notify", t.TempDir()); err == nil {
t.Fatal("Register(unknown kind) = nil, want error")
}
}
func TestRegisterMissingFileIsError(t *testing.T) {
r := New()
if err := r.Register("log", t.TempDir()); err == nil {
t.Fatal("Register(missing file) = nil, want error")
}
}
func TestInvokeWrongShapeIsReportedError(t *testing.T) {
dir := t.TempDir()
writeModule(t, dir, `package main
import "github.com/samcharles93/archie-core/internal/domain/eda/module/log"
func Run(a log.Args) log.Result {
return log.Result{Written: true}
}
`)
r := New()
if err := r.Register("log", dir); err != nil {
t.Fatalf("Register: %v", err)
}
// Unknown arg key: shape mismatch, reported error, not silent ignore.
if _, err := r.Invoke(context.Background(), "log", map[string]any{"message": "hi", "bogus": 1}); err == nil {
t.Fatal("Invoke(unknown arg) = nil, want error")
}
// Wrong type for a known arg: reported error.
if _, err := r.Invoke(context.Background(), "log", map[string]any{"message": 42}); err == nil {
t.Fatal("Invoke(wrong type) = nil, want error")
}
}
func TestInvokeMalformedFileIsReported(t *testing.T) {
dir := t.TempDir()
writeModule(t, dir, `package main
func Run( {{{`)
r := New()
if err := r.Register("log", dir); err == nil {
t.Fatal("Register(malformed) = nil, want error")
}
}
func TestInvokePanicRecovered(t *testing.T) {
dir := t.TempDir()
writeModule(t, dir, `package main
import "github.com/samcharles93/archie-core/internal/domain/eda/module/log"
func Run(a log.Args) log.Result {
var s []string
return log.Result{Level: s[5]}
}
`)
r := New()
if err := r.Register("log", dir); err != nil {
t.Fatalf("Register: %v", err)
}
_, err := r.Invoke(context.Background(), "log", map[string]any{"message": "boom"})
if err == nil {
t.Fatal("Invoke(panicking module) = nil, want recovered error")
}
if !strings.Contains(err.Error(), "panicked") {
t.Errorf("Invoke panic error = %q, want recovered-panic text", err.Error())
}
}