Commit Graph
28 Commits
Author SHA1 Message Date
sam 4237b560b0 fix(eda): type-aware actions classifier, one stable-identifier grammar, loader tests that fail for the named reason 2026-09-23 21:48:52 +10:00
sam 71d026feb2 fix(yaegiutil): refuse incomplete Yaegi interface wrappers at load
The generated symbol tables carried 25 hand-added nil guards across three
packages, re-applied by hand after every regeneration because `yaegi extract`
does not emit them. Regeneration silently dropped them, and three package docs
instructed the next agent to put them back.

The guards also answered the wrong way. A nil WResolve returned ("", nil), so a
malformed secret engine served an empty secret and reported success; a nil
WStop reported a clean shutdown for a module that never ran one.

Reject the wrapper where it enters the daemon instead. Yaegi type-checks
"var X Iface = impl{}" and refuses an incomplete impl on its own, but the
wrapper type is exported into the symbol table, so interpreted code can build
one directly and struct-literal semantics leave the omitted method nil.
Resolve now names the missing method and skips the plugin.

Generated tables are used verbatim again. Also names the wanted type on a
failed assertion, which formatted as "want <nil>" for every interface target.
2026-09-20 12:05:39 +10:00
sam 23a301f89f refactor(plugin): delete the unused Registry type
Registry carried no plugin identity, discovery, compatibility or
lifecycle, so it was never the seed for the approved internal/domain/plugin
package. Host, Manifest and Module are, and they stay.

Regenerating pluginextract strips the interface-wrapper nil guards, so
those are restored by hand; the only intended change to the generated
table is dropping the Registry symbol.
2026-09-20 11:38:15 +10:00
sam 74682465ca chore(plugin): delete the unwired capability-host health surface (#882)
* chore(plugin): delete the unwired capability-host health surface

plugin.Host.Health, plugin.Host.Manifests, ModuleStatus and their private
safeHealth helper were implemented, exhaustively tested, and called from nowhere
but their own tests. AGENTS.md forbids speculative surfaces, and no settled
design places plugin module status anywhere that exists:

- docs/prds/status-health-surface.md Phase 2 is self-described as "a proposed
  resolution, awaiting Sam's sign-off -- not yet implemented", and does not
  mention plugin status even in that unapproved list.
- The readiness subsystem enumerates exactly five probes -- state_db, config,
  disk, model, gateway (internal/app/archied/readiness.go) -- with no plugin
  probe.
- docs/architecture/plugins-and-extensions.md names no operator-visible plugin
  status surface.
- The only bead naming it is closed.

plugin.Host itself stays: it is the daemon's live capability host, constructed
and Start/Stop-ed from internal/app/archied. Only the unconsumed observability
methods go.

host_contract_test.go encoded the defect -- it asserted plugin.Host *defines*
"Health" and "Manifests" -- so those two names are removed from its list, which
is now exactly the shipped lifecycle: Register, Start, Stop. The three tests
covering the deleted surface are removed with it.

The Yaegi symbol table (internal/plugin/pluginextract) was regenerated rather
than hand-edited, so ModuleStatus leaves the reflected type set with it.

Reachability evidence: Host.Health and Host.Manifests appear only in their own
tests; plugin.Host is otherwise reached from bootstrap.go and gateway.go for
Register/Start/Stop. Note that "task reachability" does NOT flag these -- the
address-taken constructor is a known blind spot of the whole-program RTA.

* chore(plugin): delete the dead Module.Health chain wired to nothing

Extends the capability-host observability deletion: plugin.Module.Health
was a required interface method with real implementations in
tools/provider/mcp and tools/provider/builtin, aggregated by
tools/provider.Registry.Health via a provider-package-local safeHealth,
and backed by per-module LifecycleState bookkeeping in plugin.Host
(registeredModule.state, setModuleState). None of it had a caller before
this change -- Host.Health was the only thing that would ever have read
a registered module's Health(), and Host.Health was already deleted as
unreachable. registeredModule.state was write-only: nothing ever read it.

Deleted:
- plugin.Module.Health, plugin.Health/HealthStatus (+ consts),
  plugin.LifecycleState (+ consts), legacyModule.Health,
  registeredModule.state, Host.setModuleState and its call sites.
- tools/provider.Registry.Health, tools/provider.safeHealth.
- tools/provider/mcp.Provider.Health, tools/provider/builtin.Provider.Health.
- Tests exclusively covering the above, and fakes' now-unnecessary Health
  methods (plugin/host_test.go, pluginextract/reachability_test.go,
  tools/provider/{provider,optional}_test.go, tools/provider/mcp/provider_test.go,
  app/agentworker/mcp_providers_test.go).

internal/plugin/pluginextract's generated file was regenerated via
`go generate ./internal/plugin/...` and the yaegi wrapper nil-guards
(dropped by fresh generation, as documented in reachability_test.go)
were reapplied by hand to WManifest/WStart/WStop/WName/WVersion,
following this repo's established post-regen pattern.
2026-09-19 14:10:39 +10:00
sam 4b250ba929 test(plugin): stop asserting documentation prose in a test
TestPluginEngineRuleIsCanonicalAndSharedByAgentInstructions read
ARCHITECTURE.md and CLAUDE.md and failed if their prose did not contain a
particular heading and link. That is not a behaviour check: it makes an
agent-instruction file load-bearing for the build, so editing documentation
can break compilation of the test suite, and a doc drift becomes a red gate
that says nothing about the code.

The genuine invariant -- that an engine family has one owner that both
registers and resolves it -- is asserted on the AST by
TestEngineFamilyInspectionRejectsInvalidShapes, which is untouched.

Removes the readTestFile helper it was the only caller of.
2026-09-14 08:09:14 +00:00
sam 3d37a437ff fix(extract): regenerate stale Yaegi symbol tables; fix wfextract directive
Two pre-existing issues discovered while scoping the Module-position slice
(archie-core-e7bo), unrelated to the playbook engine work:

1. wfextract's //go:generate directive still targeted
   internal/workflow, a path gone since the domain migration -- the
   committed generated file was hand-patched for imports only, never
   regenerated. 'go generate ./...' failed repo-wide, silently.

2. Regeneration surfaced real drift on three surfaces:
   - wfextract: the committed table predated the migration and was
     missing everything added since -- routing YAML loaders (t2db.9/.10/
     .11), the review types, and the Forger/Reviewer/Trees interfaces
     that TaskContext exposes to every interpreted stage plugin. A stage
     plugin referencing tc.Trees or the review types could not resolve
     them.
   - pluginextract: internal/plugin/host.go (f73b2d1) added the
     Host/Module/Manifest/Permission/Health/CapabilityKind/LifecycleState
     /ModuleStatus types and NewHost/AdaptLegacy/HostAPIVersion, none of
     which were ever extracted. A daemon plugin written against the
     capability-host API (bootstrap.go:707 LoadDir) silently had no way
     to reference them.
   - secretextract: symbol exports were current; only the hand-applied
     nil-guards and comments differ from raw generator output.

Fix: update the wfextract directive to internal/domain/workflow and
regenerate all four extract surfaces via 'go generate ./...'. Re-apply
the documented hand-written nil-guards on interface wrappers (Yaegi
wrappers omit them natively), restating the IMPORTANT note in wfextract's
package doc (it now has interface wrappers for the first time). Delete
the stale hand-patched wfextract generated file in favour of the
correctly-named regeneration.

Regression coverage (the previously-missing types are now reachable from
interpreted code, proven by tests that fail against the old tables):
- wfextract/reachability_test.go: interprets a stage plugin referencing
  Trees/Reviewer/LoadPlaybookDirs and calls it; plus a Go-side nil-guard
  test on the three wrappers.
- pluginextract/reachability_test.go: interprets a daemon plugin building
  a Host from a Go-side Module and calls it end to end; plus nil-guard
  tests on the _Module/_Plugin wrappers.
- secretextract keeps its existing wrapper nil-guard tests.

Verified red-then-green: after a fresh 'go generate ./...' wipes the
guards, the guard tests fail with nil-pointer panics; after re-applying,
they pass. task check clean (113/113 UI, exit 0).
2026-09-03 03:30:23 +10:00
sam 0fc20319c7 fix(lint): noctx fixes and dead-code cleanup
- install_script_test.go: exec.Command -> exec.CommandContext (noctx)
- architecture_test.go, store_test.go: go fix removed now-unused
  inspectEngineFamilies/parsePackageFiles/stringPtr helpers.
2026-08-31 21:35:58 +10:00
sam 805c508d41 fix(app): repoint capability-host wiring contract test at internal/app/archied
Second architecture test (after contract_test.go) that still grepped
cmd/archied for wiring strings following the cmd/archied ->
internal/app/archied extraction. task check now passes clean.
2026-08-31 21:27:07 +10:00
sam 3129422a79 fix(plugin): add missing CLAUDE.md reference to plugin engine rule
Test TestPluginEngineRuleIsCanonicalAndSharedByAgentInstructions was
missing entirely, letting CLAUDE.md drift out of sync with
ARCHITECTURE.md#plugin-engine-rule-strict. Added the test and the
missing reference.
2026-08-31 21:25:20 +10:00
sam 5465335140 refactor: remove unused Operator param. 2026-08-29 00:20:40 +10:00
sam fd1276cd4b fix: go.mod bumped to 1.27.0 and stupid tests removed 2026-08-28 23:44:41 +10:00
sam ff4af61e2b refactor: extract daemon bootstrap into phased wiring 2026-08-16 03:12:18 +10:00
sam 6108fc5548 fix: use errors.Is/As for context errors and cleanup 2026-07-30 00:06:10 +10:00
sam a73119c7cc Merge remote-tracking branch 'origin/fix/43-loaddir-no-timeout-around-yaegi-eval-can-hang-daemon-startup' 2026-07-29 23:15:58 +10:00
sam d9f4de068b chore: golangci-lint --fix + formatters (gci, gofumpt, gofix, goimports)
Auto-fix lint issues and reformat imports across all files.
gci: group imports per project convention
gofumpt: format unused imports with blank lines
gofix: Go standard fixes
goimports: organize imports

Remaining 54 issues require manual judgment (not auto-fixable).
2026-07-29 03:23:57 +10:00
archie 255f580cf6 test: failing repro for #43 2026-07-27 03:01:56 +00:00
sam f73b2d1c8c feat: memory manager sync/prefetch/scan/shutdown + broad in-flight epic work
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:

- Memory manager (18.7-18.16): background sync pipeline, prefetch with
  timeout, memory-write notification, threat scanning, shutdown drain
  with abandoned-write tracking, frozen snapshot pattern. Wired
  ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
  and notification actually apply to writes, not just exist as unused
  primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
  internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
  multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
2026-07-27 03:34:46 +10:00
sam e8f78dd0d5 test: regression tests for all audit gaps — 6 enhanced + 4 new packages
Telegram: splitLongMessage (8 cases), isChatAllowed, updateMetadata (4 cases),
  panicRecoveryMiddleware, updateLoggingMiddleware, authorizedMessage, stop-while-stopped
Container: error paths, payload round-trip, overwrite, minimal payload
Events: buffer defaults, subscribe-after-close, double-close safety,
  slow-subscriber isolation, multi-subscriber ordering, publish timestamping
NATS RPC: envelope edge cases, JSON round-trip, request timeout, rollback
NellDB: empty store (6 ops), concurrent enqueue/claim (20 goroutines),
  context cancellation, oversized payload clipping, unmarshalable data,
  status transitions, events ordering, recover stale, clear terminal
Gateway tasks: empty/long title, idempotent enqueue, chat label

New packages with tests (formerly zero-test):
  internal/gate (220 lines), internal/gate/gateextract,
  internal/plugin/pluginextract, internal/workflow/wfextract

All 37 packages pass. Fixes regression gaps from audit.
2026-07-26 10:05:44 +10:00
sam f6d51416f0 chore: replace 607 em-dashes with double hyphens project-wide
Includes MCP crash-after-init test fix (crash moved from handle() to serve()
entry point so transport sees immediate subprocess death).
2026-07-26 09:53:33 +10:00
sam 9df02f2b60 fix: golangci-lint baseline to zero findings (39→0)
19 files touched. All fixes are in code — no lint configuration
weakened:
- Added unchecked error handling in defer cleanup paths
- Removed dead variables and unused functions
- Added missing imports (io in gitea.go)
- Simplified conditions (TrimSuffix, De Morgan, slices.Contains)

Also fixes JetStream TempDir teardown race (archie-core-340):
t.Cleanup(srv.Shutdown) now registers after t.TempDir() so
server shutdown (including JetStream files) completes before
temp dir removal. Added srv.WaitForShutdown() for deterministic
cleanup ordering.

Closes: archie-core-756, archie-core-340
2026-07-25 23:51:13 +10:00
sam d80401d595 feat: add config overlay and shared RPC/Yaegi packages
- Introduce `-config-overlay` flag and `config.LoadOverlay()` to merge a base config with an overlay file, only overriding fields the overlay sets. This enables docker-compose to mount the base config from the host and apply only deployment-specific overrides.
- Create `internal/natsrpc` package with reusable NATS request/reply primitives (Envelope, Client, Call, RegisterAll, Respond). Migrate storerpc, worktreerpc, and forgerpc to use it, eliminating duplicated marshal/unmarshal and error handling.
- Create `internal/yaegiutil` package with shared interpreter construction (stdlib + extra symbols), generic export resolution (Resolve), and panic recovery (Safe). Migrate all Yaegi-based surfaces (gate scripts, daemon plugins, skill plugins, workflow stages, skill scripts) to use it, reducing boilerplate and centralizing panic safety.
- Simplify `config.docker.toml` to contain only overlay fields (paths, container config) and remove the duplicated repos list – the base config is now the single source of truth.
- Update docker-compose.yml to pass both -config and -config-overlay flags and mount the base config from the host.
- Add tests for LoadOverlay, DockerConfigOverlaysExampleConfig, Envelope, and yaegiutil utilities.
2026-07-24 12:57:10 +10:00
sam fd360fcc0b fix: resolve 8 remaining design-decision audit items
Fixes (4 items):
  S4 — Add TypeScript, Deno, Bun cache ecosystems to
       cacheVolumesByEcosystem. TypeScript shares node+pnpm volumes;
       Deno and Bun get dedicated cache volumes.
  R4 — Share single Docker client between container.Pool and
       storage.DockerBackend. NewPool now accepts an optional
       DockerClient in Config; if nil, creates its own. Pool
       tracks ownership (ownCli) and only closes its own client.
  S2 — Wire Storage.Teardown() call in daemon.process() after
       workflow.Run(). Docker backend is a no-op; this closes the
       interface contract for future backends.
  R5 — Emit observability event on AugmentRegistry failure.
       Previously only logged a warning; now emits a
       'registry_augment_failed' event with error detail.
       Promoted log level to Error.

Documentation (4 items):
  D1 — Document PluginRegistry as reserved for future daemon
       extension points (forge resolvers, storage backends, etc.)
  S1 — Fix /data/ layout comment to reflect actual task.json path
       (/data/worktree/task.json, not /data/task.json)
  S3 — Add TODO(PRD §3) for session.jsonl, memory.jsonl, plugins/
       volume — these need agent-side implementation first
  N6 — Document that VolumeInspectOptions{} requires moby v25+
       (go.mod pins compatible version)
  N7 — Document that _Plugin.IValue is required by Yaegi interface
       bridging; warn that go generate strips nil-guards

Test: TestCacheMountsForTypeScript verifies typescript ecosystem
returns node+pnpm volumes sorted by destination.

Verification: single client.New(client.FromEnv) in main.go;
all 21 storage tests pass; full suite green.
2026-07-22 05:00:37 +10:00
sam 8a8e676bd6 fix: address 6 'Soon' resilience audit findings
R2  (resilience): skip broken stage plugins, don't abort daemon startup.
    BuildWorkflow now skips individual broken plugins with a warning
    instead of hard-failing. BuildRegistry and AugmentRegistry skip
    broken skills with a warning instead of returning an error. One
    bad plugin in one skill no longer prevents the daemon from starting
    or blocks other skills from registering their workflows.

R1  (resilience): log skipped daemon plugins in LoadDir. Every failure
    path now emits a WARN log via slog.Default() — unreadable files,
    compile errors, missing Plugin exports, wrong types. Operators can
    now see WHY a plugin was skipped. PRD says 'logged and skipped';
    code now matches spec.

N3  (resilience): log symbol-table load errors separately from plugin
    source errors. A broken pluginextract.Symbols no longer causes
    silent skip of ALL plugins — each failure is logged with the
    symbol table index that failed.

N4  (resilience): centralize Yaegi interpreter setup in
    newStageInterpreter(). Each plugin file still needs its own
    interpreter (package main with duplicate Stage() symbols cannot
    co-evaluate), but the setup is now explicit and documented.

N5  (resilience): document tuple fallback positional-field fragility.
    The func() (string, func(...)) fallback in loadStage now warns
    that it constructs Stage with positional fields and will silently
    zero-value any future Stage fields.

R3  (resilience): document nil-client edge case in ensureVolume.

Tests: TestBuildWorkflowSkipsBrokenPlugin (broken plugin skipped,
surviving stages execute), TestBuildRegistrySkipsBrokenSkill (bad
skill doesn't block others, built-in fallbacks preserved).
2026-07-22 04:55:26 +10:00
sam 04175a1a6a fix: address 9 'Now' adversarial audit findings
P1  (panic):      nil Storage guard in daemon.process()
C1  (correctness): check WriteTaskJSON error — don't start blind
C2  (correctness): move 'working' label after container setup
C3  (correctness): warn on duplicate workflow names in registry
C4  (correctness): test that generated _Plugin wrapper has nil-guards
C5  (correctness): .agents/skills as file (ENOTDIR) not fatal
C6  (correctness): ecosystem case-insensitive in cacheMounts
N1  (correctness): TOCTOU race — treat 'already exists' as success
N2  (correctness): recover from panicking plugin.Name() at startup

--- Details ---

P1: d.Storage.Setup() is now guarded by a nil check. In normal
operation Storage is always set when ContainerPool is set, but
the invariant is now explicit and enforced.

C1: container.WriteTaskJSON() error was discarded with _. Now it
returns early with a park transition — the container never starts
without a boot-time brief.

C2: SetStateLabel('working') was called before container setup.
If Acquire failed, the task was parked but the label stayed
'working' permanently. Now the label is set only after successful
container setup (or immediately if no containers are configured).

C3: BuildRegistry and AugmentRegistry now log a warning when two
skills declare the same workflow name and one overwrites the other.

C4: TestGeneratedWrapperHasNilGuards verifies that nil WName/
WVersion don't panic. If go generate strips the nil-guards, this
test catches it before the daemon crashes.

C5: skill.Catalog() now returns nil (no skills) instead of
propagating ENOTDIR errors when .agents/skills is a file.

C6: cacheMounts() normalizes ecosystem to lowercase. 'Python',
'NODE', 'Go' all resolve correctly now.

N1: ensureVolume() treats Docker's 'already exists' error as
success, closing the TOCTOU window between VolumeInspect and
VolumeCreate.

N2: safePluginInfo() wraps plugin.Name()/Version() calls in a
recover. A plugin with a panicking Name() no longer crashes the
daemon at startup.
2026-07-22 04:50:11 +10:00
sam 9bd61e61ac fix: nil-guard _Plugin wrapper, add adversarial tests
Two bugs found by adversarial tests:

1. _Plugin wrapper panicked on nil WName/WVersion — the daemon calls
   Name()/Version() at startup to log loaded plugins, so a plugin with
   nil func fields would crash the daemon. Fixed with nil guards
   returning empty strings.

2. LoadDir returned nil for empty directories instead of an empty
   slice — inconsistent with the contract (nil means 'no plugin dir
   configured', empty means 'no plugins found').

Added 9 adversarial tests that attempt to break LoadDir:
- Nil function fields (must not panic on load or call)
- Empty name/version (valid per interface contract)
- Wrong export type (int instead of Plugin — must skip)
- Partial interface implementation (missing method — must skip)
- Empty directory (returns empty slice, not nil)
- Nested subdirectories (must not be traversed)
- Panic in Name() function (deferred to call time, loader survives)
- Unused import (must not crash eval)
- Go build tags (must be ignored in Yaegi)

Why: per test-first-build skill, every build/test failure results in
a test that would have caught it. Adversarial tests alongside
confirmation tests prevent silent breakage.
2026-07-22 04:12:56 +10:00
sam f38ba5d515 feat: wire Layer 2 daemon plugin loader into archied startup
Daemon plugins (Yaegi-interpreted .go files) are now loaded from the
configured plugin_dir at startup. Each file must export a 'Plugin'
variable using the _Plugin wrapper struct, which bridges Yaegi types
to the Go plugin.Plugin interface.

Changes:
- internal/plugin/pluginextract/ — generated Yaegi symbol table for
  the plugin package (mirrors wfextract pattern), including the
  _Plugin interface wrapper that lets interpreted code implement
  Go interfaces
- LoadDir now accepts optional symbol tables via variadic parameter,
  avoiding an import cycle between plugin and pluginextract
- Plugin tests rewritten as behavioral (not structural), verifying
  actual Yaegi interpretation of valid, invalid, and missing plugins
- config.PluginDir field for specifying the plugin directory
- daemon.PluginRegistry field for holding loaded plugins
- main.go wires plugin loading: LoadDir at startup, register into
  daemon, log each loaded plugin

Why: the Plugin interface and LoadDir existed but weren't connected
to the daemon. Skills define workflows; daemon plugins are the
extension point for forge resolvers, storage backends, and other
core-daemon concerns.
2026-07-22 04:11:02 +10:00
sam d3c843e8f5 feat: close final two gaps — agent skill loading + core plugin registry
Agent skill loading: HandleMessage now loads .agents/skills/<workflow>/SKILL.md
from its own workspace before running stages. Skill body prepended to each
stage's Mission. Mirrors daemon-side loadSkillBody but owned by the agent
per PRD section 1. workflowToSkillDir mapping added.

Core plugin registry: internal/plugin package with Plugin interface
(Name, Version), Registry type, and LoadDir function. Yaegi-interprets
.go files from ~/.config/archie/plugins/ at startup. Failed plugins are
skipped. PRD section 5 Layer 2.

All 21 PRD claims now have tests. Zero failing tests.
2026-07-22 03:37:15 +10:00
sam 39b66fc1ef test: add failing regression tests for the two remaining gaps
1. Agent doesn't load its own skills:
   TestHandleMessageLoadsSkillsFromWorktree — creates a worktree with
   .agents/skills/tdd-bugfix/SKILL.md, sends a bare mission to
   HandleMessage, verifies the agent loaded the skill body from its
   workspace. Fails at runtime: HandleMessage never reads .agents/skills/,
   the Mission reaches the runner unchanged.

2. Core plugin registry nonexistent:
   TestPluginInterfaceExists, TestRegistryExists,
   TestLoadPluginsFromConfigDir — fails to compile: Plugin interface,
   Registry type, and LoadDir function are undefined. PRD section 5
   Layer 2 entirely aspirational.
2026-07-22 03:35:03 +10:00