Commit Graph
18 Commits
Author SHA1 Message Date
sam bbbb527721 feat(webui): finish redesign dashboard task and secret settings 2026-10-13 16:41:35 +11:00
sam 8b28857ff3 feat(secretengine): serve secret engines as go-plugin extensions
A secret engine is now a subprocess serving secretengine.v1, installed through
the package store's new extensions family and enabled by extension-settings.
bws leaves the binary, as do the Yaegi engine loader and its examples.
2026-10-04 00:40:23 +10:00
sam 96e5d7b84b refactor: strip doc and issue narration from code comments
Remove PRD/doc paths, bead IDs, phase notes and deleted-test references
from comments; drop test-only hooks left by the test deletion. Includes
in-progress archied bootstrap, gateway and query changes.
2026-10-03 21:47:46 +10:00
sam ff424b70ac refactor: testing from the beginning. a journey in 150k lines of tests to none 2026-10-03 21:16:25 +10:00
sam c835aabb1e feat(plugins): load plugin and secret-engine files without a restart
docs/prds/plugin-settings-live.md: a `plugin-settings` change now applies
live for plugin_dir, module_dir and secret_engine_dir, and a file dropped
into any of those directories loads on the reconciliation tick.

- PluginSettingsKind moves to ApplyMode "live" and joins
  runtimeResourceKinds, so a stored directory change re-layers immediately.
- A reconciler re-runs the directory loads and the layering on the
  apply-status restamp interval. Files are content-hashed, so only new and
  changed files load.
- plugin.Host grows Add and Replace for the running state (Register keeps
  its pre-Start rule), and plugin.LoadFile / secret.Registry.LoadFile are
  the single-file load paths the boot loader and the reconciler share.
- Yaegi cannot unload, so a removed file keeps running and the removal is
  reported through apply status against the live version until restart; a
  clean pass clears it.
- A plugin-settings document that changes skills_dir is refused before the
  layering, since skills_dir has no live consumer.
- applystatus.Reporter.AppliedVersion lets the reconciliation report an
  outcome without resetting the version to zero.
2026-10-02 10:01:35 +10:00
sam 1e7f478d2f fix(yaegiutil): refuse incomplete Yaegi interface wrappers at load
The generated symbol tables carried 25 hand-added nil guards across three
packages, re-applied by hand after every regeneration because `yaegi extract`
does not emit them. Regeneration silently dropped them, and three package docs
instructed the next agent to put them back.

The guards also answered the wrong way. A nil WResolve returned ("", nil), so a
malformed secret engine served an empty secret and reported success; a nil
WStop reported a clean shutdown for a module that never ran one.

Reject the wrapper where it enters the daemon instead. Yaegi type-checks
"var X Iface = impl{}" and refuses an incomplete impl on its own, but the
wrapper type is exported into the symbol table, so interpreted code can build
one directly and struct-literal semantics leave the omitted method nil.
Resolve now names the missing method and skips the plugin.

Generated tables are used verbatim again. Also names the wanted type on a
failed assertion, which formatted as "want <nil>" for every interface target.
2026-09-20 12:05:39 +10:00
sam 494cf02c1c fix(secret): recover a panicking secret engine instead of crashing
Registry.Resolve and Getenv called engine.Resolve directly with no panic
recovery, so a defective interpreted engine (sops/Vault plugin) took the
whole daemon down at boot when resolving a provider credential
(provider_secrets.go's only caller has no recovery of its own). Wrap both
call sites in yaegiutil.Safe, the same recovery every other Yaegi
extension surface in this repo already uses.
2026-09-15 04:38:51 +10:00
sam da28ddb5e6 refactor(ui): sever the archie-ui binary from every daemon runtime package
Completes the objective deletion gate for the UI Service
(archie-core-8cda.5.6): go list -deps ./cmd/archie-ui now links zero
banned packages -- no modernc.org/sqlite, no internal/store, gateway,
channels, memory, secret, agentexec, tools, skill, yaegiutil, curator,
or the workflow engine. The closure shrinks from 68 to 26 archie-core
packages. Each cut is a producer-side move; no consumer-side defensive
shims.

- Store contracts (TaskStore family, value types, sentinels) moved from
  internal/store to internal/domain/storecontract -- the approved contract
  location per state-store-contract rev. 2d, and NOT internal/contracts:
  that tree is buf generate output and tools/proto.sh check deletes
  hand-written files there. internal/store keeps aliases; sentinel
  message strings are byte-identical and remain the wire contract.
- Chat contracts (ChatContract, sessions, turns, streaming, commands)
  moved from internal/gateway to internal/domain/messaging; gateway keeps
  aliases. ToolCallEvent rendering helpers become gateway functions
  (methods cannot live on alias types).
- Task vocabulary (Task, statuses, mid-run Store, Definition) split into
  internal/domain/workflow/task; the pipeline engine stays in package
  workflow with aliases, so daemon-side callers are unchanged.
- overlay.DeniedKeys moved to internal/infrastructure/configuration:
  webui referenced the constant and that dragged the overlay SQLite
  store in.
- webui.Server's concrete runtime handles become webui-owned view
  interfaces with daemon-side bootstrap adapters: channel lifecycle
  (internal/channels/status), MemoryStatus, CuratorStatus, and a
  SkillCatalog seam replacing the direct skill.DefaultRoots/CatalogRoots
  calls. Each degrades to an empty page when unwired.
- config.SecretRef moved into internal/config (config vocabulary);
  internal/secret keeps an alias and Registry.Resolve absorbs the
  zero-value short-circuit.

Docs: state-store-contract rev. 2d relocation record; ui-service-boundary
deletion gate marked satisfied; ARCHITECTURE.md package map rows.
2026-09-10 03:03:33 +10:00
sam 43c6f257ed fix(extract): regenerate stale Yaegi symbol tables; fix wfextract directive
Two pre-existing issues discovered while scoping the Module-position slice
(archie-core-e7bo), unrelated to the playbook engine work:

1. wfextract's //go:generate directive still targeted
   internal/workflow, a path gone since the domain migration -- the
   committed generated file was hand-patched for imports only, never
   regenerated. 'go generate ./...' failed repo-wide, silently.

2. Regeneration surfaced real drift on three surfaces:
   - wfextract: the committed table predated the migration and was
     missing everything added since -- routing YAML loaders (t2db.9/.10/
     .11), the review types, and the Forger/Reviewer/Trees interfaces
     that TaskContext exposes to every interpreted stage plugin. A stage
     plugin referencing tc.Trees or the review types could not resolve
     them.
   - pluginextract: internal/plugin/host.go (f73b2d1) added the
     Host/Module/Manifest/Permission/Health/CapabilityKind/LifecycleState
     /ModuleStatus types and NewHost/AdaptLegacy/HostAPIVersion, none of
     which were ever extracted. A daemon plugin written against the
     capability-host API (bootstrap.go:707 LoadDir) silently had no way
     to reference them.
   - secretextract: symbol exports were current; only the hand-applied
     nil-guards and comments differ from raw generator output.

Fix: update the wfextract directive to internal/domain/workflow and
regenerate all four extract surfaces via 'go generate ./...'. Re-apply
the documented hand-written nil-guards on interface wrappers (Yaegi
wrappers omit them natively), restating the IMPORTANT note in wfextract's
package doc (it now has interface wrappers for the first time). Delete
the stale hand-patched wfextract generated file in favour of the
correctly-named regeneration.

Regression coverage (the previously-missing types are now reachable from
interpreted code, proven by tests that fail against the old tables):
- wfextract/reachability_test.go: interprets a stage plugin referencing
  Trees/Reviewer/LoadPlaybookDirs and calls it; plus a Go-side nil-guard
  test on the three wrappers.
- pluginextract/reachability_test.go: interprets a daemon plugin building
  a Host from a Go-side Module and calls it end to end; plus nil-guard
  tests on the _Module/_Plugin wrappers.
- secretextract keeps its existing wrapper nil-guard tests.

Verified red-then-green: after a fresh 'go generate ./...' wipes the
guards, the guard tests fail with nil-pointer panics; after re-applying,
they pass. task check clean (113/113 UI, exit 0).
2026-09-03 03:30:23 +10:00
sam 88a995a736 test(coverage): add tests for workintake, secret, forge, natsrpc
Closes coverage gaps flagged in load-bearing packages:
- internal/domain/workintake: 14.3% -> 94.6%
- internal/secret/enginehost: 0% -> 100%
- internal/secret/secretextract: 10% -> 100%
- internal/forge: 36.1% -> 77.0%
- internal/natsrpc: 28.3% -> 91.3%
2026-08-31 21:24:25 +10:00
sam a5a878c2d0 feat(secret): add sops, vault/openbao and age engine plugins
CLI-based, env-configured, cache-less engines resolving SecretRef values
through the existing Yaegi plugin surface. Fresh resolve per call, so
values updated in the backend are picked up without daemon restart —
closing the gap that motivated the remote secret-input epic (#504),
whose value-entry web form duplicated the Bitwarden app for bws.

- examples/secret-engines/: sops (SOPS_FILE), vault (VAULT_ADDR/TOKEN,
  key "path/field" or VAULT_SECRET_PATH prefix), age (AGE_FILE/
  AGE_IDENTITY, key=value lines). //go:build ignore keeps them out of
  the Go toolchain; yaegi treats the tag as a comment and loads them.
- enginehost: host-side Run helper registered as a Yaegi symbol. yaegi's
  interpreted os/exec cannot pass env to child processes, so plugins
  shell out through host code; bounded by a 30s timeout like the bws
  engine.
- secret.LoadDir seeds the interpreter with the host environment: yaegi's
  interpreted os package keeps a private env otherwise, and plugin
  os.Getenv would silently see nothing.
- cmd/archied wires enginehost + unrestricted symbols into the engine
  plugin loader, matching skillscript's interpretation posture.
- tests load the real shipped plugin files with fake CLIs on PATH;
  added CONTEXT.md glossary for the remote secret-input vocabulary.
2026-08-18 00:32:57 +10:00
sam 3ae59c99c0 fix(secrets): wire provider credentials through engines 2026-07-31 08:45:58 +10:00
sam 6108fc5548 fix: use errors.Is/As for context errors and cleanup 2026-07-30 00:06:10 +10:00
sam 65aa5b462d chore: cleanup 2026-07-28 18:46:50 +10:00
sam 4ee8579ddc refactor: phase 2 of lint remediation — mechanical fixes (qa4)
130 -> 96. forcetypeassert, predeclared and usetesting are now at zero.

Fanned out across four isolated worktrees partitioned by package so no
two workers could touch the same file: internal/tools (20 assertions),
internal/webui, internal/memory + internal/ratelimit, internal/secret.
Four stragglers outside those scopes fixed inline (internal/pairing,
internal/forge, internal/channels/telegram, internal/tools/mcp).

Unchecked type assertions became comma-ok with a t.Fatalf naming the
actual type. One site in a goroutine uses t.Errorf + return instead,
since t.Fatalf from a non-test goroutine is undefined behaviour.

predeclared renames chose names from meaning rather than adding a
suffix: max -> maxQueryLen, maxMatchLen, maxRequests, alphabetLen, and
nextBackoff's max -> ceiling. `real` -> `base` in the webui tests.

os.Setenv became t.Setenv, which restores on cleanup, so the paired
deferred Unsetenv calls went with it.

Note for future fan-outs: the agent worktrees branched from origin/main
(a7388a0), not from this branch, so every diff arrived based on a tree
without the phase 1 autofix. They were applied with `git apply --3way`
and the phase 1 changes to internal/memory/manager.go were verified to
survive — in particular the repaired `ctx, cancel =` assignment. A plain
`git apply` would have silently reverted it.
2026-07-28 01:37:20 +10:00
sam 6fe1a7d037 fix: lint cleanup across repo (errcheck, staticcheck, ineffassign, unused)
- daemon.go: real bug — maybeRetryParked resolved fg := d.forgeFor(task)
  (matching markDead's pattern) but called d.Forge.SetStateLabel instead
  of fg.SetStateLabel, silently using the wrong forge client for
  multi-identity/multi-forge task routing. Fixed to use fg throughout.
- telegram.go: updateThreadID existed but was never called anywhere;
  wired it into updateLoggingMiddleware so debug logs include thread_id.
- Two SA9003 empty-branch tests (nell adapter_test.go, tools
  toolentry_test.go) were dead assertions with no body; implemented the
  actual check instead of deleting them.
- memory manager_test.go: mockProvider.lastConfigSchema/lastBackupPaths
  were declared but never recorded; wired GetConfigSchema/BackupPaths to
  populate them and strengthened the aggregation tests to assert on
  per-provider content, not just counts.
- Removed genuinely dead gateway messageevent_test.go helpers (intPtr/
  i64Ptr, marked //go:fix inline, already fully superseded by the new()
  builtin used throughout the file).
- Remaining errcheck findings (defer Close/Flush/Write, os.Setenv, etc.
  in test files) fixed with the existing `_ = ` convention already used
  elsewhere in the codebase.

go build ./..., go test ./... (all packages ok), and
golangci-lint run ./... all clean.
2026-07-27 03:49:33 +10:00
sam 2b1045c0ad feat: secrets-engine registry with yaegi extension interface (archie-core-xec)
Adds internal/secret: a pluggable Engine interface (Name/Version/Resolve)
resolved through a Registry, so secrets backends (sops, Bitwarden Secrets
Manager, Vault/OpenBao) aren't hard-coded into the binary. Only the "env"
engine is compiled in; custom engines are Yaegi-interpreted .go plugins
loaded via Registry.LoadDir, mirroring internal/plugin's conventions
(secretextract provides the generated, nil-guarded symbol table for the
Yaegi/Go interface boundary).

config.Forge.TokenEnv is replaced end-to-end with config.Forge.Token
(secret.SecretRef{Engine, Key}) -- finalize() defaulting, forge token
resolution in cmd/archied/main.go, and per-identity validation that now
requires an explicit token instead of silently resolving to "". No
backwards-compat shim.

Provider.APIKeyEnv, NATSConfig.TokenEnv, and TelegramConfig.TokenEnv are
deliberately not migrated here -- they're forwarded by name to
ai-sdk/runtime and subprocess workers rather than resolved centrally,
which needs its own eager-resolve-then-inject design pass
(archie-core-xec.4). The three shipped engine plugins (sops, bws,
vault/openbao) are tracked separately as archie-core-xec.1-3.

Adversarially reviewed by a fresh agent with no implementation context;
fixed the two findings that survived (missing per-identity token
validation, underdocumented nil-wrapper Resolve edge case).
2026-07-26 14:49:32 +10:00
sam 146acf5466 feat: secret engine registry foundation (archie-core-xec)
internal/secret package:
- Engine interface (Name, Version, Resolve) for pluggable secret backends
- SecretRef type replacing TokenEnv/APIKeyEnv string fields
- ResolveOrEnv for backward-compatible env-var fallback
- Built-in env engine registered by default
- Registry with Register/Get/Resolve/LoadDir (yaegi-based)
- DefaultRegistry() singleton for one-off resolution
- 12 tests, race-clean

Scoped to foundation only. Provider/NATS/Telegram migration deferred
to xec.4 (separate bead — those call sites forward env var names to
subprocesses/runtimes and need eager-resolve-then-inject, not a simple
field rename).

40/40 packages pass with -race.
2026-07-26 14:20:19 +10:00