14 Commits
Author SHA1 Message Date
sam f4c304cfba refactor: log through the injected logger with context; NATS handlers root in the registering context (#1927) 2026-11-13 03:56:12 +11:00
sam 60c0f19e63 fix(forge): a close refused for open dependencies is a typed error across the boundary (#1922) 2026-11-13 03:47:34 +11:00
sam 0880e6a01c feat(natsrpc): propagate trace context across NATS and into task runs 2026-10-25 23:03:55 +11:00
sam cb6b8618c2 refactor: cut remaining long comments down to what the code does 2026-10-03 22:06:57 +10:00
sam 780bc44d82 refactor: testing from the beginning. a journey in 150k lines of tests to none 2026-10-03 21:16:25 +10:00
sam 05529d48d5 refactor(state-store): cut daemon/Gateway over to the remote State Store (archie-core-8cda.4.6) (#774)
* refactor(state-store): delete NATS storerpc; agent store is gRPC-only (archie-core-8cda.4.6)

The agent is fully on the gRPC State Store contract, so the legacy NATS
storerpc path is dead (docs/prds/state-store-contract.md §12 step 4):
- delete internal/storerpc (Server, Client, subjects) and its tests
- drop the storerpc registration from registerTaskRPCServers and the
  now-unused store param
- make the agent transport's State Store URL required and Store() return
  only the gRPC staterpc.Client (no NATS fallback)
- rework agentworker/agent transport tests to back the store with a real
  gRPC State Store server instead of the deleted NATS storerpc server

The agent binary now fails fast when STATE_STORE_URL is absent. Agent store
calls (workflow.Store) go over gRPC to the archie-state-store service.

* refactor(state-store): delete daemon/gateway in-process store ownership (archie-core-8cda.4.6)

The daemon and gateway no longer open archie.db in-process: the single SQLite
file is owned by the standalone archie-state-store process, and both consumers
dial its gRPC State StateStore contract (docs/prds/state-store-contract.md
§12 step 7, no dual-store ownership).

- openStores no longer opens the production task store or seeds b.st; it only
  wires secrets/forge and the gateway conversation store
- openStateStoreAdapter now requires [services.state].target (the old local
  default is deleted) and resolves b.stateStore as the remote *staterpc.Client
- delete the in-process startStateStoreServer (the multiplexed gRPC server and
  its per-task bearer-token registry) from the daemon composition
- RunGateway now resolves the same remote State Store adapter; its session
  SQLite stays Gateway-owned and untouched
- daemon.ConnectedStateStore carries the configured target plus the resolved
  bearer token, which containerEnv injects as STATE_STORE_URL / STATE_STORE_TOKEN
  so archie-agent authenticates to the same remote service
- readiness store probe reads b.stateStore

PRD-consistent choice (noted): with the in-process server gone, per-task token
generation is removed; the daemon/gateway/agent all present the single
[services.state].target_token (or STATE_STORE_TOKEN secret) that the standalone
store validates.

* docs(webui): require [services.state].target in config and deployment templates (archie-core-8cda.4.6)

After the in-process store path is deleted, archied and archie-gateway dial the
standalone archie-state-store gRPC service via [services.state].target and never
open archie.db themselves (docs/prds/state-store-contract.md §12 step 7).

- config.example.toml documents [services.state].target as required, with the
  loopback vs Docker-bridge gateway + target_token rule
- deployment templates set [services.state] per topology: bridge gateway +
  token for container-mode agents, loopback for host-only
- deployments/README.md and systemd-user-service.md describe start commands
  for both topologies and drop the stale "not yet consumed by the daemon" note

* style: gofumpt alignment for State Store daemon literal; appease noctx/staticcheck (archie-core-8cda.4.6)

* docs(state-store): refresh stale in-process references post-cutover (archie-core-8cda.4.6)
2026-09-07 05:06:12 +10:00
sam ba672965c6 test(coverage): add tests for workintake, secret, forge, natsrpc
Closes coverage gaps flagged in load-bearing packages:
- internal/domain/workintake: 14.3% -> 94.6%
- internal/secret/enginehost: 0% -> 100%
- internal/secret/secretextract: 10% -> 100%
- internal/forge: 36.1% -> 77.0%
- internal/natsrpc: 28.3% -> 91.3%
2026-08-31 21:24:25 +10:00
sam 18d1af1eeb fix(natsrpc): make RegisterAll wait for the server to see subscriptions
RegisterAll returned as soon as nc.Subscribe had queued each SUB frame on
the client. Delivery to the server is asynchronous, so a caller that
registered a responder and immediately issued a request could beat its own
subscription and get nats.ErrNoResponders from a responder that was, by
then, listening.

This made TestRunTaskExecutesBootstrapWorkflowEndToEnd flaky in CI: it
passed in isolation and failed under full-suite load, when the scheduler
was contended enough for the request to win the race.

Flushing after subscribing gives Register a real guarantee -- a nil error
now means the subjects are reachable. That applies to production startup
too, not just tests: archied registers storerpc, forgerpc and worktreerpc
the same way.

The duplicated rollback loop is now a single unsubscribeAll closure, which
also becomes the returned unsubscribe func.

Verified: three consecutive full-suite runs with no archie-agent failure,
plus five targeted runs of the rpc packages. Remaining gateway and
telegram failures are pre-existing and unrelated.
2026-07-29 10:57:18 +10:00
sam 6fe1a7d037 fix: lint cleanup across repo (errcheck, staticcheck, ineffassign, unused)
- daemon.go: real bug — maybeRetryParked resolved fg := d.forgeFor(task)
  (matching markDead's pattern) but called d.Forge.SetStateLabel instead
  of fg.SetStateLabel, silently using the wrong forge client for
  multi-identity/multi-forge task routing. Fixed to use fg throughout.
- telegram.go: updateThreadID existed but was never called anywhere;
  wired it into updateLoggingMiddleware so debug logs include thread_id.
- Two SA9003 empty-branch tests (nell adapter_test.go, tools
  toolentry_test.go) were dead assertions with no body; implemented the
  actual check instead of deleting them.
- memory manager_test.go: mockProvider.lastConfigSchema/lastBackupPaths
  were declared but never recorded; wired GetConfigSchema/BackupPaths to
  populate them and strengthened the aggregation tests to assert on
  per-provider content, not just counts.
- Removed genuinely dead gateway messageevent_test.go helpers (intPtr/
  i64Ptr, marked //go:fix inline, already fully superseded by the new()
  builtin used throughout the file).
- Remaining errcheck findings (defer Close/Flush/Write, os.Setenv, etc.
  in test files) fixed with the existing `_ = ` convention already used
  elsewhere in the codebase.

go build ./..., go test ./... (all packages ok), and
golangci-lint run ./... all clean.
2026-07-27 03:49:33 +10:00
sam f73b2d1c8c feat: memory manager sync/prefetch/scan/shutdown + broad in-flight epic work
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:

- Memory manager (18.7-18.16): background sync pipeline, prefetch with
  timeout, memory-write notification, threat scanning, shutdown drain
  with abandoned-write tracking, frozen snapshot pattern. Wired
  ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
  and notification actually apply to writes, not just exist as unused
  primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
  internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
  multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
2026-07-27 03:34:46 +10:00
sam e8f78dd0d5 test: regression tests for all audit gaps — 6 enhanced + 4 new packages
Telegram: splitLongMessage (8 cases), isChatAllowed, updateMetadata (4 cases),
  panicRecoveryMiddleware, updateLoggingMiddleware, authorizedMessage, stop-while-stopped
Container: error paths, payload round-trip, overwrite, minimal payload
Events: buffer defaults, subscribe-after-close, double-close safety,
  slow-subscriber isolation, multi-subscriber ordering, publish timestamping
NATS RPC: envelope edge cases, JSON round-trip, request timeout, rollback
NellDB: empty store (6 ops), concurrent enqueue/claim (20 goroutines),
  context cancellation, oversized payload clipping, unmarshalable data,
  status transitions, events ordering, recover stale, clear terminal
Gateway tasks: empty/long title, idempotent enqueue, chat label

New packages with tests (formerly zero-test):
  internal/gate (220 lines), internal/gate/gateextract,
  internal/plugin/pluginextract, internal/workflow/wfextract

All 37 packages pass. Fixes regression gaps from audit.
2026-07-26 10:05:44 +10:00
sam f6d51416f0 chore: replace 607 em-dashes with double hyphens project-wide
Includes MCP crash-after-init test fix (crash moved from handle() to serve()
entry point so transport sees immediate subprocess death).
2026-07-26 09:53:33 +10:00
sam 9df02f2b60 fix: golangci-lint baseline to zero findings (39→0)
19 files touched. All fixes are in code — no lint configuration
weakened:
- Added unchecked error handling in defer cleanup paths
- Removed dead variables and unused functions
- Added missing imports (io in gitea.go)
- Simplified conditions (TrimSuffix, De Morgan, slices.Contains)

Also fixes JetStream TempDir teardown race (archie-core-340):
t.Cleanup(srv.Shutdown) now registers after t.TempDir() so
server shutdown (including JetStream files) completes before
temp dir removal. Added srv.WaitForShutdown() for deterministic
cleanup ordering.

Closes: archie-core-756, archie-core-340
2026-07-25 23:51:13 +10:00
sam d80401d595 feat: add config overlay and shared RPC/Yaegi packages
- Introduce `-config-overlay` flag and `config.LoadOverlay()` to merge a base config with an overlay file, only overriding fields the overlay sets. This enables docker-compose to mount the base config from the host and apply only deployment-specific overrides.
- Create `internal/natsrpc` package with reusable NATS request/reply primitives (Envelope, Client, Call, RegisterAll, Respond). Migrate storerpc, worktreerpc, and forgerpc to use it, eliminating duplicated marshal/unmarshal and error handling.
- Create `internal/yaegiutil` package with shared interpreter construction (stdlib + extra symbols), generic export resolution (Resolve), and panic recovery (Safe). Migrate all Yaegi-based surfaces (gate scripts, daemon plugins, skill plugins, workflow stages, skill scripts) to use it, reducing boilerplate and centralizing panic safety.
- Simplify `config.docker.toml` to contain only overlay fields (paths, container config) and remove the duplicated repos list – the base config is now the single source of truth.
- Update docker-compose.yml to pass both -config and -config-overlay flags and mount the base config from the host.
- Add tests for LoadOverlay, DockerConfigOverlaysExampleConfig, Envelope, and yaegiutil utilities.
2026-07-24 12:57:10 +10:00