Commit Graph
23 Commits
Author SHA1 Message Date
sam e3ca6c536c fix(container): scope agent containers, volumes and the egress relay to the instance 2026-10-19 10:15:47 +11:00
sam a45300d626 refactor: testing from the beginning. a journey in 150k lines of tests to none 2026-10-03 21:16:25 +10:00
sam 619a9a8913 feat(container): apply container-runtime-policies live
The container runtime policies kind was restart-required because the pool
captured image, pull policy, network, max uptime and max concurrency at
construction and the dispatcher fixed its slot count at the start of a drain
pass. Both consumers now re-read the published config:

- the pool's settings come from a live source read per acquire, with the
  boot-resolved registry credential and Docker construction values preserved;
  a live default-image change is pulled rather than assumed present;
- the daemon owns one long-lived dispatcher that accepts a new limit: running
  tasks keep their slots, a raise admits queued work immediately, and a lower
  limit is applied as tasks finish;
- the control-plane descriptor is live, the kind joins runtimeResourceKinds,
  and the apply/reload paths resize the dispatcher after publishing. The
  ContainerConfig fields whose consumers now re-read are added to the reload
  allowlist; registry_auth stays restart-required.

Regenerate the docs artifacts. contracts.json also carries the voice
media-type description a prior commit left stale.
2026-10-02 11:39:46 +10:00
sam 893abf7eca feat(container): send registry auth on image pull
ImagePull was called with empty options, so a private registry answered 401
on every pull regardless of containers.pull_policy. docker login did not help
either: it writes ~/.docker/config.json, which only the CLI reads.

Add containers.registry_auth, a SecretRef reusing the credential shape the
rest of the file uses, whose resolved value is a Docker registry.AuthConfig
JSON document. archied resolves it at boot and the pool base64url-encodes it
into the X-Registry-Auth header ImagePullOptions carries.

A missing or unresolvable credential degrades to the anonymous pull that
existed before the field (with a warning), so a deployment with no private
registry is unaffected; a half-named reference is refused by
configuration.Validate. The field is file-owned and json:"-", so the
container-runtime-policies layering has to preserve it explicitly, the way it
preserves Profiles.
2026-10-02 09:35:28 +10:00
sam c431834e2b feat(eda): workflow inputs, optional repository and agent profiles
Workflow definitions declare inputs, repository (none, optional or
required; default required) and profile. A binding assigns each input
from a mapped parameter or a constant and may take its repository from a
parameter, which must name a configured repo; both are checked when the
binding is saved and again at dispatch. Inputs travel as structured task
data (tasks.inputs, the task brief, and a JSON block in the agent's
mission), never as body text.

A task with no repository runs in a per-task scratch workspace with no
clone and no publication grant. The agent.run step runs one mission
without a repository and finishes the task in the new completed status.

Agent profiles live under [containers.profiles.<name>]: an image and an
allowlist over the tools archie adds (MCP, scripts, plugins).
[containers].image is the default profile; an unconfigured profile parks
the task. The workflow is now pinned before the container starts so the
profile can choose its image.

Also stop two archied tests writing embedded NATS state into the package
directory, refuse an empty state_dir for embedded NATS, and drop the NATS
store files an earlier lane commit tracked.
2026-09-24 20:29:33 +10:00
sam c9a4f4b7e1 fix(container): stop a late max-uptime callback resurrecting teardown state 2026-09-22 01:57:59 +10:00
sam 1dc212e50e feat(gateway): give /status a real daemon-health surface (#892)
* feat(gateway): give /status a real daemon-health surface

Phase 2 of archie-core-wp9s. Phase 1 split the commands apart (health vs
work) but left /status with one aggregate queue line and a static runtime
block; it could not answer the questions an operator actually asks when
something feels wrong. /status now reports, between the queue line and the
runtime block:

- Broker: the live state of this process's own NATS connection
  (eventbus/nats.Client.Connected), not a fresh dial -- a probe would
  report the probe's reachability, not the daemon's.
- Containers: the pool's own active counter against the configured cap
  (Pool.Active/Cap), not a Docker listing, which would include containers
  this pool does not own. A zero cap means unlimited, so it is not shown
  as a ratio.
- Channels: the channel manager's real lifecycle state, including a
  failed adapter's reason (the documented decision: read the existing
  manager rather than re-deriving health per adapter).
- Chat model: last-known outcome of every chat-model call this process
  made, recorded once in sendChatTurn (the single choke point) and read
  back by /status. The second documented decision: never a live probe,
  so /status stays instant and reports what happened -- or says plainly
  that nothing has been attempted yet.
- Last poll: when the most recent poll pass began, stamped on both poll
  paths (Run's single-identity loop and runIdentities' per-identity
  goroutines). A pass that hangs leaves the stamp stale, which is the
  signal; stamping completion would leave a wedged poller looking
  healthy.

Truthful or absent: a section this process has no source for is omitted
rather than rendered as a zero. The standalone Gateway owns no poll loop,
pool or channel manager, so its /status reports broker and chat-model
health only; aggregating across processes would need a new daemon RPC and
is deliberately not built. Version stays out (/version owns it).

/status and /tasks descriptions now describe what each command actually
does on both published surfaces (Telegram menu/help and the shared
specs), pinned by a test that also fails if the two surfaces drift.

Sources are read through functions, not captured values: occupancy and
channel state move, and the daemon is built after the gateways, so a
source captured at construction would freeze /status on boot state.

Left in place deliberately: the Runtime (provider/model) block, which
Phase 1 flagged and which duplicates /model and /whoami. Removing it is a
product call on a command Sam uses daily, not a mechanical follow-up.

* docs(prds): record the two open /status health decisions as settled

Both were flagged in PR #892 as needing the maintainer's call; both are now
answered, and the PRD is the place a follow-up would look:

- The Runtime (provider/model) block stays. It is the one line in /status the
  ticket's own rule says should not be there, but /status is read daily to see
  the active model at a glance, so the duplication is deliberate. Recorded as a
  settled product decision so it is not removed later as an oversight.
- Health stays per-process. Pooling the daemon's and the Gateway's chat-model
  outcomes would need a new daemon<->gateway health RPC for one status line, and
  a per-process line is true where an aggregate would be a claim about a process
  the answering one cannot see.

/agents' published description is still left to archie-core-mxls, unchanged.
2026-09-20 00:30:23 +10:00
sam b38c14f6e1 fix(container): claim container teardown once instead of cancelling a timer
Timer.Stop cannot unwind a max-uptime callback that has already begun, so
cancelling still let the reaper and Release both stop and remove the same
container, which is the duplicate teardown and the misleading 'No such
container' warnings this was meant to remove.

Teardown is now claimed once per container: whichever of the two gets there
first talks to Docker and the loser skips to its own bookkeeping. Release
still claims after the grace period, so MaxUptime keeps biting during it.

Also table-drives the event pump priming cases, per the repo's test directive.
2026-09-19 00:13:36 +10:00
sam 0e1e5303ed fix(container): keep the max-uptime cap active during the grace period
The reaper was cancelled at the top of Release, before the grace-period
sleep, which let a container outlive MaxUptime by GracePeriod. MaxUptime is
documented as a cap from creation enforced regardless of task state, and a
grace period is task state, so the cancel belongs after the sleep.

Also asserts the disarmed reaper directly rather than only inferring it from
the absence of later Docker calls.
2026-09-19 00:06:26 +10:00
sam dedafd7d43 fix(container): cancel a container's max-uptime reaper on release
The timer stayed armed after Release removed the container, so an hour later
it stopped and removed an ID Docker had already forgotten and logged two
warnings that read like a failed teardown.
2026-09-18 23:59:40 +10:00
sam 7c5b81fd53 fix(container): enforce MaxUptime lifetime cap with a timer
WIRE: Config.MaxUptime and Acquire documented a hard container lifetime cap
(PRD section 4), but the code only wrapped ContainerCreate/ContainerStart in
a MaxUptime deadline that was cancelled when Acquire returned, so no running
container was ever killed. Wire the documented cap with a per-container
timer that stops and removes the container once the cap elapses.
2026-09-15 13:33:48 +10:00
sam 990a7c499c fix(container): stop a cancelled task's container immediately
Release() always slept the configured GracePeriod and always used
Docker's default ~10s graceful SIGTERM-then-SIGKILL timeout, even
when the release was happening because /stop had just cancelled the
task's context. A cancelled task's container -- and the in-flight
generation/tool calls inside it -- could sit doing nothing for up to
GracePeriod+~10s before actually dying, so /stop reported success
while work visibly continued for tens of seconds to a couple minutes.

releaseDecision(ctx, grace) now skips the grace sleep and requests an
immediate SIGKILL (Timeout: 0) when ctx is already cancelled; a live
ctx (normal completion) keeps the original behaviour byte for byte.

Closes archie-core-1786749700580-5-e3878224.
2026-08-18 09:05:21 +10:00
sam 4c0a06df6d fix(container): write the task boot brief under .git
WriteTaskJSON wrote task.json to the worktree root -- the same worktree
the agent commits from. CommitAll stages with go-git's All option, and
go-git's Add does not honour .gitignore or .git/info/exclude, so the
brief was swept into the agent's commit and pushed onto every task
branch. That is how a task.json for issue #57 ended up committed in
the repo root.

This is the identical failure the prepared sentinel hit; worktree.go
already documents the fix. Nothing under .git can ever be tracked, so
put the brief there rather than papering over it with an exclude.
2026-07-31 01:13:08 +10:00
sam 15d746c2f6 style: phase 1 of lint remediation — autofix pass (qa4)
golangci-lint run --fix plus the formatters: errorlint, gocritic,
sloglint, usestdlibvars, usetesting, gci, gofumpt, goimports.
160 issues -> 130.

One autofix was wrong and had to be reverted by hand. gocritic rewrote

    ctx, cancel = context.WithTimeout(ctx, m.prefetchTimeout)

to `:=` inside an if-block in internal/memory/manager.go. That does not
compile, which is the good outcome — had it compiled, the shadowed ctx
would have scoped the timeout to the if-block and the prefetch deadline
would silently never have applied. Restored the assignment and commented
why it must not be a declaration.

Worth recording: --fix is not a safe no-op. It broke the build on a
24-file change, and the failure mode it was one compiler error away from
was a silent behaviour change.
2026-07-28 01:37:20 +10:00
archie ef12eb6141 fix: baseline gate repair (go test ./... -count=1) 2026-07-26 20:05:07 +00:00
sam 6fe1a7d037 fix: lint cleanup across repo (errcheck, staticcheck, ineffassign, unused)
- daemon.go: real bug — maybeRetryParked resolved fg := d.forgeFor(task)
  (matching markDead's pattern) but called d.Forge.SetStateLabel instead
  of fg.SetStateLabel, silently using the wrong forge client for
  multi-identity/multi-forge task routing. Fixed to use fg throughout.
- telegram.go: updateThreadID existed but was never called anywhere;
  wired it into updateLoggingMiddleware so debug logs include thread_id.
- Two SA9003 empty-branch tests (nell adapter_test.go, tools
  toolentry_test.go) were dead assertions with no body; implemented the
  actual check instead of deleting them.
- memory manager_test.go: mockProvider.lastConfigSchema/lastBackupPaths
  were declared but never recorded; wired GetConfigSchema/BackupPaths to
  populate them and strengthened the aggregation tests to assert on
  per-provider content, not just counts.
- Removed genuinely dead gateway messageevent_test.go helpers (intPtr/
  i64Ptr, marked //go:fix inline, already fully superseded by the new()
  builtin used throughout the file).
- Remaining errcheck findings (defer Close/Flush/Write, os.Setenv, etc.
  in test files) fixed with the existing `_ = ` convention already used
  elsewhere in the codebase.

go build ./..., go test ./... (all packages ok), and
golangci-lint run ./... all clean.
2026-07-27 03:49:33 +10:00
sam f73b2d1c8c feat: memory manager sync/prefetch/scan/shutdown + broad in-flight epic work
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:

- Memory manager (18.7-18.16): background sync pipeline, prefetch with
  timeout, memory-write notification, threat scanning, shutdown drain
  with abandoned-write tracking, frozen snapshot pattern. Wired
  ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
  and notification actually apply to writes, not just exist as unused
  primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
  internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
  multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
2026-07-27 03:34:46 +10:00
sam e8f78dd0d5 test: regression tests for all audit gaps — 6 enhanced + 4 new packages
Telegram: splitLongMessage (8 cases), isChatAllowed, updateMetadata (4 cases),
  panicRecoveryMiddleware, updateLoggingMiddleware, authorizedMessage, stop-while-stopped
Container: error paths, payload round-trip, overwrite, minimal payload
Events: buffer defaults, subscribe-after-close, double-close safety,
  slow-subscriber isolation, multi-subscriber ordering, publish timestamping
NATS RPC: envelope edge cases, JSON round-trip, request timeout, rollback
NellDB: empty store (6 ops), concurrent enqueue/claim (20 goroutines),
  context cancellation, oversized payload clipping, unmarshalable data,
  status transitions, events ordering, recover stale, clear terminal
Gateway tasks: empty/long title, idempotent enqueue, chat label

New packages with tests (formerly zero-test):
  internal/gate (220 lines), internal/gate/gateextract,
  internal/plugin/pluginextract, internal/workflow/wfextract

All 37 packages pass. Fixes regression gaps from audit.
2026-07-26 10:05:44 +10:00
sam f6d51416f0 chore: replace 607 em-dashes with double hyphens project-wide
Includes MCP crash-after-init test fix (crash moved from handle() to serve()
entry point so transport sees immediate subprocess death).
2026-07-26 09:53:33 +10:00
sam 9df02f2b60 fix: golangci-lint baseline to zero findings (39→0)
19 files touched. All fixes are in code — no lint configuration
weakened:
- Added unchecked error handling in defer cleanup paths
- Removed dead variables and unused functions
- Added missing imports (io in gitea.go)
- Simplified conditions (TrimSuffix, De Morgan, slices.Contains)

Also fixes JetStream TempDir teardown race (archie-core-340):
t.Cleanup(srv.Shutdown) now registers after t.TempDir() so
server shutdown (including JetStream files) completes before
temp dir removal. Added srv.WaitForShutdown() for deterministic
cleanup ordering.

Closes: archie-core-756, archie-core-340
2026-07-25 23:51:13 +10:00
sam 6ebbd3b2cf fix: add missing LinkBranch stubs to test doubles
Fix go vet/test failures by adding LinkBranch methods to stubForge, prCapturingForge, testForge, and fakeForge. Update test data from 'sam/archie' to 'acme/widget' and state label defaults from 'archie:*' to 'agent:*'. Add Taskfile.yml for project build automation. Implement container self-network detection so spawned agents join the daemon's Docker network.
2026-07-24 13:32:50 +10:00
sam 4edad31b66 fix: wire dead code — GracePeriod, task.json, plugin specs
GracePeriod now enforced in Release(): if > 0, container stays alive
for the configured duration after task completion before being killed,
per PRD section 1.

WriteTaskJSON now called from daemon.process() before container
Acquire, writing the task payload to <workspace>/task.json per
PRD section 3 /data/ layout.

PluginSpecs populated from TaskContext.SkillPlugins into the
Request sent to the agent. PluginSpec type added to protocol.go.
pluginSpecs() converts skill.Plugin → agentexec.PluginSpec for
transport. PRD section 5 Layer 1.
2026-07-22 03:31:27 +10:00
sam fab7a0ae6f feat: close Gaps 2, 6, and 8
Gap 8 (frontmatter plugins key): add Plugins []string to
Metadata.Archie struct. Parses the plugins list from SKILL.md
YAML frontmatter per PRD section 5.

Gap 2 (grace period): add GracePeriod field to container.Config,
distinct from MaxUptime. MaxUptime = total container lifetime cap.
GracePeriod = idle time after task completion before kill.
Documented per PRD sections 1 and 4.

Gap 6 (task.json): add TaskPayload struct and WriteTaskJSON
function to internal/container. Daemon writes the task payload
to <workspace>/task.json before container start, matching the
PRD section 3 /data/task.json boot brief.
2026-07-22 03:25:44 +10:00