Commit Graph
137 Commits
Author SHA1 Message Date
sam 19a6311f6c feat(telegram): /restart to reload config without dropping tasks (16.35)
Operator escape hatch: reload gateway config (notably the sender
allowlist) or recover a wedged adapter from chat, with no shell access to
the host.

Start becomes a supervisor loop that relaunches the bot on request, so a
restart is scoped to the gateway: the daemon keeps running and in-flight
agent tasks are untouched. Exiting the process would also have restarted
under the unless-stopped policy, but would have killed running work.

Two failure modes are handled deliberately:

- The handler runs on the very bot the supervisor is about to stop, so it
  hands off over a buffered channel and returns instead of tearing down
  inline, which would deadlock the restart it just requested. A duplicate
  request while one is in flight is dropped, not queued.
- A failed config reload logs and relaunches on the previous settings, so
  a bad config edit cannot take chat down permanently -- which would also
  remove the means to fix it.

Authorisation is inherited from authorizedMessage: only allowlisted
senders reach any handler, so /restart is admin-only by construction.
main.go supplies Reload, which re-reads config from disk and refreshes
token and allowlist; other fields still need a full daemon restart, since
they are wired at construction.
2026-07-27 05:21:58 +10:00
sam cb147b7a37 feat(telegram): deny-by-default sender allowlist + command menu (16.33, 16.34)
Allowlist (16.33). telegram.New was called with a nil allowlist and
isChatAllowed returned true whenever the list was empty, so any Telegram
user who found the bot could talk to it; AllowedChatIDs was not plumbed
into config, so there was no way to close it. Chat tools run with the
daemon's own authority, so this had to be fixed before wiring any real
tools to the chat agent.

Now chat.telegram.allowed_user_ids configures it and an empty list denies
everyone: a bot handle is public, so failing open is not a safe default.
The check matches the sender (from.id) rather than the chat, so adding
the bot to a group cannot be used to inherit access. Rejections log the
sender id and answer once, so a legitimate user missing from the list can
tell the bot is misconfigured rather than dead. Startup warns when the
gateway is enabled with an empty list.

The previous TestIsChatAllowed asserted 'empty allowlist allows all' --
it encoded the vulnerability -- and is replaced by fail-closed tests.

Command menu (16.34). SetMyCommands was never called, so Telegram showed
no menu and the commands Router implements were undiscoverable; asked
about /commands the LLM confidently answered it had none. The published
list mirrors what Router actually handles. Failure is logged, not fatal:
it costs discoverability, not function.
2026-07-27 05:14:34 +10:00
sam 285c618f74 fix(chat): drain FullStream, not TextStream — streaming deadlocked every reply
Streamed chat replies hung on the first token: the draft rendered an
empty cursor, the typing indicator never cleared, and the turn never
completed.

RouteStream consumed stream.TextStream, but ai-sdk's channel contract
makes FullStream authoritative and requires draining it until close --
its writes are synchronous, so an unread FullStream applies backpressure
and stalls the producer. Once its buffer filled, the producer blocked,
TextStream stopped yielding, and the range loop blocked forever.

TextStream was the wrong source regardless: it is explicitly best-effort
and drops deltas when not drained, so a reply reconstructed from it can
silently lose text. Range over FullStream and collect StreamPartTextDelta
parts, exactly as the SDK documents.

Also bumps ai-sdk v0.1.17 -> v0.1.18 (build, full suite and lint clean).
2026-07-27 05:04:04 +10:00
sam 3e3911a423 feat(telegram): native rich messages + streamed replies (16.32)
Upgrades go-telegram/bot v1.18.0 -> v1.22.0 and uses the Bot API's rich
message support instead of hand-rolled formatting.

The previous commit converted LLM Markdown into Telegram's HTML subset,
on the premise that the API could not take Markdown directly and had no
table element. Both are true of v1.18.0 but not of v1.22.0, which adds
sendRichMessage/sendRichMessageDraft taking InputRichMessage{Markdown},
plus RichBlockTableCell for native tables. markdownToHTML and its tests
are deleted: LLM Markdown, tables included, now goes to Telegram as-is.

Replies also stream. runtime.ChatStream already returned a TextStream
and sendRichMessageDraft renders partial output, so the only missing
piece was a gateway seam: gateway.LLMStreamResponder plus Router.
RouteStream, which degrades to the blocking Route when unset so any
adapter can call it unconditionally. Local commands answer from local
state and are not streamed; an unrecognised /command is, since Route
hands it to the LLM. main.go derives both responders from one shared
turn implementation.

Draft updates are throttled to one per 700ms to respect per-chat rate
limits, and every streaming/rich path falls back rather than dropping a
reply: a rejected rich send retries unformatted, and a failed draft
update is skipped since the final message is authoritative.
2026-07-27 04:57:36 +10:00
sam 38a746a891 fix: make agent-container Docker network explicitly configurable
archied's container-mode agent tasks were all failing with "nats: no
responders available for request" in production. Root cause: pool.go's
selfNetwork() auto-detects the daemon's own Docker network by looking up
a container named after os.Hostname(), then joins spawned agent
containers to that same network so they can resolve "nats" by name.
On carina this detection was silently failing (wrong/stale hostname vs.
actual container ID) and returning "", so every per-task agent container
landed on the default bridge network instead — unable to resolve "nats",
never subscribing, hence "no responders" on every request.

selfNetwork() swallowed every failure silently (only logged on success),
so this had no diagnostic trail.

Fix: added Containers.Network config field (containers.network in
config.toml) so deployments can name the network explicitly instead of
relying on fragile self-detection; selfNetwork() remains as a fallback
but now logs a warning on every failure path so this doesn't silently
recur. Wired through container.Config and cmd/archied/main.go.
2026-07-27 04:25:28 +10:00
sam f73b2d1c8c feat: memory manager sync/prefetch/scan/shutdown + broad in-flight epic work
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:

- Memory manager (18.7-18.16): background sync pipeline, prefetch with
  timeout, memory-write notification, threat scanning, shutdown drain
  with abandoned-write tracking, frozen snapshot pattern. Wired
  ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
  and notification actually apply to writes, not just exist as unused
  primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
  internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
  multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
2026-07-27 03:34:46 +10:00
sam 2b1045c0ad feat: secrets-engine registry with yaegi extension interface (archie-core-xec)
Adds internal/secret: a pluggable Engine interface (Name/Version/Resolve)
resolved through a Registry, so secrets backends (sops, Bitwarden Secrets
Manager, Vault/OpenBao) aren't hard-coded into the binary. Only the "env"
engine is compiled in; custom engines are Yaegi-interpreted .go plugins
loaded via Registry.LoadDir, mirroring internal/plugin's conventions
(secretextract provides the generated, nil-guarded symbol table for the
Yaegi/Go interface boundary).

config.Forge.TokenEnv is replaced end-to-end with config.Forge.Token
(secret.SecretRef{Engine, Key}) -- finalize() defaulting, forge token
resolution in cmd/archied/main.go, and per-identity validation that now
requires an explicit token instead of silently resolving to "". No
backwards-compat shim.

Provider.APIKeyEnv, NATSConfig.TokenEnv, and TelegramConfig.TokenEnv are
deliberately not migrated here -- they're forwarded by name to
ai-sdk/runtime and subprocess workers rather than resolved centrally,
which needs its own eager-resolve-then-inject design pass
(archie-core-xec.4). The three shipped engine plugins (sops, bws,
vault/openbao) are tracked separately as archie-core-xec.1-3.

Adversarially reviewed by a fresh agent with no implementation context;
fixed the two findings that survived (missing per-identity token
validation, underdocumented nil-wrapper Resolve edge case).
2026-07-26 14:49:32 +10:00
sam f4787a819d feat: email channel + Channel interface with config schema validation
Email channel (internal/channels/email):
- SMTP server receiving inbound emails on configurable port
- Multipart body extraction (text/plain from MIME)
- SMTP relay for outbound replies with STARTTLS + AUTH
- 8 tests, race-clean

Channel interface (internal/channels/channel.go):
- ConfigSchema() — JSON Schema for channel configuration
- ValidateConfig() — validates channel-specific config at startup
- All three channels (Telegram, Email, Webhook) implement it
- Compile-time guard: var _ channels.Channel = (*Gateway)(nil)

Config: ChatConfig.Email added (listen_addr, relay_addr).
Daemon: email gateway wired into main.go behind chat.email block.

40 packages pass with -race.
2026-07-26 13:36:56 +10:00
sam 20a1210d0e feat: webhook platform + persona registry (items 5+6)
Webhook platform (internal/channels/webhook):
- Gateway adapter with HTTP server, HMAC validation, dot-notation template extraction
- Per-route configuration with secret-based signing
- Deliver-to-origin mode for sync responses
- 11 tests, race-clean

Persona registry (internal/gateway/persona.go):
- PersonaRegistry with per-session active persona selection
- DefaultPersonas(): helpful, concise, technical, creative, teacher
- Persona prompt prepended to system prompt in LLMResponder
- Thread-safe, 5 tests

Config: ChatConfig.WebhookAddr added for webhook listen address.
All 39 packages pass with -race.

Closes abg.5.3, abg.34.
2026-07-26 10:39:58 +10:00
sam b261ee4b91 feat: context compression for long-running sessions
CompressionConfig with threshold-based triggering (default 50% of 128k window).
Protects first 3 and last 20 messages. Truncated middle replaced with statistical
summary (turns, tokens removed, topic range). CompressHistory() returns
CompressedView with before/after token counts. Integrated into Telegram
LLMResponder — fetches 100-message window, compresses before LLM call.

Matches Hermes config: threshold=0.5, protect_first=3, protect_last=20.
All 38 packages pass with -race.

Closes abg.33.
2026-07-26 10:36:24 +10:00
sam 948efde036 feat: session persistence — message storage, history retrieval, search
SessionStore extended with SaveMessage, RecentMessages, SearchMessages.
NellDB-backed implementation uses a separate DocDB collection for messages
keyed by session:seq. Conversation history loaded into LLM context for each
Telegram message. Responses persisted. Substring search fallback (vector
search deferred to memory provider embedding integration).

Adapter.Store() exported to share the underlying NellDB engine for session
collections without opening a second log.

Telegram LLMResponder now:
- Saves inbound messages to session store
- Loads 20 most recent messages as chat.Message history
- Sends full conversation context to llm.Chat()
- Persists the assistant response

All 38 packages pass with -race.
2026-07-26 10:33:13 +10:00
sam 1d76560911 feat: wire Telegram LLM routing — non-command messages routed to AI
LLMResponder adapter wraps llm.Chat() with model from cfg.Models[chat] (falls
back to builder). Gateway-local commands (/status, /models, /model, /spawn)
still handled directly. LLM created before gateway so it's available at
router construction time. All 38 packages pass.
2026-07-26 10:29:07 +10:00
sam 0314955587 feat: wire memory, guardrails, and tool registry into daemon agent loop
Memory:
- MemoryManager created in main.go with built-in file-backed provider
- SystemPromptBlock() injected into every agent stage request via ExtraRules
- memoryPrompt() helper on Daemon, threaded through TaskContext.SystemPrompt

Guardrails:
- GuardrailEngine created from DefaultGuardrailConfig in main.go
- RecordSuccess/RecordFailure called after every agent stage run
- Threaded through TaskContext.Guardrails for per-task guardrail enforcement

Tool registry:
- NewRegistry() created at startup, MCP servers started from config
- Passed to Daemon.ToolRegistry for dynamic tool discovery

Daemon changes:
- Memory, Guardrails, ToolRegistry fields added (nil-safe, backward compat)
- TaskContext gains SystemPrompt + Guardrails fields
- Agent stage injects memory context into ExtraRules
- Agent stage records guardrail state after each run

Telegram LLM routing deferred (TODO comment) — gateway commands work.
All 38 packages pass with -race.
2026-07-26 10:25:07 +10:00
sam f6d51416f0 chore: replace 607 em-dashes with double hyphens project-wide
Includes MCP crash-after-init test fix (crash moved from handle() to serve()
entry point so transport sees immediate subprocess death).
2026-07-26 09:53:33 +10:00
sam 4c1cc151a5 feat: NellDB storage backend replacing SQLite (archie-core-abg.30)
Extracted store.TaskStore interface (20 methods) covering task
lifecycle, queries, and observability. Separate WorkflowStore for
the agent RPC proxy.

New internal/nell/ package:
  - OpenStore(path, nodeID) — persistent logstore.LogStore backend
  - OpenMemory(nodeID) — in-memory store for tests
  - Full TaskStore implementation via sdk.DocDB with MVCC

All consumers updated to interface:
  - daemon.Daemon.Store, webui.Server.Store, storerpc.Server.Store
  - workflow.TaskContext.Store (WorkflowStore)
  - cmd/archied uses nell.OpenStore(cfg.DBPath, cfg.BotUser)

7 integration tests. Build + 32 packages pass.

Closes: archie-core-abg.30
2026-07-26 01:06:05 +10:00
sam ae744d1cc4 feat: /spawn gateway command for chat-initiated tasks
TaskCreator interface on gateway.Router allows chat users to create
tasks via /spawn <title>. Implemented as gateway-local command (no LLM
needed). StoreTaskCreator wraps the daemon's store and uses the first
configured repo as the task workspace with timestamp-based synthetic
issue numbers to avoid collisions with Gitea-issued tasks.

Wired into archied's gateway startup: when repos are configured,
router.Tasks is populated so /spawn works. When no repos are
configured, /spawn returns 'not configured'.

Closes: archie-core-abg.13
2026-07-25 23:54:06 +10:00
sam f36b6f2e41 feat: gateway architecture with Telegram channel
internal/gateway/ — Gateway interface, Router with gateway-local
command dispatch (commands handled directly, no LLM needed).
ModelManager interface for /model and /models support.

internal/channels/telegram/ — Telegram Gateway using go-telegram/bot
v1.18.0 (ported from GopherClaw). Webhook support, middleware,
allowed-chat filtering, long-message splitting.

internal/config/ — ChatConfig + TelegramConfig optional TOML block.

Cmd wiring in archied/main.go: starts Telegram gateway when
[chat.telegram] token_env is configured.

Gateway-local commands implemented: /status, /help, /model, /models.
Write actions (/spawn, /approve, /cancel) follow in abg.13.

Closes: archie-core-abg.5, archie-core-abg.14, archie-core-abg.15
2026-07-25 23:50:52 +10:00
sam 6ebbd3b2cf fix: add missing LinkBranch stubs to test doubles
Fix go vet/test failures by adding LinkBranch methods to stubForge, prCapturingForge, testForge, and fakeForge. Update test data from 'sam/archie' to 'acme/widget' and state label defaults from 'archie:*' to 'agent:*'. Add Taskfile.yml for project build automation. Implement container self-network detection so spawned agents join the daemon's Docker network.
2026-07-24 13:32:50 +10:00
sam d80401d595 feat: add config overlay and shared RPC/Yaegi packages
- Introduce `-config-overlay` flag and `config.LoadOverlay()` to merge a base config with an overlay file, only overriding fields the overlay sets. This enables docker-compose to mount the base config from the host and apply only deployment-specific overrides.
- Create `internal/natsrpc` package with reusable NATS request/reply primitives (Envelope, Client, Call, RegisterAll, Respond). Migrate storerpc, worktreerpc, and forgerpc to use it, eliminating duplicated marshal/unmarshal and error handling.
- Create `internal/yaegiutil` package with shared interpreter construction (stdlib + extra symbols), generic export resolution (Resolve), and panic recovery (Safe). Migrate all Yaegi-based surfaces (gate scripts, daemon plugins, skill plugins, workflow stages, skill scripts) to use it, reducing boilerplate and centralizing panic safety.
- Simplify `config.docker.toml` to contain only overlay fields (paths, container config) and remove the duplicated repos list – the base config is now the single source of truth.
- Update docker-compose.yml to pass both -config and -config-overlay flags and mount the base config from the host.
- Add tests for LoadOverlay, DockerConfigOverlaysExampleConfig, Envelope, and yaegiutil utilities.
2026-07-24 12:57:10 +10:00
sam e4368f70af feat: persistent volumes with TTL for container/worktree storage (archie-core-j7v)
Add [containers].volume_ttl (default 72h). Persistent-storage repos get a
per-repo bare Git cache (--reference-if-able --dissociate) so task
worktrees clone fast without repeated full network clones, while staying
fully independent of the cache's lifecycle — cache expiry can never
corrupt an active or parked task's worktree. Docker repo volumes are
labeled as Archie-owned and swept non-forced (a volume still attached to a
running container survives and retries next cycle); the repo cache
directory is swept by mtime. Both cleanups run at daemon startup and each
poll cycle.

Filed archie-core-756 (repo-wide golangci-lint baseline) and archie-core-340
(intermittent JetStream TempDir teardown flake) as separate follow-ups
rather than scope-creeping them into this change.
2026-07-24 10:22:11 +10:00
sam 75f237d1ad feat: daemon hands off tasks to archie-agent in sandboxed mode (archie-core-5zh.8)
daemon.process() branches on ContainerPool != nil: sandboxed tasks publish
a taskrun.Request to archie.taskrun.<id> and await the response via the
new runViaAgent method, instead of running workflow.Route/workflow.Run
locally. Non-container mode is unchanged. Every durable side effect
(Store, Forge, git push) already landed via the storerpc/forgerpc/
worktreerpc proxies during archie-agent's run by the time runViaAgent
returns — it only parks the task itself on transport/setup failure, never
duplicating a terminal state archie-agent's own workflow.Run already
recorded.

Also fixes a real correctness gap: with concurrent multi-task dispatch
(archie-core-u64/g1l) plus containerized archie-agent, a shared queue-group
subscription could let one container answer a sibling's taskrun message
while having a *different* task's worktree mounted. archie-agent now reads
its boot-time task.json and subscribes to its own exact per-task subject
when present, falling back to the shared queue group only in dev/pool mode
(no task.json).

Adds agentexec.ProvidersFromConfig, deduped from cmd/archied's former
executionProviders.

Closes the archie-core-5zh epic: archie-agent runs an entire task's
workflow per invocation instead of one stage per round trip, with all
credentials staying daemon-side throughout.
2026-07-24 09:14:56 +10:00
sam 81ce920434 feat: register storerpc/forgerpc/worktreerpc servers in archied (archie-core-5zh.9)
Add registerTaskRPCServers() in cmd/archied/main.go: subscribes all three
RPC servers on the daemon's NATS connection whenever [nats] is configured,
using the same Store/Forge/worktree.Manager instances the daemon already
owns. archie-agent containers now have a live responder for the Store,
Forge, and worktree Prepare/Push calls proxied in archie-core-5zh.1-3.

Test spins an embedded NATS server and round-trips real storerpc/forgerpc/
worktreerpc clients against the registered handlers.
2026-07-24 08:59:04 +10:00
sam 4be28e0e48 feat: optional NATS token authentication (archie-core-l1q)
Add token_env to [nats] config, resolved and validated at daemon startup
(fails early if configured but the named env var is unset). Container
workers get NATS_URL/NATS_TOKEN; archie-agent honors both while CLI-supplied
URL still takes precedence. Anonymous auth remains the default for
single-node deployments.

Unblocks archie-core-5zh (promote archie-agent to per-task process).
2026-07-24 07:51:06 +10:00
sam f8df051b19 feat: bounded concurrent task dispatch in daemon (archie-core-u64)
Daemon previously processed one task at a time. Add bounded concurrent
dispatch for SQLite and NATS agent modes, reusing [containers].max_concurrency
(0 = unlimited). Same-repository tasks stay FIFO-serialized to avoid worktree
conflicts; different repositories now run concurrently.

Unblocks archie-core-l1q (NATS transport) and archie-core-g1l
(per-repo allow_concurrent override).
2026-07-24 07:42:13 +10:00
sam edd05aea6b feat: add Gitea forge support alongside GitHub
Implements the Forge interface for Gitea instances (tested against
git.catlow.cloud, Gitea v1.26.4). The forge.New factory now dispatches
on forge type, and config validation accepts 'gitea'.

Changes:
- internal/forge/gitea.go — GiteaClient implementing all 11 Forge
  methods: issue polling (assigned + label), comments, PR creation,
  PR state reconciliation, issue closure, reactions, state labels,
  push verification. Handles edge cases: not-found PRs return
  'closed', GetUserByName errors from the Gitea SDK are mapped to
  user-friendly messages.
- internal/forge/forge.go — New() factory dispatching on forge type
  (github/gitea). GitHub constructor renamed to NewGitHub.
- internal/forge/github_test.go — updated to use New() signature
- internal/config/config.go — forge.type accepts 'gitea', BotEmail
  defaults change per forge type (gitea uses @gitea.local)
- cmd/archied/main.go — passes cfg.Forge.Type to forge.New()

Dependencies: added code.gitea.io/sdk/gitea v0.25.1

End-to-end verified against live Gitea instance: daemon starts clean,
registry built (5 workflows), web UI listening, no errors or warnings.
2026-07-22 05:15:09 +10:00
sam fd360fcc0b fix: resolve 8 remaining design-decision audit items
Fixes (4 items):
  S4 — Add TypeScript, Deno, Bun cache ecosystems to
       cacheVolumesByEcosystem. TypeScript shares node+pnpm volumes;
       Deno and Bun get dedicated cache volumes.
  R4 — Share single Docker client between container.Pool and
       storage.DockerBackend. NewPool now accepts an optional
       DockerClient in Config; if nil, creates its own. Pool
       tracks ownership (ownCli) and only closes its own client.
  S2 — Wire Storage.Teardown() call in daemon.process() after
       workflow.Run(). Docker backend is a no-op; this closes the
       interface contract for future backends.
  R5 — Emit observability event on AugmentRegistry failure.
       Previously only logged a warning; now emits a
       'registry_augment_failed' event with error detail.
       Promoted log level to Error.

Documentation (4 items):
  D1 — Document PluginRegistry as reserved for future daemon
       extension points (forge resolvers, storage backends, etc.)
  S1 — Fix /data/ layout comment to reflect actual task.json path
       (/data/worktree/task.json, not /data/task.json)
  S3 — Add TODO(PRD §3) for session.jsonl, memory.jsonl, plugins/
       volume — these need agent-side implementation first
  N6 — Document that VolumeInspectOptions{} requires moby v25+
       (go.mod pins compatible version)
  N7 — Document that _Plugin.IValue is required by Yaegi interface
       bridging; warn that go generate strips nil-guards

Test: TestCacheMountsForTypeScript verifies typescript ecosystem
returns node+pnpm volumes sorted by destination.

Verification: single client.New(client.FromEnv) in main.go;
all 21 storage tests pass; full suite green.
2026-07-22 05:00:37 +10:00
sam 04175a1a6a fix: address 9 'Now' adversarial audit findings
P1  (panic):      nil Storage guard in daemon.process()
C1  (correctness): check WriteTaskJSON error — don't start blind
C2  (correctness): move 'working' label after container setup
C3  (correctness): warn on duplicate workflow names in registry
C4  (correctness): test that generated _Plugin wrapper has nil-guards
C5  (correctness): .agents/skills as file (ENOTDIR) not fatal
C6  (correctness): ecosystem case-insensitive in cacheMounts
N1  (correctness): TOCTOU race — treat 'already exists' as success
N2  (correctness): recover from panicking plugin.Name() at startup

--- Details ---

P1: d.Storage.Setup() is now guarded by a nil check. In normal
operation Storage is always set when ContainerPool is set, but
the invariant is now explicit and enforced.

C1: container.WriteTaskJSON() error was discarded with _. Now it
returns early with a park transition — the container never starts
without a boot-time brief.

C2: SetStateLabel('working') was called before container setup.
If Acquire failed, the task was parked but the label stayed
'working' permanently. Now the label is set only after successful
container setup (or immediately if no containers are configured).

C3: BuildRegistry and AugmentRegistry now log a warning when two
skills declare the same workflow name and one overwrites the other.

C4: TestGeneratedWrapperHasNilGuards verifies that nil WName/
WVersion don't panic. If go generate strips the nil-guards, this
test catches it before the daemon crashes.

C5: skill.Catalog() now returns nil (no skills) instead of
propagating ENOTDIR errors when .agents/skills is a file.

C6: cacheMounts() normalizes ecosystem to lowercase. 'Python',
'NODE', 'Go' all resolve correctly now.

N1: ensureVolume() treats Docker's 'already exists' error as
success, closing the TOCTOU window between VolumeInspect and
VolumeCreate.

N2: safePluginInfo() wraps plugin.Name()/Version() calls in a
recover. A plugin with a panicking Name() no longer crashes the
daemon at startup.
2026-07-22 04:50:11 +10:00
sam 7dbb17a4cb feat: pluggable storage backends with Docker volumes and /data/ layout
Introduces the storage.Backend interface for pluggable container mount
management. The Docker implementation (DockerBackend) is the MVP —
no NFS/SMB/S3, just Docker volumes + bind mounts.

/data/ volume layout:
  /data/worktree/  — bind mount of host worktree (always)
  /data/repo/      — per-repo Docker volume (when persistent_storage = true)
  /data/cache/go/  — shared GOMODCACHE volume
  /data/cache/node/— shared npm cache volume
  /data/cache/pnpm/— shared pnpm store volume
  /data/cache/pip/ — shared pip cache volume
  /data/cache/cargo/— shared Rust cargo cache volume
  /data/task.json  — boot-time brief (existing WriteTaskJSON)

Changes:
- internal/storage/: Backend interface, DockerBackend, Mount types,
  ecosystem-to-cache-volume mapping, ConvertMounts helper
- container.Pool.Acquire: now accepts []storage.Mount instead of
  hardcoding a single bind mount — mount list comes from the storage
  backend
- daemon.process(): calls d.Storage.Setup() to build the mount list,
  passes Ecosystem + PersistentStorage from repo config
- config.Repo.PersistentStorage: when true, a named Docker volume
  (archie-repo-<owner>-<repo>) is mounted at /data/repo and survives
  task completion
- main.go: creates DockerBackend when containers are enabled

21 tests: interface contract, ecosystem cache mounts (go/node/
python/rust/custom), worktree bind mount ordering, persistent
storage on/off/empty-owner guard, mount list sort+dedup, adversarial
(nonexistent worktree, empty ecosystem, unknown ecosystem).

Why: daemon-managed cache volumes mean warm GOMODCACHE/npm cache
across tasks. Per-repo persistent volumes preserve expensive build
artifacts. The interface leaves room for future backends without
rewiring container orchestration.
2026-07-22 04:28:30 +10:00
sam f38ba5d515 feat: wire Layer 2 daemon plugin loader into archied startup
Daemon plugins (Yaegi-interpreted .go files) are now loaded from the
configured plugin_dir at startup. Each file must export a 'Plugin'
variable using the _Plugin wrapper struct, which bridges Yaegi types
to the Go plugin.Plugin interface.

Changes:
- internal/plugin/pluginextract/ — generated Yaegi symbol table for
  the plugin package (mirrors wfextract pattern), including the
  _Plugin interface wrapper that lets interpreted code implement
  Go interfaces
- LoadDir now accepts optional symbol tables via variadic parameter,
  avoiding an import cycle between plugin and pluginextract
- Plugin tests rewritten as behavioral (not structural), verifying
  actual Yaegi interpretation of valid, invalid, and missing plugins
- config.PluginDir field for specifying the plugin directory
- daemon.PluginRegistry field for holding loaded plugins
- main.go wires plugin loading: LoadDir at startup, register into
  daemon, log each loaded plugin

Why: the Plugin interface and LoadDir existed but weren't connected
to the daemon. Skills define workflows; daemon plugins are the
extension point for forge resolvers, storage backends, and other
core-daemon concerns.
2026-07-22 04:11:02 +10:00
sam 8958a82233 feat: replace hardcoded workflow registry with catalog-driven skillbuild.BuildRegistry()
The daemon no longer hardcodes workflow.Registry{...} in main.go.
Instead, skillbuild.BuildRegistry() scans .agents/skills/ for catalog
entries declaring metadata.archie.workflow and builds each workflow
from its Yaegi-interpreted stage plugins.

Built-in workflows (bootstrap, implement, tdd, feasibility, default)
serve as fallbacks when no skill declares a given workflow name. A
skill that declares metadata.archie.workflow: implement overrides the
built-in — skills define workflows, plugins define stages.

Why: skills must be extension primitives. Adding a workflow should
mean creating .agents/skills/archie-wf-<name>/plugins/*.go, not
modifying Go code in main.go. The catalog is the single source of
truth for what workflows exist.
2026-07-22 04:02:58 +10:00
sam de57f72cce feat: add Docker container sandbox for archie-agent
Dockerfile: multi-stage fat image (golang:1.26 builder → ubuntu:24.04)
with Go 1.26, Python 3.12/3.13 via deadsnakes, UV, Node 22, pnpm,
TypeScript, esbuild, git, gh, golangci-lint, gofumpt.

Container pool (internal/container): manages Docker container lifecycle
via moby/moby client. Per-task containers with bind-mounted workspaces.

- Acquire: create, start, mount workspace, inject NATS + API key env
- Release: stop, remove (AutoRemove as safety net)
- Orphan recovery on daemon startup (label: archie-daemon=true)
- MaxConcurrency enforcement, MaxUptime config (unused in MVP)
- Image pull with "missing"/"always" policy

Config: [containers] block (enabled, image, max_concurrency, max_uptime,
pull_policy). Validation: requires agent.mode = "nats" and [nats] url.

Daemon: acquires container before workflow.Run, releases after.
Container environment auto-populated from provider API keys.
2026-07-22 02:39:17 +10:00
sam 8c4bb1ef1d feat: promote archie-agent to NATS-connected per-task worker
Replaces stdin/stdout subprocess transport with NATS JetStream
request-reply pattern. The Runner interface and workflow orchestration
are unchanged.

New NATSRunner (implements Runner):
- Publishes AgentRequestMessage to JetStream with X-Archie-Reply header
- Blocks on core NATS inbox subscription for response
- Same Run(ctx, workspace, req) (Result, error) contract

Rewritten archie-agent (cmd/archie-agent):
- Long-running NATS-connected process (was 28-line stdin/stdout)
- Pull-consumes from ARCHIE_TASKS with archie.agent.> filter
- Builds InProcessRunner, executes agent loop, replies to inbox
- Graceful shutdown via signal handling

Extended internal/nats:
- SubjectAgentWildcard, SubjectForAgentRequest, ReplyHeader
- Conn() accessor, NewReplyInbox(), PublishMsg() for JetStream
- Stream subjects extended to ["archie.task.>", "archie.agent.>"]

Config: "nats" added as valid agent.mode alongside inprocess/subprocess
Tests: NATSRunner round-trip with embedded NATS server + handler
2026-07-22 02:13:01 +10:00
sam bb2193e5e4 feat: add optional NATS JetStream task distribution
Adds optional NATS-based task distribution alongside the existing
SQLite flow. When [nats] is absent from config, zero behaviour change.

New internal/nats package:
- Subject constants matching PRD §2 (archie.task.{bug,feature,default,bootstrap})
- Client: Connect (stream + durable pull consumer), PublishTask
  (Nats-Msg-Id dedup), Fetch, DecodeTask
- Tests with embedded NATS server (connect/publish, dedup, subject routing)

Config: NATSConfig{URL, TokenEnv} on Config, commented-out [nats] block
in config.example.toml

Store: ClaimByIssue atomic claim by owner/repo/number for NATS consumer

Daemon: Nats client field, poll() branches to pollNATS/pollSQLite,
Cycle() branches to drainNATS/drainSQLite (NATS fetch then SQLite
ClaimNext fallback for requeued tasks), processNATSTask
(decode → EnqueueIssue → ClaimByIssue → process → Ack)

Scratch: explorer and plan agent outputs from parallel-explore phase
2026-07-22 02:02:09 +10:00
sam c0d108a5d0 feat: add Yaegi-based plugin system (gates, custom stages, skill scripts)
Implements the full PRD (docs/prds/yaegi-plugin-system.md): repos can
now define .archie/gate.go for diff-inspecting quality rules shell
commands can't express, .archie/stages/*.go for custom pipeline steps,
and skills can ship Go scripts run via a new run_go_script agent tool —
all interpreted in-process with Yaegi, with panics recovered so a bad
script can't take down the daemon. Hot reload (phase 4) needed no code:
every loader reads from disk per call and every task gets a fresh
clone, so nothing is ever cached to go stale.
2026-07-22 00:54:46 +10:00
sam 86339b4d0d feat: add subprocess agent runner 2026-07-21 23:21:24 +10:00
sam a8d005a6f6 feat: extract agent execution boundary 2026-07-21 22:16:50 +10:00
sam 251cc7022c feat: add configurable forge and workflow behavior 2026-07-21 15:03:41 +10:00