Commit Graph
46 Commits
Author SHA1 Message Date
sam 6be754787d polish: add install.sh, XDG setup, deployments/ examples, and github-token docs\n\n- Add install.sh following XDG Base Directory specification (~/.config/archie, ~/.local/share/archie, ~/.local/bin)\n- Add interactive setup for code forge (GitHub / Gitea / Standalone) and LLM provider selection (Ollama model picker / Cloud keys)\n- Implement NoopForge in internal/forge/noop.go for running Archie in standalone mode without a forge\n- Enable loginctl user linger and systemd user unit setup in install.sh\n- Add docs/github-token.md with step-by-step guidance for Classic PAT generation\n- Add top-level deployments/ directory with scenario templates (single-forge GitHub, multi-forge GitHub+Gitea, local Ollama standalone, systemd runbook) 2026-07-30 01:25:46 +10:00
sam e7d39254e7 feat: make /stop reach running agent tasks
/stop cancelled the chat turn but could not touch a task the daemon was
executing, and /cancel refused outright:

    case statusRunning:
        return fmt.Errorf("task is actively running; cannot cancel
                           until it reaches a waiting state")

So the one case worth interrupting was the one case neither command
would touch.

The daemon now holds a cancel function for every executing task,
registered in process() -- the single point every dispatch path reaches
-- so a task is stoppable for its whole life, including the slow clone,
worktree prepare and image pull that is exactly when someone realises
they asked for the wrong thing.

- /cancel <id> interrupts the work first, then records the terminal
  state. That order matters: writing the state while the task still runs
  invites it to write its own afterwards and win. A task the store calls
  running that nothing is executing (crashed daemon) still gets its state
  recorded, which is what unsticks it.
- bare /stop now stops the reply, the queue behind it, and every running
  task for the identity, reporting which of those it actually did.
  Nobody reaching for the brake should have to look up a task ID first.
- StopRunning deliberately writes no terminal state: a stopped task
  unwinds to parked, which is resumable. Forcing rejected would discard
  work worth keeping.
- Without a runtime wired, cancelling a running task is still refused --
  recording it as cancelled while it carries on would be a lie.

Container teardown detaches from the task context. Release runs on the
way out, and the main reason a task is on its way out is cancellation --
at which point the graceful ContainerStop failed instantly and the
container was left to the forced remove. Cleanup gets its own deadline.

process() gained two statements and crossed the funlen limit, so the
container setup block is extracted rather than suppressed.
2026-07-29 19:19:04 +10:00
sam 9d5040fd65 refactor: add internal/eventbus as the broker-neutral messaging contract
dependencies-and-contracts.md rule 1 says domains must not import
internal/infrastructure. internal/daemon and internal/agentexec both did,
and for exactly the thing the doc carves out: "internal/eventbus is an
approved shared application contract ... it MUST remain broker-neutral."

The Publisher/Consumer/Requester interfaces existed but lived inside the
NATS package, so a caller could depend on the behaviour yet still had to
import the implementation to name it. They now live in internal/eventbus,
and nats.Client asserts against eventbus.Bus.

The larger violation was ownership of meaning. The doc says "message schemas
stay with their owning domains; the event bus transports messages, it does
not own their meaning." The bus exported TaskEnvelope and a PublishTask
method, so the transport knew what a unit of work was, and it exported the
archie.agent.* subject helpers, so it also owned the agent protocol.

  - TaskEnvelope, its routing Kind, the label vocabulary and the task
    subjects move to internal/domain/workintake, which encodes and decodes
    its own bytes. This starts internal/domain/ per organisation.md.
  - The agent request/response/system subjects move to internal/agentexec,
    which defines what those messages mean.
  - internal/workflow keeps only the kind-to-registry mapping; the label
    table stays single and lives with the domain that reads forge issues.
  - The bus now carries opaque payloads on plain-string subjects.

Consumers declare the smallest interface they need rather than taking the
whole Bus: daemon.TaskBus is publish-unique/fetch/request, agentexec.
RequestBus is publish-request/reply-inbox, and archie-agent's stageBus is
fetch/respond. None can reach what it does not use.

Two defects this surfaced:

  - archie-agent declared its own JetStream stream with a second copy of the
    name, subjects, retention, dedup window, MaxDeliver and TTLs that
    nats.Connect already sets -- two definitions of one stream, free to
    diverge. Config now takes Subjects and FilterSubject from composition,
    which is what let the agent use the shared client and delete its
    hand-rolled setup.
  - Replying to a request needs core NATS, not the JetStream publish path:
    a reply inbox is ephemeral and belongs to no stream, so a durable
    publish to it fails. The old code hid this by using the raw connection.
    The contract now distinguishes Respond from Publish. The agentexec
    round-trip test caught it after I had made the same mistake in
    archie-agent.

Also translated the SDK's ErrNoResponders into eventbus.ErrNoResponders, so
the daemon's taskrun retry no longer imports the NATS SDK to name a
transient condition. The SDK is now confined to the nats package, the *rpc
packages (themselves NATS infrastructure), and cmd/*.

The agentexec test fake previously hand-rolled a JetStream consumer; it now
drains the client's own consumer, which is both closer to production and
avoids conflicting consumers on a work-queue stream.

Verified: go build, go vet, gofmt clean; full test suite passes.
2026-07-29 11:40:24 +10:00
sam a8efbf5046 refactor: route tasks on a typed kind, not forge labels
The event bus was inspecting forge issue labels ("bug", "feature",
"bootstrap") to pick a subject. That put a forge vocabulary inside
infrastructure and duplicated the label table already owned by
workflow.Route -- adding a label meant remembering both places, and
nothing would have caught it if you forgot.

The label table now lives once, in internal/workflow, which already owns
label semantics:

  - workflow.Kind names the routing category, with labelKinds as the
    single label vocabulary and kindWorkflows mapping kind to registry
    entry. Route consumes the ordered kind list so a "bug,feature" task
    still falls through to feasibility when no tdd workflow is
    registered, exactly as the switch did.
  - nats.TaskKind is a closed, validated set of subject tokens.
    PublishTask rejects an unknown kind rather than silently delivering
    it to the default queue; the zero kind means default, so messages
    queued before this field decode unchanged.
  - The daemon classifies at the publish site and sets TaskEnvelope.Kind.

Also fixes two defects this exposed:

  - drainNATS never reached its SQLite fallback. Replacing the old
    (nil, nil) no-message signal with ErrNoMessage in 492ca20 made that
    case continue at the top of the loop, but no-message is precisely
    when the fallback must run -- requeued tasks (waiting_human approval,
    retry-parked) never go through a NATS publish, so nothing would ever
    arrive to trigger them. go vet caught it as unreachable code.
  - config.classifyFeatureFile silently overwrote duplicate feature
    files, so config.gateway.yaml and conf.d/gateway.yaml together
    applied one at random. Its own doc comment recorded the bug
    ("or "" on success/duplicate"). The known-feature list in the error
    message is now derived from knownFeatureNames instead of being
    spelled out a second time beside it.

A test also called t.Fatalf from a non-test goroutine, where it does not
stop the test.

Verified: go build, go vet, gofmt clean. Pre-existing gateway (18) and
telegram (1) failures are unchanged in count -- they are lost routing
wiring from the parallel telegram merges, tracked separately.
2026-07-29 10:43:52 +10:00
sam f303998e14 Merge branch 'crew/lint5' 2026-07-29 10:37:36 +10:00
sam 492ca209f9 refactor(nats): rebuild as infrastructure/eventbus/nats with real boundaries
Relocating internal/nats as-is would have cemented its problems into the
approved layout, so this is a rewrite rather than a move.

What was wrong:
- Conn() *nats.Conn, PublishMsg(*nats.Msg) and Fetch() jetstream.Msg pushed
  SDK types into all four consumers, so the broker was not replaceable --
  the opposite of what dependencies-and-contracts.md requires.
- PublishTask took seven positional parameters, six of them strings, where a
  transposed argument was a silent bug.
- Labels travelled as a comma-separated string and were re-split on every
  publish.
- Fetch returned (nil, nil) for 'no message', which every caller had to
  remember to special-case.
- The logger field was stored and never used.
- Stream name, consumer name and every timeout were hardcoded constants.
- Errors were inconsistently wrapped; several paths returned bare errors.
- Connect repeated nc.Close() on each failure branch.

What replaces it:
- One concern per file: config, errors, subjects, message, client,
  publisher, consumer.
- Publisher/Consumer/Requester interfaces with compile-time assertions, so
  callers depend on behaviour and can be faked without a broker.
- A Message wrapper keeps jetstream types behind the boundary; consumers no
  longer import the NATS SDK at all.
- TaskEnvelope with []string labels and JSON methods preserving the
  comma-separated wire form for already-queued messages.
- Sentinel errors (ErrNoMessage, ErrNotConnected, ErrNoReplyAddress,
  ErrInvalidConfig) and %w wrapping with operation context throughout.
- Config with validation and defaults; a single deferred cleanup in Connect.
- CoreConn() as the one documented escape hatch, for the *rpc packages that
  are themselves NATS infrastructure.

Consumers migrated: cmd/archied, cmd/archie-agent, internal/daemon,
internal/agentexec. Behaviour, subjects and wire format unchanged.
2026-07-29 10:19:09 +10:00
sam a9e8cda4f2 fix(qa4.3): resolve nilerr, bodyclose, contextcheck, containedctx, fatcontext lint findings
- bodyclose: add deferred resp.Body.Close() in storage_test.go Docker test handler
- contextcheck: thread context through email handleSMTP/processMessage; replace
  context.Background() with context.WithoutCancel(ctx) in telegram, webhook, webui
  shutdown paths and container pool cleanup; add ctx parameter to Pool.Release
- containedctx: replace ctx context.Context struct field with stopCh chan struct{}
  in StdioTransport and SSETransport; thread lifecycle context through goroutine
  chains instead of storing it in structs
- fatcontext: lift context deadline derivation out of PrefetchContext loop
2026-07-29 10:10:42 +10:00
sam f800e1de36 refactor(worktree): replace git shell-out with go-git v6
internal/worktree no longer executes the git binary. Clone, fetch,
checkout, commit, push and diff all run in-process through go-git v6,
which also removes the package's 9 noctx findings since the Context
variants are first-class.

Security: the forge token no longer leaves the process. The old
implementation wrote a GIT_ASKPASS shell script to disk and exported the
token into the environment of every git child process. Authentication is
now an in-process credential passed via client.WithHTTPAuth. cloneURL
also stops embedding the bot user in the GitHub URL.

Dropped the per-repo object cache (PreparePersistent, prepareCache,
CleanupExpiredCaches). go-git can clone with shared alternates but has no
equivalent of --dissociate, so a cached bare repo would remain a live
dependency of every worktree built from it, and expiring one would
corrupt running tasks rather than fail cleanly. Every task now gets an
independent full clone. repo.PersistentStorage still governs the
container volume; only the git object cache is gone.

Two real defects surfaced while porting, both caught by the new tests:

  - go-git honours the host's global git config and cannot sign commits
    (no gpg fallback, only an ObjectSigner plugin). On any machine with
    commit.gpgSign=true set globally, every bot commit would have failed
    with "cannot auto-sign commit". setIdentity now pins gpgsign=false in
    the clone's own config so daemon behaviour does not depend on the
    ambient config of whatever host it runs on.
  - go-git's Add(All) does NOT honour .git/info/exclude, which is how the
    .archie-prepared sentinel was kept untracked. It was being committed
    and would have been pushed onto every task branch. The sentinel now
    lives at .git/archie-prepared, where nothing can ever track it, and a
    regression test asserts it never appears in a commit.

Diff is now merge-base relative via Commit.MergeBase, matching the three
dot semantics of `git diff origin/base...HEAD`; diffing against the remote
tip would attribute commits landed on base since the branch started to
this task and could trip the diff-size cap. Base resolution tries the
remote-tracking ref first and falls back to the short "origin/<base>"
revision, keeping git's more permissive lookup.

The test suite was rebuilt on go-git fixtures and no longer needs a git
binary installed to run.
2026-07-28 01:56:59 +10:00
sam 15d746c2f6 style: phase 1 of lint remediation — autofix pass (qa4)
golangci-lint run --fix plus the formatters: errorlint, gocritic,
sloglint, usestdlibvars, usetesting, gci, gofumpt, goimports.
160 issues -> 130.

One autofix was wrong and had to be reverted by hand. gocritic rewrote

    ctx, cancel = context.WithTimeout(ctx, m.prefetchTimeout)

to `:=` inside an if-block in internal/memory/manager.go. That does not
compile, which is the good outcome — had it compiled, the shadowed ctx
would have scoped the timeout to the if-block and the prefetch deadline
would silently never have applied. Restored the assignment and commented
why it must not be a declaration.

Worth recording: --fix is not a safe no-op. It broke the build on a
24-file change, and the failure mode it was one compiler error away from
was a silent behaviour change.
2026-07-28 01:37:20 +10:00
sam 4c0fc12450 test message 2026-07-27 04:39:27 +10:00
sam 6fe1a7d037 fix: lint cleanup across repo (errcheck, staticcheck, ineffassign, unused)
- daemon.go: real bug — maybeRetryParked resolved fg := d.forgeFor(task)
  (matching markDead's pattern) but called d.Forge.SetStateLabel instead
  of fg.SetStateLabel, silently using the wrong forge client for
  multi-identity/multi-forge task routing. Fixed to use fg throughout.
- telegram.go: updateThreadID existed but was never called anywhere;
  wired it into updateLoggingMiddleware so debug logs include thread_id.
- Two SA9003 empty-branch tests (nell adapter_test.go, tools
  toolentry_test.go) were dead assertions with no body; implemented the
  actual check instead of deleting them.
- memory manager_test.go: mockProvider.lastConfigSchema/lastBackupPaths
  were declared but never recorded; wired GetConfigSchema/BackupPaths to
  populate them and strengthened the aggregation tests to assert on
  per-provider content, not just counts.
- Removed genuinely dead gateway messageevent_test.go helpers (intPtr/
  i64Ptr, marked //go:fix inline, already fully superseded by the new()
  builtin used throughout the file).
- Remaining errcheck findings (defer Close/Flush/Write, os.Setenv, etc.
  in test files) fixed with the existing `_ = ` convention already used
  elsewhere in the codebase.

go build ./..., go test ./... (all packages ok), and
golangci-lint run ./... all clean.
2026-07-27 03:49:33 +10:00
sam f73b2d1c8c feat: memory manager sync/prefetch/scan/shutdown + broad in-flight epic work
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:

- Memory manager (18.7-18.16): background sync pipeline, prefetch with
  timeout, memory-write notification, threat scanning, shutdown drain
  with abandoned-write tracking, frozen snapshot pattern. Wired
  ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
  and notification actually apply to writes, not just exist as unused
  primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
  internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
  multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
2026-07-27 03:34:46 +10:00
sam 2b1045c0ad feat: secrets-engine registry with yaegi extension interface (archie-core-xec)
Adds internal/secret: a pluggable Engine interface (Name/Version/Resolve)
resolved through a Registry, so secrets backends (sops, Bitwarden Secrets
Manager, Vault/OpenBao) aren't hard-coded into the binary. Only the "env"
engine is compiled in; custom engines are Yaegi-interpreted .go plugins
loaded via Registry.LoadDir, mirroring internal/plugin's conventions
(secretextract provides the generated, nil-guarded symbol table for the
Yaegi/Go interface boundary).

config.Forge.TokenEnv is replaced end-to-end with config.Forge.Token
(secret.SecretRef{Engine, Key}) -- finalize() defaulting, forge token
resolution in cmd/archied/main.go, and per-identity validation that now
requires an explicit token instead of silently resolving to "". No
backwards-compat shim.

Provider.APIKeyEnv, NATSConfig.TokenEnv, and TelegramConfig.TokenEnv are
deliberately not migrated here -- they're forwarded by name to
ai-sdk/runtime and subprocess workers rather than resolved centrally,
which needs its own eager-resolve-then-inject design pass
(archie-core-xec.4). The three shipped engine plugins (sops, bws,
vault/openbao) are tracked separately as archie-core-xec.1-3.

Adversarially reviewed by a fresh agent with no implementation context;
fixed the two findings that survived (missing per-identity token
validation, underdocumented nil-wrapper Resolve edge case).
2026-07-26 14:49:32 +10:00
sam 0314955587 feat: wire memory, guardrails, and tool registry into daemon agent loop
Memory:
- MemoryManager created in main.go with built-in file-backed provider
- SystemPromptBlock() injected into every agent stage request via ExtraRules
- memoryPrompt() helper on Daemon, threaded through TaskContext.SystemPrompt

Guardrails:
- GuardrailEngine created from DefaultGuardrailConfig in main.go
- RecordSuccess/RecordFailure called after every agent stage run
- Threaded through TaskContext.Guardrails for per-task guardrail enforcement

Tool registry:
- NewRegistry() created at startup, MCP servers started from config
- Passed to Daemon.ToolRegistry for dynamic tool discovery

Daemon changes:
- Memory, Guardrails, ToolRegistry fields added (nil-safe, backward compat)
- TaskContext gains SystemPrompt + Guardrails fields
- Agent stage injects memory context into ExtraRules
- Agent stage records guardrail state after each run

Telegram LLM routing deferred (TODO comment) — gateway commands work.
All 38 packages pass with -race.
2026-07-26 10:25:07 +10:00
sam f6d51416f0 chore: replace 607 em-dashes with double hyphens project-wide
Includes MCP crash-after-init test fix (crash moved from handle() to serve()
entry point so transport sees immediate subprocess death).
2026-07-26 09:53:33 +10:00
sam 4c1cc151a5 feat: NellDB storage backend replacing SQLite (archie-core-abg.30)
Extracted store.TaskStore interface (20 methods) covering task
lifecycle, queries, and observability. Separate WorkflowStore for
the agent RPC proxy.

New internal/nell/ package:
  - OpenStore(path, nodeID) — persistent logstore.LogStore backend
  - OpenMemory(nodeID) — in-memory store for tests
  - Full TaskStore implementation via sdk.DocDB with MVCC

All consumers updated to interface:
  - daemon.Daemon.Store, webui.Server.Store, storerpc.Server.Store
  - workflow.TaskContext.Store (WorkflowStore)
  - cmd/archied uses nell.OpenStore(cfg.DBPath, cfg.BotUser)

7 integration tests. Build + 32 packages pass.

Closes: archie-core-abg.30
2026-07-26 01:06:05 +10:00
sam 399ca66e32 feat: multi-identity daemon config and core
internal/config/ — IdentityConfig struct for [[identities]] TOML
array. Backward-compatible: empty Identities uses legacy
single-identity fields unchanged. Validation for identity name,
bot_user, repos, forge type.

internal/daemon/ — IdentityRunner bundles per-identity forge client,
worktree manager, repos, and config. NewIdentityRunner constructor.
Run() starts one goroutine per identity with independent poll loops
for failure isolation. cycleForIdentity() polls identity repos via
dedicated forge client and drains shared store.

Design decision: per-identity goroutine with shared infrastructure
(store, NATS, container pool, event bus, web UI, plugin registry).

Closes: archie-core-abg.3, archie-core-abg.4
2026-07-25 23:50:58 +10:00
sam 6ebbd3b2cf fix: add missing LinkBranch stubs to test doubles
Fix go vet/test failures by adding LinkBranch methods to stubForge, prCapturingForge, testForge, and fakeForge. Update test data from 'sam/archie' to 'acme/widget' and state label defaults from 'archie:*' to 'agent:*'. Add Taskfile.yml for project build automation. Implement container self-network detection so spawned agents join the daemon's Docker network.
2026-07-24 13:32:50 +10:00
sam 1ce6489823 fix: bump branch slug limit to 60 chars 2026-07-24 11:22:34 +10:00
sam 006b4e8e54 feat: @mention detection, branch linking, stale worktree pull 2026-07-24 10:38:07 +10:00
sam e4368f70af feat: persistent volumes with TTL for container/worktree storage (archie-core-j7v)
Add [containers].volume_ttl (default 72h). Persistent-storage repos get a
per-repo bare Git cache (--reference-if-able --dissociate) so task
worktrees clone fast without repeated full network clones, while staying
fully independent of the cache's lifecycle — cache expiry can never
corrupt an active or parked task's worktree. Docker repo volumes are
labeled as Archie-owned and swept non-forced (a volume still attached to a
running container survives and retries next cycle); the repo cache
directory is swept by mtime. Both cleanups run at daemon startup and each
poll cycle.

Filed archie-core-756 (repo-wide golangci-lint baseline) and archie-core-340
(intermittent JetStream TempDir teardown flake) as separate follow-ups
rather than scope-creeping them into this change.
2026-07-24 10:22:11 +10:00
sam 75f237d1ad feat: daemon hands off tasks to archie-agent in sandboxed mode (archie-core-5zh.8)
daemon.process() branches on ContainerPool != nil: sandboxed tasks publish
a taskrun.Request to archie.taskrun.<id> and await the response via the
new runViaAgent method, instead of running workflow.Route/workflow.Run
locally. Non-container mode is unchanged. Every durable side effect
(Store, Forge, git push) already landed via the storerpc/forgerpc/
worktreerpc proxies during archie-agent's run by the time runViaAgent
returns — it only parks the task itself on transport/setup failure, never
duplicating a terminal state archie-agent's own workflow.Run already
recorded.

Also fixes a real correctness gap: with concurrent multi-task dispatch
(archie-core-u64/g1l) plus containerized archie-agent, a shared queue-group
subscription could let one container answer a sibling's taskrun message
while having a *different* task's worktree mounted. archie-agent now reads
its boot-time task.json and subscribes to its own exact per-task subject
when present, falling back to the shared queue group only in dev/pool mode
(no task.json).

Adds agentexec.ProvidersFromConfig, deduped from cmd/archied's former
executionProviders.

Closes the archie-core-5zh epic: archie-agent runs an entire task's
workflow per invocation instead of one stage per round trip, with all
credentials staying daemon-side throughout.
2026-07-24 09:14:56 +10:00
sam b6b23bbb59 feat: per-repo allow_concurrent override for same-repo dispatch (archie-core-g1l)
Add config.Repo.AllowConcurrent (allow_concurrent toml key). Repos opted in
skip the daemon's default FIFO same-repo task serialization while still
respecting [containers].max_concurrency globally. Worktrees are already keyed
per issue number, so opted-in repos don't collide on disk.
2026-07-24 07:47:10 +10:00
sam f8df051b19 feat: bounded concurrent task dispatch in daemon (archie-core-u64)
Daemon previously processed one task at a time. Add bounded concurrent
dispatch for SQLite and NATS agent modes, reusing [containers].max_concurrency
(0 = unlimited). Same-repository tasks stay FIFO-serialized to avoid worktree
conflicts; different repositories now run concurrently.

Unblocks archie-core-l1q (NATS transport) and archie-core-g1l
(per-repo allow_concurrent override).
2026-07-24 07:42:13 +10:00
sam 51ade3e503 fix: conventional commit prefix in branch names (feat/62-... not archie/issue-62) 2026-07-23 22:29:32 +10:00
sam 3898fd179c fix: descriptive branch names — archie/62-clear-finished-button instead of archie/issue-62 2026-07-23 20:25:38 +10:00
sam 423eb78fef feat: add CreateIssue to Forge interface with Gitea implementation 2026-07-23 19:12:12 +10:00
archie d2b2f29096 Implement max_retries cap on maybeRetryParked to prevent infinite parked-task loops (archie)
Implements #61
2026-07-23 18:49:38 +10:00
sam 3598498813 fix: clone worktree before container acquire to fix bind-mount race 2026-07-22 21:02:25 +10:00
sam fd360fcc0b fix: resolve 8 remaining design-decision audit items
Fixes (4 items):
  S4 — Add TypeScript, Deno, Bun cache ecosystems to
       cacheVolumesByEcosystem. TypeScript shares node+pnpm volumes;
       Deno and Bun get dedicated cache volumes.
  R4 — Share single Docker client between container.Pool and
       storage.DockerBackend. NewPool now accepts an optional
       DockerClient in Config; if nil, creates its own. Pool
       tracks ownership (ownCli) and only closes its own client.
  S2 — Wire Storage.Teardown() call in daemon.process() after
       workflow.Run(). Docker backend is a no-op; this closes the
       interface contract for future backends.
  R5 — Emit observability event on AugmentRegistry failure.
       Previously only logged a warning; now emits a
       'registry_augment_failed' event with error detail.
       Promoted log level to Error.

Documentation (4 items):
  D1 — Document PluginRegistry as reserved for future daemon
       extension points (forge resolvers, storage backends, etc.)
  S1 — Fix /data/ layout comment to reflect actual task.json path
       (/data/worktree/task.json, not /data/task.json)
  S3 — Add TODO(PRD §3) for session.jsonl, memory.jsonl, plugins/
       volume — these need agent-side implementation first
  N6 — Document that VolumeInspectOptions{} requires moby v25+
       (go.mod pins compatible version)
  N7 — Document that _Plugin.IValue is required by Yaegi interface
       bridging; warn that go generate strips nil-guards

Test: TestCacheMountsForTypeScript verifies typescript ecosystem
returns node+pnpm volumes sorted by destination.

Verification: single client.New(client.FromEnv) in main.go;
all 21 storage tests pass; full suite green.
2026-07-22 05:00:37 +10:00
sam 04175a1a6a fix: address 9 'Now' adversarial audit findings
P1  (panic):      nil Storage guard in daemon.process()
C1  (correctness): check WriteTaskJSON error — don't start blind
C2  (correctness): move 'working' label after container setup
C3  (correctness): warn on duplicate workflow names in registry
C4  (correctness): test that generated _Plugin wrapper has nil-guards
C5  (correctness): .agents/skills as file (ENOTDIR) not fatal
C6  (correctness): ecosystem case-insensitive in cacheMounts
N1  (correctness): TOCTOU race — treat 'already exists' as success
N2  (correctness): recover from panicking plugin.Name() at startup

--- Details ---

P1: d.Storage.Setup() is now guarded by a nil check. In normal
operation Storage is always set when ContainerPool is set, but
the invariant is now explicit and enforced.

C1: container.WriteTaskJSON() error was discarded with _. Now it
returns early with a park transition — the container never starts
without a boot-time brief.

C2: SetStateLabel('working') was called before container setup.
If Acquire failed, the task was parked but the label stayed
'working' permanently. Now the label is set only after successful
container setup (or immediately if no containers are configured).

C3: BuildRegistry and AugmentRegistry now log a warning when two
skills declare the same workflow name and one overwrites the other.

C4: TestGeneratedWrapperHasNilGuards verifies that nil WName/
WVersion don't panic. If go generate strips the nil-guards, this
test catches it before the daemon crashes.

C5: skill.Catalog() now returns nil (no skills) instead of
propagating ENOTDIR errors when .agents/skills is a file.

C6: cacheMounts() normalizes ecosystem to lowercase. 'Python',
'NODE', 'Go' all resolve correctly now.

N1: ensureVolume() treats Docker's 'already exists' error as
success, closing the TOCTOU window between VolumeInspect and
VolumeCreate.

N2: safePluginInfo() wraps plugin.Name()/Version() calls in a
recover. A plugin with a panicking Name() no longer crashes the
daemon at startup.
2026-07-22 04:50:11 +10:00
sam 7dbb17a4cb feat: pluggable storage backends with Docker volumes and /data/ layout
Introduces the storage.Backend interface for pluggable container mount
management. The Docker implementation (DockerBackend) is the MVP —
no NFS/SMB/S3, just Docker volumes + bind mounts.

/data/ volume layout:
  /data/worktree/  — bind mount of host worktree (always)
  /data/repo/      — per-repo Docker volume (when persistent_storage = true)
  /data/cache/go/  — shared GOMODCACHE volume
  /data/cache/node/— shared npm cache volume
  /data/cache/pnpm/— shared pnpm store volume
  /data/cache/pip/ — shared pip cache volume
  /data/cache/cargo/— shared Rust cargo cache volume
  /data/task.json  — boot-time brief (existing WriteTaskJSON)

Changes:
- internal/storage/: Backend interface, DockerBackend, Mount types,
  ecosystem-to-cache-volume mapping, ConvertMounts helper
- container.Pool.Acquire: now accepts []storage.Mount instead of
  hardcoding a single bind mount — mount list comes from the storage
  backend
- daemon.process(): calls d.Storage.Setup() to build the mount list,
  passes Ecosystem + PersistentStorage from repo config
- config.Repo.PersistentStorage: when true, a named Docker volume
  (archie-repo-<owner>-<repo>) is mounted at /data/repo and survives
  task completion
- main.go: creates DockerBackend when containers are enabled

21 tests: interface contract, ecosystem cache mounts (go/node/
python/rust/custom), worktree bind mount ordering, persistent
storage on/off/empty-owner guard, mount list sort+dedup, adversarial
(nonexistent worktree, empty ecosystem, unknown ecosystem).

Why: daemon-managed cache volumes mean warm GOMODCACHE/npm cache
across tasks. Per-repo persistent volumes preserve expensive build
artifacts. The interface leaves room for future backends without
rewiring container orchestration.
2026-07-22 04:28:30 +10:00
sam a9a3f7b3f9 feat: per-worktree dynamic registry augmentation
When a worktree already exists on disk (retry, waiting_human ->
implement handoff), the daemon now scans it for .agents/skills/ that
declare metadata.archie.workflow and augments the startup registry
with any newly discovered workflows. Worktree skills take precedence
over the startup registry for that task only.

Changes:
- skillbuild.AugmentRegistry(worktree, base) — scans a worktree
  directory for skills, builds workflows from plugins, returns a
  new registry with worktree entries merged over base. Base is never
  mutated. Mirrors BuildRegistry but starts from caller's registry
  instead of builtins.
- daemon.process() — computes worktree path, calls AugmentRegistry
  if the directory exists, uses augmented registry for routing.
  Degrades gracefully to startup registry on error.

Tests (10 new):
- Adds worktree workflows to base registry
- Worktree skills override same-named base entries
- Base registry is not mutated (copy semantics)
- Nonexistent worktree returns base unchanged
- Worktree with no .agents/skills/ returns base unchanged
- Worktree skill with no plugins does not override base
- Adversarial: concurrent augmentation is safe
- Adversarial: broken SKILL.md is skipped
- Adversarial: bare worktree (no .agents/) does not error
- Adversarial: nil base is treated as empty

Why: skills are per-project (.agents/skills/ in each repo). The
startup registry only knows about a shared skills directory. Per-
worktree augmentation means a repo can define its own workflows
without restarting the daemon.
2026-07-22 04:18:12 +10:00
sam f38ba5d515 feat: wire Layer 2 daemon plugin loader into archied startup
Daemon plugins (Yaegi-interpreted .go files) are now loaded from the
configured plugin_dir at startup. Each file must export a 'Plugin'
variable using the _Plugin wrapper struct, which bridges Yaegi types
to the Go plugin.Plugin interface.

Changes:
- internal/plugin/pluginextract/ — generated Yaegi symbol table for
  the plugin package (mirrors wfextract pattern), including the
  _Plugin interface wrapper that lets interpreted code implement
  Go interfaces
- LoadDir now accepts optional symbol tables via variadic parameter,
  avoiding an import cycle between plugin and pluginextract
- Plugin tests rewritten as behavioral (not structural), verifying
  actual Yaegi interpretation of valid, invalid, and missing plugins
- config.PluginDir field for specifying the plugin directory
- daemon.PluginRegistry field for holding loaded plugins
- main.go wires plugin loading: LoadDir at startup, register into
  daemon, log each loaded plugin

Why: the Plugin interface and LoadDir existed but weren't connected
to the daemon. Skills define workflows; daemon plugins are the
extension point for forge resolvers, storage backends, and other
core-daemon concerns.
2026-07-22 04:11:02 +10:00
sam 4edad31b66 fix: wire dead code — GracePeriod, task.json, plugin specs
GracePeriod now enforced in Release(): if > 0, container stays alive
for the configured duration after task completion before being killed,
per PRD section 1.

WriteTaskJSON now called from daemon.process() before container
Acquire, writing the task payload to <workspace>/task.json per
PRD section 3 /data/ layout.

PluginSpecs populated from TaskContext.SkillPlugins into the
Request sent to the agent. PluginSpec type added to protocol.go.
pluginSpecs() converts skill.Plugin → agentexec.PluginSpec for
transport. PRD section 5 Layer 1.
2026-07-22 03:31:27 +10:00
sam de57f72cce feat: add Docker container sandbox for archie-agent
Dockerfile: multi-stage fat image (golang:1.26 builder → ubuntu:24.04)
with Go 1.26, Python 3.12/3.13 via deadsnakes, UV, Node 22, pnpm,
TypeScript, esbuild, git, gh, golangci-lint, gofumpt.

Container pool (internal/container): manages Docker container lifecycle
via moby/moby client. Per-task containers with bind-mounted workspaces.

- Acquire: create, start, mount workspace, inject NATS + API key env
- Release: stop, remove (AutoRemove as safety net)
- Orphan recovery on daemon startup (label: archie-daemon=true)
- MaxConcurrency enforcement, MaxUptime config (unused in MVP)
- Image pull with "missing"/"always" policy

Config: [containers] block (enabled, image, max_concurrency, max_uptime,
pull_policy). Validation: requires agent.mode = "nats" and [nats] url.

Daemon: acquires container before workflow.Run, releases after.
Container environment auto-populated from provider API keys.
2026-07-22 02:39:17 +10:00
sam bb2193e5e4 feat: add optional NATS JetStream task distribution
Adds optional NATS-based task distribution alongside the existing
SQLite flow. When [nats] is absent from config, zero behaviour change.

New internal/nats package:
- Subject constants matching PRD §2 (archie.task.{bug,feature,default,bootstrap})
- Client: Connect (stream + durable pull consumer), PublishTask
  (Nats-Msg-Id dedup), Fetch, DecodeTask
- Tests with embedded NATS server (connect/publish, dedup, subject routing)

Config: NATSConfig{URL, TokenEnv} on Config, commented-out [nats] block
in config.example.toml

Store: ClaimByIssue atomic claim by owner/repo/number for NATS consumer

Daemon: Nats client field, poll() branches to pollNATS/pollSQLite,
Cycle() branches to drainNATS/drainSQLite (NATS fetch then SQLite
ClaimNext fallback for requeued tasks), processNATSTask
(decode → EnqueueIssue → ClaimByIssue → process → Ack)

Scratch: explorer and plan agent outputs from parallel-explore phase
2026-07-22 02:02:09 +10:00
sam c0d108a5d0 feat: add Yaegi-based plugin system (gates, custom stages, skill scripts)
Implements the full PRD (docs/prds/yaegi-plugin-system.md): repos can
now define .archie/gate.go for diff-inspecting quality rules shell
commands can't express, .archie/stages/*.go for custom pipeline steps,
and skills can ship Go scripts run via a new run_go_script agent tool —
all interpreted in-process with Yaegi, with panics recovered so a bad
script can't take down the daemon. Hot reload (phase 4) needed no code:
every loader reads from disk per call and every task gets a fresh
clone, so nothing is ever cached to go stale.
2026-07-22 00:54:46 +10:00
sam 86339b4d0d feat: add subprocess agent runner 2026-07-21 23:21:24 +10:00
sam a8d005a6f6 feat: extract agent execution boundary 2026-07-21 22:16:50 +10:00
sam 251cc7022c feat: add configurable forge and workflow behavior 2026-07-21 15:03:41 +10:00
sam 239b8eda42 feat: feasibility workflow — assess, PRD, human go/no-go loop
Routed via the feature label. Stages: assess (read-only agent with a
structured decide tool — verdict as a tool call, not parsed prose;
not-a-fit closes the issue as won't-do with reasons) -> PRD (read-only
agent; problem, solution, surface, acceptance criteria, non-goals) ->
deliver (PRD posted as the watched issue comment, optional [notify]
webhook POST for n8n->email, task -> waiting_human).

The daemon's cycle now watches waiting_human tasks: replies after the
watch comment (excluding the bot) are judged by an LLM (triage role,
APPROVE/REJECT/UNCLEAR + reason — never keyword matching). Approvals
requeue under implement (whose planner now treats an existing PRD as
prior context); rejections close with the reasoning acknowledged;
unclear replies keep waiting.

Supporting changes: forge.Comment returns the comment id;
forge.RepliesAfter; store watch_comment_id (additive migration),
WaitingTasks, Requeue; Route honours a pre-assigned workflow;
AgentStage.Extra for stage-specific tools.

Live: todo#7 assessed fit -> PRD delivered -> waiting_human.
2026-07-21 04:27:17 +10:00
sam 387c033b29 feat: observability — event bus, SQLite event log, live web dashboard
Adapted from tau's eventbus/metrics designs (per Sam), slimmed for a
small daemon:

- internal/events: single-type bus with mutex fan-out and bounded
  per-subscriber buffers that DROP on overflow (counted) — a stalled
  dashboard can never backpressure the task engine (tau's blocking
  delivery deliberately not copied)
- store: events table + queries powering the dashboards: task
  timelines, per-workflow outcomes/spend, stage durations and error
  counts (from stage_finish events), token spend by day
- emit points: task_queued, stage_start/finish(duration, error),
  agent_finish(status, tokens, steps, model), parked, outcome,
  pr_merged/pr_rejected
- internal/webui: read-only dashboard (embedded single page, no build
  step) — status cards, task board with click-through timelines, live
  SSE event tail with catch-up via ?since=<event id>, workflow/stage
  metrics tables. [web] listen config, default 127.0.0.1:8484, no auth
  by design (bind localhost/tailnet); "off" disables
- sink pipeline: bus -> SQLite insert (stamps row id) -> SSE broadcast

Verified live: dashboard serves real workflow stats from the existing
task history; bus overflow and event-log round-trips unit-tested.
2026-07-21 04:20:45 +10:00
sam 94a58e11b6 feat: assignee-driven pickup + repro proof on TDD PRs
- Issues are picked up by assigning them to the bot (tink-bot style)
  instead of by label; labels now only influence workflow routing
- TDD: capture the failing 'go test' output after the repro stage
  (parking if it unexpectedly passes) and post it on the PR as a code
  block — evidence the bug was reproduced before the fix
- TDD fix stage now write-protects *_test.go via agentloop.ProtectPaths
  (environmental enforcement, replacing the advisory prompt rule)
- OpenPR extracted so stages can act after the PR exists in the same
  stage (the engine stops at the first stage with an outcome)
2026-07-21 04:10:27 +10:00
sam 708e55a95b feat: implement workflow — planner/builder agent stages over agentloop
AgentStage is the reusable bridge from workflow stages to agentloop
runs: role->model selection from config, budgets, repo gate conversion,
task-row NotesStore (survives worktree cleanup, never pollutes the PR),
and park-on-anything-but-passed. Every future LLM stage (tdd,
feasibility) is an AgentStage with a different mission/gate/handler.

The implement workflow: prepare -> baseline gate (park on red base,
prevents park-storms) -> plan (read-only agentloop, plan posted to the
issue as a veto window) -> build (full toolset behind the repo's
configured gate) -> commit/push -> diff cap -> PR with the builder's
finish summary as the body.

Proven live on samcharles93/todo#3: planner 3 steps, builder 4 steps
(~31k tokens total), producing gate-verified table-driven TodoStore
tests as PR #4.
2026-07-21 04:03:06 +10:00
sam c2c8e92304 feat: archied walking skeleton — poll, store, worktree, workflow engine, bootstrap PR
Repurposes archie-core as the archie orchestrator daemon (the old
Pocketbase PM-backend plan is archived under .references/).

- cmd/archied: -config, -once (single cycle for timers/testing)
- internal/config: TOML; per-repo gate command lists; role->model refs
- internal/store: SQLite (modernc, WAL) task state machine with
  transitions audit, idempotent enqueue, crash recovery (running ->
  queued, attempt++)
- internal/forge: go-github v78 — labelled-issue polling, invitation
  auto-accept, push verification, PR create/state, comments, close
- internal/worktree: fresh clone per task, bot identity, askpass token
  handling, diff-size stats, cleanup on merge/reject
- internal/workflow: extensible engine (stages over TaskContext, shared
  step library, label routing bug->tdd feature->feasibility default->
  implement); parks always comment on the issue. Bootstrap workflow
  (deterministic marker-file PR) proves the plumbing without LLM spend.

Store and worktree round-trips covered by tests (local bare remote).
2026-07-21 03:46:06 +10:00