Replace the service bearer tokens with [tls] ca_file and cert_dir. Caller
identity comes from the verified certificate, not metadata. Without TLS,
services listen and dial on loopback only. Task containers get only the CA
and keep their grant.
A setup session started its container with "sleep infinity" and opened the
PTY with "sh -l". Neither binary is guaranteed: the spec capability whose
floor is a shell and bash is com.docker.sandbox/sbx@1, and archie's
supported map does not list it, so a Kit image need not provide either. A
missing sleep killed the container at start and a missing sh failed the
exec, both as an opaque error.
The container now runs archie-agent hold as its entrypoint. The agent
binary is archie's own and already mounted into every Kit, so the
keepalive needs nothing the image may lack and the container env stays
empty. Before opening the PTY, harnesssession probes for bash then sh and
returns ErrNoShell naming the Kit reference when neither is present.
Test: TestPickShell covers bash-preferred, sh-only, neither, and a probe
that cannot run.
archie-agent mcp is a stdio MCP server, on the official go-sdk, serving a
stage's capture tools and appending accepted calls to a captures file.
The harness runner writes the tool spec and a standard mcpServers config,
registers it through HarnessSpec.MCPConfig, and reads the captures back
into the stage result. The captures file is writable by the harness user,
so the worker validates every record again with the same acceptCapture
the built-in loop now shares, and forged, undeclared or over-cap records
are dropped. A stage with capture tools on a harness without MCP is
refused before it runs.
A Kit task's worker shares the sandbox network with its harness, so it
reaches NATS and the State Store through the relay like the harness
reaches the proxy. archie-agent relay takes repeated -forward
listen=target pairs, binds them all before serving so a port conflict
fails at start, and stops every forward when one fails. RelayArgs builds
the command from the fixed relay ports.
archie-agent relay listens on the sandbox's isolated network and joins
every connection to one fixed target, the proxy. The relay holds no
secrets and can reach nothing else, so the host needs no address on a
sandbox network at all.
* refactor(state-store): delete NATS storerpc; agent store is gRPC-only (archie-core-8cda.4.6)
The agent is fully on the gRPC State Store contract, so the legacy NATS
storerpc path is dead (docs/prds/state-store-contract.md §12 step 4):
- delete internal/storerpc (Server, Client, subjects) and its tests
- drop the storerpc registration from registerTaskRPCServers and the
now-unused store param
- make the agent transport's State Store URL required and Store() return
only the gRPC staterpc.Client (no NATS fallback)
- rework agentworker/agent transport tests to back the store with a real
gRPC State Store server instead of the deleted NATS storerpc server
The agent binary now fails fast when STATE_STORE_URL is absent. Agent store
calls (workflow.Store) go over gRPC to the archie-state-store service.
* refactor(state-store): delete daemon/gateway in-process store ownership (archie-core-8cda.4.6)
The daemon and gateway no longer open archie.db in-process: the single SQLite
file is owned by the standalone archie-state-store process, and both consumers
dial its gRPC State StateStore contract (docs/prds/state-store-contract.md
§12 step 7, no dual-store ownership).
- openStores no longer opens the production task store or seeds b.st; it only
wires secrets/forge and the gateway conversation store
- openStateStoreAdapter now requires [services.state].target (the old local
default is deleted) and resolves b.stateStore as the remote *staterpc.Client
- delete the in-process startStateStoreServer (the multiplexed gRPC server and
its per-task bearer-token registry) from the daemon composition
- RunGateway now resolves the same remote State Store adapter; its session
SQLite stays Gateway-owned and untouched
- daemon.ConnectedStateStore carries the configured target plus the resolved
bearer token, which containerEnv injects as STATE_STORE_URL / STATE_STORE_TOKEN
so archie-agent authenticates to the same remote service
- readiness store probe reads b.stateStore
PRD-consistent choice (noted): with the in-process server gone, per-task token
generation is removed; the daemon/gateway/agent all present the single
[services.state].target_token (or STATE_STORE_TOKEN secret) that the standalone
store validates.
* docs(webui): require [services.state].target in config and deployment templates (archie-core-8cda.4.6)
After the in-process store path is deleted, archied and archie-gateway dial the
standalone archie-state-store gRPC service via [services.state].target and never
open archie.db themselves (docs/prds/state-store-contract.md §12 step 7).
- config.example.toml documents [services.state].target as required, with the
loopback vs Docker-bridge gateway + target_token rule
- deployment templates set [services.state] per topology: bridge gateway +
token for container-mode agents, loopback for host-only
- deployments/README.md and systemd-user-service.md describe start commands
for both topologies and drop the stale "not yet consumed by the daemon" note
* style: gofumpt alignment for State Store daemon literal; appease noctx/staticcheck (archie-core-8cda.4.6)
* docs(state-store): refresh stale in-process references post-cutover (archie-core-8cda.4.6)
* feat(state-store): generalize storerpc as the State Store gRPC contract
Implements archie-core-8cda.4.2 against docs/prds/state-store-contract.md
rev. 2c: generate protobuf/gRPC client and server adapters for every
ratified store contract (proto/state/v1/state.proto, 40 RPCs, one
StateStoreService), add internal/infrastructure/staterpc with a bufconn
conformance suite exercising both local and remote adapters, serve the
service in-process (bridge address + mandatory per-task bearer token when
agent containers exist, loopback-only otherwise), and flip archie-agent's
workflow.Store to a long-lived gRPC client when the daemon injects
STATE_STORE_URL/STATE_STORE_TOKEN (falling back to the legacy NATS
storerpc path otherwise -- one agent process is never both).
Also drops the *sql.Tx parameter from store.BindingDispatcher.RecordDispatch
(a required wire-blocker fix: a transaction cannot cross a gRPC boundary,
and production always passed nil).
fix(store): drop RecordDispatch's unusable *sql.Tx parameter
feat(config): add services.state contract-adapter selector
* fix(state-store): apply no-mistakes review decisions
- Drop StreamTokenInterceptor (unused; the State Store service has no
streaming RPCs and nothing calls it).
- Make StateStoreTokens.Validate compare tokens in constant time
(crypto/subtle) per the auto-fix hardening.ing finding.
- Amend config.example.toml / config.Services.State doc text so it frames
[services.state].target as the ratified surface consumed by the upcoming
standalone archie-state-store binary (.4.3), not active today.
Autonomous workflows now have exactly one production boundary: a task-scoped archie-agent receives a full-task core-NATS request and runs the worker-local stage loop. The legacy [agent] section remains decode-only for migration; it no longer selects runtime behavior.
cmd/archie-agent used to own worker construction, NATS lifecycle, task execution, MCP setup, and workflow wiring. It now owns process input only and invokes internal/app/agentworker, while transport-specific NATS, boot-brief, and git-safety mechanics live under infrastructure.
The dubious-ownership fix landed twice: markWorktreeSafe in gitsafe.go
and an inline exec.Command block in main.go, both configuring the same
safe.directory on every start. Two setup paths for one invariant means a
later change to either one silently disagrees with the other.
Keep the tested helper and drop the inline block. markWorktreeSafe also
skips the call when /data/worktree is not a mounted directory, so
running archie-agent directly on a host no longer writes to the
operator's own ~/.gitconfig for a path that was never mounted.
The new test parses main.go and fails on more than one setup path, so a
future re-addition of an inline block is caught rather than merged.
Refs #468
Agent containers bind-mount the worktree from the host at
/data/worktree, where the owning UID differs from the container
user's UID. Git refuses to operate in such directories, so every
gate command that calls the Go toolchain (go build/test/vet) fails
with 'fatal: detected dubious ownership in repository' before any
real work begins.
Add 'git config --global --add safe.directory /data/worktree' at
archie-agent startup so all subsequent git-spawned operations
(baseline gate, build gate, agent shell tool invocations) work
correctly. Logs a warning if git is unavailable rather than
failing — the agent remains usable for repos with no gate config.
Part of #453 (task-observability epic #450). archie-agent logged to
os.Stderr only, and its container is created with AutoRemove: true
(internal/container/pool.go), so all of it disappeared the moment a
task's container exited -- there was no way to see why, say, a gate
command failed inside one.
Adds agentexec.SystemLogHandler: a slog.Handler that tees to whatever
it wraps (kept for an operator attached to a live container) and
additionally publishes each record, JSON-encoded as a logging.Entry,
to SubjectForSystem(taskID) -- a subject that already existed,
documented for exactly this ("log dumps, health... observability"),
with zero publishers anywhere in the codebase.
cmd/archie-agent/main.go wires it in only when this container was
spawned for one specific task (bootTaskID / .git/task.json present);
shared queue-group mode serves many unrelated tasks per process, so
there's no single task subject to ship to and keeps the stderr-only
logger.
Building this surfaced a real pre-existing bug: FeedHandler's attr
flattening applied the group chain active at Handle-time to *all*
accumulated attrs, including ones added via WithAttrs before a later
WithGroup call -- nesting them under a group they were never
actually inside, contrary to slog's own documented Handler contract.
It went unnoticed because nothing in the codebase called WithGroup
until this change needed to. Extracting the shared flattening logic
into exported FlattenAttrs/PrefixAttrs (so SystemLogHandler doesn't
duplicate it) is what surfaced it; fixed in the same change since I
was the one making it public API.
Docs on the wire format's own trap for the daemon-side consumer
(#454, not yet written): the published bytes are a marshaled
logging.Entry, meant for json.Unmarshal, not logging.decode() --
decode() parses raw slog JSON *log lines* off disk, a different and
incompatible shape. See the doc comment on SystemLogHandler.
- Reword docs and comments to drop vague "legacy" qualifiers where they no longer describe deprecation status
- Remove unused code: RecentEvents, MessageEvent.ToLegacy, index setState, stderr redaction tool, telegramOnlyCommands
- Delete or comment out tests covering removed or superseded behavior
WriteTaskJSON wrote task.json to the worktree root -- the same worktree
the agent commits from. CommitAll stages with go-git's All option, and
go-git's Add does not honour .gitignore or .git/info/exclude, so the
brief was swept into the agent's commit and pushed onto every task
branch. That is how a task.json for issue #57 ended up committed in
the repo root.
This is the identical failure the prepared sentinel hit; worktree.go
already documents the fix. Nothing under .git can ever be tracked, so
put the brief there rather than papering over it with an exclude.
dependencies-and-contracts.md rule 1 says domains must not import
internal/infrastructure. internal/daemon and internal/agentexec both did,
and for exactly the thing the doc carves out: "internal/eventbus is an
approved shared application contract ... it MUST remain broker-neutral."
The Publisher/Consumer/Requester interfaces existed but lived inside the
NATS package, so a caller could depend on the behaviour yet still had to
import the implementation to name it. They now live in internal/eventbus,
and nats.Client asserts against eventbus.Bus.
The larger violation was ownership of meaning. The doc says "message schemas
stay with their owning domains; the event bus transports messages, it does
not own their meaning." The bus exported TaskEnvelope and a PublishTask
method, so the transport knew what a unit of work was, and it exported the
archie.agent.* subject helpers, so it also owned the agent protocol.
- TaskEnvelope, its routing Kind, the label vocabulary and the task
subjects move to internal/domain/workintake, which encodes and decodes
its own bytes. This starts internal/domain/ per organisation.md.
- The agent request/response/system subjects move to internal/agentexec,
which defines what those messages mean.
- internal/workflow keeps only the kind-to-registry mapping; the label
table stays single and lives with the domain that reads forge issues.
- The bus now carries opaque payloads on plain-string subjects.
Consumers declare the smallest interface they need rather than taking the
whole Bus: daemon.TaskBus is publish-unique/fetch/request, agentexec.
RequestBus is publish-request/reply-inbox, and archie-agent's stageBus is
fetch/respond. None can reach what it does not use.
Two defects this surfaced:
- archie-agent declared its own JetStream stream with a second copy of the
name, subjects, retention, dedup window, MaxDeliver and TTLs that
nats.Connect already sets -- two definitions of one stream, free to
diverge. Config now takes Subjects and FilterSubject from composition,
which is what let the agent use the shared client and delete its
hand-rolled setup.
- Replying to a request needs core NATS, not the JetStream publish path:
a reply inbox is ephemeral and belongs to no stream, so a durable
publish to it fails. The old code hid this by using the raw connection.
The contract now distinguishes Respond from Publish. The agentexec
round-trip test caught it after I had made the same mistake in
archie-agent.
Also translated the SDK's ErrNoResponders into eventbus.ErrNoResponders, so
the daemon's taskrun retry no longer imports the NATS SDK to name a
transient condition. The SDK is now confined to the nats package, the *rpc
packages (themselves NATS infrastructure), and cmd/*.
The agentexec test fake previously hand-rolled a JetStream consumer; it now
drains the client's own consumer, which is both closer to production and
avoids conflicting consumers on a work-queue stream.
Verified: go build, go vet, gofmt clean; full test suite passes.
Relocating internal/nats as-is would have cemented its problems into the
approved layout, so this is a rewrite rather than a move.
What was wrong:
- Conn() *nats.Conn, PublishMsg(*nats.Msg) and Fetch() jetstream.Msg pushed
SDK types into all four consumers, so the broker was not replaceable --
the opposite of what dependencies-and-contracts.md requires.
- PublishTask took seven positional parameters, six of them strings, where a
transposed argument was a silent bug.
- Labels travelled as a comma-separated string and were re-split on every
publish.
- Fetch returned (nil, nil) for 'no message', which every caller had to
remember to special-case.
- The logger field was stored and never used.
- Stream name, consumer name and every timeout were hardcoded constants.
- Errors were inconsistently wrapped; several paths returned bare errors.
- Connect repeated nc.Close() on each failure branch.
What replaces it:
- One concern per file: config, errors, subjects, message, client,
publisher, consumer.
- Publisher/Consumer/Requester interfaces with compile-time assertions, so
callers depend on behaviour and can be faked without a broker.
- A Message wrapper keeps jetstream types behind the boundary; consumers no
longer import the NATS SDK at all.
- TaskEnvelope with []string labels and JSON methods preserving the
comma-separated wire form for already-queued messages.
- Sentinel errors (ErrNoMessage, ErrNotConnected, ErrNoReplyAddress,
ErrInvalidConfig) and %w wrapping with operation context throughout.
- Config with validation and defaults; a single deferred cleanup in Connect.
- CoreConn() as the one documented escape hatch, for the *rpc packages that
are themselves NATS infrastructure.
Consumers migrated: cmd/archied, cmd/archie-agent, internal/daemon,
internal/agentexec. Behaviour, subjects and wire format unchanged.
19 files touched. All fixes are in code — no lint configuration
weakened:
- Added unchecked error handling in defer cleanup paths
- Removed dead variables and unused functions
- Added missing imports (io in gitea.go)
- Simplified conditions (TrimSuffix, De Morgan, slices.Contains)
Also fixes JetStream TempDir teardown race (archie-core-340):
t.Cleanup(srv.Shutdown) now registers after t.TempDir() so
server shutdown (including JetStream files) completes before
temp dir removal. Added srv.WaitForShutdown() for deterministic
cleanup ordering.
Closes: archie-core-756, archie-core-340
daemon.process() branches on ContainerPool != nil: sandboxed tasks publish
a taskrun.Request to archie.taskrun.<id> and await the response via the
new runViaAgent method, instead of running workflow.Route/workflow.Run
locally. Non-container mode is unchanged. Every durable side effect
(Store, Forge, git push) already landed via the storerpc/forgerpc/
worktreerpc proxies during archie-agent's run by the time runViaAgent
returns — it only parks the task itself on transport/setup failure, never
duplicating a terminal state archie-agent's own workflow.Run already
recorded.
Also fixes a real correctness gap: with concurrent multi-task dispatch
(archie-core-u64/g1l) plus containerized archie-agent, a shared queue-group
subscription could let one container answer a sibling's taskrun message
while having a *different* task's worktree mounted. archie-agent now reads
its boot-time task.json and subscribes to its own exact per-task subject
when present, falling back to the shared queue group only in dev/pool mode
(no task.json).
Adds agentexec.ProvidersFromConfig, deduped from cmd/archied's former
executionProviders.
Closes the archie-core-5zh epic: archie-agent runs an entire task's
workflow per invocation instead of one stage per round trip, with all
credentials staying daemon-side throughout.
Add internal/taskrun (Request/Response wire types) and
cmd/archie-agent/taskrun.go: runTask builds a workflow.Registry from the
container's own bind-mounted worktree, routes and runs the task's entire
workflow via workflow.Route/workflow.Run, using storerpc.Client/
forgerpc.Client for Store/Forge and a new hybridTrees that splits
Prepare/Push (proxied over worktreerpc — network ops needing the daemon's
credential) from CommitAll/Diff/ChangedFiles/ChangedLines (local git
against the mount).
Add config.TaskConfig.ToConfig() to rebuild a usable Config from the
curated payload, and storage.WorktreeMountDir (/data/worktree) — critical
fix: hybridTrees.Prepare discards the RPC response's host-side directory
and returns the container's fixed local mount path instead, since archied
and archie-agent see the same files at different paths.
Wired a queue-grouped core-NATS subscription on archie.taskrun.> in
cmd/archie-agent/main.go alongside the existing per-stage consumer, so
this is reachable in the running binary. End-to-end test drives the real
"bootstrap" workflow (no LLM) through real storerpc/forgerpc/worktreerpc
servers and a real local git remote — registry build, route, prepare,
commit, push, PR creation, and confirms the authoritative task state
lands in archied's store via RPC, not just in-memory.
archie-core-5zh.8 (daemon.go handoff) is unblocked.
Add token_env to [nats] config, resolved and validated at daemon startup
(fails early if configured but the named env var is unset). Container
workers get NATS_URL/NATS_TOKEN; archie-agent honors both while CLI-supplied
URL still takes precedence. Anonymous auth remains the default for
single-node deployments.
Unblocks archie-core-5zh (promote archie-agent to per-task process).
Daemon previously processed one task at a time. Add bounded concurrent
dispatch for SQLite and NATS agent modes, reusing [containers].max_concurrency
(0 = unlimited). Same-repository tasks stay FIFO-serialized to avoid worktree
conflicts; different repositories now run concurrently.
Unblocks archie-core-l1q (NATS transport) and archie-core-g1l
(per-repo allow_concurrent override).
Replaces field-existence checks with runtime behavior verification:
Gap 1 (per-task agent): FAILING
TestHandleMessageReadsWorkflowField verifies HandleMessage produces
different results when Workflow is set vs. unset. Currently identical
— HandleMessage ignores the Workflow field. Must run full multi-stage
workflow per PRD section 1.
Gap 3 (response/system split): PASSING
TestHandleMessageRoutesSystemMessagesSeparately verifies Channel
echoes from request to response. HandleMessage now reads input Channel
and sets it on the response envelope. Subject functions verified
distinct.
Gap 5 (daemon review): PASSING
TestReviewResultBlocksOnResultWhenRejected verifies ReviewResult
hook gates OnResult. Error blocks — OnResult never called. Nil
allows — OnResult called normally.
Agent processing extracted into agentexec.HandleMessage for testability.
cmd/archie-agent delegates to it.