feat: promote archie-agent to NATS-connected per-task worker

Replaces stdin/stdout subprocess transport with NATS JetStream
request-reply pattern. The Runner interface and workflow orchestration
are unchanged.

New NATSRunner (implements Runner):
- Publishes AgentRequestMessage to JetStream with X-Archie-Reply header
- Blocks on core NATS inbox subscription for response
- Same Run(ctx, workspace, req) (Result, error) contract

Rewritten archie-agent (cmd/archie-agent):
- Long-running NATS-connected process (was 28-line stdin/stdout)
- Pull-consumes from ARCHIE_TASKS with archie.agent.> filter
- Builds InProcessRunner, executes agent loop, replies to inbox
- Graceful shutdown via signal handling

Extended internal/nats:
- SubjectAgentWildcard, SubjectForAgentRequest, ReplyHeader
- Conn() accessor, NewReplyInbox(), PublishMsg() for JetStream
- Stream subjects extended to ["archie.task.>", "archie.agent.>"]

Config: "nats" added as valid agent.mode alongside inprocess/subprocess
Tests: NATSRunner round-trip with embedded NATS server + handler
This commit is contained in:
sam committed 2026-07-22 02:13:01 +10:00
1 parent bb2193e5e4
commit 8c4bb1ef1d
11 files changed
+1309 -18

No files matched your search

+329
View File
@@ -0,0 +1,329 @@
# agentexec
Now I have all the information needed. Here is the comprehensive report.
---
## 1. All Fields of Invocation, Request, Result, Response
All types are defined in `/work/apps/archie-core/internal/agentexec/protocol.go`.
**Invocation** (lines 139-144):
```go
Version int `json:"version"`
Workspace string `json:"workspace"`
Request Request `json:"request"`
Providers map[string]Provider `json:"providers"`
```
**Request** (lines 59-74):
```go
Version int `json:"version"`
TaskID int64 `json:"task_id"`
Attempt int `json:"attempt"`
Stage string `json:"stage"`
Model string `json:"model"`
Mission string `json:"mission"`
ExtraRules string `json:"extra_rules,omitempty"`
ReadOnly bool `json:"read_only,omitempty"`
Budget Budget `json:"budget"`
Gate Gate `json:"gate"`
Preflight []Command `json:"preflight,omitempty"`
Protection Protection `json:"protection"`
Notes string `json:"notes,omitempty"`
CaptureTools []CaptureTool `json:"capture_tools,omitempty"`
```
**Result** (lines 97-111):
```go
Version int `json:"version"`
TaskID int64 `json:"task_id"`
Attempt int `json:"attempt"`
Stage string `json:"stage"`
Status string `json:"status"`
StopReason string `json:"stop_reason,omitempty"`
Changes []string `json:"changes,omitempty"`
Iterations int `json:"iterations,omitempty"`
TokensUsed int `json:"tokens_used,omitempty"`
Summary string `json:"summary,omitempty"`
Detail string `json:"detail,omitempty"`
AppendedNotes []string `json:"appended_notes,omitempty"`
Captures map[string][]json.RawMessage `json:"captures,omitempty"`
```
**Response** (lines 168-172):
```go
Version int `json:"version"`
Result Result `json:"result"`
Error string `json:"error,omitempty"`
```
Supporting types referenced above:
- **Budget** (lines 31-35): `MaxSteps int`, `MaxTokens int`, `WallClock time.Duration`
- **Gate** (lines 25-28): `Commands []Command`, `MaxConsecutiveFailures int`
- **Command** (lines 18-22): `Name string`, `Argv []string`, `ExpectFailure bool`
- **Protection** (lines 38-41): `Suffixes []string`, `Globs []string`
- **CaptureTool** (lines 45-53): `Name string`, `Description string`, `Parameters json.RawMessage`, `RequiredFields []string`, `NonEmptyStrings []string`, `BooleanFields []string`, `MaxCalls int`
- **Provider** (lines 115-119): `Class string`, `APIKeyEnv string`, `BaseURL string`
---
## 2. ServeOne Step by Step
Defined in `/work/apps/archie-core/internal/agentexec/worker.go`.
**`ServeOne` (lines 16-20)** is the public entry point. It delegates to `serveOne` with a factory closure that, given an `Invocation`, returns a runner:
```go
func ServeOne(ctx context.Context, in io.Reader, out io.Writer, log *slog.Logger) error {
return serveOne(ctx, in, out, func(invocation Invocation) Runner {
return NewInProcessRunner(NewRuntime(invocation.Providers), log)
})
}
```
**`serveOne` (lines 22-43)** does:
1. **Decode invocation** (line 24): Calls `decodeOne(in, &invocation)` which reads up to `maxProtocolBytes+1` bytes (8 MB + 1) from `in`. Rejects if the value exceeds 8 MB. JSON-decodes into the value (here, `Invocation`). Also checks there is exactly one JSON value (rejects trailing JSON values, line 58-61).
2. **Validate invocation** (line 27): Calls `invocation.Validate()` which checks:
- `Version == ProtocolVersion` (must be 1)
- `Workspace` is non-empty
- `Providers` map is non-empty, and every entry's `Class` is non-empty and `BaseURL` is parseable without userinfo/query/fragment
- The inner `Request.Validate()`: `Version` must be 1, `TaskID` > 0, `Attempt` > 0, `Stage` non-empty, `Model` non-empty
3. **Create runner** (line 30): Calls `newRunner(invocation)` -- in ServeOne's case, `NewInProcessRunner(NewRuntime(invocation.Providers), log)`.
4. **Run agent** (line 34): `runner.Run(ctx, invocation.Workspace, invocation.Request)` returns `result, runErr`.
5. **Build response** (lines 35-38): Constructs `Response{Version: ProtocolVersion, Result: result}`. If `runErr != nil`, sets `response.Error = runErr.Error()`.
6. **Encode response to stdout** (lines 39-41): `json.NewEncoder(out).Encode(response)`. Returns error if encoding fails.
7. **Return nil** on success.
Key design detail: The `serveOne` generic function separates the wire protocol (stdin/stdout JSON framing) from the runner creation strategy. The factory `func(Invocation) Runner` allows different strategies (in-process vs. subprocess vs. future NATS). Currently `ServeOne` hardcodes the in-process factory, but a future NATS-based archie-agent could call `serveOne` with a different factory or skip it entirely.
---
## 3. SubprocessRunner.Run Step by Step
Defined in `/work/apps/archie-core/internal/agentexec/subprocess.go`.
The `SubprocessRunner` struct (lines 17-27):
```go
type SubprocessRunner struct {
Command string
Args []string
Environ []string // daemon's os.Environ()
AdditionalEnv []string // operator-approved env var names
Diagnostics io.Writer // typically os.Stderr
Providers map[string]Provider
}
```
**`Run` (lines 29-94)**:
1. **Extract provider name from model** (lines 30-32): Splits `req.Model` at the first "/". E.g. `"anthropic/claude-sonnet-4"` yields providerName=`"anthropic"`. If no "/" or empty, returns error.
2. **Look up provider** (lines 34-37): Indexes `r.Providers[providerName]`. Missing returns error.
3. **Build and validate invocation** (lines 38-42): Creates `Invocation{Version, Workspace, Request, Providers}` with just that one provider entry. Validates.
4. **Check command** (lines 43-45): `strings.TrimSpace(r.Command)` must be non-empty.
5. **Marshal invocation** (lines 46-50): `json.Marshal(invocation)` + appends newline.
6. **Create OS command** (line 52): `exec.CommandContext(ctx, r.Command, r.Args...)`.
7. **Configure process cancellation** (line 53): `configureProcessCancellation(cmd)` -- on Unix this sets `Setpgid: true` and a `Cancel` function that does `syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)` (see `/work/apps/archie-core/internal/agentexec/subprocess_process_unix.go` lines 15-27). This ensures the worker process AND all its grandchildren die on context cancellation.
8. **Set stdin** (line 54): `cmd.Stdin = bytes.NewReader(payload)`.
9. **Set environment** (line 55): `WorkerEnvironment(r.Environ, r.AdditionalEnv, providers)` which (lines 155-162):
- Starts with `DefaultEnvironmentNames()`: `PATH, HOME, TMPDIR, TMP, TEMP, LANG, LC_ALL, SSL_CERT_FILE, SSL_CERT_DIR`
- Appends `r.AdditionalEnv`
- Appends each provider's `APIKeyEnv`
- Calls `SelectEnvironment` which copies only those named variables from `r.Environ`, preserving order.
10. **Capture stdout and stderr** (lines 56-62):
- Stdout: `cappedBuffer{limit: maxProtocolBytes}` (8 MB). If `r.Diagnostics != nil`, stderr goes to `io.MultiWriter(r.Diagnostics, stderr)` (tee to both the diagnostics writer and the capped buffer). Otherwise just the capped buffer (64 KB limit via `maxDiagnosticBytes`).
11. **Run the process** (lines 63-71): `cmd.Run()`. On error:
- If `context.Cause(ctx)` is set (cancellation), returns the cause wrapped with any stderr diagnostic.
- Otherwise returns the exec error with stderr output.
12. **Check stdout truncation** (lines 72-74): If `stdout.truncated` is true, return error.
13. **Decode response** (lines 75-78): `decodeOne(bytes.NewReader(stdout.Bytes()), &response)` -- same framing as the reader side.
14. **Validate response version** (lines 79-81): Must match `ProtocolVersion` (1).
15. **Validate result against request** (lines 82-91):
- If `response.Result.Version != 0`, calls `response.Result.ValidateFor(req)` -- checks Version==1, TaskID/Attempt/Stage identity match, Status non-empty.
- If `response.Error != ""`, returns result with `errors.New(response.Error)`.
- Validates one more time at line 90 for the non-error case.
16. **Return** (line 93): `response.Result, nil`.
---
## 4. NewRuntime and InProcessRunner
**`NewRuntime`** in `/work/apps/archie-core/internal/agentexec/runtime.go` (lines 7-23):
- Takes `providers map[string]Provider` (agentexec.Provider type).
- Returns `*runtime.Runtime` (from `github.com/samcharles93/ai-sdk/runtime`).
- If providers is empty, returns `nil`.
- Calls `runtime.RegisterBuiltinClasses()` (global, side-effecting).
- For each entry: creates `runtime.ProviderConfig{ID: name, Class: provider.Class, BaseURL: provider.BaseURL}`.
- If `provider.APIKeyEnv == ""`, sets `Auth = AuthConfig{Type: AuthTypeNone}`, otherwise `Auth = AuthConfig{APIKeyEnv: provider.APIKeyEnv}`.
- Calls `runtime.NewRuntime(runtime.Config{Providers: catalog})` and returns it.
**`InProcessRunner`** in `/work/apps/archie-core/internal/agentexec/inprocess.go` (lines 29-33):
```go
type InProcessRunner struct {
runtime *runtime.Runtime
log *slog.Logger
run loopFunc // func(context.Context, agentloop.Config) (agentloop.Result, error)
}
```
**`NewInProcessRunner`** (lines 35-37): Stores runtime, log, and sets `run = agentloop.Run` (the exported function from the `agentloop` package).
**`InProcessRunner.Run`** (lines 39-86):
1. Validates `req` via `req.Validate()`.
2. Checks `r.runtime != nil` and `r.run != nil`.
3. Creates `memoryNotes` struct (lines 219-228 of inprocess.go) -- implements a simple `Load`/`Append` interface. Initialized with `req.Notes`, collects `appended` entries.
4. Creates `captures` map: `map[string][]json.RawMessage`.
5. Calls `r.run(ctx, agentloop.Config{...})` with all fields mapped from `req`:
- `Runtime: r.runtime`
- `ModelRef: req.Model`
- `WorkDir: workspace`
- `Mission: req.Mission`, `ExtraRules: req.ExtraRules`
- `Notes: notes` (the memoryNotes)
- `Gate: toAgentGate(req.Gate)` (converts agentexec.Gate -> agentloop.GateConfig)
- `Preflight: toAgentCommands(req.Preflight)`
- `Budget: agentloop.Budget(req.Budget)` (direct cast)
- `ReadOnly: req.ReadOnly`
- `ProtectPaths: protectionMatcher(...)` (closure that checks suffixes/globs)
- `Extra: mergeToolSets(captureToolSet(req.CaptureTools, captures), scriptToolSet(workspace))`
- `Logger: r.logger(req)` (structured with task/attempt/stage/model)
6. Builds `Result` from the returned `agentloop.Result` fields.
7. Checks `context.Cause(ctx)` for cancellation -- if cancelled, returns result with cancellation cause as error.
8. Returns `result, err`.
---
## 5. Provider Types
There are **two separate `Provider` structs** with identical fields but different packages and tags:
**`agentexec.Provider`** (`/work/apps/archie-core/internal/agentexec/protocol.go`, lines 115-119):
```go
type Provider struct {
Class string `json:"class"`
APIKeyEnv string `json:"api_key_env,omitempty"`
BaseURL string `json:"base_url,omitempty"`
}
```
Used for: JSON serialization to the subprocess worker, and as the type consumed by `NewRuntime` and `SubprocessRunner.Providers`.
**`config.Provider`** (`/work/apps/archie-core/internal/config/config.go`, lines 125-129):
```go
type Provider struct {
Class string `toml:"class"`
APIKeyEnv string `toml:"api_key_env"`
BaseURL string `toml:"base_url"`
}
```
Used for: TOML deserialization from the config file.
**Bridge** (`/work/apps/archie-core/cmd/archied/main.go`, lines 182-188):
```go
func executionProviders(cfg config.Config) map[string]agentexec.Provider {
providers := make(map[string]agentexec.Provider, len(cfg.Providers))
for name, p := range cfg.Providers {
providers[name] = agentexec.Provider{Class: p.Class, APIKeyEnv: p.APIKeyEnv, BaseURL: p.BaseURL}
}
return providers
}
```
The daemon composition root converts config.Provider to agentexec.Provider by field-wise copy and then passes them to both `NewRuntime` and `SubprocessRunner`.
---
## 6. How drainNATS Interacts with SubprocessRunner
**Current state: drainNATS only affects task distribution, not agent execution.**
The flow:
1. **Daemon.Cycle** (daemon.go lines 94-103): If `d.Nats != nil`, calls `d.drainNATS(ctx)`, else `d.drainSQLite(ctx)`.
2. **drainNATS** (lines 126-151): Calls `d.Nats.Fetch(ctx)` to get a JetStream message. If found, calls `processNATSTask`. Falls back to `drainSQLite` logic (SQLite `ClaimNext` then `d.process`) if no NATS message.
3. **processNATSTask** (lines 219-253): Decodes `TaskMessage`, enqueues to SQLite, claims the task, then calls `d.process(ctx, task)` -- the **same** `process` function used by the SQLite path.
4. **d.process** (lines 472-493): Routes the task to the correct workflow, then calls:
```go
workflow.Run(ctx, wf, &workflow.TaskContext{
...
Agent: d.Agent, // line 488 -- the agentexec.Runner interface
...
})
```
5. **workflow.Run** (workflow.go line 145): Iterates stages. For LLM-driven stages, calls `AgentStage.Stage()` (agent.go line 110):
```go
res, err := tc.Agent.Run(ctx, tc.Dir, req)
```
This `Agent` field is the `agentexec.Runner` interface. It was wired in `cmd/archied/main.go` (lines 124-138):
- If `cfg.Agent.Mode == "subprocess"`: creates `&agentexec.SubprocessRunner{...}` with `Command`, `Environ`, `AdditionalEnv`, `Diagnostics`, `Providers`.
- If `cfg.Agent.Mode == "inprocess"`: creates `agentexec.NewInProcessRunner(llm, log)`.
6. **SubprocessRunner.Run** spawns a fresh `archie-agent` binary per stage, sends JSON on stdin, reads JSON from stdout.
7. **The daemon's NATS `processNATSTask` does NOT interact with SubprocessRunner directly** -- it only fetches the task message from NATS, enqueues it to SQLite, and claims it. The actual agent execution happens through `d.Agent.Run` which remains either in-process or subprocess regardless of the NATS path.
---
## Design Implications for NATSRunner and NATS-based archie-agent
**Runner interface** (`/work/apps/archie-core/internal/agentexec/inprocess.go`, line 20-22):
```go
type Runner interface {
Run(ctx context.Context, workspace string, req Request) (Result, error)
}
```
A new `NATSRunner` would need to implement this same interface. It would:
1. Accept a NATS connection, a subject prefix, and the provider map (or the whole Invocation fields).
2. In `Run(ctx, workspace, req)`:
- Build an `Invocation` (same as `SubprocessRunner.Run` does on lines 38-42).
- Publish it to a NATS subject (e.g. `archie.stage.request.<taskID>`).
- Include a reply subject in the message headers or as part of the payload.
- Wait for the response on the reply subject (with the context's deadline).
- Decode the `Response`, validate, return `Result`.
3. No need for stdin/stdout marshalling or process management.
**New archie-agent** (`cmd/archie-agent/main.go`): Instead of `ServeOne(ctx, os.Stdin, os.Stdout, log)`, it would:
1. Connect to NATS.
2. Subscribe to a subject like `archie.stage.request.>`.
3. For each message: decode `Invocation` from the message body, call `serveOne`-style logic (validate, build runner, run, build response), publish `Response` back to the reply subject, ack.
4. Could be a long-running daemon with a pool of subscribers.
**Key integration point**: The daemon composition root (`cmd/archied/main.go` lines 124-138) would gain a third branch:
```go
case "nats":
agentRunner = &agentexec.NATSRunner{
Conn: natsClient.Conn(), // or a new NATS connection
Subject: "archie.stage.request",
Providers: providers,
}
```
And in `config.go` the `Agent.Mode` validation (lines 298-305) would accept `"nats"` in addition to `"inprocess"` and `"subprocess"`.
**The `serveOne` generic function** in `worker.go` (lines 22-43) is already well-factored: it separates wire protocol from runner creation. A NATS-based archie-agent could reuse `serveOne` by providing a factory that creates an in-process runner, or it could have its own message handler that calls the same `Validate` -> `Runner.Run` -> `Response` logic.
+322
View File
@@ -0,0 +1,322 @@
# nats-reply
Now I have a complete understanding. Here is the comprehensive report.
---
## NATS Agent Communication Reply Pattern -- Design Report
### Sources Read
- `/work/apps/archie-core/internal/nats/client.go` -- existing JetStream Client, PublishTask, Fetch, Connect
- `/work/apps/archie-core/internal/nats/subjects.go` -- existing subject constants
- `/work/apps/archie-core/internal/daemon/daemon.go` -- current NATS integration in the daemon
- `/work/apps/archie-core/internal/config/config.go` -- NATSConfig struct
- `/work/apps/archie-core/go.mod` -- nats.go v1.52.0, nats-server/v2 v2.14.3
- `/work/apps/archie-core/internal/agentexec/inprocess.go` -- Runner interface definition (line 20)
- `/work/apps/archie-core/internal/agentexec/subprocess.go` -- SubprocessRunner.Run (blocks on exec.Command)
- `/work/apps/archie-core/internal/agentexec/protocol.go` -- Request, Result, Invocation, Response types
- `/work/apps/archie-core/internal/workflow/agent.go` -- how AgentStage calls `tc.Agent.Run(ctx, tc.Dir, req)` (line 110)
- `/home/sam/go/pkg/mod/github.com/nats-io/nats.go@v1.52.0/jetstream/publish.go` -- PublishMsg internals
- `/home/sam/go/pkg/mod/github.com/nats-io/nats.go@v1.52.0/jetstream/message.go` -- jetstream.Msg interface (Reply(), Headers(), etc.)
- `/home/sam/go/pkg/mod/github.com/nats-io/nats.go@v1.52.0/context.go` -- RequestMsgWithContext, requestWithContext
- `/home/sam/go/pkg/mod/github.com/nats-io/nats.go@v1.52.0/nats.go` -- Msg struct, NewInbox, createNewRequestAndSend
- `/work/apps/archie-core/.scratch/plan-nats-design.md` -- the original Phase 1 NATS task distribution design document
---
### Key Finding: `PublishMsg` Eats the Reply Field
The most critical technical finding is in `jetstream/publish.go` (line 229-232) and `context.go` (line 40-84). When you call:
```go
js.PublishMsg(ctx, &nats.Msg{Subject: subj, Reply: "my.reply", Data: data})
```
The `PublishMsg` implementation calls `js.conn.RequestMsgWithContext(ctx, m)`, which internally calls `createNewRequestAndSend(subj, hdr, data)` (new style) or `oldRequestWithContext` (old style). Both paths create a **fresh inbox** for the JetStream PubAck response and call `nc.publish(subj, respInbox, ...)` -- the reply subject you set on the `nats.Msg` is **completely ignored**. The wire-level `Reply` on the published NATS message is the PubAck inbox, not your custom reply subject.
There is a second issue: even if the Reply did pass through, it would need to point at an inbox the JetStream server could reach. And `PublishMsgAsync` explicitly rejects messages with `Reply` set (`ErrAsyncPublishReplySubjectSet`, line 336-337 of publish.go).
**This means: the reply-to address must be carried outside the `nats.Msg.Reply` field -- in custom headers or the message body.**
---
### 1. Subject Naming Pattern
**Requests** (published by the daemon to JetStream):
```
archie.agent.<task_id>.<stage>
```
Examples: `archie.agent.42.planner`, `archie.agent.42.builder`, `archie.agent.17.analyst`
**Responses** (published by the agent worker back to the daemon):
Use core NATS reply inboxes, generated by `nc.NewInbox()`:
```
_INBOX.<random-hex-string>
```
These are ephemeral, one-shot subjects. No JetStream stream is involved on the response path.
**Why not a deterministic subject for responses?** The daemon needs to wait for exactly one response from a specific agent execution. An inbox subscription is the NATS-idiomatic way: it creates a unique subject per request, the daemon subscribes to it before publishing, and no other subscriber can accidentally consume the response.
**New subject constants to add to `internal/nats/subjects.go`:**
```go
const SubjectAgentRequest = "archie.agent." // prefix, full subject is built with task_id and stage
func SubjectForAgentRequest(taskID int64, stage string) string {
return fmt.Sprintf("archie.agent.%d.%s", taskID, stage)
}
```
---
### 2. JetStream or Core NATS?
**Hybrid: JetStream for requests, core NATS for responses.**
| Aspect | Requests | Responses |
|--------|----------|-----------|
| Transport | JetStream (`js.PublishMsg`) | Core NATS (`nc.Publish`) |
| Persistence | Yes (survives daemon crash) | No (ephemeral) |
| Dedup | Yes (Nats-Msg-Id header) | Not needed |
| Delivery guarantee | At-least-once (consumer retries) | Best-effort (daemon subscribed) |
**Rationale:**
- The **request** (agentexec.Request) is a large, important payload. If the daemon crashes after publishing but before the response arrives, JetStream retains the message. A restarted daemon (or a different daemon on the same stream) can pick it up.
- The **response** is an ephemeral, one-shot delivery. If the daemon disconnects while waiting (crash, network blip), the response is lost -- but that is acceptable because the agent will eventually time out waiting for a result confirmation and the daemon can re-issue the request.
- Using JetStream for responses would require a second pull consumer and polling -- incompatible with the blocking `Runner.Run` contract.
- The existing `nc.Conn` is already available via `js.Conn()`, so no extra connection is needed.
---
### 3. Correlating Requests with Responses
Use **reply inbox** correlation -- the standard NATS request-reply pattern:
1. Daemon calls `nc.NewInbox()` to get a unique reply subject
2. Daemon subscribes to that inbox via `nc.SubscribeSync(inbox)` with `AutoUnsubscribe(1)`
3. Daemon sets the inbox address in a **custom header** `X-Archie-Reply` on the JetStream message
4. Daemon publishes the agent request via `js.PublishMsg`
5. Daemon blocks on `sub.NextMsg(timeout)` waiting for the response
6. Agent worker receives the JetStream message, reads the `X-Archie-Reply` header
7. Agent worker processes the request (may take minutes)
8. Agent worker publishes the result to the reply inbox via `nc.Publish(replySubject, jsonResult)`
9. Daemon receives the response and returns `agentexec.Result`
**No separate correlation ID is needed.** The reply inbox IS the correlation mechanism. The `agentexec.Response` already carries `TaskID`, `Attempt`, and `Stage` for validation (`ValidateFor` in `agentexec/protocol.go:176-190`), so we get double-checking on the identity of the response without any new identifiers.
---
### 4. Changes to `internal/nats`
#### 4a. New message types
Add to `client.go` (or a new `agent.go`):
```go
// AgentRequestMessage is the NATS payload for an agent stage execution request.
type AgentRequestMessage struct {
TaskID int64 `json:"task_id"`
Attempt int `json:"attempt"`
Stage string `json:"stage"`
Workspace string `json:"workspace"`
Request agentexec.Request `json:"request"`
Providers map[string]agentexec.Provider `json:"providers,omitempty"`
}
// AgentResponseEnvelope is the response payload published to the reply inbox.
type AgentResponseEnvelope struct {
Version int `json:"version"`
Result agentexec.Result `json:"result"`
Error string `json:"error,omitempty"`
}
```
#### 4b. New methods on `Client`
```go
// ReplyHeader is the custom header carrying the daemon's reply inbox.
const ReplyHeader = "X-Archie-Reply"
// NewReplyInbox creates a unique inbox and returns a sync subscription.
func (c *Client) NewReplyInbox() (*nats.Subscription, error) {
sub, err := c.conn.SubscribeSync(c.conn.NewInbox())
if err != nil {
return nil, err
}
sub.AutoUnsubscribe(1) // exactly one response expected
return sub, nil
}
// PublishAgentRequest publishes an agent execution request to JetStream
// with the reply inbox set in the X-Archie-Reply header.
func (c *Client) PublishAgentRequest(ctx context.Context, req agentexec.Request, providers map[string]agentexec.Provider, workspace string, replyInbox string) error {
msg := AgentRequestMessage{
TaskID: req.TaskID,
Attempt: req.Attempt,
Stage: req.Stage,
Workspace: workspace,
Request: req,
Providers: providers,
}
data, err := json.Marshal(msg)
if err != nil {
return err
}
subject := SubjectForAgentRequest(req.TaskID, req.Stage)
headers := nats.Header{}
headers.Set(ReplyHeader, replyInbox)
msgID := fmt.Sprintf("%s%d/%d/%s", msgPrefix, req.TaskID, req.Attempt, req.Stage)
headers.Set("Nats-Msg-Id", msgID)
_, err = c.js.PublishMsg(ctx, &nats.Msg{
Subject: subject,
Data: data,
Header: headers,
})
return err
}
```
#### 4c. New stream or extend existing
The existing `ARCHIE_TASKS` stream has subjects `["archie.task.>"]`. Agent requests use `archie.agent.<task_id>.<stage>`, which does **not** match `archie.task.>`.
**Recommendation: Extend the existing stream's subjects** by adding `"archie.agent.>"` to `StreamConfig.Subjects`. This is simpler than creating a second stream and avoids managing two stream lifecycles:
```go
stream, err := js.CreateOrUpdateStream(ctx, jetstream.StreamConfig{
Name: streamName,
Subjects: []string{"archie.task.>", "archie.agent.>"},
Storage: jetstream.FileStorage,
Retention: jetstream.WorkQueuePolicy,
Duplicates: dedupWindow,
})
```
The same WorkQueuePolicy works: when an agent receives and acks the request, it is removed from the stream. If the daemon crashes, the message stays until the agent acks or `MaxDeliver` is reached.
**Alternative: Create a dedicated `ARCHIE_AGENT` stream.** This would allow different retention/deduplication settings (e.g., a `LimitsPolicy` with a `MaxAge` of 1 hour vs the current WorkQueue). But this adds complexity. Start with the extension approach.
#### 4d. NATSRunner: bridge between Runner interface and NATS
Create `/work/apps/archie-core/internal/agentexec/nats.go`:
```go
type NATSRunner struct {
Nats *nats.Client
Providers map[string]agentexec.Provider
Log *slog.Logger
}
func (r *NATSRunner) Run(ctx context.Context, workspace string, req agentexec.Request) (agentexec.Result, error) {
if err := req.Validate(); err != nil {
return agentexec.Result{}, err
}
// 1. Create reply inbox
replySub, err := r.Nats.NewReplyInbox()
if err != nil {
return agentexec.Result{}, fmt.Errorf("nats reply inbox: %w", err)
}
defer replySub.Unsubscribe()
// 2. Compute timeout from budget
timeout := defaultAgentTimeout
if req.Budget.WallClock > 0 {
timeout = req.Budget.WallClock
}
// 3. Publish request with reply header
if err := r.Nats.PublishAgentRequest(ctx, req, r.Providers, workspace, replySub.Subject); err != nil {
return agentexec.Result{}, fmt.Errorf("publish agent request: %w", err)
}
// 4. Wait for reply (blocking, matching SubprocessRunner.Run behavior)
reply, err := replySub.NextMsg(timeout)
if err != nil {
return agentexec.Result{}, fmt.Errorf("agent reply timeout: %w", err)
}
// 5. Decode response
var envelope AgentResponseEnvelope
if err := json.Unmarshal(reply.Data, &envelope); err != nil {
return agentexec.Result{}, fmt.Errorf("decode agent response: %w", err)
}
if envelope.Error != "" {
return envelope.Result, errors.New(envelope.Error)
}
if err := envelope.Result.ValidateFor(req); err != nil {
return agentexec.Result{}, err
}
return envelope.Result, nil
}
```
---
### 5. Simplest Possible Reply Pattern
**Respondent's note: the paragraphs below are the core answer.**
The simplest pattern that works with the existing JetStream setup:
1. **Daemon side** (NATSRunner, blocking like SubprocessRunner):
- `sub, _ := nc.SubscribeSync(nc.NewInbox())` -- creates a unique reply subject as a sync subscription
- `sub.AutoUnsubscribe(1)` -- exactly one reply expected
- Publish to JetStream: `js.PublishMsg(ctx, &nats.Msg{Subject: "archie.agent.<taskID>.<stage>", Header: {"X-Archie-Reply": [inbox]}, Data: payload})`
- Block: `msg, err := sub.NextMsg(budget.WallClock)` -- returns `agentexec.Result` on success, error on timeout
- Unsubscribe the inbox
2. **Agent side** (archie-agent worker, or any consumer):
- Receive via JetStream pull consumer
- `replyTo := msg.Headers().Get("X-Archie-Reply")`
- Process the request (the existing agentloop)
- Result: `nc.Publish(replyTo, jsonBytes)` -- a core NATS publish, no JetStream involved
3. **No additional JetStream consumers, streams, or streams needed.** The existing `nc.Conn` (exposed as `js.Conn()`) handles both directions.
**NATS API calls involved:**
| Operation | API Call | Package |
|-----------|----------|---------|
| Create reply inbox | `nc.NewInbox()` / `nc.SubscribeSync(inbox)` | `nats.go` core |
| Publish request | `js.PublishMsg(ctx, &nats.Msg{...})` | `jetstream` |
| Wait for reply | `sub.NextMsg(timeout)` | `nats.go` core |
| Send reply (agent) | `nc.Publish(reply, data)` | `nats.go` core |
**Total new lines of code in `internal/nats`:** approximately 100-120 lines (two message structs, two new helper functions, one new subject function, and the header constant).
**Total new lines for `NATSRunner`:** approximately 50-60 lines.
---
### Configuration and Wiring
The daemon already supports `agent.mode` with values `"inprocess"` and `"subprocess"`. Add `"nats"` as a third mode.
In `cmd/archied/main.go`, when `cfg.Agent.Mode == "nats"`:
```go
runner := &agentexec.NATSRunner{
Nats: natsClient,
Providers: agentexec.ProviderConfig(cfg.Providers),
Log: log.With("runner", "nats"),
}
```
No changes are needed to the `workflow` package or the daemon's `process` method -- they already consume `tc.Agent.Run(ctx, tc.Dir, req)` via the `agentexec.Runner` interface.
---
### Critical Technical Decision: Reply Subject in Headers
The `X-Archie-Reply` header approach was chosen over alternatives:
- **Reply in `nats.Msg.Reply` field**: Does not work -- `PublishMsg` internally uses the Reply field for the JetStream PubAck response and overwrites whatever you set.
- **Reply in message body**: Would require the agent to parse the JSON body before knowing where to respond -- a chicken-and-egg problem if the body is corrupted or the format changes.
- **Correlation ID in headers + deterministic response subject**: Would require provisioning a consumer on the response subject and polling -- breaks the blocking `Runner.Run` contract.
The reply inbox approach gives us:
- No polling (daemon blocks on sync subscription)
- Automatic correlation (each request gets a unique inbox)
- No consumer lifecycle management for responses
- Matching the existing `SubprocessRunner` blocking contract exactly
+188
View File
@@ -0,0 +1,188 @@
# workflow
Here is the full analysis across all five files.
---
## 1. Fields AgentStage reads from TaskContext to build the Request
The `AgentStage.Stage()` method (lines 47-148 of `internal/workflow/agent.go`) constructs a `agentexec.Request` and reads the following from `TaskContext`:
| Request field (line) | TaskContext source | Notes |
|---|---|---|
| `Version` (94) | Constant `agentexec.ProtocolVersion` | Not from TaskContext |
| `TaskID` (96) | `tc.Task.ID` | |
| `Attempt` (97) | `tc.Task.Attempt` | |
| `Stage` (98) | `a.Name` (the AgentStage field) | Not from TaskContext |
| `Model` (99) | `tc.Cfg.Models[a.Role]` | Falls back to `tc.Cfg.Models["builder"]` at line 54 if the role key is empty; returns an error at line 57 if still empty |
| `Mission` (100) | `a.Mission(tc)` | The Mission func receives the full `*TaskContext` — implement's plan/build missions read `tc.Task.Title`, `tc.Task.Body`, `tc.Task.Plan`, `tc.Task.IssueNumber`, `tc.Repo.FullName()` |
| `ExtraRules` (101) | `a.ExtraRules` | Static field on AgentStage, not read from TaskContext per se |
| `ReadOnly` (102) | `a.ReadOnly` | Static field |
| `Budget` (103) | `tc.Cfg.Budgets.MaxSteps`, `.MaxTokens`, `.WallClock.Std()` | Overridden by `a.MaxSteps` if > 0 (line 65-67) |
| `Gate` (104) | `a.Gate(tc)` | The Gate function receives TaskContext; implement's build stage calls `GateFromRepo(tc.Repo, tc.Cfg.Budgets)` (agent.go line 152-161) |
| `Preflight` (105) | `tc.Repo.ResolvedPreflight()` | Split into `agentexec.Command` structs (lines 86-92) |
| `Protection` (106) | `tc.Repo.Protect` + `a.ProtectGlobs(tc)` | Combined; set to zero Protection if `a.ReadOnly` is true (line 82-84) |
| `Notes` (107) | `tc.Task.Notes` | |
| `CaptureTools` (108) | `a.CaptureTools(tc)` | The CaptureTools function receives TaskContext |
At line 110, `tc.Agent.Run(ctx, tc.Dir, req)` is called, which also reads `tc.Dir` (the worktree directory path) and `tc.Agent` (the Runner interface).
The Mission function for the **plan** stage (implement.go lines 48-64) reads `tc.Task.Plan`, `tc.Task.IssueNumber`, `tc.Task.Title`, `tc.Task.Body`, `tc.Repo.FullName()`.
The Mission function for the **build** stage (implement.go lines 83-93) reads `tc.Task.IssueNumber`, `tc.Task.Title`, `tc.Task.Body`, `tc.Task.Plan`, `tc.Repo.FullName()`.
---
## 2. What happens after AgentStage.Run returns — Result processing
The full post-Run sequence is in `agent.go` lines 110-146:
1. **Line 111-113: Early-return on hard transport error.** If `err != nil && res.Version == 0`, the error is wrapped and returned immediately — the result is unusable (version 0 signals no valid response).
2. **Line 114-116: Validate.** `res.ValidateFor(req)` checks that the result is consistent with the request (e.g., tool call parameters match what was requested). Returns validation error.
3. **Lines 117-124: Persist appended notes.** Any entries in `res.AppendedNotes` are appended to `tc.Task.Notes` (prefixed with "- ") and saved via `tc.Store.Update(ctx, tc.Task)`. If the store write fails, the error is returned.
4. **Lines 125-127: Return on agent-level error.** If `err != nil` (and we've already passed the version-zero check), wraps and returns it as `"agent run: ..."`. This catches cases where the agent loop ran but produced an error.
5. **Lines 129-130: Accumulate usage.** `tc.Task.TokensUsed += res.TokensUsed; tc.Task.Iterations += res.Iterations`.
6. **Lines 131-134: Emit event.** `tc.Emit("agent_finish", a.Name, res.Summary, ...)` with status, stop_reason, tokens, iterations, model.
7. **Lines 136-142: Check status.** If `res.Status != agentexec.StatusPassed`, it returns an error of the form `"agent {status} ({stop_reason}): {detail or summary clipped to 2000 chars}"`. This handles "blocked", "idle", "parked", etc. — any non-passing status causes the stage to fail.
8. **Lines 143-145: OnResult callback.** If the stage's `OnResult` func is non-nil, it is called with `(tc, res)`. The implement workflow's plan stage OnResult (lines 65-72 of implement.go) sets `tc.Task.Plan = res.Summary` and posts it as a forge comment. The build stage OnResult (lines 94-97 of implement.go) sets `tc.BuildSummary = res.Summary`.
9. **Line 146: Return nil** if no OnResult or OnResult succeeded.
---
## 3. Full list of stage names in the implement workflow; which call AgentStage
From `implement.go` lines 38-109, the stages in order:
| Order | Stage name | Calls AgentStage? | Implementation |
|---|---|---|---|
| 1 | `"prepare"` | No | `StagePrepareWorktree()` — clones repo, checks out branch (steps.go:21-31) |
| 2 | `"repo-stages"` | No | `StageRepoStages()` — runs Yaegi custom stages from `.archie/stages/*.go` (steps.go:84-104) |
| 3 | `"baseline"` | No | `StageBaselineGate()` — verifies the gate commands pass at base commit before any agent work (implement.go:16-30) |
| 4 | `"plan"` | **Yes** | `AgentStage{...}` — read-only planner, max 15 steps, posts the plan as a comment on the issue (implement.go:43-73) |
| 5 | `"build"` | **Yes** | `AgentStage{...}` — builder with gate, no ReadOnly, mission includes the plan, OnResult captures `tc.BuildSummary` (implement.go:75-98) |
| 6 | `"commit-push"` | No | `StageCommitPush(...)` — commits all worktree changes and pushes the branch (steps.go:51-58) |
| 7 | `"custom-gate"` | No | `StageYaegiGate()` — evaluates `.archie/gate.go` for project-specific rules (steps.go:111-151) |
| 8 | `"diff-cap"` | No | `StageDiffCap()` — parks if the diff exceeds configured line cap (steps.go:63-76) |
| 9 | `"open-pr"` | No | `StageOpenPR(...)` — opens the pull request with BuildSummary as body (steps.go:170-174) |
Two stages call AgentStage: **"plan"** (read-only, no gate, posts plan comment) and **"build"** (read-write, has gate, captures summary).
---
## 4. Every TaskContext.Forge call in steps.go and implement.go
All invocations of `tc.Forge.*` or `tc.TaskContext.Forge.*`:
**In `implement.go`:**
- **Line 68** — `tc.Forge.Comment(context.Background(), tc.Task.Owner, tc.Task.Repo, tc.Task.IssueNumber, body)` — Post the plan as an issue comment (inside the plan stage's `OnResult`). Uses `context.Background()` rather than the stage's context.
**In `steps.go`:**
- **Line 160** — `tc.Forge.CreatePR(ctx, t.Owner, t.Repo, title, tc.Branch, tc.Repo.BaseBranch(), body)` — Create the pull request (inside `OpenPR()` helper, called by `StageOpenPR`).
**In `workflow.go` (not steps.go or implement.go, but part of the workflow engine that uses `tc.Forge`):**
- **Line 195** — `tc.Forge.SetStateLabel(...)` — Set the state label on the issue when a workflow finishes with an outcome (inside `finish()`).
- **Line 221** — `tc.Forge.SetStateLabel(...)` — Set the "parked" label on the issue (inside `park()`).
- **Line 225** — `tc.Forge.Comment(ctx, t.Owner, t.Repo, t.IssueNumber, body)` — Post the park reason as an issue comment (inside `park()`).
Total distinct forge operations during workflow execution: **Comment** (plan posting, park notification), **CreatePR** (opening PRs), **SetStateLabel** (three call sites: finish outcome, park, and park again for the label).
Note: Forge is also used in `daemon.go` outside the workflow engine — `AcceptInvitations`, `VerifyPush`, `React`, `IssuesWithLabel`, `AssignedIssues`, `PRState`, `RepliesAfter`, `CloseIssue`, `SetStateLabel` (for queue/working labels). But in the workflow stages themselves, only the three operations above.
---
## 5. If agent execution moves to NATS: what stays in archied vs. moves to archie-agent
The central boundary is `tc.Agent.Run(ctx, tc.Dir, req)` at `agent.go:110`. This is the single point where workflow execution crosses from deterministic orchestration into LLM-driven agent execution.
### Stays in archied (all non-agent orchestration)
- **Workflow engine** (`workflow.go`): the `Run()` loop that iterates stages, persists task state after each stage, handles outcomes/parking/finish.
- **All deterministic stages** (steps.go, implement.go):
- `StagePrepareWorktree` — git clone and branch checkout via `tc.Trees`
- `StageRepoStages` — Yaegi custom stage discovery and execution
- `StageBaselineGate` — runs shell gate commands at base commit
- `StageCommitPush` — git commit and push via `tc.Trees`
- `StageYaegiGate` — Yaegi gate evaluation
- `StageDiffCap` — diff line counting
- `StageOpenPR` — forge PR creation
- **AgentStage Result processing** (`agent.go` lines 110-146) — validation, note appending, token/iteration accumulation, event emission, OnResult callbacks (plan posting to forge, BuildSummary capture).
- **TaskContext construction** (`daemon.go:481-492`) — the daemon builds TaskContext from store, config, trees, forge, etc.
- **All forge interactions** (`tc.Forge.*`) — commenting on issues, setting labels, creating PRs.
- **Store operations** (`tc.Store.*`) — task persistence, transitions, claims.
- **Task queueing and polling** (`daemon.go`): `poll`, `pollNATS`, `pollSQLite`, `checkWaiting`, `reconcilePRs`, `drainSQLite`, `drainNATS`.
### Moves to archie-agent (the LLM execution layer)
- **The actual `tc.Agent.Run(ctx, tc.Dir, req)` call** — the entire agent loop: LLM chat completion, tool execution (read/write files in the worktree, run shell commands, execute gates), budget tracking, stop reason detection.
- **The worktree directory (`tc.Dir`)** and all file I/O within it. The agent's file read/write/edit tools operate on `tc.Dir`. If archie-agent is a separate process, it needs access to the same worktree (either via shared filesystem, or by serializing file operations back to archied, or by archied exposing a filesystem server).
- **Preflight command execution** (lines 86-92 of agent.go) — `tc.Repo.ResolvedPreflight()` commands are baked into the `Request.Preflight` field and executed by the Runner; they would run in archie-agent.
- **Gate execution** (the gate commands in `Request.Gate`) — run by the Runner during the agent loop.
### What the NATS bridge layer must carry
The NATS message would need to convey:
**From archied to archie-agent (the Request):**
- TaskID, Attempt, Stage name, Model reference
- Mission string (already a string — the produced mission text)
- ExtraRules, ReadOnly flag
- Budget (MaxSteps, MaxTokens, WallClock)
- Gate (command list, max failures)
- Preflight (command list)
- Protection (suffixes + globs)
- Notes
- CaptureTools definitions
- **Worktree access** — either the path (if shared filesystem) or the repo coordinates + branch + commit SHA for the agent to clone locally
**From archie-agent to archied (the Result):**
- Version, Status, StopReason
- Summary, Detail
- TokensUsed, Iterations
- AppendedNotes (string list)
- Full tool call records (for CaptureTools)
- File modifications list (if archied needs to track what changed)
### Architectural observation about worktree access
The critical coupling is `tc.Dir` being passed directly to `tc.Agent.Run` (line 110). Currently the agent loop reads and writes files inside that directory. If archie-agent runs as a separate process:
- **Option A (shared filesystem)**: Both archied and archie-agent mount the same worktree directory. ARCHied passes the path, archie-agent writes directly. Simple but requires shared storage.
- **Option B (archied as filesystem proxy)**: Archie-agent sends file read/write/stat requests back to archied over NATS (or a separate channel). Looser coupling but higher latency and complexity.
- **Option C (archied streams the Request, agent clones independently)**: Archie-agent receives the repo/branch/commit via NATS and clones its own worktree copy. This avoids file-routing latency but requires git credentials in the agent process.
### Summary diagram
```
archied (stays) NATS bridge archie-agent (moves)
─────────────────────── ────────── ──────────────────────
Poll → Enqueue → Claim
Build TaskContext
↓
workflow.Run()
prepare (git clone)
repo-stages
baseline (gate verify)
plan (AgentStage) ───── Request ────→ ──────→ agent loop (LLM + file I/O)
←──── Result ──── ←── gate/preflight execution
OnResult: post plan comment
build (AgentStage) ───── Request ────→ ──────→ agent loop (LLM + file I/O)
←──── Result ──── ←── gate/preflight execution
OnResult: capture BuildSummary
commit-push (git)
custom-gate (yaegi)
diff-cap
open-pr (forge CreatePR)
finish/outcome
```
The workflow engine, stage orchestration, all forge/store interactions, and AgentStage result processing stay in archied. Only the `tc.Agent.Run()` call — the LLM agent loop with its file and command execution — moves to archie-agent.
+169 -8
View File
@@ -1,15 +1,33 @@
// Command archie-agent executes one unprivileged autonomous agent stage from
// a versioned JSON invocation on stdin and writes one response to stdout.
// Command archie-agent is a long-running NATS-connected worker that executes
// autonomous agent stages. It subscribes to archie.agent.> on the ARCHIE_TASKS
// JetStream stream, runs the agent loop for each received request, and publishes
// the result back to the daemon's reply inbox.
package main
import (
"context"
"encoding/json"
"flag"
"fmt"
"log/slog"
"os"
"os/signal"
"syscall"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/samcharles93/archie-core/internal/agentexec"
arnats "github.com/samcharles93/archie-core/internal/nats"
)
const (
streamName = "ARCHIE_TASKS"
consumerName = "archie-agent"
dedupWindow = 2 * time.Minute
pollTimeout = 5 * time.Second
ackWait = 30 * time.Minute
)
func main() {
@@ -17,12 +35,155 @@ func main() {
}
func run() int {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
log := slog.New(slog.NewJSONHandler(os.Stderr, nil))
if err := agentexec.ServeOne(ctx, os.Stdin, os.Stdout, log); err != nil {
log.Error("agent worker failed", "err", err)
natsURL := flag.String("nats-url", "", "NATS server URL (required)")
consumer := flag.String("consumer", consumerName, "JetStream consumer name")
flag.Parse()
if *natsURL == "" {
fmt.Fprintln(os.Stderr, "error: -nats-url is required")
flag.Usage()
return 1
}
return 0
log := slog.New(slog.NewJSONHandler(os.Stderr, nil))
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
// Connect to NATS.
nc, err := nats.Connect(*natsURL)
if err != nil {
log.Error("nats connect failed", "err", err)
return 1
}
defer nc.Close()
log.Info("nats connected", "url", *natsURL)
// JetStream setup.
js, err := jetstream.New(nc)
if err != nil {
log.Error("jetstream init failed", "err", err)
return 1
}
stream, err := js.CreateOrUpdateStream(ctx, jetstream.StreamConfig{
Name: streamName,
Subjects: []string{"archie.task.>", "archie.agent.>"},
Storage: jetstream.FileStorage,
Retention: jetstream.WorkQueuePolicy,
Duplicates: dedupWindow,
})
if err != nil {
log.Error("stream setup failed", "err", err)
return 1
}
cons, err := stream.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
Name: *consumer,
Durable: *consumer,
FilterSubject: arnats.SubjectAgentWildcard,
AckPolicy: jetstream.AckExplicitPolicy,
MaxDeliver: 3,
AckWait: ackWait,
InactiveThreshold: 24 * time.Hour,
})
if err != nil {
log.Error("consumer setup failed", "err", err)
return 1
}
log.Info("archie-agent ready", "nats", *natsURL, "consumer", *consumer)
// Main loop: fetch, run, reply, ack.
for {
if ctx.Err() != nil {
log.Info("archie-agent shutting down")
return 0
}
batch, err := cons.Fetch(1, jetstream.FetchMaxWait(pollTimeout))
if err != nil {
if ctx.Err() != nil {
return 0
}
log.Error("fetch failed", "err", err)
time.Sleep(1 * time.Second)
continue
}
var msg jetstream.Msg
for m := range batch.Messages() {
if m != nil {
msg = m
break
}
}
if err := batch.Error(); err != nil {
log.Error("fetch batch error", "err", err)
continue
}
if msg == nil {
continue
}
// Process the message.
if err := handle(ctx, msg, nc, log); err != nil {
log.Error("handle failed", "err", err)
msg.Nak()
continue
}
msg.Ack()
}
}
func handle(ctx context.Context, msg jetstream.Msg, nc *nats.Conn, log *slog.Logger) error {
// Decode the request.
var req agentexec.AgentRequestMessage
if err := json.Unmarshal(msg.Data(), &req); err != nil {
return fmt.Errorf("decode request: %w", err)
}
replyTo := msg.Headers().Get(arnats.ReplyHeader)
if replyTo == "" {
return fmt.Errorf("missing %s header", arnats.ReplyHeader)
}
log.Info("processing stage",
"task", req.TaskID,
"attempt", req.Attempt,
"stage", req.Stage,
"reply_to", replyTo,
)
// Build the agent runner and execute.
llm := agentexec.NewRuntime(req.Providers)
if llm == nil {
return fmt.Errorf("no providers configured in request")
}
runner := agentexec.NewInProcessRunner(llm, log)
result, runErr := runner.Run(ctx, req.Workspace, req.Request)
// Build and publish the response.
resp := agentexec.AgentResponseEnvelope{
Version: req.Request.Version,
Result: result,
}
if runErr != nil {
resp.Error = runErr.Error()
}
data, err := json.Marshal(resp)
if err != nil {
return fmt.Errorf("encode response: %w", err)
}
if err := nc.Publish(replyTo, data); err != nil {
return fmt.Errorf("publish response: %w", err)
}
log.Info("stage complete",
"task", req.TaskID,
"stage", req.Stage,
"status", result.Status,
)
return nil
}
+10
View File
@@ -134,6 +134,16 @@ func run() int {
}
case "inprocess":
agentRunner = agentexec.NewInProcessRunner(llm, log)
case "nats":
if natsClient == nil {
log.Error("agent.mode is nats but [nats] is not configured")
return 1
}
agentRunner = &agentexec.NATSRunner{
Nats: natsClient,
Providers: providers,
Log: log,
}
}
}
d := &daemon.Daemon{
+2 -2
View File
@@ -37,8 +37,8 @@ api_key_env = "OPENAI_API_KEY"
class = "ollama"
[agent]
mode = "inprocess" # "subprocess" runs each autonomous stage through archie-agent
command = "archie-agent" # executable path for subprocess mode
mode = "inprocess" # "subprocess" runs each stage through archie-agent (stdin/stdout); "nats" sends stages over NATS
command = "archie-agent" # executable path for subprocess mode (not used in nats mode)
# Subprocess mode isolates the protocol and process lifecycle, not OS privileges.
# The worker still runs as the daemon user until the Docker sandbox phase lands.
# env = ["GOCACHE"] # extra environment variable names forwarded to the worker
+107
View File
@@ -0,0 +1,107 @@
package agentexec
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"time"
natsio "github.com/nats-io/nats.go"
arnats "github.com/samcharles93/archie-core/internal/nats"
)
// AgentRequestMessage is the NATS payload for an agent stage execution request.
type AgentRequestMessage struct {
TaskID int64 `json:"task_id"`
Attempt int `json:"attempt"`
Stage string `json:"stage"`
Workspace string `json:"workspace"`
Request Request `json:"request"`
Providers map[string]Provider `json:"providers,omitempty"`
}
// AgentResponseEnvelope is the response payload published to the reply inbox.
type AgentResponseEnvelope struct {
Version int `json:"version"`
Result Result `json:"result"`
Error string `json:"error,omitempty"`
}
const defaultAgentTimeout = 30 * time.Minute
// NATSRunner sends agent stage execution requests over NATS JetStream and
// waits for replies on core NATS inboxes. It implements Runner.
type NATSRunner struct {
Nats *arnats.Client
Providers map[string]Provider
Log *slog.Logger
}
// Run publishes a stage execution request to JetStream with an X-Archie-Reply
// header, blocks on the sync inbox subscription, and returns the result.
func (r *NATSRunner) Run(ctx context.Context, workspace string, req Request) (Result, error) {
if err := req.Validate(); err != nil {
return Result{}, err
}
// 1. Create reply inbox with auto-unsubscribe after one message.
replySub, err := r.Nats.NewReplyInbox()
if err != nil {
return Result{}, fmt.Errorf("nats reply inbox: %w", err)
}
defer replySub.Unsubscribe()
// 2. Compute timeout from budget.
timeout := defaultAgentTimeout
if req.Budget.WallClock > 0 {
timeout = req.Budget.WallClock
}
// 3. Publish request with reply header.
msg := AgentRequestMessage{
TaskID: req.TaskID,
Attempt: req.Attempt,
Stage: req.Stage,
Workspace: workspace,
Request: req,
Providers: r.Providers,
}
data, err := json.Marshal(msg)
if err != nil {
return Result{}, err
}
subject := arnats.SubjectForAgentRequest(req.TaskID, req.Stage)
headers := natsio.Header{}
headers.Set(arnats.ReplyHeader, replySub.Subject)
if _, err := r.Nats.PublishMsg(ctx, &natsio.Msg{
Subject: subject,
Data: data,
Header: headers,
}); err != nil {
return Result{}, fmt.Errorf("publish agent request: %w", err)
}
// 4. Wait for reply (blocking, matching SubprocessRunner.Run behaviour).
reply, err := replySub.NextMsg(timeout)
if err != nil {
return Result{}, fmt.Errorf("agent reply: %w", err)
}
// 5. Decode response.
var envelope AgentResponseEnvelope
if err := json.Unmarshal(reply.Data, &envelope); err != nil {
return Result{}, fmt.Errorf("decode agent response: %w", err)
}
if envelope.Error != "" {
return envelope.Result, errors.New(envelope.Error)
}
if err := envelope.Result.ValidateFor(req); err != nil {
return envelope.Result, err
}
return envelope.Result, nil
}
+130
View File
@@ -0,0 +1,130 @@
package agentexec
import (
"context"
"encoding/json"
"log/slog"
"testing"
"time"
"github.com/nats-io/nats-server/v2/server"
natssrv "github.com/nats-io/nats-server/v2/test"
natsio "github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
arnats "github.com/samcharles93/archie-core/internal/nats"
)
func startEmbeddedNATS(t *testing.T) *server.Server {
t.Helper()
// RunRandClientPortServer doesn't enable JetStream. Enable it separately.
srv := natssrv.RunRandClientPortServer()
t.Cleanup(srv.Shutdown)
if err := srv.EnableJetStream(&server.JetStreamConfig{StoreDir: t.TempDir()}); err != nil {
t.Fatalf("enable jetstream: %v", err)
}
return srv
}
// handler subscribes to archie.agent.> and replies with a canned result.
func startHandler(t *testing.T, nc *natsio.Conn) {
t.Helper()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
stream, err := js.CreateOrUpdateStream(context.Background(), jetstream.StreamConfig{
Name: "ARCHIE_TASKS",
Subjects: []string{"archie.task.>", "archie.agent.>"},
Storage: jetstream.FileStorage,
Retention: jetstream.WorkQueuePolicy,
})
if err != nil {
t.Fatal(err)
}
cons, err := stream.CreateOrUpdateConsumer(context.Background(), jetstream.ConsumerConfig{
Name: "test-agent",
FilterSubject: arnats.SubjectAgentWildcard,
AckPolicy: jetstream.AckExplicitPolicy,
MaxDeliver: 1,
})
if err != nil {
t.Fatal(err)
}
go func() {
for {
batch, err := cons.Fetch(1, jetstream.FetchMaxWait(5 * time.Second))
if err != nil {
return
}
for msg := range batch.Messages() {
if msg == nil {
continue
}
var req AgentRequestMessage
json.Unmarshal(msg.Data(), &req)
replyTo := msg.Headers().Get(arnats.ReplyHeader)
resp := AgentResponseEnvelope{
Version: req.Request.Version,
Result: Result{
Version: ProtocolVersion,
TaskID: req.TaskID,
Attempt: req.Attempt,
Stage: req.Stage,
Status: StatusPassed,
Summary: "handler-reply",
TokensUsed: 42,
},
}
data, _ := json.Marshal(resp)
nc.Publish(replyTo, data)
msg.Ack()
}
}
}()
}
func TestNATSRunnerRoundTrip(t *testing.T) {
srv := startEmbeddedNATS(t)
url := srv.ClientURL()
ctx := context.Background()
client, err := arnats.Connect(ctx, url, slog.New(slog.DiscardHandler))
if err != nil {
t.Fatal(err)
}
defer client.Close()
// Start the handler that simulates archie-agent.
startHandler(t, client.Conn())
runner := &NATSRunner{
Nats: client,
Providers: map[string]Provider{"openai": {Class: "openai", APIKeyEnv: "TEST_KEY"}},
Log: slog.New(slog.DiscardHandler),
}
req := Request{
Version: ProtocolVersion,
TaskID: 42,
Attempt: 1,
Stage: "test",
Model: "openai/gpt",
Mission: "test mission",
Budget: Budget{MaxSteps: 5, MaxTokens: 100},
}
result, err := runner.Run(ctx, "/tmp/test-workspace", req)
if err != nil {
t.Fatal(err)
}
if result.Status != StatusPassed {
t.Fatalf("expected passed, got %s", result.Status)
}
if result.Summary != "handler-reply" {
t.Fatalf("expected 'handler-reply', got %q", result.Summary)
}
if result.TokensUsed != 42 {
t.Fatalf("expected 42 tokens, got %d", result.TokensUsed)
}
}
+2 -2
View File
@@ -296,9 +296,9 @@ func Load(path string) (Config, error) {
cfg.Agent.Command = "archie-agent"
}
switch cfg.Agent.Mode {
case "inprocess", "subprocess":
case "inprocess", "subprocess", "nats":
default:
return cfg, fmt.Errorf("config: agent.mode %q is invalid (want inprocess or subprocess)", cfg.Agent.Mode)
return cfg, fmt.Errorf("config: agent.mode %q is invalid (want inprocess, subprocess, or nats)", cfg.Agent.Mode)
}
if cfg.Agent.Mode == "subprocess" && strings.TrimSpace(cfg.Agent.Command) == "" {
return cfg, fmt.Errorf("config: agent.command is required in subprocess mode")
+24 -1
View File
@@ -54,7 +54,7 @@ func Connect(ctx context.Context, url string, log *slog.Logger) (*Client, error)
stream, err := js.CreateOrUpdateStream(ctx, jetstream.StreamConfig{
Name: streamName,
Subjects: []string{"archie.task.>"},
Subjects: []string{"archie.task.>", "archie.agent.>"},
Storage: jetstream.FileStorage,
Retention: jetstream.WorkQueuePolicy,
Duplicates: dedupWindow,
@@ -81,9 +81,32 @@ func Connect(ctx context.Context, url string, log *slog.Logger) (*Client, error)
return &Client{conn: nc, js: js, stream: stream, consumer: consumer, log: log}, nil
}
// Conn returns the underlying NATS connection for core NATS operations
// (inbox subscriptions, direct publishes).
func (c *Client) Conn() *nats.Conn { return c.conn }
// PublishMsg publishes a message to JetStream with optional headers.
func (c *Client) PublishMsg(ctx context.Context, msg *nats.Msg) (*jetstream.PubAck, error) {
return c.js.PublishMsg(ctx, msg)
}
// Close drains and closes the NATS connection.
func (c *Client) Close() { c.conn.Close() }
// NewReplyInbox creates a unique inbox subject and returns a synchronous
// subscription configured to auto-unsubscribe after one message.
func (c *Client) NewReplyInbox() (*nats.Subscription, error) {
sub, err := c.conn.SubscribeSync(c.conn.NewInbox())
if err != nil {
return nil, err
}
if err := sub.AutoUnsubscribe(1); err != nil {
sub.Unsubscribe()
return nil, err
}
return sub, nil
}
// PublishTask publishes a discovered issue to the appropriate NATS subject.
// The Nats-Msg-Id header enables JetStream dedup within the dedup window.
func (c *Client) PublishTask(ctx context.Context, owner, repo string, number int, title, body, labels string) error {
+26 -5
View File
@@ -1,10 +1,15 @@
// Package nats manages the NATS JetStream connection for task distribution.
// When configured, the daemon publishes discovered issues to NATS subjects
// and consumes them back via a JetStream pull consumer. When not configured
// (nil client), the existing SQLite ClaimNext flow is used unchanged.
// Package nats manages the NATS JetStream connection for task distribution
// and agent communication. When configured, the daemon publishes discovered
// issues to NATS subjects and consumes them back via a JetStream pull
// consumer. Agent workers consume stage execution requests and reply via
// core NATS inboxes. When not configured (nil client), the existing SQLite
// ClaimNext flow is used unchanged.
package nats
import "strings"
import (
"fmt"
"strings"
)
// Task distribution subjects. Each subject encodes the workflow type so
// future multi-daemon deployments can filter by workflow.
@@ -35,3 +40,19 @@ func SubjectForLabels(labels []string) string {
}
return SubjectTaskDefault
}
// Agent communication subjects.
const (
// SubjectAgentWildcard matches all agent stage execution requests.
SubjectAgentWildcard = "archie.agent.>"
)
// SubjectForAgentRequest returns the JetStream subject for a stage request.
func SubjectForAgentRequest(taskID int64, stage string) string {
return fmt.Sprintf("archie.agent.%d.%s", taskID, stage)
}
// ReplyHeader is the custom NATS header carrying the daemon's reply inbox.
// JetStream's PublishMsg consumes the Reply field internally for PubAck, so
// the reply address is carried via this header instead.
const ReplyHeader = "X-Archie-Reply"