Files
archie/docs/prds/external-agent-harness.md
T

415 lines
25 KiB
Markdown

# External agent harnesses
**Status:** Approved **Scope:** the harness model and, in "Session contract",
the duplex session that the setup terminal and the run-scoped live view both
use. Code cites this file for both (`ui/src/harness/SetupTerminal.vue`,
`internal/infrastructure/postgres/queries/harness_secrets.sql`).
## Decision
A workflow stage can run on an external coding-agent CLI instead of the built-in
agent loop: Claude Code, Codex, Pi, OMP, GitHub Copilot CLI, or any other CLI
packaged as a Docker Sandbox Kit v3. Each authenticates with an API key or the
operator's subscription login, and can carry the operator's own skills, plugins,
subagents and instructions.
Two boundaries hold regardless of what the harness does:
- **The real credential never enters the container.** The container sees
sentinel values; archie's egress proxy substitutes the real credential on
outbound requests and performs OAuth refresh on the host.
- **Archie's workflow guarantees are enforced outside the harness.** Git,
protected paths, read-only stages, gates and budgets are checked by archie
around the process and against the worktree afterwards.
## Kits
A harness is packaged as a Kit v3 image: an OCI image whose
`vnd.docker.sandbox.kit.descriptor` manifest annotation declares what the
harness needs. Archie reads descriptors with the published
`github.com/docker/sandbox-kit-spec/v3/spec` package and does not maintain its
own descriptor format. Published Kits run unmodified.
Archie is a partial Kit runtime. It implements these capability types:
| Type | Archie behaviour |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------- |
| `credential@1` | Proxy-managed only, described below. |
| `network-policy@1` | Enforced by the egress proxy. |
| `volume@1` | A Docker volume keyed by WorkflowExecution, deleted with it. |
| `lifecycle@1` | Install and startup hooks run on container create. |
| `agent-sessions@1` | The headless `prompt` and `resume` invocations for the stage. |
| `agent-skills@1` | Discovery paths for bundled skills and optional shared skills; host access is always read-only. |
| `agent-skill@1` | Selected image-bundled skills copied into the declared discovery paths. |
| `agent-context@1` | The resolved discovery directory, defaulting beside the workspace; preserve instructions outside Archie-managed sections. |
| `resources@1` | Container CPU and memory limits. |
Every other type fails closed: a required entry refuses the launch, an optional
one is skipped and recorded on the step. `credential@1` with `passthrough: true`
and any credential for a forge service are refused.
The operator's own setup (plugins, subagents, instructions, settings) is a mixin
Kit, published with the harness Kit as a Kit set: the set merges their layers
into one image. It is versioned image content, rewritten on every create, never
state a run can modify.
## Contract
A harness run is a second implementation of `agentexec.Runner`. It takes the
same `Request` and returns the same `Result`, so workflows, the daemon and the
State Store do not know which runner ran a stage.
`agent-sessions@1` supplies the invocation. Archie supplies, per CLI, an output
adapter that translates the CLI's JSON output stream into `ToolCallReporter`
calls, `Usage` and a session ID. A CLI with no adapter can run stages with no
capture tools, reporting no usage; its token budget falls back to wall clock.
Workflow definition validation rejects, at load time, a stage whose needs the
selected harness cannot meet: capture tools without an adapter, or a gate retry
without a `resume` verb.
## Selection
A harness is an agent profile. `docs/prds/event-automation.md` defines a profile
as the execution environment a workflow selects; a harness profile names one Kit
pinned by digest, a workload Kit or a published Kit set, in place of a container
image. Archie runs that Kit's image and composes no Kits itself; a bare mixin is
refused. Everything else about profiles holds: a workflow names its profile, a
stage may name a different one, and a profile holds no secrets.
`archie-agent` builds the stage's runner from the stage's profile: the ai-sdk
agent loop for an image profile, the harness runner for a Kit profile. Image
profiles, and every workflow already written against them, are unchanged. A Kit
profile named by no workflow is a validation error.
A stage naming a different profile than its workflow's is a separate run, not a
mid-container runner switch: a Kit task runs in one container provisioned for
one profile (Container layout), so a stage that needs a different one is its own
callee, started the way `workflow.call` already starts one
(`docs/prds/event-automation.md` "Workflows call workflows"), with its own agent
profile resolved off its own task row. This needs no mechanism beyond
`workflow.call` plus per-task profile resolution (`daemon.pinTaskProfile`), both
of which already treat every task's profile independently of any caller's -- see
`TestWorkflowCallerConformance` (staterpc) for the callee's independent
`Workflow` field and `TestWorkflowCallGivesACalleeItsOwnProfile` (daemon) for
the Kit-vs-image runner selection that independence produces.
Profiles are org resources in the State Store, so a Kit profile belongs to one
org, is granted to its workspaces like any profile, and applies without a
restart.
## Organisations
`docs/prds/orgs-and-access.md` is the boundary; a harness adds nothing that
crosses it.
- A Kit profile, its credential bindings and its OAuth tokens belong to one org.
No record is shared between orgs.
- A harness run is a run like any other: it acts as the workflow's identity,
under the run credential issued at dispatch.
- The egress proxy injects a credential only when the run credential carries the
secret the binding names. A Kit that declares a service the identity is not
granted gets no credential: a required one refuses the launch, an optional one
is skipped.
- A Kit's network policy is what the Kit asks for; the identity's network grants
are what the org allows. The proxy enforces the intersection.
- Concurrency slots are counted per binding within its org.
## Credentials
A credential binding maps a Kit's `credential@1` `service` name to an org
secret, which the org grants to identities like any secret. The secret holds one
of:
- **API key:** a value in the org's secret store.
- **OAuth:** tokens captured by the setup terminal, stored in the State Store
under the at-rest envelope of `binding-secret-encryption.md` with its own
domain separator.
The egress proxy resolves the binding for each request against the run
credential. The container gets the Kit's sentinel in the named variable or
credential file. OAuth refresh happens at the proxy, which intercepts the Kit's
`tokenEndpoint`, so a refreshed token is written to the State Store and never
returned to the container.
Runs sharing an OAuth binding share its subscription's rate limit.
`max_concurrent` on the binding (default 1) caps its simultaneous runs. A stage
waiting for a slot is `pending`, not `running`.
## Session contract
A **session** is a duplex PTY into a Kit container the daemon owns. One concept
covers both targets; only the address, the lifetime and the established egress
phase differ.
| | Setup session | Run session (deferred) |
| --------- | ------------------------------------------------------------- | ------------------------------ |
| Address | `{org, profile}` | `{org, task, attempt}` |
| Container | ephemeral, from the profile's Kit | the task's running container |
| Purpose | run the CLI's own login so the proxy captures the tokens | watch and optionally take over |
| Lifetime | bounded by the session TTL; the container is removed on close | bounded by the task |
- **Transport.** The daemon serves `harness.v1.HarnessService`, a bidi gRPC
stream, and the dashboard implements its `HarnessTerminal` interface
(`internal/webui`) over that client; the browser's WebSocket route is
unchanged. The contract lives on the daemon because the daemon owns
containers: the dashboard process must not link `internal/container`,
`internal/app/archied` or `internal/agentexec`.
- **Authorisation.** Opening a session is the `update` action on the secret,
decided by the dashboard's authorizer before the route runs
(`internal/webui/authorize.go`). The dashboard then dials the daemon with the
harness service token. The daemon does not re-authorise a human; it trusts the
asserted `{org, profile}` exactly as the task-action RPC trusts an asserted
actor. Because that authority is a shell in a container holding an org's
credential, the harness listener requires `HARNESS_TOKEN` even on loopback,
unlike the State Store's loopback convention.
- **Egress phase.** Token interception is filtered by phase (`oauth.go`'s
`rule.runtime == atRun`), and `EnterRuntime` is one-way. A setup session
therefore follows a run's lifecycle exactly: install hooks at the install
phase, `EnterRuntime` before the shell. A credential whose token endpoint must
be captured declares `phase: runtime`; a credential capturing at install phase
is captured by the install hooks that run before the switch.
- **Credentials.** Capture is gated: the proxy intercepts a token endpoint only
for a service the session is granted (`Proxy.granted`). A setup session has no
task, so the daemon registers a session grant: its proxy token resolves to
`{org, the Kit's declared credential services the org binds}`. That is the
intersection the `update`-on-secret authorisation covers. The real token never
enters the container: the response is sentinelized and stored as the org's
harness secret.
- **Container discipline.** The setup container runs a keepalive entrypoint with
an empty `Config.Env`; the PTY is an exec as the Kit's harness user with the
harness env and nothing inherited from any worker. No worktree is mounted. It
is acquired through the container pool, so it carries the `archie-daemon`
label, the concurrency cap and the max-uptime reaper; the session TTL is the
smaller bound. Closing revokes the egress session, removes the sandbox network
and the Kit volumes, then releases the container.
- **Kit unavailable.** The daemon serves the contract only when it has a
container pool and a Kit launcher; otherwise there is no listener, and a
client's refused connection is gRPC's `Unavailable`. The dashboard's existing
failure surface is the honest degrade; `/api/harness`'s `terminal` field means
"a client is configured", not "the daemon can run a Kit".
- **Run target.** `RunTarget{task_id, attempt}` opens a read-only view of a
running task's sandbox; it ships as the peek. The daemon resolves the pair
itself, so the dashboard never learns a container name: the task store says
whether the task is `running` and which attempt is current, and the Kit
launcher names the container `kitrun.TaskContainer(task_id, attempt)` (one
container per attempt, removed when the run ends; volumes remain task-scoped
for retries). A finished or unknown task, a task that is not running, an
attempt other than the running one, or a container that is gone is `NotFound`
with the reason in the message; an unreachable task store or container engine
is `Unavailable`. The view follows the container's stdout and stderr,
including the harness output forwarded by the worker, carries no stdin and no
exec, and ends when the task's container ends or the client leaves, so it
confers no authority over the container. A read-only tail of the existing
per-attempt logs was rejected: those refresh only on task state changes and
are the daemon's account, not the sandbox's. Mid-stream failures are shown as
a reason in the terminal before it closes.
- **Authorisation.** The dashboard route `GET /api/tasks/{id}/sandbox` is
decided as `read-logs` on the task before the handler runs, so a caller sees
a sandbox exactly when they may read that task's log. The daemon checks that
the task belongs to the asserted org before following output.
- **Grants.** The view takes no grant of its own. The run's egress session
(its proxy token, install then `EnterRuntime`) already belongs to the run
and is revoked by `Launcher.Release` when the task ends; the view reads
after that phase switch and dies with the container.
- **Credentials.** The sandbox holds sentinels, never real credentials: API
keys are `egress.Sentinel` (`credentials.go`), and an OAuth token set is
sentinelised in the response and swapped back by the proxy only on a granted
service's resource host in the matching phase (`oauth.go`, `injectOAuth`).
The proxy runs in the daemon outside the container, so a shell in the
container sees only sentinels.
- **Takeover (interactive PTY into a running task) is out of scope for this
bead.** It needs the dangerous-command authority (`archie-core-sod1`) to
gate it, and that authority is not wired for it: the harness listener has no
approval step; `ToolSetOptions.Approval` in `internal/agentexec/modelloop`
is never set by a caller, so task runs never build approval-required tools;
approvals reach a human only on the chat paths (dashboard ask, Telegram,
Gateway). Takeover would also need an audit record naming the actor, task,
attempt and open/close times, and revocation tied to task end (the exec is
closed with the container). Those follow the setup session's shape
(`harnesssession.Session`, `container.ExecPTY`) once the authority exists.
- **Audit.** Opening and closing a session and a successful capture are recorded
with the asserted org and profile. A run view's open and close are logged with
the task and attempt; it reads output only, so it has no takeover record.
Each session is the only interactive path into a harness container.
## Egress
Only Kit profiles run on the egress path. An image profile keeps the network
access its identity is granted, because workflows such as a network investigator
need raw DNS, ICMP and arbitrary TCP that no HTTP proxy carries.
- **Sandbox network.** Each Kit run gets its own internal Docker bridge on which
the host holds no address. Host services, including those listening on every
interface, are unreachable from it by any address.
- **Relay.** One relay container, `archie-agent relay`, sits on the sandbox
networks under a fixed alias and on the host-reachable network. It forwards
three fixed ports, to the proxy, NATS and the State Store, and can reach
nothing else. It holds no secrets.
- **Proxy.** The proxy runs inside `archied`, listening on the host gateway of
the host-reachable network. It terminates TLS with a per-daemon CA installed
into the container at create, identifies the container by a per-run proxy
password distinct from the worker's run grant, and applies that run's network
policy and credential inject rules. Upstream dials refuse loopback, and refuse
internal addresses unless the policy names the exact IP, so an allowed name
cannot resolve back into the host.
## Container layout
A Kit task runs in one container, from the Kit composition's image, on the
task's sandbox network. The worker and every harness invocation share it; they
are separated by user, not by container, so a run's cost scales with the run and
not with how many agents it fans out to.
- **Worker.** Archie mounts its own `archie-agent` binary read-only and runs it
as the container's entrypoint, as root. It holds the task's NATS and State
Store credentials in its own environment and in root-only files, runs the
workflow, and serves `archie-agent mcp`. The Kit image's entrypoint is not
PID 1.
- **Harness.** Every invocation runs as the Kit's user (the image's `User`,
default uid 1000) with an explicit environment: the Kit's variables, the proxy
environment and the credential sentinels, and nothing inherited from the
worker. That user cannot read the worker's environment or files, or trace it.
Sibling invocations in one run share the user and the worktree.
- **Hooks.** `lifecycle@1` install and startup hooks run as the user each
declares, before the worker accepts work.
- **Mounts.** The task worktree at the Kit's workspace directory, owned by the
Kit's user, and the read-only skills store. Nothing else from the host.
- **Network.** The relay forwards three fixed ports on the sandbox network: the
egress proxy, NATS and the State Store. The worker reaches NATS and the State
Store with its credentials; a harness can open those ports but holds no
credential either accepts.
- **Volumes.** `volume@1` paths persist for the life of the WorkflowExecution,
which covers session resume across gate iterations and retries. They are not
shared between executions.
## Archie tools inside the harness
Capture tools return a stage's structured results through `archie-agent mcp`, a
stdio MCP server. The same command forwards central-tool discovery and calls
over Streamable HTTP to `http://archie.tools/mcp` through the egress proxy.
Claude Code, Codex, Pi, OMP and GitHub Copilot CLI register it using their
native MCP configuration. Registration files are per-stage and outside the
worktree; the Kit's bundled settings remain intact. A CLI that exits without
initializing the required MCP server fails the stage.
`archied` handles that exact endpoint inside the authenticated task proxy; it is
not a listener or an upstream network destination. It is available only during
task runtime, never during install or a setup session. The harness supplies a
sentinel, and the proxy injects a separate random gateway token held only on the
host. Ending the proxy session revokes that token and cancels active calls.
The proxy reaches `archie-gateway` through its authenticated gRPC contract. The
Gateway derives the task, execution principal and org from a live run grant on
every request. Its tool list is bounded by the profile in that org and the
package's accepted tool authority, checked again at invocation. Workspace tools,
chat file delivery, conversation-only tools and tools that require an
unavailable approver are not served. Tool result limits apply without exposing
host spill paths; media is returned as MCP content.
Task MCP providers run in the Gateway with task-bound sessions and sampling.
Sampling uses the task's model and credentials, records task usage and refuses
revoked grants. It does not use the chat identity or chat model. The worker does
not start another copy of those providers for Kit tasks. The harness's own
built-in tools remain outside Archie's tool-result limits.
## Enforcement around the harness
| Property | Enforcement |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Model never runs git | No forge credential can be bound. Archie records `HEAD` and every ref before the run; any change fails the step (`failed`, policy class). The workflow commits the stage's changes. |
| Protected paths | A diff touching a `Protection` path fails the step. |
| Read-only stage | Any worktree diff fails the step. |
| Gate | Archie runs the stage's gate after the harness exits. On failure it resumes the session with the gate output, up to the stage's iteration budget. |
| Budget | Archie kills the process on wall-clock timeout, or when streamed usage passes the token budget. |
| Cancellation | Cancelling the step kills the process group, then the container. |
| Egress | Only the proxy is reachable; the Kit's network policy decides the rest. |
Harness settings that deny git or protected paths are added where the CLI
supports them, to save wasted turns. They are never the enforcement.
## Execution tree
A harness run is one `agent` StepExecution. Subagents the harness spawns
internally are not recorded as children. Their usage is included in the step's
`tokens_used` when the adapter reports it.
## Agents never hold a harness
An Agent reaches a harness only by invoking a Workflow whose stage names it. No
Agent tool, chat tool or capability exposes a harness, its setup terminal or its
credential. The stage boundary is what carries the container, the git and path
checks, the gate and the budget; a harness outside a stage has none of them.
## Out of scope
- Chat through a harness. The gateway chat loop stays on the built-in runtime.
- `network-policy@2` HTTP method and path rules; a Kit requiring it is refused.
- Recording the harness's internal subagents as StepExecutions.
- Cost in currency.
- Choosing a harness automatically per task.
## Plan
1. Descriptor resolution with the Kit spec package; capability support table and
fail-closed refusal.
2. Egress proxy: internal network, per-daemon CA, per-container token,
`network-policy@1`.
3. `credential@1`: API-key bindings, sentinel rendering, inject rules.
4. Harness `Runner`: container create with worktree, volumes, lifecycle hooks
and the mounted `archie-agent`; `agent-sessions@1` invocation; worktree
checks for git, protected paths and read-only; budget and cancellation.
5. Claude Code output adapter; `archie-agent mcp`; gate-and-resume loop.
6. Kit profiles, runner selection from the stage's profile, and definition
validation.
7. OAuth bindings as org secrets: token-endpoint interception, encrypted
storage, per-binding concurrency; dashboard setup terminal.
8. Output adapters for Codex, Pi, OMP and GitHub Copilot CLI.
## Verification
- Every named adapter discovers and calls a permitted central tool through the
proxy and records a local capture. A guessed tool outside the profile or
accepted package authority is refused.
- The gateway token never appears in the container, generated MCP files,
worktree, logs or events. It is not the container's proxy password.
- A permission change after discovery takes effect at the next call; ending the
proxy session cancels active calls, and a revoked grant cannot execute tools
or sample a model.
- Bundled skills remain visible with shared skills missing or disabled; shared
mounts are per-skill and read-only. Explicit context directories preserve
non-managed instructions and refuse symlinked staging paths.
- The same workflow definition completes on `builtin` and on the Claude Code Kit
with the same `Result` shape and captures.
- After `/login` in the setup terminal, the container's credential file holds
only sentinels, the State Store holds the encrypted token, and a task run
authenticates.
- A real API key or OAuth token never appears in the container environment,
filesystem, worktree, task log or event stream.
- A request to a host outside the Kit's runtime allow list fails, including a
direct connection that bypasses the proxy settings.
- A harness instructed to `git commit` fails its step, and the task branch is
unchanged.
- A harness edit to a protected test file fails the step in a TDD fix stage.
- A read-only feasibility stage that writes a file fails.
- A file a run writes into a Kit-managed path such as the CLI's settings file is
gone on the next execution.
- A Kit requiring an unsupported capability is refused at launch with the
capability named.
- A stage with a failing gate resumes the same session with the gate output,
then succeeds or exhausts its iterations.
- Two tasks sharing an OAuth binding with `max_concurrent = 1` run one at a
time; the second waits as `pending`.
- Stopping a run kills the harness process and removes the container.
- A harness invocation cannot read the worker's NATS or State Store credential
from its environment, its files or `/proc`.
- A run fanning out to many agent invocations uses one container.
- A Kit profile in org A cannot be selected, and its binding cannot be resolved,
by a workflow in org B.
- A harness run whose identity is not granted a Kit's required credential is
refused at launch; the proxy never injects a secret outside the run
credential.
- The firewall investigation example runs unchanged on an image profile with
direct network access.