docs(prds): harnesses run as Kit v3 images behind a credential-injecting egress proxy

This commit is contained in:
sam committed 2026-09-26 01:54:35 +10:00
1 parent f74827795d
commit 22d36fef75
2 files changed
+162 -108

No files matched your search

+1 -1
View File
@@ -328,7 +328,7 @@
"url": "/dev/prds/external-agent-harness/",
"title": "External agent harnesses",
"section": "PRDs",
"body": "# External agent harnesses\n\n**Status:** Draft\n**Authority:** `docs/architecture/agent-system.md` (agent execution as a data\nboundary); settles the harness non-goal deferred by\n`docs/prds/service-decomposition.md`.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (a harness\nrun is one `agent` StepExecution).\n\n## Decision\n\nA workflow stage can run on an external coding-agent CLI instead of the\nbuilt-in agent loop. Supported harnesses are Claude Code, Codex, Pi, OMP and\nGitHub Copilot CLI. Each authenticates with either an API key or the\noperator's own subscription login, and can run with the operator's own\nskills, plugins, subagents and instructions.\n\nArchie's guarantees do not move into the harness. Git, protected paths,\nread-only stages, gates and budgets are enforced by archie around the\nharness process and checked against the worktree afterwards. They never\ndepend on the harness obeying its instructions.\n\n## Contract\n\nA harness is a second implementation of `agentexec.Runner`. It takes the same\n`Request` and returns the same `Result`, so workflows, the daemon and the\nState Store do not know which runner ran a stage.\n\nEach adapter owns, for one CLI:\n\n- the non-interactive invocation and its streamed JSON output;\n- translating stream events into `ToolCallReporter` calls and `Usage`;\n- resuming the same session with follow-up input;\n- the capabilities it declares: `mcp`, `resume`, `usage`.\n\nAn adapter that cannot report usage reports zero and says so in\n`StopReason`; token budgets for that stage are enforced by wall clock only.\nWorkflow definition validation rejects a stage whose requirements the chosen\nharness does not declare, at load time, not mid-run.\n\n## Selection\n\nOrder of precedence, first match wins:\n\n1. the stage's `runner` in the workflow definition;\n2. the agent profile's `runner`;\n3. `builtin`, the ai-sdk agent loop.\n\n`runner` names a `[harnesses.\u003cname\u003e]` entry. `archie-agent` reads the entry\nwhen it builds the stage's runner; that is the field's only consumer. An\nunknown name and an entry named by no stage and no profile are both\nvalidation errors.\n\n```toml\n[harnesses.claude-sub]\nadapter = \"claude-code\"\nimage = \"registry.local/archie-agent-claude:latest\"\nhome = \"claude-sam\" # a harness home, below\nmax_concurrent = 1\n\n[harnesses.codex-api]\nadapter = \"codex\"\nimage = \"registry.local/archie-agent-codex:latest\"\napi_key = { engine = \"env\", key = \"OPENAI_API_KEY\" }\nmax_concurrent = 4\n```\n\n## Credentials and customisation\n\nAn entry authenticates one of two ways. Setting both is a validation error.\n\n- **API key:** `api_key` is a secret reference, injected into the container\n as the environment variable the adapter names. The key is never written\n into the worktree or a harness home.\n- **Subscription:** `home` names a **harness home**, a directory under\n archie's state directory that the operator logs into once with\n `archied harness login \u003chome\u003e`. That command runs the CLI's own login flow\n with its config directory pointed at the home.\n\nA harness home is also where the operator's own setup lives: skills,\nplugins, subagent definitions, global instructions, and the CLI's settings\nfile. Archie mounts it read-write as the CLI's config directory, because\nsubscription tokens refresh in place. Archie never edits it. Archie's own\nmission and tools reach the harness through the prompt and the invocation,\nso they do not touch the operator's files.\n\n`max_concurrent` caps simultaneous runs per entry, default 1. Runs sharing a\nhome share its refresh state and its subscription's rate limit; the cap is\nper home, summed across entries that name it. A stage waiting for a slot is\n`pending`, not `running`.\n\n## Archie tools inside the harness\n\nCapture tools, which are how a stage returns structured results, are served\nto the harness by `archie-agent mcp`, a stdio MCP server started inside the\ncontainer and registered through the adapter's invocation flags. It serves\nthe stage's `CaptureTools` and the central tools the agent profile allows.\nIt does not serve the `workspace` toolset, for the same reason the built-in\nrunner withholds it. The runner's `ToolLimits` bound the results it returns.\nThe harness's own built-in tools are outside archie's limits.\n\nA harness that does not declare `mcp` can only run stages with no capture\ntools.\n\n## Enforcement around the harness\n\nThe harness runs as a subprocess of `archie-agent` in the task-scoped\ncontainer, in the task worktree. Archie holds these properties by\nconstruction and by checking the worktree after the process exits:\n\n| Property | Enforcement |\n| ----------------- | -------------------------------------------------------------------------------------Line truncated
"body": "# External agent harnesses\n\n**Status:** Draft\n**Authority:** `docs/architecture/agent-system.md` (agent execution as a data\nboundary); settles the harness non-goal deferred by\n`docs/prds/service-decomposition.md`.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (a harness\nrun is one `agent` StepExecution), `docs/prds/binding-secret-encryption.md`\n(the at-rest envelope reused for harness credentials).\n\n## Decision\n\nA workflow stage can run on an external coding-agent CLI instead of the\nbuilt-in agent loop: Claude Code, Codex, Pi, OMP, GitHub Copilot CLI, or any\nother CLI packaged as a Docker Sandbox Kit v3. Each authenticates with an API\nkey or the operator's subscription login, and can carry the operator's own\nskills, plugins, subagents and instructions.\n\nTwo boundaries hold regardless of what the harness does:\n\n- **The real credential never enters the container.** The container sees\n sentinel values; archie's egress proxy substitutes the real credential on\n outbound requests and performs OAuth refresh on the host.\n- **Archie's workflow guarantees are enforced outside the harness.** Git,\n protected paths, read-only stages, gates and budgets are checked by archie\n around the process and against the worktree afterwards.\n\n## Kits\n\nA harness is packaged as a Kit v3 image: an OCI image whose\n`vnd.docker.sandbox.kit.descriptor` manifest annotation declares what the\nharness needs. Archie reads descriptors with the published\n`github.com/docker/sandbox-kit-spec/v3/spec` package and does not maintain\nits own descriptor format. Published Kits run unmodified.\n\nArchie is a partial Kit runtime. It implements these capability types:\n\n| Type | Archie behaviour |\n| ------------------- | ----------------------------------------------------------------------------------------- |\n| `credential@1` | Proxy-managed only, described below. |\n| `network-policy@1` | Enforced by the egress proxy. |\n| `volume@1` | A Docker volume keyed by WorkflowExecution, deleted with it. |\n| `lifecycle@1` | Install and startup hooks run on container create. |\n| `agent-sessions@1` | The headless `prompt` and `resume` invocations for the stage. |\n| `agent-skills@1` | The operator's skills store, mounted read-only whatever the Kit asks. |\n| `agent-context@1` | Archie's stage instructions, delivered where the Kit declares. |\n| `resources@1` | Container CPU and memory limits. |\n\nEvery other type fails closed: a required entry refuses the launch, an\noptional one is skipped and recorded on the step. `credential@1` with\n`passthrough: true` and any credential for a forge service are refused.\n\nThe operator's own setup (plugins, subagents, instructions, settings) is a\nmixin Kit the operator builds `FROM` or alongside the harness Kit. It is\nversioned image content, rewritten on every create, never state a run can\nmodify.\n\n## Contract\n\nA harness run is a second implementation of `agentexec.Runner`. It takes the\nsame `Request` and returns the same `Result`, so workflows, the daemon and\nthe State Store do not know which runner ran a stage.\n\n`agent-sessions@1` supplies the invocation. Archie supplies, per CLI, an\noutput adapter that translates the CLI's JSON output stream into\n`ToolCallReporter` calls, `Usage` and a session ID. A CLI with no adapter can\nrun stages with no capture tools, reporting no usage; its token budget falls\nback to wall clock.\n\nWorkflow definition validation rejects, at load time, a stage whose needs the\nselected harness cannot meet: capture tools without an adapter, or a gate\nretry without a `resume` verb.\n\n## Selection\n\nOrder of precedence, first match wins:\n\n1. the stage's `runner` in the workflow definition;\n2. the agent profile's `runner`;\n3. `builtin`, the ai-sdk agent loop.\n\n`runner` names a harness resource. `archie-agent` reads it when it builds the\nstage's runner; that is its only consumer. An unknown name, and a harness\nnamed by no stage and no profile, are validation errors.\n\nA harness resource holds the workload Kit reference pinned by digest, zero or\nmore mixin Kit references, the credential bindings it uses, and\n`max_concurrent` (default 1). Harness resources and credential bindings are\nState Store resources managed from the dashboard, so they apply without a\nrestart.\n\n## Credentials\n\nA credential binding is keyed by the Kit's `credential@1` `service` name and\nholds one of:\n\n- **API key:** a `{ engine, key }` secret reference resolved through\n `internal/secret`.\n- **OAutLine truncated
},
{
"url": "/dev/prds/identity-decision/",
+161 -107
View File
@@ -5,38 +5,71 @@
boundary); settles the harness non-goal deferred by
`docs/prds/service-decomposition.md`.
**Compounds with:** `docs/prds/execution-tree-state-machine.md` (a harness
run is one `agent` StepExecution).
run is one `agent` StepExecution), `docs/prds/binding-secret-encryption.md`
(the at-rest envelope reused for harness credentials).
## Decision
A workflow stage can run on an external coding-agent CLI instead of the
built-in agent loop. Supported harnesses are Claude Code, Codex, Pi, OMP and
GitHub Copilot CLI. Each authenticates with either an API key or the
operator's own subscription login, and can run with the operator's own
built-in agent loop: Claude Code, Codex, Pi, OMP, GitHub Copilot CLI, or any
other CLI packaged as a Docker Sandbox Kit v3. Each authenticates with an API
key or the operator's subscription login, and can carry the operator's own
skills, plugins, subagents and instructions.
Archie's guarantees do not move into the harness. Git, protected paths,
read-only stages, gates and budgets are enforced by archie around the
harness process and checked against the worktree afterwards. They never
depend on the harness obeying its instructions.
Two boundaries hold regardless of what the harness does:
- **The real credential never enters the container.** The container sees
sentinel values; archie's egress proxy substitutes the real credential on
outbound requests and performs OAuth refresh on the host.
- **Archie's workflow guarantees are enforced outside the harness.** Git,
protected paths, read-only stages, gates and budgets are checked by archie
around the process and against the worktree afterwards.
## Kits
A harness is packaged as a Kit v3 image: an OCI image whose
`vnd.docker.sandbox.kit.descriptor` manifest annotation declares what the
harness needs. Archie reads descriptors with the published
`github.com/docker/sandbox-kit-spec/v3/spec` package and does not maintain
its own descriptor format. Published Kits run unmodified.
Archie is a partial Kit runtime. It implements these capability types:
| Type | Archie behaviour |
| ------------------- | ----------------------------------------------------------------------------------------- |
| `credential@1` | Proxy-managed only, described below. |
| `network-policy@1` | Enforced by the egress proxy. |
| `volume@1` | A Docker volume keyed by WorkflowExecution, deleted with it. |
| `lifecycle@1` | Install and startup hooks run on container create. |
| `agent-sessions@1` | The headless `prompt` and `resume` invocations for the stage. |
| `agent-skills@1` | The operator's skills store, mounted read-only whatever the Kit asks. |
| `agent-context@1` | Archie's stage instructions, delivered where the Kit declares. |
| `resources@1` | Container CPU and memory limits. |
Every other type fails closed: a required entry refuses the launch, an
optional one is skipped and recorded on the step. `credential@1` with
`passthrough: true` and any credential for a forge service are refused.
The operator's own setup (plugins, subagents, instructions, settings) is a
mixin Kit the operator builds `FROM` or alongside the harness Kit. It is
versioned image content, rewritten on every create, never state a run can
modify.
## Contract
A harness is a second implementation of `agentexec.Runner`. It takes the same
`Request` and returns the same `Result`, so workflows, the daemon and the
State Store do not know which runner ran a stage.
A harness run is a second implementation of `agentexec.Runner`. It takes the
same `Request` and returns the same `Result`, so workflows, the daemon and
the State Store do not know which runner ran a stage.
Each adapter owns, for one CLI:
`agent-sessions@1` supplies the invocation. Archie supplies, per CLI, an
output adapter that translates the CLI's JSON output stream into
`ToolCallReporter` calls, `Usage` and a session ID. A CLI with no adapter can
run stages with no capture tools, reporting no usage; its token budget falls
back to wall clock.
- the non-interactive invocation and its streamed JSON output;
- translating stream events into `ToolCallReporter` calls and `Usage`;
- resuming the same session with follow-up input;
- the capabilities it declares: `mcp`, `resume`, `usage`.
An adapter that cannot report usage reports zero and says so in
`StopReason`; token budgets for that stage are enforced by wall clock only.
Workflow definition validation rejects a stage whose requirements the chosen
harness does not declare, at load time, not mid-run.
Workflow definition validation rejects, at load time, a stage whose needs the
selected harness cannot meet: capture tools without an adapter, or a gate
retry without a `resume` verb.
## Selection
@@ -46,87 +79,91 @@ Order of precedence, first match wins:
2. the agent profile's `runner`;
3. `builtin`, the ai-sdk agent loop.
`runner` names a `[harnesses.<name>]` entry. `archie-agent` reads the entry
when it builds the stage's runner; that is the field's only consumer. An
unknown name and an entry named by no stage and no profile are both
validation errors.
`runner` names a harness resource. `archie-agent` reads it when it builds the
stage's runner; that is its only consumer. An unknown name, and a harness
named by no stage and no profile, are validation errors.
```toml
[harnesses.claude-sub]
adapter = "claude-code"
image = "registry.local/archie-agent-claude:latest"
home = "claude-sam" # a harness home, below
max_concurrent = 1
A harness resource holds the workload Kit reference pinned by digest, zero or
more mixin Kit references, the credential bindings it uses, and
`max_concurrent` (default 1). Harness resources and credential bindings are
State Store resources managed from the dashboard, so they apply without a
restart.
[harnesses.codex-api]
adapter = "codex"
image = "registry.local/archie-agent-codex:latest"
api_key = { engine = "env", key = "OPENAI_API_KEY" }
max_concurrent = 4
```
## Credentials
## Credentials and customisation
A credential binding is keyed by the Kit's `credential@1` `service` name and
holds one of:
An entry authenticates one of two ways. Setting both is a validation error.
- **API key:** a `{ engine, key }` secret reference resolved through
`internal/secret`.
- **OAuth:** tokens captured by the setup terminal, stored in the State Store
under the at-rest envelope of `binding-secret-encryption.md` with its own
domain separator.
- **API key:** `api_key` is a secret reference, injected into the container
as the environment variable the adapter names. The key is never written
into the worktree or a harness home.
- **Subscription:** `home` names a **harness home**, a directory under
archie's state directory that the operator logs into once with
`archied harness login <home>`. That command runs the CLI's own login flow
with its config directory pointed at the home.
The egress proxy resolves the binding for each request. The container gets
the Kit's sentinel in the named variable or credential file. OAuth refresh
happens at the proxy, which intercepts the Kit's `tokenEndpoint`, so a
refreshed token is written to the State Store and never returned to the
container.
A harness home is also where the operator's own setup lives: skills,
plugins, subagent definitions, global instructions, and the CLI's settings
file. Archie mounts it read-write as the CLI's config directory, because
subscription tokens refresh in place. Archie never edits it. Archie's own
mission and tools reach the harness through the prompt and the invocation,
so they do not touch the operator's files.
Runs sharing an OAuth binding share its subscription's rate limit.
`max_concurrent` is enforced per binding, summed across harnesses that use
it. A stage waiting for a slot is `pending`, not `running`.
`max_concurrent` caps simultaneous runs per entry, default 1. Runs sharing a
home share its refresh state and its subscription's rate limit; the cap is
per home, summed across entries that name it. A stage waiting for a slot is
`pending`, not `running`.
## Setup terminal
The dashboard opens an operator-only terminal (xterm.js over a WebSocket to a
PTY) into an ephemeral container built from the harness's Kits, on the same
egress proxy. The operator runs the CLI's own login. The proxy captures the
tokens at the token endpoint into the credential binding. The container is
removed when the terminal closes. No worktree is mounted into it.
The terminal requires the administrative role. It is the only interactive
path into a harness container.
## Egress proxy
The egress proxy runs inside `archied`. Task and setup containers attach to
an internal Docker network whose only reachable endpoint is the proxy, so
egress outside the policy fails at the network, not by convention. The proxy
terminates TLS with a per-daemon CA installed into the container at create,
identifies the container by a per-container proxy token, and applies that
container's resolved `network-policy@1` and `credential@1` inject rules.
## Container layout
- The task worktree is mounted at the Kit's workspace directory. Nothing else
from the host is mounted, except the read-only skills store.
- Archie mounts its own `archie-agent` binary read-only, so published Kits
need not contain it. `archie-agent` drives the harness as a subprocess and
serves `archie-agent mcp`.
- `volume@1` paths persist for the life of the WorkflowExecution, which
covers session resume across gate iterations and retries. They are not
shared between executions.
## Archie tools inside the harness
Capture tools, which are how a stage returns structured results, are served
to the harness by `archie-agent mcp`, a stdio MCP server started inside the
container and registered through the adapter's invocation flags. It serves
the stage's `CaptureTools` and the central tools the agent profile allows.
It does not serve the `workspace` toolset, for the same reason the built-in
runner withholds it. The runner's `ToolLimits` bound the results it returns.
The harness's own built-in tools are outside archie's limits.
A harness that does not declare `mcp` can only run stages with no capture
tools.
by `archie-agent mcp`, a stdio MCP server registered through the Kit's MCP
configuration or the adapter's flags. It serves the stage's `CaptureTools`
and the central tools the agent profile's `AllowTools` permits, never the
`workspace` toolset. The runner's `ToolLimits` bound its results; the
harness's own built-in tools are outside archie's limits.
## Enforcement around the harness
The harness runs as a subprocess of `archie-agent` in the task-scoped
container, in the task worktree. Archie holds these properties by
construction and by checking the worktree after the process exits:
| Property | Enforcement |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Model never runs git | No forge credential can be bound. Archie records `HEAD` and every ref before the run; any change fails the step (`failed`, policy class). The workflow commits the stage's changes. |
| Protected paths | A diff touching a `Protection` path fails the step. |
| Read-only stage | Any worktree diff fails the step. |
| Gate | Archie runs the stage's gate after the harness exits. On failure it resumes the session with the gate output, up to the stage's iteration budget. |
| Budget | Archie kills the process on wall-clock timeout, or when streamed usage passes the token budget. |
| Cancellation | Cancelling the step kills the process group, then the container. |
| Egress | Only the proxy is reachable; the Kit's network policy decides the rest. |
| Property | Enforcement |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| Model never runs git | The container has no forge credential. Archie records `HEAD` and every ref before the run; any change fails the step (`failed`, policy class). The workflow commits the stage's changes. |
| Protected paths | A diff touching a `Protection` path fails the step. |
| Read-only stage | Any worktree diff fails the step. |
| Gate | Archie runs the stage's gate after the harness exits. On failure it resumes the same session with the gate output, up to the stage's iteration budget. A harness without `resume` gets one iteration. |
| Budget | Archie kills the process on wall-clock timeout, or when streamed usage passes the token budget. |
| Cancellation | Cancelling the step kills the process group. |
Harness settings that deny git or protected paths are added where an
adapter supports them, to save wasted turns. They are never the
enforcement.
## Images
The default `archie-agent` image bundles no third-party CLI. A harness entry
names an image that contains both the CLI and the `archie-agent` binary.
The repository ships one Dockerfile build target per supported adapter,
built on the `agent` image. Pull policy follows `containers.pull_policy`.
Harness settings that deny git or protected paths are added where the CLI
supports them, to save wasted turns. They are never the enforcement.
## Execution tree
@@ -137,42 +174,59 @@ step's `tokens_used` when the adapter reports it.
## Agents never hold a harness
An Agent reaches a harness only by invoking a Workflow whose stage names it.
No Agent tool, chat tool or capability exposes a harness, its home or its
credential. The stage boundary is what carries the container, the git and
path checks, the gate and the budget; a harness outside a stage has none of
them.
No Agent tool, chat tool or capability exposes a harness, its setup terminal
or its credential. The stage boundary is what carries the container, the git
and path checks, the gate and the budget; a harness outside a stage has none
of them.
## Out of scope
- Chat through a harness. The gateway chat loop stays on the built-in
runtime.
- `network-policy@2` HTTP method and path rules; a Kit requiring it is
refused.
- Recording the harness's internal subagents as StepExecutions.
- Cost in currency.
- Choosing a harness automatically per task.
## Plan
1. Harness `Runner` with the Claude Code adapter; worktree checks for git,
protected paths and read-only; budget and cancellation.
2. `archie-agent mcp` serving capture tools; gate-and-resume loop.
3. `[harnesses.*]` config, selection precedence and definition validation.
4. API key credentials; harness homes and `archied harness login`;
per-home concurrency.
5. Codex adapter, then Pi, OMP and GitHub Copilot CLI.
6. Dockerfile targets per adapter.
1. Descriptor resolution with the Kit spec package; capability support
table and fail-closed refusal.
2. Egress proxy: internal network, per-daemon CA, per-container token,
`network-policy@1`.
3. `credential@1`: API-key bindings, sentinel rendering, inject rules.
4. Harness `Runner`: container create with worktree, volumes, lifecycle
hooks and the mounted `archie-agent`; `agent-sessions@1` invocation;
worktree checks for git, protected paths and read-only; budget and
cancellation.
5. Claude Code output adapter; `archie-agent mcp`; gate-and-resume loop.
6. Harness resources, selection precedence and definition validation.
7. OAuth bindings: token-endpoint interception, encrypted storage, per-binding
concurrency; dashboard setup terminal.
8. Output adapters for Codex, Pi, OMP and GitHub Copilot CLI.
## Verification
- The same workflow definition completes on `builtin` and on each adapter
with the same `Result` shape and captures.
- A harness instructed to `git commit` fails its step, and the task branch
is unchanged.
- The same workflow definition completes on `builtin` and on the Claude Code
Kit with the same `Result` shape and captures.
- After `/login` in the setup terminal, the container's credential file holds
only sentinels, the State Store holds the encrypted token, and a task run
authenticates.
- A real API key or OAuth token never appears in the container environment,
filesystem, worktree, task log or event stream.
- A request to a host outside the Kit's runtime allow list fails, including a
direct connection that bypasses the proxy settings.
- A harness instructed to `git commit` fails its step, and the task branch is
unchanged.
- A harness edit to a protected test file fails the step in a TDD fix stage.
- A read-only feasibility stage that writes a file fails.
- A file a run writes into a Kit-managed path such as the CLI's settings file
is gone on the next execution.
- A Kit requiring an unsupported capability is refused at launch with the
capability named.
- A stage with a failing gate resumes the same session with the gate output,
then succeeds or exhausts its iterations.
- Two tasks naming the same home with `max_concurrent = 1` run one at a
- Two tasks sharing an OAuth binding with `max_concurrent = 1` run one at a
time; the second waits as `pending`.
- An API key never appears in the worktree, the harness home, the task log,
or the event stream.
- Stopping a run kills the harness process within the step's cancel path.
- Stopping a run kills the harness process and removes the container.