From f74827795d032d4475aa2eb02257ef05c2b2373d Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Sat, 26 Sep 2026 01:45:10 +1000 Subject: [PATCH] docs(prds): fold review findings into execution-tree and harness PRDs --- docs/data/generated/dev-docs.json | 4 ++-- docs/prds/execution-tree-state-machine.md | 8 +++++++- docs/prds/external-agent-harness.md | 10 ++++++---- 3 files changed, 15 insertions(+), 7 deletions(-) diff --git a/docs/data/generated/dev-docs.json b/docs/data/generated/dev-docs.json index 46bc1d5cb..e0a61672b 100644 --- a/docs/data/generated/dev-docs.json +++ b/docs/data/generated/dev-docs.json @@ -316,7 +316,7 @@ "url": "/dev/prds/execution-tree-state-machine/", "title": "Execution tree and enforced state machine", "section": "PRDs", - "body": "# Execution tree and enforced state machine\n\n**Status:** Draft\n**Authority:** `docs/architecture/agent-system.md` (canonical workflow model);\nsettles the \"enforced WorkflowExecution state machine\" item in\n`docs/architecture/migration-decisions.md` section 4.\n**Compounds with:** `docs/prds/task-run-detail.md` (attempt attribution),\n`docs/prds/state-store-contract.md` (wire contract and task-scoped grants).\n\n## Decision\n\nA WorkflowExecution becomes the root of a tree of StepExecutions. Every stage\nrun, and every agent call a stage makes, is its own durable record with a\nparent, an attempt, a status and an outcome. Both levels move only through a\ndeclared transition table, enforced by the State Store in the same transaction\nthat writes the audit row and the domain event.\n\n## Problem\n\n- A run is one `tasks` row. The only per-step record is the mutable `stage`\n column, overwritten as `workflow.Run` advances, plus observability events.\n A failed attempt cannot say which step failed or what its children did\n without replaying the event stream.\n- `Store.Transition` guards on the expected `from` status, but any `from` to\n any `to` is accepted. A `queued` to `merged` write succeeds.\n- Stop cancels one context in `daemon.runningTasks`. Nothing persisted says\n which work was in flight, so an agent call that outlives its stage (a\n reviewer, a container) is invisible to the store.\n- `workflow.Run` writes `tc.Store.Update` and discards the error, then emits\n `KindStageStart` separately. State and events can diverge.\n\n## Model\n\n```text\nWorkflowExecution (tasks row, id E)\n└── attempt N\n ├── StepExecution stage \"plan\" depth 0\n ├── StepExecution stage \"implement\" depth 0\n │ └── StepExecution agent \"implement\" depth 1\n └── StepExecution stage \"review\" depth 0\n ├── StepExecution agent \"review:1\" depth 1\n └── StepExecution agent \"review:2\" depth 1\n```\n\n- A StepExecution belongs to exactly one WorkflowExecution and one attempt.\n `parent_id` is null for a stage and names the enclosing StepExecution\n otherwise. Depth is derived from the parent, never supplied by the caller.\n- `kind` is `stage` or `agent`. A new kind needs a named consumer before it is\n added.\n- A retry starts a new attempt. Earlier attempts' StepExecutions are never\n rewritten.\n\n## Transition tables\n\nWorkflowExecution statuses are the existing `taskstate` constants, persisted\nstrings unchanged. `taskstate` gains the one table every writer checks:\n\n| From | To |\n| --------------- | ---------------------------------------------------------- |\n| `queued` | `running`, `closed_wont_do` |\n| `running` | `queued` (crash recovery), `waiting_human`, `pr_open`, `completed`, `parked`, `closed_wont_do` |\n| `waiting_human` | `queued` (approve), `closed_wont_do` |\n| `parked` | `queued` (retry), `dead` (abandon), `closed_wont_do` |\n| `pr_open` | `merged`, `rejected`, `queued` (remediation), `closed_wont_do` |\n| terminal | none |\n\nStepExecution statuses:\n\n| From | To |\n| --------- | ------------------------------------------------------ |\n| `pending` | `running`, `cancelled` |\n| `running` | `succeeded`, `failed`, `cancelled`, `interrupted` |\n| terminal | none |\n\n`interrupted` means the process running the step died or shut down. It is\ndistinct from `cancelled`, which an operator or a parent asked for, and from\n`failed`, which the step reported.\n\nA StepExecution cannot reach `running` while its WorkflowExecution is not\n`running`, and cannot start under a terminal parent.\n\n## Enforcement\n\n- The table lives in `taskstate` (no dependencies), so the store, daemon and\n gateway share one copy.\n- The State Store rejects an illegal transition with a new sentinel,\n `ErrIllegalTransition`, carried across the wire through `mapError` like\n `ErrStaleTransition`. A stale `from` stays `ErrStaleTransition`.\n- Each transition is one transaction: guarded row update, audit row, domain\n event row. The events table is written by the store, not emitted beside it.\n `KindStageStart` and `KindStageFinish` are produced by the step transition.\n- `workflow.Run` stops discarding persistence errors. A failed step write\n parks the execution; the step does not run unrecorded.\n\n## Cancellation\n\nStopping a run is one store call, `CancelExecution(E, reason)`. In one\ntransaction it moves every non-terminal StepExecution of the current attempt\nto `cancelled` and the WorkflowExecution to `closed_wont_do` or `parked`,\nwhichever the operator action names. The daemon then cancels the in-memory\ncontext. A worker whose step write returns `ErrStaleTransition` because its\nstep is already `cancelled` stops without writing further.\n\nThe store is the record of cancellation. The context cancel is the delivery\nmechanism, so a lost cancel still leaves the store correct and the worker's\nnext write fails.\n\n## Crash recovery\n\n`RecoverStale` moves every `running` StepExecution of an interrupted\nexecution to `interrupted` in the same transaction that requeues the\nexecution. The next attempt starts with fresh steps.\n\n## Wire contract\n\nNew State Store RPCs, each with a matching Go method on a narrow consumer\nfacade:\n\n- `StartStep(execution, attempt, parent, kind, name) -\u003e step id`\n- `FinishStep(step, from, to, detail, tokens_used)`\n- `CancelExecution(execution, reason, to)`\n- `ListSteps(execution, attempt)`\n\n`TaskGrants.UnaryInterceptor` authorises a task-scoped token for `StartStep`\nand `FinishStep` on its own execution only. The container records its own\nagent calls. `CancelExecution` and `ListSteps` require the administrative\ntoken.\n\n## Storage\n\nOne new table, `step_executions`: `id`, `execution_id`, `attempt`,\n`parent_id`, `depth`, `kind`, `name`, `status`, `detail`, `tokens_used`,\n`started_at`, `finished_at`. Indexed on `(execution_id, attempt, id)`.\nExisting rows gain no backfilled steps. Their history stays in `transitions`\nand `events`.\n\n`tasks.stage` is written from the step transition while any reader still uses\nit, then dropped once the dashboard reads `ListSteps`.\n\n## Out of scope\n\n- Running sibling steps in parallel. The tree records concurrency; changing\n the stage runner is separate work.\n- Cost in currency. `tokens_used` per step is the only usage field.\n- Workflow definition versioning and the `tasks` to `workflow_executions`\n rename.\n- A harness contract for swappable coding agents.\n\n## Plan\n\n1. `taskstate` transition tables with table-driven tests covering every\n legal and a sample of illegal pairs.\n2. `ErrIllegalTransition` enforced in the Postgres store and mapped over the\n wire.\n3. `step_executions` migration, `StartStep`/`FinishStep`, grant rules;\n `workflow.Run` records stages; events written in the transition\n transaction.\n4. Agent-call children recorded from the agent runtime.\n5. `CancelExecution`; daemon stop and chat/dashboard actions route through it.\n6. `RecoverStale` interrupts running steps.\n7. Dashboard run detail reads `ListSteps`; `tasks.stage` dropped.\n\n## Verification\n\n- An illegal WorkflowExecution or StepExecution transition returns\n `ErrIllegalTransition` over gRPC, and `errors.Is` matches on the client.\n- Killing the daemon mid-stage, then restarting, leaves that stage\n `interrupted` and the execution `queued` with a new attempt.\n- Stopping a run during review leaves every review child `cancelled`, and a\n late `FinishStep` from the container returns `ErrStaleTransition`.\n- A task-scoped token cannot `StartStep` on another execution.\n- For every step transition there is exactly one event row, written in the\n same transaction.\n" + "body": "# Execution tree and enforced state machine\n\n**Status:** Draft\n**Authority:** `docs/architecture/agent-system.md` (canonical workflow model);\nsettles the \"enforced WorkflowExecution state machine\" item in\n`docs/architecture/migration-decisions.md` section 4.\n**Compounds with:** `docs/prds/task-run-detail.md` (attempt attribution),\n`docs/prds/state-store-contract.md` (wire contract and task-scoped grants).\n\n## Decision\n\nA WorkflowExecution becomes the root of a tree of StepExecutions. Every stage\nrun, and every agent call a stage makes, is its own durable record with a\nparent, an attempt, a status and an outcome. Both levels move only through a\ndeclared transition table, enforced by the State Store in the same transaction\nthat writes the audit row and the domain event.\n\n## Problem\n\n- A run is one `tasks` row. The only per-step record is the mutable `stage`\n column, overwritten as `workflow.Run` advances, plus observability events.\n A failed attempt cannot say which step failed or what its children did\n without replaying the event stream.\n- `Store.Transition` guards on the expected `from` status, but any `from` to\n any `to` is accepted. A `queued` to `merged` write succeeds.\n- Stop cancels one context in `daemon.runningTasks`. Nothing persisted says\n which work was in flight, so an agent call that outlives its stage (a\n reviewer, a container) is invisible to the store.\n- `workflow.Run` writes `tc.Store.Update` and discards the error, then emits\n `KindStageStart` separately. State and events can diverge.\n\n## Model\n\n```text\nWorkflowExecution (tasks row, id E)\n└── attempt N\n ├── StepExecution stage \"plan\" depth 0\n ├── StepExecution stage \"implement\" depth 0\n │ └── StepExecution agent \"implement\" depth 1\n └── StepExecution stage \"review\" depth 0\n ├── StepExecution agent \"review:1\" depth 1\n └── StepExecution agent \"review:2\" depth 1\n```\n\n- A StepExecution belongs to exactly one WorkflowExecution and one attempt.\n `parent_id` is null for a stage and names the enclosing StepExecution\n otherwise. Depth is derived from the parent, never supplied by the caller.\n- `kind` is `stage` or `agent`. A new kind needs a named consumer before it is\n added.\n- A retry starts a new attempt. Earlier attempts' StepExecutions are never\n rewritten.\n\n## Transition tables\n\nWorkflowExecution statuses are the existing `taskstate` constants, persisted\nstrings unchanged. `taskstate` gains the one table every writer checks:\n\n| From | To |\n| --------------- | ---------------------------------------------------------- |\n| `queued` | `running`, `closed_wont_do` |\n| `running` | `queued` (crash recovery), `waiting_human`, `pr_open`, `completed`, `parked`, `closed_wont_do` |\n| `waiting_human` | `queued` (approve), `closed_wont_do` |\n| `parked` | `queued` (retry), `dead` (abandon), `closed_wont_do` |\n| `pr_open` | `merged`, `rejected`, `queued` (remediation), `closed_wont_do` |\n| terminal | none |\n\nStepExecution statuses:\n\n| From | To |\n| --------- | ------------------------------------------------------ |\n| `pending` | `running`, `cancelled` |\n| `running` | `succeeded`, `failed`, `cancelled`, `interrupted` |\n| terminal | none |\n\n`interrupted` means the process running the step died or shut down. It is\ndistinct from `cancelled`, which an operator or a parent asked for, and from\n`failed`, which the step reported.\n\nA StepExecution cannot reach `running` while its WorkflowExecution is not\n`running`, and cannot start under a terminal parent.\n\n## Enforcement\n\n- The table lives in `taskstate` (no dependencies), so the store, daemon and\n gateway share one copy.\n- Every store method that changes `tasks.status` checks the table and writes\n the audit and event rows, not only `Transition`. That includes the claim,\n `ParkTask`, `Requeue`, `RetryTask`, `BeginRemediation` and `RecoverStale`\n paths, several of which change status without an audit row.\n- The State Store rejects an illegal transition with a new sentinel,\n `ErrIllegalTransition`, carried across the wire through `mapError` like\n `ErrStaleTransition`. A stale `from` stays `ErrStaleTransition`.\n- Each transition is one transaction: guarded row update, audit row, domain\n event row. The events table is written by the store, not emitted beside it.\n `KindStageStart` and `KindStageFinish` are produced by the step transition\n and published to the bus after commit.\n- `workflow.Run` stops discarding persistence errors. A failed step write\n parks the execution; the step does not run unrecorded.\n\n## Cancellation\n\nStopping a run is one store call, `CancelExecution(E, reason)`. In one\ntransaction it moves every non-terminal StepExecution of the current attempt\nto `cancelled` and the WorkflowExecution to `closed_wont_do` or `parked`,\nwhichever the operator action names. The daemon then cancels the in-memory\ncontext. A worker whose step write returns `ErrStaleTransition` because its\nstep is already `cancelled` stops without writing further.\n\nThe store is the record of cancellation. The context cancel is the delivery\nmechanism, so a lost cancel still leaves the store correct and the worker's\nnext write fails.\n\n## Crash recovery\n\n`RecoverStale` moves every `running` StepExecution of an interrupted\nexecution to `interrupted` in the same transaction that requeues the\nexecution. The next attempt starts with fresh steps.\n\n## Wire contract\n\nNew State Store RPCs, each with a matching Go method on a narrow consumer\nfacade:\n\n- `StartStep(execution, attempt, parent, kind, name) -\u003e step id`\n- `FinishStep(step, from, to, detail, tokens_used)`\n- `CancelExecution(execution, reason, to)`\n- `ListSteps(execution, attempt)`\n\n`TaskGrants.UnaryInterceptor` authorises a task-scoped token for `StartStep`\nand `FinishStep` on its own execution only. The container records its own\nagent calls. `CancelExecution` and `ListSteps` require the administrative\ntoken.\n\n## Storage\n\nOne new table, `step_executions`: `id`, `execution_id`, `attempt`,\n`parent_id`, `depth`, `kind`, `name`, `status`, `detail`, `tokens_used`,\n`started_at`, `finished_at`. Indexed on `(execution_id, attempt, id)`.\nDeleting an execution (archive, clearing terminal tasks) deletes its steps.\nExisting rows gain no backfilled steps. Their history stays in `transitions`\nand `events`.\n\n`tasks.stage` is written from the step transition while any reader still uses\nit, then dropped once the dashboard reads `ListSteps`.\n\n## Out of scope\n\n- Running sibling steps in parallel. The tree records concurrency; changing\n the stage runner is separate work.\n- Cost in currency. `tokens_used` per step is the only usage field.\n- Workflow definition versioning and the `tasks` to `workflow_executions`\n rename.\n- A harness contract for swappable coding agents.\n\n## Plan\n\n1. `taskstate` transition tables with table-driven tests covering every\n legal and a sample of illegal pairs.\n2. `ErrIllegalTransition` enforced in the Postgres store and mapped over the\n wire.\n3. `step_executions` migration, `StartStep`/`FinishStep`, grant rules;\n `workflow.Run` records stages; events written in the transition\n transaction.\n4. Agent-call children recorded from the agent runtime.\n5. `CancelExecution`; daemon stop and chat/dashboard actions route through it.\n6. `RecoverStale` interrupts running steps.\n7. Dashboard run detail reads `ListSteps`; `tasks.stage` dropped.\n\n## Verification\n\n- An illegal WorkflowExecution or StepExecution transition returns\n `ErrIllegalTransition` over gRPC, and `errors.Is` matches on the client.\n- Killing the daemon mid-stage, then restarting, leaves that stage\n `interrupted` and the execution `queued` with a new attempt.\n- Stopping a run during review leaves every review child `cancelled`, and a\n late `FinishStep` from the container returns `ErrStaleTransition`.\n- A task-scoped token cannot `StartStep` on another execution.\n- For every step transition there is exactly one event row, written in the\n same transaction.\n" }, { "url": "/dev/prds/extensions-center/", @@ -328,7 +328,7 @@ "url": "/dev/prds/external-agent-harness/", "title": "External agent harnesses", "section": "PRDs", - "body": "# External agent harnesses\n\n**Status:** Draft\n**Authority:** `docs/architecture/agent-system.md` (agent execution as a data\nboundary); settles the harness non-goal deferred by\n`docs/prds/service-decomposition.md`.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (a harness\nrun is one `agent` StepExecution).\n\n## Decision\n\nA workflow stage can run on an external coding-agent CLI instead of the\nbuilt-in agent loop. Supported harnesses are Claude Code, Codex, Pi, OMP and\nGitHub Copilot CLI. Each authenticates with either an API key or the\noperator's own subscription login, and can run with the operator's own\nskills, plugins, subagents and instructions.\n\nArchie's guarantees do not move into the harness. Git, protected paths,\nread-only stages, gates and budgets are enforced by archie around the\nharness process and checked against the worktree afterwards. They never\ndepend on the harness obeying its instructions.\n\n## Contract\n\nA harness is a second implementation of `agentexec.Runner`. It takes the same\n`Request` and returns the same `Result`, so workflows, the daemon and the\nState Store do not know which runner ran a stage.\n\nEach adapter owns, for one CLI:\n\n- the non-interactive invocation and its streamed JSON output;\n- translating stream events into `ToolCallReporter` calls and `Usage`;\n- resuming the same session with follow-up input;\n- the capabilities it declares: `mcp`, `resume`, `usage`.\n\nAn adapter that cannot report usage reports zero and says so in\n`StopReason`; token budgets for that stage are enforced by wall clock only.\nWorkflow definition validation rejects a stage whose requirements the chosen\nharness does not declare, at load time, not mid-run.\n\n## Selection\n\nOrder of precedence, first match wins:\n\n1. the stage's `runner` in the workflow definition;\n2. the agent profile's `runner`;\n3. `builtin`, the ai-sdk agent loop.\n\n`runner` names a `[harnesses.\u003cname\u003e]` entry. An entry with no consumer\n(named by no stage and no profile) is a validation warning, and an unknown\nname is a validation error.\n\n```toml\n[harnesses.claude-sub]\nadapter = \"claude-code\"\nimage = \"registry.local/archie-agent-claude:latest\"\nhome = \"claude-sam\" # a harness home, below\nmax_concurrent = 1\n\n[harnesses.codex-api]\nadapter = \"codex\"\nimage = \"registry.local/archie-agent-codex:latest\"\napi_key = { engine = \"env\", key = \"OPENAI_API_KEY\" }\nmax_concurrent = 4\n```\n\n## Credentials and customisation\n\nAn entry authenticates one of two ways. Setting both is a validation error.\n\n- **API key:** `api_key` is a secret reference, injected into the container\n as the environment variable the adapter names. The key is never written\n into the worktree or a harness home.\n- **Subscription:** `home` names a **harness home**, a directory under\n archie's state directory that the operator logs into once with\n `archied harness login \u003chome\u003e`. That command runs the CLI's own login flow\n with its config directory pointed at the home.\n\nA harness home is also where the operator's own setup lives: skills,\nplugins, subagent definitions, global instructions, and the CLI's settings\nfile. Archie mounts it read-write as the CLI's config directory, because\nsubscription tokens refresh in place. Archie never edits it. Archie's own\nmission and tools reach the harness through the prompt and the invocation,\nso they do not touch the operator's files.\n\n`max_concurrent` caps simultaneous runs per entry, default 1. Runs sharing a\nhome share its refresh state and its subscription's rate limit; the cap is\nper home, summed across entries that name it. A stage waiting for a slot is\n`pending`, not `running`.\n\n## Archie tools inside the harness\n\nCapture tools, which are how a stage returns structured results, are served\nto the harness by `archie-agent mcp`, a stdio MCP server started inside the\ncontainer and registered through the adapter's invocation flags. It serves\nthe stage's `CaptureTools` and the central tools the agent profile allows.\nIt does not serve the `workspace` toolset, for the same reason the built-in\nrunner withholds it.\n\nA harness that does not declare `mcp` can only run stages with no capture\ntools.\n\n## Enforcement around the harness\n\nThe harness runs as a subprocess of `archie-agent` in the task-scoped\ncontainer, in the task worktree. Archie holds these properties by\nconstruction and by checking the worktree after the process exits:\n\n| Property | Enforcement |\n| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |\n| Model never runs git | The container has no forge credential. Archie records `HEAD` and every ref before the run; any change fails the step (`failed`, policy class). The workflow commits the stage's changes. |\n| Protected paths | A diff touching a `Protection` path fails the step. |\n| Read-only stage | Any worktree diff fails the step. |\n| Gate | Archie runs the stage's gate after the harness exits. On failure it resumes the same session with the gate output, up to the stage's iteration budget. A harness without `resume` gets one iteration. |\n| Budget | Archie kills the process on wall-clock timeout, or when streamed usage passes the token budget. |\n| Cancellation | Cancelling the step kills the process group. |\n\nHarness settings that deny git or protected paths are added where an\nadapter supports them, to save wasted turns. They are never the\nenforcement.\n\n## Images\n\nThe default `archie-agent` image bundles no third-party CLI. A harness entry\nnames an image that contains both the CLI and the `archie-agent` binary.\nThe repository ships one Dockerfile build target per supported adapter,\nbuilt on the `agent` image. Pull policy follows `containers.pull_policy`.\n\n## Execution tree\n\nA harness run is one `agent` StepExecution. Subagents the harness spawns\ninternally are not recorded as children. Their usage is included in the\nstep's `tokens_used` when the adapter reports it.\n\n## Agents never hold a harness\n\nAn Agent reaches a harness only by invoking a Workflow whose stage names it.\nNo Agent tool, chat tool or capability exposes a harness, its home or its\ncredential. The stage boundary is what carries the container, the git and\npath checks, the gate and the budget; a harness outside a stage has none of\nthem.\n\n## Out of scope\n\n- Chat through a harness. The gateway chat loop stays on the built-in\n runtime.\n- Recording the harness's internal subagents as StepExecutions.\n- Cost in currency.\n- Choosing a harness automatically per task.\n\n## Plan\n\n1. Harness `Runner` with the Claude Code adapter; worktree checks for git,\n protected paths and read-only; budget and cancellation.\n2. `archie-agent mcp` serving capture tools; gate-and-resume loop.\n3. `[harnesses.*]` config, selection precedence and definition validation.\n4. API key credentials; harness homes and `archied harness login`;\n per-home concurrency.\n5. Codex adapter, then Pi, OMP and GitHub Copilot CLI.\n6. Dockerfile targets per adapter.\n\n## Verification\n\n- The same workflow definition completes on `builtin` and on each adapter\n with the same `Result` shape and captures.\n- A harness instructed to `git commit` fails its step, and the task branch\n is unchanged.\n- A harness edit to a protected test file fails the step in a TDD fix stage.\n- A read-only feasibility stage that writes a file fails.\n- A stage with a failing gate resumes the same session with the gate output,\n then succeeds or exhausts its iterations.\n- Two tasks naming the same home with `max_concurrent = 1` run one at a\n time; the second waits as `pending`.\n- An API key never appears in the worktree, the harness home, the task log,\n or the event stream.\n- Stopping a run kills the harness process within the step's cancel path.\n" + "body": "# External agent harnesses\n\n**Status:** Draft\n**Authority:** `docs/architecture/agent-system.md` (agent execution as a data\nboundary); settles the harness non-goal deferred by\n`docs/prds/service-decomposition.md`.\n**Compounds with:** `docs/prds/execution-tree-state-machine.md` (a harness\nrun is one `agent` StepExecution).\n\n## Decision\n\nA workflow stage can run on an external coding-agent CLI instead of the\nbuilt-in agent loop. Supported harnesses are Claude Code, Codex, Pi, OMP and\nGitHub Copilot CLI. Each authenticates with either an API key or the\noperator's own subscription login, and can run with the operator's own\nskills, plugins, subagents and instructions.\n\nArchie's guarantees do not move into the harness. Git, protected paths,\nread-only stages, gates and budgets are enforced by archie around the\nharness process and checked against the worktree afterwards. They never\ndepend on the harness obeying its instructions.\n\n## Contract\n\nA harness is a second implementation of `agentexec.Runner`. It takes the same\n`Request` and returns the same `Result`, so workflows, the daemon and the\nState Store do not know which runner ran a stage.\n\nEach adapter owns, for one CLI:\n\n- the non-interactive invocation and its streamed JSON output;\n- translating stream events into `ToolCallReporter` calls and `Usage`;\n- resuming the same session with follow-up input;\n- the capabilities it declares: `mcp`, `resume`, `usage`.\n\nAn adapter that cannot report usage reports zero and says so in\n`StopReason`; token budgets for that stage are enforced by wall clock only.\nWorkflow definition validation rejects a stage whose requirements the chosen\nharness does not declare, at load time, not mid-run.\n\n## Selection\n\nOrder of precedence, first match wins:\n\n1. the stage's `runner` in the workflow definition;\n2. the agent profile's `runner`;\n3. `builtin`, the ai-sdk agent loop.\n\n`runner` names a `[harnesses.\u003cname\u003e]` entry. `archie-agent` reads the entry\nwhen it builds the stage's runner; that is the field's only consumer. An\nunknown name and an entry named by no stage and no profile are both\nvalidation errors.\n\n```toml\n[harnesses.claude-sub]\nadapter = \"claude-code\"\nimage = \"registry.local/archie-agent-claude:latest\"\nhome = \"claude-sam\" # a harness home, below\nmax_concurrent = 1\n\n[harnesses.codex-api]\nadapter = \"codex\"\nimage = \"registry.local/archie-agent-codex:latest\"\napi_key = { engine = \"env\", key = \"OPENAI_API_KEY\" }\nmax_concurrent = 4\n```\n\n## Credentials and customisation\n\nAn entry authenticates one of two ways. Setting both is a validation error.\n\n- **API key:** `api_key` is a secret reference, injected into the container\n as the environment variable the adapter names. The key is never written\n into the worktree or a harness home.\n- **Subscription:** `home` names a **harness home**, a directory under\n archie's state directory that the operator logs into once with\n `archied harness login \u003chome\u003e`. That command runs the CLI's own login flow\n with its config directory pointed at the home.\n\nA harness home is also where the operator's own setup lives: skills,\nplugins, subagent definitions, global instructions, and the CLI's settings\nfile. Archie mounts it read-write as the CLI's config directory, because\nsubscription tokens refresh in place. Archie never edits it. Archie's own\nmission and tools reach the harness through the prompt and the invocation,\nso they do not touch the operator's files.\n\n`max_concurrent` caps simultaneous runs per entry, default 1. Runs sharing a\nhome share its refresh state and its subscription's rate limit; the cap is\nper home, summed across entries that name it. A stage waiting for a slot is\n`pending`, not `running`.\n\n## Archie tools inside the harness\n\nCapture tools, which are how a stage returns structured results, are served\nto the harness by `archie-agent mcp`, a stdio MCP server started inside the\ncontainer and registered through the adapter's invocation flags. It serves\nthe stage's `CaptureTools` and the central tools the agent profile allows.\nIt does not serve the `workspace` toolset, for the same reason the built-in\nrunner withholds it. The runner's `ToolLimits` bound the results it returns.\nThe harness's own built-in tools are outside archie's limits.\n\nA harness that does not declare `mcp` can only run stages with no capture\ntools.\n\n## Enforcement around the harness\n\nThe harness runs as a subprocess of `archie-agent` in the task-scoped\ncontainer, in the task worktree. Archie holds these properties by\nconstruction and by checking the worktree after the process exits:\n\n| Property | Enforcement |\n| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |\n| Model never runs git | The container has no forge credential. Archie records `HEAD` and every ref before the run; any change fails the step (`failed`, policy class). The workflow commits the stage's changes. |\n| Protected paths | A diff touching a `Protection` path fails the step. |\n| Read-only stage | Any worktree diff fails the step. |\n| Gate | Archie runs the stage's gate after the harness exits. On failure it resumes the same session with the gate output, up to the stage's iteration budget. A harness without `resume` gets one iteration. |\n| Budget | Archie kills the process on wall-clock timeout, or when streamed usage passes the token budget. |\n| Cancellation | Cancelling the step kills the process group. |\n\nHarness settings that deny git or protected paths are added where an\nadapter supports them, to save wasted turns. They are never the\nenforcement.\n\n## Images\n\nThe default `archie-agent` image bundles no third-party CLI. A harness entry\nnames an image that contains both the CLI and the `archie-agent` binary.\nThe repository ships one Dockerfile build target per supported adapter,\nbuilt on the `agent` image. Pull policy follows `containers.pull_policy`.\n\n## Execution tree\n\nA harness run is one `agent` StepExecution. Subagents the harness spawns\ninternally are not recorded as children. Their usage is included in the\nstep's `tokens_used` when the adapter reports it.\n\n## Agents never hold a harness\n\nAn Agent reaches a harness only by invoking a Workflow whose stage names it.\nNo Agent tool, chat tool or capability exposes a harness, its home or its\ncredential. The stage boundary is what carries the container, the git and\npath checks, the gate and the budget; a harness outside a stage has none of\nthem.\n\n## Out of scope\n\n- Chat through a harness. The gateway chat loop stays on the built-in\n runtime.\n- Recording the harness's internal subagents as StepExecutions.\n- Cost in currency.\n- Choosing a harness automatically per task.\n\n## Plan\n\n1. Harness `Runner` with the Claude Code adapter; worktree checks for git,\n protected paths and read-only; budget and cancellation.\n2. `archie-agent mcp` serving capture tools; gate-and-resume loop.\n3. `[harnesses.*]` config, selection precedence and definition validation.\n4. API key credentials; harness homes and `archied harness login`;\n per-home concurrency.\n5. Codex adapter, then Pi, OMP and GitHub Copilot CLI.\n6. Dockerfile targets per adapter.\n\n## Verification\n\n- The same workflow definition completes on `builtin` and on each adapter\n with the same `Result` shape and captures.\n- A harness instructed to `git commit` fails its step, and the task branch\n is unchanged.\n- A harness edit to a protected test file fails the step in a TDD fix stage.\n- A read-only feasibility stage that writes a file fails.\n- A stage with a failing gate resumes the same session with the gate output,\n then succeeds or exhausts its iterations.\n- Two tasks naming the same home with `max_concurrent = 1` run one at a\n time; the second waits as `pending`.\n- An API key never appears in the worktree, the harness home, the task log,\n or the event stream.\n- Stopping a run kills the harness process within the step's cancel path.\n" }, { "url": "/dev/prds/identity-decision/", diff --git a/docs/prds/execution-tree-state-machine.md b/docs/prds/execution-tree-state-machine.md index e4b3cb420..def7f6c07 100644 --- a/docs/prds/execution-tree-state-machine.md +++ b/docs/prds/execution-tree-state-machine.md @@ -83,12 +83,17 @@ A StepExecution cannot reach `running` while its WorkflowExecution is not - The table lives in `taskstate` (no dependencies), so the store, daemon and gateway share one copy. +- Every store method that changes `tasks.status` checks the table and writes + the audit and event rows, not only `Transition`. That includes the claim, + `ParkTask`, `Requeue`, `RetryTask`, `BeginRemediation` and `RecoverStale` + paths, several of which change status without an audit row. - The State Store rejects an illegal transition with a new sentinel, `ErrIllegalTransition`, carried across the wire through `mapError` like `ErrStaleTransition`. A stale `from` stays `ErrStaleTransition`. - Each transition is one transaction: guarded row update, audit row, domain event row. The events table is written by the store, not emitted beside it. - `KindStageStart` and `KindStageFinish` are produced by the step transition. + `KindStageStart` and `KindStageFinish` are produced by the step transition + and published to the bus after commit. - `workflow.Run` stops discarding persistence errors. A failed step write parks the execution; the step does not run unrecorded. @@ -131,6 +136,7 @@ token. One new table, `step_executions`: `id`, `execution_id`, `attempt`, `parent_id`, `depth`, `kind`, `name`, `status`, `detail`, `tokens_used`, `started_at`, `finished_at`. Indexed on `(execution_id, attempt, id)`. +Deleting an execution (archive, clearing terminal tasks) deletes its steps. Existing rows gain no backfilled steps. Their history stays in `transitions` and `events`. diff --git a/docs/prds/external-agent-harness.md b/docs/prds/external-agent-harness.md index b9799ca3f..03211f14f 100644 --- a/docs/prds/external-agent-harness.md +++ b/docs/prds/external-agent-harness.md @@ -46,9 +46,10 @@ Order of precedence, first match wins: 2. the agent profile's `runner`; 3. `builtin`, the ai-sdk agent loop. -`runner` names a `[harnesses.]` entry. An entry with no consumer -(named by no stage and no profile) is a validation warning, and an unknown -name is a validation error. +`runner` names a `[harnesses.]` entry. `archie-agent` reads the entry +when it builds the stage's runner; that is the field's only consumer. An +unknown name and an entry named by no stage and no profile are both +validation errors. ```toml [harnesses.claude-sub] @@ -95,7 +96,8 @@ to the harness by `archie-agent mcp`, a stdio MCP server started inside the container and registered through the adapter's invocation flags. It serves the stage's `CaptureTools` and the central tools the agent profile allows. It does not serve the `workspace` toolset, for the same reason the built-in -runner withholds it. +runner withholds it. The runner's `ToolLimits` bound the results it returns. +The harness's own built-in tools are outside archie's limits. A harness that does not declare `mcp` can only run stages with no capture tools.