Clears the five open CodeQL alerts. harnessCredential parsed uid and gid
as 32-bit and os.Chown converted them to int, which wraps on a 32-bit
build; they are now parsed to 31 bits, so an out-of-range id is refused
where it is read. mergeMapping sized its map by len(base)+len(override);
it now hints len(base) and lets the map grow.
agent-context@1 was listed as supported, but a Kit's contributed guidance
never reached the agent. At container create archie now writes the
workload's profile file (CLAUDE.md, AGENTS.md, ...) beside the workspace
with a Kits index: a staged body is pointed at where the image holds it,
an inline one gets its own file the index points at. The profile never
inlines a Kit's text. The files are written as root, so a run can read
but not change them, and are rewritten on every create.
The workspace moves to /archie/workspace so the profile sits in a real
parent directory rather than /, which some CLIs stop short of when they
look upward for instruction files.
Part of archie-core-egkf.6.
A Kit's agent-skills@1 path now gets the shared skills store
(skills_dir/.agents/skills) bind-mounted read-only, whatever mode the Kit
asks for; with no skills_dir the host withholds the mount. skills_dir is
read live on every launch. skill.StoreDir exposes the store path so the
launcher does not copy the package's directory constant.
Part of archie-core-egkf.6.
kitrun ran the first Kit of a profile's list as the container image, so
every mixin's layers (its tools, skills and staged context) were dropped
without a refusal. A profile's `kit` is now one digest-pinned reference:
a workload Kit, or a published Kit set, which the Kit spec's frontend
merges from a workload and its mixins into one image. archie composes no
Kits itself, and a bare mixin is refused with a pointer to publishing a
set. kit.Compose had no production caller left and is removed.
BREAKING CHANGE: `[containers.profiles.X] kit` and the agent-profiles
document's `kit` are a string, not a list. A stored list fails to decode
and must be rewritten as the one workload or set reference.
Part of archie-core-egkf.6.
The egress proxy now holds a Kit's OAuth tokens on the host. A refresh
carrying the sentinel refresh token goes upstream with the org's stored
one; a successful response is written to the State Store and returned to
the container with sentinels only. Requests to the Kit's resource hosts
carrying the sentinel access token get the stored one.
The org's token set is read or written only for a run whose credential
carries the service, so a Kit declaring a service never reaches the org
secret by itself. The launcher grants a bound OAuth service without a
config secret, and Register refuses a required OAuth credential when no
harness secret store is wired. A provider that does not rotate refresh
tokens keeps the stored one.
Part of archie-core-egkf.7 (#1208). Credential-file rendering and
per-binding concurrency remain.
A worktree terminal cleanup keeps (uncaptured or unreadable work) now has a
reap path: archiving calls Daemon.RemoveWorktree for the owning identity's
trees, and the keep logs name archive as the way out. The no-change cleanup
test reports completed through one direct transition instead of reaching
merged via a faked pr_open hop.
Recover the OAuth storage slice from f6079e9d and the missing workflow persistence error handling from f07c314e, adapted to the current step contract. Require encryption for OAuth token storage and verify RPC round trips and task-grant denial.
Plan item 4 of docs/prds/execution-tree-state-machine.md (archie-core-1216),
plus the callee half of CancelExecution that plan item 5 deferred until the
tree could decide it.
Every agent call a stage makes records itself: AgentStage's bridge, the
baseline-fix auto-fix run, the adversarial reviewer and a workflow.call each
start a child StepExecution of kind agent or call under the stage's own step
-- TaskContext.StepID, which the engine sets before the stage body starts and
clears when it returns -- and finish it with the call's own outcome and the
tokens it accounted. A child's recording write failure parks the execution
the same way a stage's does: an agent call that ran unrecorded is the
invisibility the step table exists to end. A stage invoked outside the
engine's recorded run -- the stage unit tests -- records no child and runs
unchanged.
The call step is what makes the callee half of the PRD's Cancellation section
decidable, and this lands it: a call step closes when its callee ends, so an
open call step is a callee the caller waits on (wait:true) and a closed one
is a wait:false callee that runs on. CancelExecution reads the open call
steps before its sweep closes them and cancels each named callee through its
own CancelExecution in the caller's write's aftermath -- the callee's own
steps, its own events, its own move to the same destination, and the callee's
own wait:true callees beyond it, the depth the enqueue enforces bounding the
recursion. A callee that cannot be cancelled is reported and the caller's own
record stands.
StartStep refuses a call step whose callee is not this run's own (the
callee's call_parent_task_id must name the recording execution): a
compromised container could otherwise record a step naming a foreign
execution and cancel it. StartStepRequest gains called_execution_id (field 6;
non-breaking, the column and its named consumer already exist) and
StepStart carries it.
PRD decisions recorded here:
- The reviewer records as one agent child named "review": the Reviewer
contract runs one subagent per review, the PRD model's review:1/review:2
are illustrative of depth-1 children. A completed report closes it
succeeded, a not-run review failed with the skip reason; the contract
carries no usage, so tokens are 0.
- A failed-callee call step closes as failed with the callee's detail when
the wait ends otherwise; the single open case is the caller itself being
cancelled, whose CancelExecution already swept this step -- a late write
must not pretend the wait is still the caller's business.
- A call step's callee is cancelled to the same destination the caller's
action named (parked stays parked, closed_wont_do ends the whole chain),
and a callee that cannot be cancelled is reported while the caller's own
record stands either way.
- The agent child's detail is the result summary on success and the error's
text on failure, with the call's own TokensUsed -- the PRD's only usage
field, untracked per call for the reviewer contract.
docs/prds/execution-tree-state-machine.md: step_executions (0017) is now the
durable per-step record; the dashboard's run detail (GET /api/tasks/{id}/
attempts) still folded stage_start/stage_finish events into a rail by hand,
and tasks.stage duplicated what StartStep/FinishStep already record properly.
Added the missing read:
- ListSteps(execution_id, attempt) -- attempt 0 lists every attempt of the
execution, oldest first, so the rail renders every attempt from one call.
New storecontract.StepReader (separate from StepRecorder: this is a
dashboard/API/dispatch action per the PRD's wire contract, never
task-scoped-grant callable), postgresdb.ListStepExecutions, staterpc
client/server/values.go plumbing, wired through state_store.go's Deps and
archieui/compose.go's Server.Steps the same optional-capability pattern
every other dashboard read uses.
- internal/webui/api_tasks_attempts.go: stageViewsFromSteps replaces
foldStages/finishStage/stageOutcome. Attempt bounds, event_count and which
attempt numbers exist stay sourced from events (an attempt can carry
events -- agent calls -- with no step), only the stage list itself moves.
The wire JSON contract is unchanged, so no frontend change was needed;
`task ui` confirms ui/dist still matches a fresh build.
- A running step past the task's own current attempt reads interrupted, not
running: RecoverStale (archie-core, still open per #1214) is what corrects
a crashed run's stale "running" row, and until it does this is the honest
read rather than trusting a row that may be stale.
Dropped tasks.stage and every reader, no fallback:
- Migration 0018 drops the column; tasks.sql's INSERT/UPDATE/RETURNING lists
and the sqlc-generated code no longer carry it.
- workflow.Task.Stage is gone; workflow.Run's in-flight "which stage is this"
bookkeeping moves to TaskContext.Stage (execution-time only, never
persisted -- park/finish's event tagging is its only reader).
- pb.Task.stage (wire field 10) is `[deprecated = true]`, not deleted (never
reuse a field number); staterpc's taskProto/taskValue no longer map it.
- taskactions.Task.Stage and the retry event's "previous_stage" data field
are gone: the operator-action package had no reader for the column beyond
echoing it.
- The chat /tasks command's per-task "workflow/stage" display drops to
workflow-only (ChatTaskSummary.Stage removed, workflowStage() deleted):
it was a genuine reader with no replacement in scope for this change.
Read the archie-web-ui skill and the state-store/daemon docs/development
pages first, per instruction. Stayed out of cancel/stop code throughout
(Pi's concurrent CancelExecution work); the only shared files touched
(workflow.go, daemon-adjacent SQL) got the smallest possible diff.
Plan item 6 of docs/prds/execution-tree-state-machine.md (archie-core-1214),
the crash-recovery sweep the step table made decidable. RecoverStale used to be
one bare UPDATE of tasks running -> queued; the steps an interrupted execution
left running stayed running forever, so the next attempt started under steps
that were never closed.
Recovery is now one transaction: the running executions are locked, every step
each one's current attempt left running moves to the step transition table's
interrupted outcome with one stage_finish event row carrying the
interruption marker -- the same document stageOutcome reads, so the attempt
view renders the interruption the way the pre-step-table producer wrote it --
each execution's audit row lands in the same write, and the existing
running->queued requeue query runs last inside it. Both edges are pinned in
SQL and pinned to their tables (the execution pair beside claim and
BeginRemediation, the step pair in its own test, because the two tables are
different tables), so a query edit cannot move a write off a table.
Deliberately running-only, per the PRD's letter: pending rows never persist
outside StartStep's own transaction, so a crashed attempt leaves only running
rows, and a finished step is never rewritten. The interrupted steps' events
render as the PRD's interrupted -- distinct from cancelled, which an operator
or parent asked for, and from failed, which the step reported.
The wait:true-callee and agent-child sweeps ride #1216's tree, and the
dashboard's ListSteps read is plan item 7; both untouched here, and the
RecoverStale signature and RPC are unchanged.
Plan item 5 of docs/prds/execution-tree-state-machine.md (archie-core-1217).
Until now a stop was a context cancel plus a caller-side transition, and the
step rows a worker still held running could never be told: the execution moved
while the work carried on, and the old interrupt-first ordering existed only
because the record could not defend itself. The step table can: the PRD's
ordering -- the store is the record, the context cancel is the delivery --
became safe the moment a worker's next step write fails ErrStaleTransition
against a cancelled step.
- storecontract.ExecutionCanceller and the CancelExecution RPC: one
transaction moves every non-terminal StepExecution of the execution's
current attempt to cancelled (one event row each, in the same
transaction), and the execution itself to the status the operator action
names, under the shared transition table with its audit row. Staleness is
decided before legality, the way guardTransition decides it. Earlier
attempts are never rewritten. Deliberately administrative: the
deny-by-default grant rule covers it, pinned by a test.
- taskactions.Service routes stop, cancel, abandon and reject through it and
the old Transition/CancelTask combinations are deleted; the daemon's
in-memory cancel is demoted to the delivery hook the PRD names, called
after the store record.
- The gateway's chat controller records first and delivers second, the same
order, and records even when no runtime is wired here (a standalone
gateway cannot reach the daemon's goroutines): the record is what unsticks
a lost-cancel task, and the delivery is best effort. StopRunning and
TaskRuntime.CancelRunningTasks are deleted outright -- no production
caller remained once the task_action tool owned stopping, and the daemon's
CancelRunningTasks/runningTasks.stopAll go with them.
- The gateway's chat adapter writes through CancelExecution instead of a
bare Transition, so chat /cancel and the dashboard's action service land
on the same path.
PRD decisions recorded here:
- Each cancelled step's event row is stage_finish with {cancelled: true} and
the reason as its detail -- the exact-one-event-per-step-transition rule
covers the pending -> cancelled transition too, even for a step that never
ran.
- A parked cancel carries the needs_human class, matching the Transition
path this replaces; the transient class stays with parkRunningTask's
unwind parks, which now meet ErrStaleTransition for cancelled runs and are
ignored there.
- Cancelling a wait:true callee is deferred to #1216's agent-call children:
the callee's own tree and the wait flag are what make
"cancelling a caller cancels each callee it waits on" decidable, and the
callee writes it cannot fake are this issue's half of the contract.
- RecoverStale's interrupted sweep is plan item 6, untouched here.
docs/prds/external-agent-harness.md (Credentials) and orgs-and-access.md:
egress.Resolver had no production implementation, and no run carried a
credential naming the org secrets its identity is granted, so a Kit's
required credential@1 was refused at the proxy and no harness run
authenticated (archie-core-egkf.11).
Two things, as scoped (this does not attempt #1026's full run-credential
consolidation replacing WorktreeGrantIssuer/StateStoreGrantIssuer -- that
PRD item is real, separate, and still open; this is deliberately narrower):
1. A run credential naming granted secrets: config.CredentialBinding maps a
Kit credential@1 service to a secret; config.Config/IdentityConfig gain
Org and GrantedCredentials (a per-identity allowlist of service names,
reusing the existing per-identity config-override pattern DiffCapLines
already uses, not a new Cedar/Authorizer integration -- KindSecret policy
objects exist in internal/domain/access but assembling a full Principal
is #1026's job, not a two-line addition here). config.ContainerConfig.
BoundCredentials computes the declared (Kit) ∩ granted (identity) ∩
org-matched intersection Sam's own rule requires: neither side widens
the other, proven by a mutation-checked test flipping each of the three
independently.
2. egress.GrantResolver: the production Resolver, holding only what a
caller explicitly Grant-ed one run, replaced wholesale (not merged) on
a repeated Grant, gone entirely on RevokeGrant. kitrun.Launcher.Launch
computes the intersection per launch and resolves real values through
the existing secret registry (SecretResolver), Grants it before the
proxy session opens, and Release/error-path rollback (a new shared
Launcher.release) always revokes it.
Course-corrected mid-implementation (flagged by Sam): credential bindings
first landed as a boot-time config.ContainerConfig.Credentials field
captured once into Launcher.Bindings -- exactly the "parses but is not a
resource" trap CLAUDE.md warns about, and inconsistent with agent-profiles
(#1207) making a Kit profile apply without a restart. Rebuilt as its own
live control-plane resource, CredentialBindingsKind ("credential-bindings"),
following the agent-profiles/container-runtime-policies split precisely:
config.ContainerConfig.Credentials is json:"-", [containers].credentials
is only its seed, and kitrun.Launcher now holds *config.Holder (not a
captured slice) and reads Containers.Credentials fresh on every Launch.
Org/GrantedCredentials were already read fresh per dispatch (d.configFor
inside runKitTask, never cached), so no equivalent fix was needed there.
Deliberately not built: identities and grants as their own State Store
resource (orgs-and-access.md's "Org owns ... identities and their grants").
Identities are entirely config.toml-defined today with no control-plane
resource at all; moving GrantedCredentials alone into one would put half
an identity in two places. That migration is identities' own scope, not
credential bindings'.
Plan item 3 of docs/prds/execution-tree-state-machine.md (archie-core-1220), on
top of the transition tables and ErrIllegalTransition enforcement (1214/1215,
archie-core-qdbo.5):
- Migration 0017 adds step_executions -- id, org_id, workspace_id, execution_id,
attempt, parent_id, depth, kind, name, called_execution_id, status, detail,
tokens_used, started_at, finished_at, indexed on (execution_id, attempt, id).
Ownership fields are stamped from the execution's row; depth is derived from
the parent, never supplied by a caller. Existing rows gain no backfilled
steps.
- task.Store (the workflow engine's mid-run facade) gains StartStep/FinishStep
(5 methods, within the 8 cap), with storecontract.StepRecorder fronting them
on the server; deadlineStore wraps both with its per-call timeout.
- The store enforces each write as one transaction: the guarded row write
under the shared step transition table plus the transition's own
stage_start/stage_finish event row. Staleness is decided before legality,
the way guardTransition decides it, and the store returns the persisted
event for the caller's post-commit publish -- the EmitDurable convention
(the daemon's event sink skips already-persisted rows by assigned ID), which
is why workflow.Run stops emitting stage events beside the store and no
longer discards step-write errors: a failed start or finish parks the
execution, and the stage does not run unrecorded. An interrupted stage
records interrupted rather than failed; a recording failure on that
shutdown path is logged and left to RecoverStale, which owns the
interrupted-execution sweep in plan item 6.
- The run credential of docs/prds/orgs-and-access.md authorises StartStep and
FinishStep on its own execution only: both requests carry the execution ID
the grant check verifies (FinishStep carries it because the interceptor
never reads the database), and FinishStep verifies handler-side that the
step really is that execution's. authorizesTaskScopedCall's sanctioned
widenings are now held as a lookup table so the next one stops growing a
switch.
- The PRD's wire contract is StartStep(execution, attempt, parent, kind, name)
-> step id; the responses also carry the persisted event (needed for the
post-commit publish) and FinishStep's request carries execution_id (needed
for the grant check) -- both noted here as deliberate additions to the PRD's
shorthand. called_execution_id exists in the table but not yet on the wire:
its named consumer is the workflow-call recording of plan item 4 (#1216).
Conformance drives both adapters; grant tests pin the own-execution scoping;
the wire sentinel set gains ErrInvalidStep (InvalidArgument, "step execution
invalid") for a step kind no consumer names.
docs/prds/external-agent-harness.md ("Contract"): workflow definition
validation must reject a stage whose needs the selected harness cannot
meet -- capture tools without an adapter, or a gate retry without a
resume verb. The step vocabulary had no static way to declare either.
Added task.WorkflowInterface.Needs{Captures, GateRetries}, parsed and
validated (GateRetries >= 0) the same way Inputs/Repository already are.
Only a Kit profile is checked against it: an image profile's built-in
agent loop always supports both, so Needs constrains only the harness path.
Two enforcement points, chosen for where each fact is actually known
without a network fetch at save time:
- Captures vs adapter is knowable from config alone (profile.Adapter, the
agentexec.harnessAdapters table), so daemon.pinTaskProfile checks it
right where it already resolves the profile (validateProfileMeetsNeeds),
parking the task the same way an unconfigured profile already does.
- GateRetries vs the Kit's actual agent-sessions resume verb is only known
once the Kit composition is fetched, which kitrun.Launcher.Launch
already does -- so kit.ValidateNeeds runs there (new kitrun.Request.
GateRetries field, threaded from the workflow's parsed Needs in
daemon.runKitTask) rather than inventing an image-fetching workflow-save
validator.
Deliberately NOT built: "a Kit profile named by no workflow is a
validation error" (external-agent-harness.md "Selection"). Read literally
("a Kit profile [that is] named by no workflow"), this is a
cross-resource check -- an agent-profiles entry unreferenced by anything
in workflow-definitions -- and no control-plane Definition validator can
see another resource today (Definition.Validate is a pure []byte->error
function by design, matching every existing resource kind). Building
that crossing is a real, separate piece of infrastructure with its own
ordering questions (does defining a Kit profile before its workflow now
fail?), not a two-line addition, so it is left for a follow-on rather
than bolted on here.
Also fixes a test bug caught by rerunning with a short -timeout per
instruction: TestRunKitTaskThreadsGateRetriesToTheLauncher's fake NATS
subscriber never responded, so runKitTask's call into runViaAgent blocked
forever waiting for a reply bounded by a zero (unconfigured) wall-clock
budget. Fixed by responding, matching the sibling test's pattern.
The per-kind request-ID dedup was fixed in 9cd6a2aa for the SQLite store and
carried into the PostgreSQL rewrite (038b8035) from day one: ResourceByRequest
filters WHERE kind = $1 AND request_id = $2 and resource_history carries
UNIQUE(kind, request_id). What neither rewrite kept is the test that pins the
cross-kind half of the contract -- the deleted internal/store suite had one, and
the Postgres suite only covers same-kind replay.
Pin it, and prove the failure mode it guards against: with the kind filter
dropped from the query, a request ID reused across two kinds silently returns
the first kind's history row under the second kind's write -- wrong kind, wrong
value, no error (archie-core-1179's latent hazard). Restore the filter and both
kinds write their own revisions, with per-kind replay still idempotent.
Readiness probe test asserts each subsystem registers exactly once instead of
an ordered five-item list. Config descriptor test drops the exact-count check
and its stale settings.js rationale. Container defaults test references the
production constant.
Refs #1229#1231#1232
Every caller-supplied status write (Transition, ParkTask, Requeue,
RetryTask, BeginRemediation) now checks the shared taskstate transition
table inside the write's transaction, after the from guard decides
staleness. An off-table pair returns the new storecontract.ErrIllegalTransition,
carried over gRPC through mapError/unmapError on (FailedPrecondition,
"illegal transition"). Claim, BeginRemediation and RecoverStale pin
their pairs in SQL; a test pins those pairs to the table so a query
edit cannot move a write off it. Requeue/RetryTask share one guarded
requeue body. (archie-core-qdbo.5)
The execution transition table refuses most of the pairs shipped fixtures
wrote to build a finished task -- running -> merged, queued -> parked,
queued -> waiting_human, queued -> merged -- so those fixtures fail the
moment the store enforces it.
internal/taskstate/taskstatetest seeds a status by walking the table's
legal edges (shortest route, computed from taskstate.CanTransition), so a
fixture cannot write an illegal pair and fails loudly when no route
exists. The fixtures in internal/daemon, internal/app/archied,
internal/webui and internal/infrastructure/postgres go through it, and
the table's own test names running -> merged as refused.
The transition table moved the no-change outcomes to StatusCompleted, so a
run that opens no pull request now ends in a status the per-workflow stats
did not count at all: it was not merged, not in review, not parked, and the
workflows page would have shown it as a run that went nowhere. The count
travels the whole State Store path, since every dashboard consumer reads it
through the gRPC client rather than the local store.
- WorkflowStats filters `completed` beside merged and pr_open; the SQL is
the count, so a wrong filter reads as zero rather than failing, and
TestWorkflowStatsCountsCompletedRuns holds it against a real store.
- storecontract.WorkflowStat gains Completed (json "completed"), the proto
message gains field 9, and workflowStatProto/workflowStatValue carry it;
TestWorkflowStatProtoCarriesEveryField round-trips every field by name.
- The gateway's chat task report names the status.
- The dashboard counts a completed run as delivered in GatePulseCard's
totals and badge numerator, and the WorkflowStat/WorkflowStats/WorkflowRow
interfaces carry the field.
- TestOutcomeSitesAreTransitionsFromRunning is the guard that keeps the
producer and the table agreeing: it re-reads this package's tc.Outcome
assignment sites from source and requires each status to be one
internal/taskstate allows from running, so a new or moved site has to be
adjudicated against the table instead of discovered on a live run.
Assemble turns an admitted Kit composition and the workload image's
config into the harness spec (the image's user, never root; the image
env plus the workspace, egress and credential sentinels; the launch
argv and agent-sessions verbs), lifecycle hooks with their declared users
and restricted environments (the image-derived baseline, egress settings,
and only the runtime variables each hook declares), the Kit's files, and
volume names stable per execution.
A Kit task's worker shares the sandbox network with its harness, so it
reaches NATS and the State Store through the relay like the harness
reaches the proxy. archie-agent relay takes repeated -forward
listen=target pairs, binds them all before serving so a port conflict
fails at start, and stops every forward when one fails. RelayArgs builds
the command from the fixed relay ports.
A session carries its run and its Kit's credential@1 requests. The proxy
applies apiKey inject rules only in the credential's phase and only to
hosts its rule names, after the network policy has admitted the request,
so a key never reaches a host its Kit did not declare. The real value
comes from a Resolver that returns only secrets the run credential
carries: a required credential that is unbound or unresolvable stops the
request before anything is sent, and an optional one passes only when it
is genuinely unbound. Named keys appear in the sandbox as a sentinel;
inject-only keys have no presence.
Each sandbox gets an internal bridge on which the host holds no address,
so host services listening on every interface are unreachable; the relay
is attached under a fixed alias and is the one way out. ProxyEnv gives a
sandbox container its proxy URL and CA trust. The Docker integration test
proves allowed egress works and that the proxy listener, host services and
the internet are unreachable directly; it fails if the host-address
inhibition is removed.
archie-agent relay listens on the sandbox's isolated network and joins
every connection to one fixed target, the proxy. The relay holds no
secrets and can reach nothing else, so the host needs no address on a
sandbox network at all.
CONNECT tunnels and absolute-form http requests authenticate to a
per-container session, are checked against the session's phase policy,
and are forwarded with verified upstream TLS. A tunnel serves only its
admitted host: mismatched SNI and Host headers are refused. Upstream dials
refuse loopback, and refuse internal addresses unless the policy names the
exact IP, so an allowed name cannot resolve back into the host.
Decodes published descriptors from the manifest annotation, composes one
workload with its mixins through the Kit spec package, and applies the
runtime's support table: unsupported required capabilities refuse the
launch, optional ones are skipped and recorded, and forge or passthrough
credentials are refused. A composition must carry a headless prompt verb.
Every install serves from Postgres, so the legacyread/legacyimport packages,
the archie-state-store import command, the boot gate that refused to serve
over unimported SQLite files, and db_path (which only located those files)
go. Migration 0010 drops the import_completion table.
ListCaptures and ListUndispatchedCaptures had no callers once the client
moved to the streaming RPCs; they were kept only for mixed-version rollouts.
Binding.secret was unused since signing moved to Source; its number and
name are reserved.
Each credential is now only a {engine, key} secret reference. A stale
token_env in a file is reported at startup as an unrecognised key.
nats.token_env stays (no secret-ref form), and providers.api_key_env stays:
it is the carrier the resolver writes each resolved key's env name into.