Commit Graph
27 Commits
Author SHA1 Message Date
sam 862ac46e8b feat(tasks): attribute events, changes and config to the attempt that produced them
The events table had no attempt column, so two attempts of the same task were
indistinguishable: a retried task's stage events interleaved into one timeline and
nothing could say which run produced a given result.

- events gains an `attempt` column through the existing presence-gated
  migrateTasks loop rather than a version bump, so user_version stays 3 and an
  existing archie.db is migrated in place. A legacy-database test builds the
  pre-change schema and proves both that existing rows survive unattributed
  (attempt = 0) and that a new value round-trips.
- Event.attempt = 11 and ReadTaskLogRequest.stage = 9 are field additions to
  existing messages: no new RPC, no new table, and no change to any canonical error
  message that crosses the state-store wire.
- Every task event the workflow engine emits now carries the attempt. Producers
  that have no attempt yet (task_queued, binding_dispatch_failure) stay
  unattributed rather than guessing one.
- Changed files are captured as a durable changes_captured event at commit and push
  time, reusing the existing three-dot diff primitive: paths from FilePatches(),
  counts from Stats(), base_sha recorded as the merge base actually diffed rather
  than the base tip, a 200-file cap with totals kept complete. Consumption goes
  through an unexported optional interface so workflow.Trees -- the yaegi capability
  surface for interpreted stage plugins -- is not widened. A capture is also taken
  once the pull request exists, so the capture can link it.
- The per-attempt configuration is captured as a durable config_captured event
  carrying the non-secret config.TaskConfig, instead of a new table plus two new
  state-store RPCs: events are already this system's per-attempt provenance stream,
  so a table would have given one feature two provenance mechanisms. Redaction is
  proven against canary secrets. Measured payload: 539 bytes typical, 2333 bytes at
  20 models.
- Captures carry a schema marker and are refused unless their key set matches the
  declared payload shape, so drift surfaces as "recorded but could not be read"
  rather than as a capture of zeroes.
- Three read endpoints serve the attempt rail, the changed-file captures and a raw
  debug view, mapping to UI-owned response types rather than dumping store types.
- The task-log read gains a stage filter, bound to the stage logger for that
  stage's own lines; agent and tool output records no stage.

A task-scoped grant may already insert events on its own task, so this provenance
is exactly as trustworthy as the rest of that task's event stream. That limit is
documented rather than papered over.
2026-09-18 18:32:59 +10:00
sam 537ba9a3fb fix(agentworker): hand the worktree back on every sandbox exit
A container run only restored ownership of the bind-mounted worktree on the
CommitAll/Push success path (archie-core#520). The agent runs as root inside
the container, so every other way a run can end -- a gate failure that parks
the task, a stage error, an early return -- left the worktree owned by UID 0
on the host. The daemon is the process that later cleans and resets that
directory as its own non-root user, and it cannot even unlink inside a
root-owned directory, so the next attempt failed before it could start no
matter what the worktree refresh did.

runTask now defers the same reconcileOwnership the commit and push paths
already call, so it runs at every exit. The retry recoveries in worktree
package cannot fix this from the daemon side: a non-root process cannot
chown, and removing the directory needs write access to it.

Observed through a supplementary group -- a non-root process may move a file
it owns into a group it is a member of -- so the test proves the chown ran
without needing root.

Residual, filed as archie-core-zcg8: a container that is SIGKILLed never
runs this defer.
2026-09-18 01:49:35 +10:00
sam 0b3dc6c01f refactor(forge): delete the unwired RepliesAfter surface
DELETE: IssueForge.RepliesAfter and its Reply type had no production caller --
the waiting_human status exists and is reachable, but automatic detection of a
human reply never landed, so the intended poller was never built. Neither
implementation paginated either (both read one page of 50), so the method
would have been wrong if wired as-is.

The manual path is untouched: waiting_human remains reachable and an operator
requeues through the existing task-action flow. Removing the interface method
also removes seven test fakes that existed only to satisfy it.

No behavioural test accompanies this: a pure deletion has no assertion to
make fail, and the compiler enforces completeness across every implementation.
2026-09-15 16:54:16 +10:00
sam d7b809e61b fix(agentworker): wire configured tool-result policy into worker's LoopRunner
TaskConfig now carries the daemon's ToolPolicy (MaxResultChars/SpillDir)
via ForTask/ToConfig, and runTask applies it to the constructed
LoopRunner. Previously the worker always built the agent loop with a
zero-value ToolLimits, so tool results in every task's agent stage were
never capped or spilled despite the configured 50k default (agentexec-1,
tools-core-11).
2026-09-15 10:14:50 +10:00
sam 669f98a059 fix(daemon): carry kind/label routing bindings to the process that routes
An operator's kind/label -> workflow overrides were decoded, validated and
installed into the DAEMON's package-level state via
workflow.SetKindWorkflows/SetLabelWorkflows -- but workflow.Route's only
production caller is app/agentworker/task_execution.go, in the separate
archie-agent process. The bindings never crossed the boundary, so every
configured override was silently ignored and routing always fell back to the
built-in defaults (bug -> tdd, feature -> feasibility, bootstrap -> bootstrap).

Carry the resolved bindings on taskrun.Request, mirroring how MCPServers already
crosses this boundary, and apply them in the worker before Route is called.
Resolved values are carried rather than file paths, so the worker does not need
the routing files to be readable in its sandbox. nil still means "use the
built-in defaults", so this is backward compatible for any request built by an
older daemon.

routeTask is a named seam for that install-then-route step. It mutates
package-level routing state, which is sound here because each archie-agent
process is task-scoped and ephemeral; if the worker ever served more than one
task that assumption would need revisiting.

Tests are mutation-verified: removing either the two Set calls in routeTask or
the two request fields in runViaAgent reproduces the exact assertion failures
the lane observed red-first, so neither test is vacuous.

Found by an adversarial package sweep (temp tracker: workflow-engine-1,
.local/issue-tracker/).
2026-09-15 02:13:33 +10:00
sam bdf73ed371 feat(workflow): review calibration -- checked properties, style nits out of contract
Closes archie-core-h019.5. The shipped contract already made only a confirmed
error-level finding block, and kept zero-findings distinct from not-run; what
was missing was the other half of calibration, and that is what a reviewer that
cries wolf needs.

- ReviewReport gains Checked []ReviewCheck {Property, Evidence}: the properties
  the reviewer verified clean. Evidence is required, for the same reason a
  finding requires a failure scenario -- a check without evidence is an
  assertion, not a verification.
- The reviewer records checks through a typed record_checked tool, mirroring
  record_finding, so it cannot end its turn having silently skipped the
  checklist.
- Style/formatting/naming nits are stated out of contract, in the ReviewFinding
  type doc and the reviewer prompt: they are the padding that trains an
  operator to skim.
- The PR-body findings section renders the cleared properties, so a
  zero-finding review arrives backed by what was checked rather than reading
  the same as a review that never looked.

Design: docs/prds/review-calibration.md.
2026-09-11 20:18:06 +10:00
sam ad1e6ea0d3 feat(gateway): operator-triggered PR review with selectable reviewer model
Add a channel-neutral review_pr command that runs the adversarial reviewer
against an existing pull request and returns structured findings. The Web UI
chat and Telegram invoke the same command; the reviewer model is models.reviewer
(falling back to models.builder), configurable independently of planner/builder.

- forge: PullRequest + PullRequestReader.GetPullRequest (github, gitea)
- worktree: Manager.CheckoutPR materialises a PR head for Diff/Snapshot
- agentworker: NewOperatorReviewer + ReviewerModel (single model-resolution rule)
- gateway: review_pr tool, ChatPRReviewer, wired into TurnRunner for every channel
- archied: prReviewer adapter with identity-scoped repo allow-list and
  per-PR single-flight, so duplicate/concurrent reviews are deterministic

Closes archie-core-h019.8.
2026-09-10 01:09:31 +10:00
sam 05529d48d5 refactor(state-store): cut daemon/Gateway over to the remote State Store (archie-core-8cda.4.6) (#774)
* refactor(state-store): delete NATS storerpc; agent store is gRPC-only (archie-core-8cda.4.6)

The agent is fully on the gRPC State Store contract, so the legacy NATS
storerpc path is dead (docs/prds/state-store-contract.md §12 step 4):
- delete internal/storerpc (Server, Client, subjects) and its tests
- drop the storerpc registration from registerTaskRPCServers and the
  now-unused store param
- make the agent transport's State Store URL required and Store() return
  only the gRPC staterpc.Client (no NATS fallback)
- rework agentworker/agent transport tests to back the store with a real
  gRPC State Store server instead of the deleted NATS storerpc server

The agent binary now fails fast when STATE_STORE_URL is absent. Agent store
calls (workflow.Store) go over gRPC to the archie-state-store service.

* refactor(state-store): delete daemon/gateway in-process store ownership (archie-core-8cda.4.6)

The daemon and gateway no longer open archie.db in-process: the single SQLite
file is owned by the standalone archie-state-store process, and both consumers
dial its gRPC State StateStore contract (docs/prds/state-store-contract.md
§12 step 7, no dual-store ownership).

- openStores no longer opens the production task store or seeds b.st; it only
  wires secrets/forge and the gateway conversation store
- openStateStoreAdapter now requires [services.state].target (the old local
  default is deleted) and resolves b.stateStore as the remote *staterpc.Client
- delete the in-process startStateStoreServer (the multiplexed gRPC server and
  its per-task bearer-token registry) from the daemon composition
- RunGateway now resolves the same remote State Store adapter; its session
  SQLite stays Gateway-owned and untouched
- daemon.ConnectedStateStore carries the configured target plus the resolved
  bearer token, which containerEnv injects as STATE_STORE_URL / STATE_STORE_TOKEN
  so archie-agent authenticates to the same remote service
- readiness store probe reads b.stateStore

PRD-consistent choice (noted): with the in-process server gone, per-task token
generation is removed; the daemon/gateway/agent all present the single
[services.state].target_token (or STATE_STORE_TOKEN secret) that the standalone
store validates.

* docs(webui): require [services.state].target in config and deployment templates (archie-core-8cda.4.6)

After the in-process store path is deleted, archied and archie-gateway dial the
standalone archie-state-store gRPC service via [services.state].target and never
open archie.db themselves (docs/prds/state-store-contract.md §12 step 7).

- config.example.toml documents [services.state].target as required, with the
  loopback vs Docker-bridge gateway + target_token rule
- deployment templates set [services.state] per topology: bridge gateway +
  token for container-mode agents, loopback for host-only
- deployments/README.md and systemd-user-service.md describe start commands
  for both topologies and drop the stale "not yet consumed by the daemon" note

* style: gofumpt alignment for State Store daemon literal; appease noctx/staticcheck (archie-core-8cda.4.6)

* docs(state-store): refresh stale in-process references post-cutover (archie-core-8cda.4.6)
2026-09-07 05:06:12 +10:00
sam 576e22b554 feat(state-store): generalize storerpc as the State Store gRPC contract (archie-core-8cda.4.2) (#770)
* feat(state-store): generalize storerpc as the State Store gRPC contract

Implements archie-core-8cda.4.2 against docs/prds/state-store-contract.md
rev. 2c: generate protobuf/gRPC client and server adapters for every
ratified store contract (proto/state/v1/state.proto, 40 RPCs, one
StateStoreService), add internal/infrastructure/staterpc with a bufconn
conformance suite exercising both local and remote adapters, serve the
service in-process (bridge address + mandatory per-task bearer token when
agent containers exist, loopback-only otherwise), and flip archie-agent's
workflow.Store to a long-lived gRPC client when the daemon injects
STATE_STORE_URL/STATE_STORE_TOKEN (falling back to the legacy NATS
storerpc path otherwise -- one agent process is never both).

Also drops the *sql.Tx parameter from store.BindingDispatcher.RecordDispatch
(a required wire-blocker fix: a transaction cannot cross a gRPC boundary,
and production always passed nil).

fix(store): drop RecordDispatch's unusable *sql.Tx parameter
feat(config): add services.state contract-adapter selector

* fix(state-store): apply no-mistakes review decisions

- Drop StreamTokenInterceptor (unused; the State Store service has no
  streaming RPCs and nothing calls it).
- Make StateStoreTokens.Validate compare tokens in constant time
  (crypto/subtle) per the auto-fix hardening.ing finding.
- Amend config.example.toml / config.Services.State doc text so it frames
  [services.state].target as the ratified surface consumed by the upcoming
  standalone archie-state-store binary (.4.3), not active today.
2026-09-07 03:01:33 +10:00
sam 5c0efaceaf refactor(workflow): relocate Task/Status/Source and define workflow.Store (#754)
* refactor(workflow,store): relocate Task/Status/Source and define workflow.Store

Move store.Task, store.Status*/store.Source*, and the 3-method workflow
store interface out of internal/store into internal/domain/workflow as
workflow.Task/workflow.Status/workflow.Source plus a consumer-owned
workflow.Store, superseding store.WorkflowStore. internal/domain/workflow
now imports zero internal/store symbols; internal/store persists the
domain type instead (infrastructure depends on domain), and its Store
satisfies workflow.Store.

Updates every production and test caller of the relocated symbols across
daemon, webui, gateway, agentworker, agenttransport/nats, storerpc,
taskrun, and worktreerpc, and removes TestRecordDispatchViaExplicitTx per
docs/prds/state-store-contract.md (rev. 2c) sections 4 and 12 step 2.

Two workflow-package integration tests that exercise the real *store.Store
move to an external workflow_test package (store_integration_test.go) to
avoid a workflow(test)->store->workflow(prod) import cycle now that store
depends on workflow.

* no-mistakes(document): docs(state-store): mark §4.8 relocation done in state-store-contract
2026-09-07 00:39:26 +10:00
sam c3e9e4c43e feat(workflow): adversarial self-review stage before PR open
Implements archie-core-h019.3 (reviewer isolation) and h019.4 (wiring),
per docs/prds/adversarial-self-review.md.

- internal/domain/workflow: new Reviewer interface and StageReview,
  inserted between StageDiffCap and StageOpenPR in the implement
  workflow. Fail-closed: a review that didn't run, or a confirmed
  error-level finding, parks the task (store.StatusParked) with
  findings in Detail; warn/plausible findings never block. Inert
  unless Repo.ReviewEnabled.
- internal/worktree: Manager.Snapshot exports HEAD's tracked files into
  a fresh directory with no .git, for the reviewer's isolated
  workspace. Trees interface gains Snapshot; hybridTrees delegates to
  it.
- internal/app/agentworker: subagentReviewer implements Reviewer using
  ai-sdk's agent.Subagent (nested, in-process, non-streaming
  generation) -- conversation isolation is structural (Subagent.Run
  never carries message history), workspace isolation is the snapshot
  above. The reviewer's toolset is read-only (read/grep/find) rooted
  at the snapshot, never the worker's own toolset, so it cannot reach
  the worktreerpc publication grant. Findings are captured via a typed
  record_finding tool appending to a slice the stage owns, read even
  when Subagent.Run errors (a truncated run's captured findings
  survive).
- internal/config: Repo.ReviewEnabled (default false) and
  Models["reviewer"] (falls back to Models["builder"], no schema
  change).

Judgment calls: Reviewer.Review never returns a Go error -- every
failure folds into ReviewReport per the PRD's fail-closed contract,
which the stage can act on uniformly. A Subagent.Run error with zero
captured findings resolves to ReviewStatusNotRun (park); a
Subagent.Run error with >=1 captured findings resolves to
ReviewStatusCompleted with those findings, since the PRD's own example
("three defects survive a step-limit truncation") requires them to
still be evaluated for blocking rather than discarded.
2026-08-31 21:51:33 +10:00
sam a825db283c fix: harden task and chat execution
Task runs now preserve real model context limits, provider usage, baseline-repair spend, commit identity, durable agent-finish telemetry, and useful timeline details without imposing a token cap. Dashboard forge links are canonical per owning GitHub or Gitea identity. Chat turn claims acquire SQLite writer intent before reading, eliminating the concurrent SQLITE_BUSY snapshot race.
2026-08-25 05:48:17 +10:00
sam 694a977ffe fix(agentworker): reconcile task clone ownership after commits 2026-08-25 03:28:40 +10:00
sam 5d00ec7b0c refactor(agentexec): remove legacy execution paths
Autonomous workflows now have exactly one production boundary: a task-scoped archie-agent receives a full-task core-NATS request and runs the worker-local stage loop. The legacy [agent] section remains decode-only for migration; it no longer selects runtime behavior.
2026-08-24 14:56:57 +10:00
sam 73a83a133b chore: apply gofumpt formatting 2026-08-23 15:49:40 +00:00
sam ed33ab5380 feat(agent): stamp archie-agent's build and report it back to the daemon
archied's own build has carried a compiled-in gatewayVersion and an
internal/installtype stamp since archie-core-522/b5d08d0, but
archie-agent's build had neither -- its Taskfile.yml and Dockerfile
build steps passed no -ldflags at all, so even a daemon that wanted to
verify what agent build is actually in service had nothing to check
against. daemonRunningVersions said as much explicitly: the agent was
left out of every verification on purpose, because nothing self-reported
a real value.

Every archie-agent process is task-scoped and ephemeral (spawned per
task, dies after), not a long-lived process the daemon can poll on
demand the way it polls Telegram's gateway. So there's no "ask the
agent its version" request -- the only channel back is whatever a
worker includes in the taskrun.Response it already sends on every
completed task.

Stamp archie-agent's build the same way archied's is (installtype,
plus a new agentworker.Version()), have executeTaskRequest report both
in its Response, and have the daemon record the latest observation in
a new mutex-guarded AgentStatus, fed from runViaAgent on every
successful response only -- never off a request that never reached an
agent, or one describing a failed/parked task. daemonRunningVersions
now includes the agent component once (and only once) something has
actually been observed, so Report.Verify gets a real value to check
future update claims against instead of omitting the component
entirely.

Part 1/3 of archie-core-1786751948782-3-837a8fd0 point 4. Closes
archie-core-1786751948782-3-837a8fd0.1.
2026-08-23 23:18:00 +10:00
sam 537651fb4c Merge pull request #562 from samcharles93/fix-agentworker-json-import
fix(agentworker): move JSONL decode into storage, unblocking CI
2026-08-23 03:00:32 +10:00
sam ab5a60c1b8 fix(agentworker): move JSONL decode into storage, unblocking CI
TestApplicationOwnsNoTransportWireMechanics was failing on main since
d7801bb: persistent_runner.go imported encoding/json to decode its
own project-memory JSONL log, which the architecture test treats as
a transport-wire-mechanics violation. That ban exists to keep NATS
message encoding out of the app layer -- this was local disk I/O, not
wire traffic, but the test can't tell the difference from an import
alone, and it shouldn't have to: the fix is to not decode JSON in
agentworker at all.

storage.ReadJSONLines is AppendJSONLine's read-side counterpart,
added to the package that already owns this format (see
organisation.md, "a package owns its own format end to end").
persistent_runner.go now calls it instead of scanning and decoding
itself, and no longer imports encoding/json or bufio.

This was blocking every deploy.yml run on main, independent of and
found while shipping the video-generation PR (#561).
2026-08-23 02:57:41 +10:00
sam dbb6550ab3 fix(worktree): bind publication to dispatch grants
Worktree RPC used to accept repository coordinates from the sandbox. It now accepts only an opaque per-dispatch grant, and the daemon derives the repository path and branch from grant-owned state.

Repository worktree paths also move from an ambiguous flattened key to encoded owner/repo components. Origin-matched legacy clones migrate safely; mismatched legacy paths are preserved.
2026-08-22 22:06:54 +10:00
sam 99738845b6 fix(container): persist npm cache for per-task MCP servers
Ephemeral task containers re-resolve and re-download npx-launched MCP
server packages from the npm registry on every task, since each task
gets a fresh container with no persistent cache -- a worse amplifier
of a registry outage than the daemon-restart case d4bb914b already
fixed.

Extract the daemon's npm-cache env logic into mcp.NpmCacheEnv, shared
by both the daemon's own long-lived npx MCP servers and archie-agent's
per-task ones. Mount a new archie-cache-mcp-npm Docker volume at
/data/cache/mcp-npm on every task container regardless of ecosystem
(MCP servers are daemon-wide config, not tied to the task repo's
language), and point npx-launched per-task stdio transports at it.

Closes archie-core-qk8m.
2026-08-22 21:13:33 +10:00
sam d7801bb7d3 feat(storage): persist agent project state 2026-08-22 21:00:20 +10:00
sam 5da1d5d006 fix(agentworker,daemon): reconcile worktree ownership before push
Agent containers run as root (no USER in the Dockerfile, no
userns-remap), so a commit the agent writes to the bind-mounted
worktree lands owned by UID 0 on the host. The daemon then reads
those same loose objects to push, running as its own non-root host
user, and fails with "openat objects/../..: permission denied".

containerEnv() now passes the daemon's own UID/GID as
WORKTREE_UID/WORKTREE_GID. hybridTrees.Push -- the single point
where a commit's objects are handed back for the actual push --
chowns the worktree to that owner first, via os.Lchown so a
legitimately-tracked-but-dangling symlink doesn't abort a push that
would otherwise have succeeded.

Closes archie-core-1786749700606-6-bf6aa32d.
2026-08-18 09:05:31 +10:00
sam 16ec8393fe style: apply gofumpt formatting 2026-08-17 00:02:31 +10:00
sam 20bff1a367 feat(agentexec,workflow): surface tool calls on the task timeline
archie-core-467 wired tool-call visibility into the interactive chat
gateway, but a workflow task's own transcript never showed what its
tools actually did -- only stage/agent-finish events existed, so
there was no way to confirm an agent's summary matched what it really
ran.

Adds a ToolCallReporter, invoked once per completed call from
toolExecute (the single choke point every tool call passes through
before reaching a model), reported via a new ToolSetOptions.OnToolCall
hook. Runner.Run grows a report parameter to carry it in: InProcessRunner
wires it through, since it executes in the same process as its tools;
SubprocessRunner and NATSRunner accept and ignore it, since their tool
calls happen in a different process with nothing here to observe them.

The workflow engine turns each report into a tool_call event on the
task's own Bus (agent.go, implement.go), riding the same NATS
event-forwarding archie-core-518 already built to get container
events onto the daemon's timeline.

Scoped to the central tool registry (read/write/edit/grep/shell/...)
built via BuildToolSet; capture tools (structured finish output) and
run_go_script build their ToolSet entries directly and are not yet
covered.
2026-08-15 12:53:41 +10:00
sam a237a35532 fix(agentworker): back off before refetching a nak'd stage message
On a stage-handle failure, the main loop logged the error, called
Nak (which requests redelivery), and looped straight back into
FetchStage with no delay -- unlike the fetch-error branch three lines
above, which already backs off for a second. A deterministic failure
(a broken gate, a malformed request) therefore refetched and failed
again immediately, flooding the log with one "handle failed" line per
loop iteration instead of per genuine retry -- hundreds of identical
lines within the same second in production.

Adds the same time.Second backoff after Nak.
2026-08-15 12:53:31 +10:00
sam d9a9b17153 fix(agentworker): forward workflow events across the NATS worker boundary
A workflow run through the container/NATS path never wired
workflow.TaskContext.Bus, so every tc.Emit call (stage_start,
stage_finish, outcome, parked) was a silent no-op. archied's own
in-process path sets Bus and always worked; anything dispatched to an
archie-agent worker did not, which meant the dashboard's per-task
timeline showed "No events yet" for every task regardless of status
or stage -- confirmed live across a Working task and two differently
parked tasks.

Bridges the gap the same way system logs already cross this boundary:
ForwardTaskEvents relays a per-task in-process *events.Bus to a new
NATS subject (SubjectForEvents), and a new daemon-side subscriber
(subscribeAgentEvents) republishes decoded events on the daemon's own
Bus, landing them in the same persist-and-broadcast pipeline an
in-process workflow already used.

Closes #518
2026-08-15 10:34:25 +10:00
sam 45345cdfb6 refactor(agent): move worker runtime into application layer
cmd/archie-agent used to own worker construction, NATS lifecycle, task execution, MCP setup, and workflow wiring. It now owns process input only and invokes internal/app/agentworker, while transport-specific NATS, boot-brief, and git-safety mechanics live under infrastructure.
2026-08-15 00:43:12 +10:00