Commit Graph
236 Commits
Author SHA1 Message Date
sam 082fbd416b feat(logging): stamp trace and span ids on log records 2026-10-25 22:55:57 +11:00
sam a24eadee8e refactor(controlplane): move the store-backed server out of internal/app 2026-10-25 22:49:27 +11:00
sam e99c49cee6 fix(archieui): reach the sandbox and setup terminal on the daemon's listener 2026-10-24 11:24:49 +11:00
sam 98e7378f2d refactor(servicekit): stop linking the State Store and control plane into archie-messaging 2026-10-17 17:22:53 +11:00
sam aa012a7c6e feat(identity): add browser setup and user accounts 2026-10-16 02:41:24 +11:00
sam cf3455ef2e feat(rpc): mTLS between services from operator-issued certificates
Replace the service bearer tokens with [tls] ca_file and cert_dir. Caller
identity comes from the verified certificate, not metadata. Without TLS,
services listen and dial on loopback only. Task containers get only the CA
and keep their grant.
2026-10-16 02:37:51 +11:00
sam 417d379ede feat(releaseupdate): transactional self-update through archied (#1633)
archied update install verifies the release zip, checks every binary
reports the release, pulls the agent image by digest, backs up the
database in-process and stages the binaries before handing off to
archied update watchdog in its own process. The watchdog restarts units
in release order, requires archied healthy for 120s and otherwise rolls
back binaries and image policy, restoring the database only when a
migration ran. Boot recovery resumes the frozen watchdog.

Deletes scripts/archie-update-install, scripts/archie-update-watchdog,
the updater's --from-source and --auto, CommandInstaller,
ScriptInstaller and the update-image and update-topology subcommands.
2026-10-12 03:21:02 +11:00
sam a8706941ed fix(prreview): restore the benchmark runner on configured models and public PRs 2026-10-11 04:11:40 +11:00
sam df554b3056 fix(daemon): keep model provider keys out of agent containers via the egress proxy 2026-10-10 17:12:09 +11:00
sam 6110693aa8 fix(harness): keep the setup container alive with archie's own binary
A setup session started its container with "sleep infinity" and opened the
PTY with "sh -l". Neither binary is guaranteed: the spec capability whose
floor is a shell and bash is com.docker.sandbox/sbx@1, and archie's
supported map does not list it, so a Kit image need not provide either. A
missing sleep killed the container at start and a missing sh failed the
exec, both as an opaque error.

The container now runs archie-agent hold as its entrypoint. The agent
binary is archie's own and already mounted into every Kit, so the
keepalive needs nothing the image may lack and the container env stays
empty. Before opening the PTY, harnesssession probes for bash then sh and
returns ErrNoShell naming the Kit reference when neither is present.

Test: TestPickShell covers bash-preferred, sh-only, neither, and a probe
that cannot run.
2026-10-09 13:05:05 +11:00
sam 3777779acc test(archieui): restore the archie-ui import-boundary check
Only the daemon owns containers, and the dashboard consumes a harness
session over its gRPC contract. Go imports are atomic, so a single
reference relinks a runtime, and the boundary holds only while something
re-measures it. cmd/archie-ui/architecture_test.go enforced this until it
was purged with the test suite; nothing enforced it since.

The restored check runs go list -deps . and fails if the dashboard links
internal/container, internal/app/archied or internal/agentexec. It is
transitive, which is what "links" means, and runs in the existing test
step.
2026-10-09 11:42:39 +11:00
sam bba5c9d7de feat(harness): serve a daemon-side setup session to the dashboard
The dashboard's setup terminal was a 503: it had a WebSocket route, an
authorizer and an xterm.js surface, but nothing owned a container. The
daemon owns containers, so the session contract lands there and the
dashboard consumes it over a new gRPC contract.

One session concept, two targets. A setup session is addressed by Kit
profile and is ephemeral; a run session (archie-core-qi91) is addressed by
task and attempt and is bounded by the task. This slice serves setup and
answers Unimplemented for run.

The contract, the authorisation, the lifetime and the egress phase are
settled in docs/prds/external-agent-harness.md, "Session contract". The
short version: the daemon serves harness.v1.HarnessService; the dashboard
authorizes the operator as an update on the secret and then dials the
daemon with the harness service token, required even on loopback because a
session is a shell holding an org's credential; the setup container runs
no worker, mounts no worktree, holds a keepalive, and carries an empty
container environment. Install hooks run at the install phase and the
session enters runtime before the shell, so a runtime-phase credential's
token endpoint is the one the proxy intercepts.

Capture is gated by the egress proxy on a granted service, and a setup
session has no task, so the daemon registers a session grant: the
session's proxy token resolves to the org's bindings for the Kit's
declared services. The real token never enters the container: the token
endpoint response is sentinelized and stored as the org's harness secret.

Every teardown path releases the pool slot, revokes the egress session and
the grant, removes the sandbox network, and removes the session's volumes.
The listener requires HARNESS_TOKEN and stops with a 10s grace before
Stop, so an open session cannot hold shutdown to the watchdog leash.
2026-10-09 01:38:41 +11:00
sam 6d75bb2bc1 feat(releaseupdate): archied update check finds releases through go-selfupdate; archie-update-check is deleted 2026-10-07 01:12:11 +11:00
sam efa1e5cf85 feat(messaging): serve MessagingService.Deliver and deliver chat schedules through it 2026-10-05 18:01:33 +11:00
sam 27ac50113e feat(releaseupdate): install from release artifacts with journaled rollback, topology check and channels 2026-10-05 17:02:29 +11:00
sam b24186ba9a feat(presence): services card on the status page and archied status 2026-10-04 03:29:05 +11:00
sam d2ae235628 refactor(archieplaybooks): fold archie-playbooks into archied playbooks lint|serve 2026-10-04 01:20:33 +10:00
sam fa58050281 refactor(archiegateway): own package for the Gateway service
archie-gateway no longer links the daemon, the web UI or archied. Shared
runtime layering, watches, catalog, NATS discovery and the State Store client
live in servicekit; chat task profiles in chattask. The Gateway waits for the
daemon's NATS endpoint instead of starting a second broker, and chat
approvals go through the task-action client instead of a local service with
no daemon behind it. Version stamping uses buildinfo only.
2026-10-04 00:59:02 +10:00
sam 1400f204ae refactor(statestore): own package for the State Store service; access reset moves to archie-state-store 2026-10-04 00:46:11 +10:00
sam 3c338a9722 refactor: strip doc and issue narration from code comments
Remove PRD/doc paths, bead IDs, phase notes and deleted-test references
from comments; drop test-only hooks left by the test deletion. Includes
in-progress archied bootstrap, gateway and query changes.
2026-10-03 21:47:46 +10:00
sam a45300d626 refactor: testing from the beginning. a journey in 150k lines of tests to none 2026-10-03 21:16:25 +10:00
sam 958c20c473 test(messaging): widen the config-read gate past explicit config.Config types
The gate that keeps a PRD-forbidden [models]/[providers] read out of the
Messaging Service only blamed identifiers whose parameter or var type spelled
config.Config. A value reached through an inferred declaration
("cfg := doc.Config") or held in a struct field therefore left the read
intact while the rule saw no holder -- the same class of miss that let voice
transcription ship behind a green link gate.

Widen the holder detection to track the config type through inferred short
variable declarations and vars, struct fields and receivers, type-preserving
expressions, and range variables over a slice or map of config, then assert
each previously-inert shape red first.
2026-10-02 11:44:51 +10:00
sam 0b1a852650 fix(messaging): gate the boundary on config reads, not only package links
The deletion gate checked which packages the messaging binary links, so a
process that reads a decoded [models]/[providers] value passed while
building an LLM client in-process. Go imports are atomic, but a config
read leaves no import to measure.

- add the model-provider capability packages (transcription, embedding)
  to the banned-link table and classify links through a testable
  bannedLink rule.
- scan internal/app/archiemessaging's production sources for a read of a
  top-level config section the PRD forbids, so the projection that built
  the transcriber cannot come back.
- unit-test both rules, including a synthetic copy of the old wiring, so
  the live gate cannot rot into a no-op and still pass.

Red first: TestMessagingReadsNoBannedConfigSection failed on cfg.Models
and cfg.Providers before the capability moved, and the link gate failed on
internal/infrastructure/transcription.
2026-10-02 11:12:59 +10:00
sam 00e5b431e4 feat(webui): let the operator choose the retry worktree mode
A retry now carries the operator's worktree choice (refresh onto the base
branch or continue the work already pushed on the task branch) from the
dashboard through the task action to the daemon dispatch, and persists it
on the task row so it survives a reload. The daemon resolves the mode from
the task, never from the workflow; BeginRemediation records continue
explicitly so a remediation still resumes its pull request branch.
2026-10-02 10:53:21 +10:00
sam 15946dacfb feat(taskactions): one approval path answers the review gate from every surface
Decision 1 of docs/prds/pr-review-operator-response.md: the dashboard's
POST /api/tasks/{id}/action is the authoritative surface, chat's /approve
is a shortcut that posts every offered finding, and both reach the same
Service.Apply so one operator intent cannot be recorded as two decisions.

The action body now carries instructions (required by rereview) and the
finding keys an approve posts. taskactions.Service answers an offer
through the guarded RespondReviewGate write: approve records the
selection on the offer, rereview records the instructions, increments the
round and clears the offer, and a rereview with no instructions, with no
offer, or at the cap is refused as a conflict with nothing written. Chat
reaches the same service: chatTaskControllerAdapter.ApproveChatTask
delegates to b.taskActions() instead of its own requeue, which is what
removed the handler's hardcoded implement handoff -- feasibility's deliver
stage now names implement on the task before it waits, the invariant the
handler needs.

The payload crosses every hop it must: the NATS action request, the
gateway.v1 gRPC contract (regenerated), the LocalChatAdapter, and the
webui handler, which answers 400 for a rereview without instructions and
409 for the cap. A rereview event kind names its actor the way the
approval and rejection trios do.

taskactions.Task gains review_gate and rereview_rounds, and the two
surfaces' tests assert they agree.
2026-10-02 02:27:59 +10:00
sam 01c98009bc feat(store): key control-plane resources by (org, kind)
resources and resource_history are one document per org and kind, with
per-org advisory locks, request-ID idempotency and audit keys. The
control-plane Query/History/Command/Watch requests carry org_id; a request
naming none acts in the default org until the caller's org is derived
server-side.
2026-09-30 03:34:02 +10:00
sam 65674c665a test(state-store): parallelize real-process smoke tests 2026-09-30 02:14:29 +10:00
sam 04a1d48511 feat(pr-review): benchmark live pipeline through generic workflow spawn 2026-09-29 00:35:34 +10:00
sam 5c91575166 feat(pr-review): operator/chat trigger queues a pr-review task
review_pr self-omitted since 7299b923 deleted the synchronous reviewer it
called. It is rebuilt on the task enqueue path everything else uses: the
tool resolves the identity's configured repo, then queues a pr-review task
naming the pull request through the workflow's declared pr_number input,
which is the same generic input mechanism a watched-repos binding uses.

That input has to reach the store with the task, so
EnqueueChatTaskRequest gains inputs_json and the tasks insert writes the
inputs column directly. StampTaskBinding no longer writes it: the insert is
now the column's only writer, and a chat task must not acquire binding
provenance just to carry an input.

ChatPRReviewer, its result shape and the reviewer wiring are deleted with
the path they served.
2026-09-28 18:24:36 +10:00
sam c8f9571448 feat(access): archied access reset --org/--instance on the State Store host (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam 7b37438e59 feat(prreview): benchmark harness scoring findings on the Martian problems
Scores a reviewer's findings on the 38 runnable Martian Code-Review-Bench
PRs: the deterministic core turns findings into posted comments, an
independent judge model matches them to the goldens, and the run reports
golden micro-recall and golden-only precision. The reviewer is a findings
fixture directory until the pr-review workflow lands.

Run with: task bench:prreview -- -findings DIR
2026-09-26 10:30:12 +10:00
sam 928d7041e6 feat(agentexec): serve capture tools to harnesses over MCP
archie-agent mcp is a stdio MCP server, on the official go-sdk, serving a
stage's capture tools and appending accepted calls to a captures file.
The harness runner writes the tool spec and a standard mcpServers config,
registers it through HarnessSpec.MCPConfig, and reads the captures back
into the stage result. The captures file is writable by the harness user,
so the worker validates every record again with the same acceptCapture
the built-in loop now shares, and forged, undeclared or over-cap records
are dropped. A stage with capture tools on a harness without MCP is
refused before it runs.
2026-09-26 03:28:58 +10:00
sam 2f866ac6d8 feat(egress): relay forwards the proxy, NATS and the State Store
A Kit task's worker shares the sandbox network with its harness, so it
reaches NATS and the State Store through the relay like the harness
reaches the proxy. archie-agent relay takes repeated -forward
listen=target pairs, binds them all before serving so a port conflict
fails at start, and stops every forward when one fails. RelayArgs builds
the command from the fixed relay ports.
2026-09-26 03:10:11 +10:00
sam 3394d36086 feat(egress): relay forwarding a sandbox network to the egress proxy
archie-agent relay listens on the sandbox's isolated network and joins
every connection to one fixed target, the proxy. The relay holds no
secrets and can reach nothing else, so the host needs no address on a
sandbox network at all.
2026-09-26 02:33:41 +10:00
sam 8c55b5d4e4 chore(state-store): remove the one-time SQLite import and db_path
Every install serves from Postgres, so the legacyread/legacyimport packages,
the archie-state-store import command, the boot gate that refused to serve
over unimported SQLite files, and db_path (which only located those files)
go. Migration 0010 drops the import_completion table.
2026-09-25 00:40:48 +10:00
sam f68355ca77 Merge branch 'pg/foundation' into worktree-agent-a96503bc8828fc104
# Conflicts:
#	docs/data/generated/dev-docs.json
#	internal/app/archied/state_store_test.go
#	internal/app/archied/state_store_toolcalls_test.go
#	internal/contracts/state/v1/state.pb.go
#	internal/daemon/dispatch_bindings_test.go
#	internal/infrastructure/bindingcipher/cipher_test.go
#	internal/infrastructure/captureintake/receiver_test.go
#	internal/infrastructure/edastore/contracts_binding.go
#	internal/infrastructure/edastore/testsupport_test.go
#	internal/infrastructure/postgres/edastore.go
#	internal/infrastructure/postgres/edastore_test.go
#	internal/infrastructure/staterpc/error_test.go
#	internal/store/store.go
#	internal/webui/api_binding_test.go
2026-09-24 19:47:00 +10:00
sam 0495af7c3d docs(store): drop residual SQLite references; fix lane lint findings 2026-09-24 19:39:06 +10:00
sam bf3e688bfe refactor(store)!: remove the SQLite task store, the PocketBase event store and the SQLite session store
The State Store, event capture and Gateway conversations already serve from
PostgreSQL. This deletes internal/store and internal/infrastructure/edastore,
the SQLite gateway session store and openProductionTaskStore, and the -db
recovery path on a legacy task file; backup, restore, validate and rollback
work on the database database_url names.

- The binding-secret cipher moves to internal/infrastructure/bindingcipher and
  ToolCall to internal/infrastructure/postgres.
- NewSessionStoreMemory is now a pure in-memory store that passes the shared
  SessionStore conformance suite.
- Tests run against PostgreSQL through internal/infrastructure/postgres/pgstore;
  legacyimport/legacyfixture writes the legacy files the import and the boot
  gate read.
- The cron dialect is owned in internal/domain/scheduling, so
  github.com/pocketbase/pocketbase and pocketbase/dbx leave go.mod.
  modernc.org/sqlite stays for the one-time import (legacyread).
- The UI and Messaging architecture gates ban the Postgres and legacy readers
  in place of internal/store.
2026-09-24 14:43:50 +10:00
sam 3fce04bf5d merge: t2db.40 sources and signing into pg/foundation
Resolves against t2db.36: captures carry both event_type and unsigned (proto field 10), the State Store serves sources and event types from the Postgres EDA store, the capture row shows event type and signature together, and the staterpc sentinels map both ways from one table.
2026-09-24 14:38:40 +10:00
sam 963f967c56 feat(eda): sources own signing, with UUIDv7 paths and approved unsigned option
A source is the capture endpoint: a UUIDv7 path by default or a unique
URL-safe custom path, signed by default with an Archie-generated secret
(encrypted via the bindings cipher), or unsigned after an approval.
Intake verifies X-Hub-Signature-256/X-Signature-256 against the source
secret before redaction; an unauthenticated event on a signed source is
captured but never dispatched, an event on an approved unsigned source
dispatches and is marked unsigned. The binding secret is retired; migration
0006 derives sources from existing binding and capture source strings,
moving each binding's secret onto its source. SourceStore crosses the
State Store gRPC contract. The dashboard gains a sources panel and flags
unsigned in the capture inspector, binding list and task timeline.
2026-09-24 14:26:21 +10:00
sam a31ac8f675 Merge branch 'worktree-agent-a2b33953bdffebf56' into worktree-giggly-marinating-lamport
# Conflicts:
#	docs/data/generated/dev-docs.json
#	internal/app/archied/state_store_pool.go
2026-09-24 10:06:00 +10:00
sam 970a65fe36 feat(archied): serve the State Store, event store and Gateway conversations from Postgres
The State Store's task and event-capture stores and the conversation store
read by archied and served by archie-gateway now open on database_url.
Every serving process refuses to start when legacy SQLite data exists and no
import completion is recorded, naming archie-state-store import; a fresh
install records completion itself. The Gateway holds its Postgres serve
claim for its lifetime and releases it before the pool closes.
2026-09-24 10:01:54 +10:00
sam 9f5f072fe7 Merge branch 'worktree-agent-a091696d945ed3377' into worktree-giggly-marinating-lamport
# Conflicts:
#	cmd/archie-state-store/recovery.go
2026-09-24 09:56:10 +10:00
sam fe819d489a feat(archied): recovery commands on PostgreSQL; fail-closed pre-update snapshot, no automatic schema rollback
Without -db, backup/restore/validate/rollback resolve database_url from the
config and work on the whole PostgreSQL database. Restore claims every service
role; rollback claims the State Store role; validate refuses a schema newer
than the binary. The installer refuses a daemon update it cannot snapshot
first; the watchdog never restores the database and prints the explicit
offline restore instead.
2026-09-24 09:50:10 +10:00
sam cdb34e2295 fix(state-store): real-process tests give the State Store a database_url 2026-09-24 09:48:58 +10:00
sam 42dcc80754 feat(state-store): one-time import of the legacy SQLite stores into Postgres
archie-state-store import copies the State Store task database, the
PocketBase event store and the Gateway conversation store, all named from
db_path, into Postgres in one transaction. The bead asked for an archied
command; offline maintenance of a store belongs to the binary that owns it,
so it sits beside backup/restore/validate/rollback instead.

It refuses rather than guesses: while the state-store or gateway ownership
claim is held, when a domain it imports already holds rows (an allowlist of
that domain's tables, checked after Migrate), on a completed prior import,
on FK orphans, alias case collisions, duplicate bindings, events whose (at,
id) order microsecond storage would change, schema gaps, and any value the
target column cannot hold, naming table, row, value and rule. Historical ids
are kept and every identity sequence is set from AUTOINCREMENT's high-water
mark. After loading it compares key sets, event order, resource blobs and
binding secrets byte for byte and the next id of every sequence, checks the
source files were not written while read, and only then writes an
import_completion row (migration 0005) in the same transaction.

postgres.ImportComplete and legacyimport.HasLegacyData are the two halves
of the boot gate: legacy data without a completed import must not serve.
2026-09-24 09:45:26 +10:00
sam 65697bd4ae refactor(state-store): remove the PocketBase admin view
stateadmin served read-only PocketBase views over the SQLite task file on -admin-addr. PocketBase is SQLite-only and the State Store moves to Postgres (archie-core-7r0x), so the view goes; operators use a Postgres tool of their choice.
2026-09-24 01:09:42 +10:00
sam 62645d1a93 feat(playbooks): archie-playbooks serve, a language server publishing lint findings as diagnostics 2026-09-23 22:22:56 +10:00
sam 0ad5d2acd9 feat(playbooks): lint an EDA playbook directory with -eda-dir 2026-09-23 22:11:01 +10:00
sam 1b28bba234 feat(workflow): routing and playbook findings carry the key's file:line 2026-09-23 21:41:56 +10:00