Commit Graph
981 Commits
Author SHA1 Message Date
sam 4ffebb855d feat(webui): five-place top bar with Settings, New task and one connection pill
Dashboard, Tasks, Workflows, Events and Settings replace the Work, Agent
and System menus. Configuration pages move under /settings/* with
redirects from their old paths; Logs and Appearance leave the bar and stay
reachable from Jump to, the Dashboard and the account menu. New task opens
the Start work form in a dialog and lands on the new task. One pill reports
the live stream's state.

Drops docs/prds/dashboard-navigation-groups.md: web UI design is not
tracked in PRDs.
2026-09-25 15:09:18 +10:00
sam 9344647719 fix(webui): multiplex dashboard live updates over one stream 2026-09-25 14:05:34 +10:00
sam c5faad5915 feat(controlplane): field-level sys_audit table, audit view on settings pages, whole-task time limit 2026-09-25 09:54:30 +10:00
sam 7cb921378b fix(webui): open control-plane watch streams before the first change 2026-09-25 09:37:36 +10:00
sam 174e7c3cde fix(taskactions): abandon leaves the forge issue open 2026-09-25 08:59:54 +10:00
sam cc98831e15 fix(daemon): park a task whose agent container exits before replying 2026-09-25 08:31:46 +10:00
sam 95afdac206 fix(update): treat the runtime as the release version; stamp it in dist-zip
The update check still read the retired archie/v* tag stream, so a host on
1.41.1 was offered runtime 1.35.0, and the installer built the agent image
from that stale tag. Both now use the release's archied/v* tag. The dist-zip
build also stamps runtimeVersion, so /version stops reporting 'dev'.
2026-09-25 01:58:20 +10:00
sam 5f915931ef test: remove layout-pinned route-registry and task-meta fixture tests
TestDashboardPagesRegistryCoversEveryRoute parsed ui/src/router/index.ts as
text, and TestTaskMetaPayloadMatchesFixture plus ui/test/task-meta-snapshot
pinned a formatted JSON fixture. They failed on formatting, not behaviour.
2026-09-25 01:32:20 +10:00
sam 5cd1079066 fix(ci): make the gate tolerate formatting and give deploy a PG 18 client
The page-registry test parsed the route table one line per route and the
task-meta fixture was byte-compared, so the prettier reformat broke both.
The registry test now splits route objects and ignores comments; the
fixture compares compacted JSON. Also drops a stale router comment that
described the removed redirects, rebuilds ui/dist, regenerates the
contracts artifact, keeps prettier off generated output, and installs
postgresql-client-18 in deploy.yml's gate as quality.yml already does.
2026-09-25 01:29:23 +10:00
sam 2e00777f07 refactor: format markdown files using prettier, fix spelling concerns, reduce Taskfile bloat 2026-09-25 01:10:28 +10:00
sam 2051ca94d2 chore(state-store): remove the one-time SQLite import and db_path
Every install serves from Postgres, so the legacyread/legacyimport packages,
the archie-state-store import command, the boot gate that refused to serve
over unimported SQLite files, and db_path (which only located those files)
go. Migration 0010 drops the import_completion table.
2026-09-25 00:40:48 +10:00
sam a2deca6612 chore(staterpc): remove deprecated unary capture RPCs and Binding.secret
ListCaptures and ListUndispatchedCaptures had no callers once the client
moved to the streaming RPCs; they were kept only for mixed-version rollouts.
Binding.secret was unused since signing moved to Source; its number and
name are reserved.
2026-09-25 00:00:16 +10:00
sam f624d23f3c chore(config): remove token_env aliases for forge, artifacts and telegram
Each credential is now only a {engine, key} secret reference. A stale
token_env in a file is reported at startup as an unrecognised key.
nats.token_env stays (no secret-ref form), and providers.api_key_env stays:
it is the carrier the resolver writes each resolved key's env name into.
2026-09-24 23:58:26 +10:00
sam 9da6108617 chore(config): remove dead [agent], [oidc], [image] and containers.enabled keys
None had a production consumer: [agent] and containers.enabled were decoded
only so old files kept loading, [oidc] was never read (archie-ui takes OIDC
from flags), and [image] was validated but no provider exists.
2026-09-24 23:52:06 +10:00
sam 8c46c9c3a6 test: check scanner error in openSSEStream 2026-09-24 22:27:56 +10:00
sam 335fb4bb22 Merge branch 'main' into pg/foundation 2026-09-24 22:21:24 +10:00
sam 55f16fa7d0 fix: reject values that exceed Int32 for all SQL LIMIT params 2026-09-24 21:45:27 +10:00
sam 6f3289fd33 docs(development): checklists for every kind of change
docs/development/ holds one page per kind of change (settings, State Store
and persistence, task lifecycle, workflows and steps, event automation,
dashboard, agent execution and containers, processes and wiring, merging
work), each naming every layer the change must reach and pointing at the
document that owns the design. The docs build puts them in the development
set; docs/index.md and CLAUDE.md link them.

Also adds the container-profile case to the validator agreement test, the
check the profiles work skipped.
2026-09-24 21:03:34 +10:00
sam f538f457ce fix(controlplane): stored container policies replace file profiles; add profile image and workflow examples
A stored container-runtime-policies document that carries Profiles now
replaces the file's profiles instead of merging into them, so a profile
deleted in the dashboard stays deleted; one stored before profiles existed
still inherits the file's. Profiles drop their json tags so the stored
document keys them like its other fields.

Adds examples/profiles/Dockerfile.agent-net (agent image plus network
lookup tools) with a build-only agent-net compose service,
examples/workflows/firewall-investigate.yaml with a test that parses every
example against the registered step vocabulary, and names the postgres
service in the compose header.
2026-09-24 20:58:08 +10:00
sam f3b8ddecb0 fix(eda): clear completed tasks; document the tool allowlist's reach and unchecked profile names
ClearTerminalTasks now deletes completed tasks, with a test that holds its
SQL list to taskstate.Terminal. The AllowTools comment says agentloop's
built-in file tools are outside the allowlist and why. The PRD model
diagram lists only what a profile holds (image, tools), and the profile
field and AgentProfile say a workflow's profile name is resolved at
dispatch, not when the workflow is saved.
2026-09-24 20:53:51 +10:00
sam 47fae672f1 feat(eda): workflow inputs, optional repository and agent profiles
Workflow definitions declare inputs, repository (none, optional or
required; default required) and profile. A binding assigns each input
from a mapped parameter or a constant and may take its repository from a
parameter, which must name a configured repo; both are checked when the
binding is saved and again at dispatch. Inputs travel as structured task
data (tasks.inputs, the task brief, and a JSON block in the agent's
mission), never as body text.

A task with no repository runs in a per-task scratch workspace with no
clone and no publication grant. The agent.run step runs one mission
without a repository and finishes the task in the new completed status.

Agent profiles live under [containers.profiles.<name>]: an image and an
allowlist over the tools archie adds (MCP, scripts, plugins).
[containers].image is the default profile; an unconfigured profile parks
the task. The workflow is now pinned before the container starts so the
profile can choose its image.

Also stop two archied tests writing embedded NATS state into the package
directory, refuse an empty state_dir for embedded NATS, and drop the NATS
store files an earlier lane commit tracked.
2026-09-24 20:29:33 +10:00
sam 2393cade0f Merge branch 'pg/foundation' into worktree-agent-a96503bc8828fc104
# Conflicts:
#	docs/data/generated/dev-docs.json
#	internal/app/archied/state_store_test.go
#	internal/app/archied/state_store_toolcalls_test.go
#	internal/contracts/state/v1/state.pb.go
#	internal/daemon/dispatch_bindings_test.go
#	internal/infrastructure/bindingcipher/cipher_test.go
#	internal/infrastructure/captureintake/receiver_test.go
#	internal/infrastructure/edastore/contracts_binding.go
#	internal/infrastructure/edastore/testsupport_test.go
#	internal/infrastructure/postgres/edastore.go
#	internal/infrastructure/postgres/edastore_test.go
#	internal/infrastructure/staterpc/error_test.go
#	internal/store/store.go
#	internal/webui/api_binding_test.go
2026-09-24 19:47:00 +10:00
sam f4b8707ba9 docs(store): drop residual SQLite references; fix lane lint findings 2026-09-24 19:39:06 +10:00
sam bbaca2a39d Merge remote-tracking branch 'origin/pg/foundation' into worktree-agent-a62864fcfbdb43f5a
# Conflicts:
#	internal/app/archied/state_store.go
#	internal/contracts/state/v1/state.pb.go
#	internal/daemon/daemon.go
#	internal/infrastructure/captureintake/receiver.go
#	internal/infrastructure/captureintake/receiver_test.go
#	internal/infrastructure/legacyimport/load.go
#	internal/infrastructure/postgres/edastore_test.go
#	internal/infrastructure/postgres/postgresdb/edastore.sql.go
#	internal/infrastructure/postgres/queries/edastore.sql
#	internal/infrastructure/staterpc/client.go
#	internal/infrastructure/staterpc/values.go
#	internal/webui/api_binding.go
#	internal/webui/api_binding_test.go
#	ui/dist/assets/index.js
#	ui/src/bindings/BindingEditor.vue
#	ui/src/bindings/BindingRow.vue
2026-09-24 19:34:31 +10:00
sam 89b1140d56 docs(proto): name the Postgres State Store in the state contract comments 2026-09-24 14:47:53 +10:00
sam ecf243fc19 feat(config): add state_dir for the daemon's host state
The embedded NATS store and endpoint file, the task-log registry and the
readiness disk probe resolve from state_dir (default: the data home's archie
directory) instead of the directory of db_path. db_path now only locates the
legacy SQLite files for the one-time import and its boot gate.
2026-09-24 14:47:21 +10:00
sam e3ee044c5e refactor(store)!: remove the SQLite task store, the PocketBase event store and the SQLite session store
The State Store, event capture and Gateway conversations already serve from
PostgreSQL. This deletes internal/store and internal/infrastructure/edastore,
the SQLite gateway session store and openProductionTaskStore, and the -db
recovery path on a legacy task file; backup, restore, validate and rollback
work on the database database_url names.

- The binding-secret cipher moves to internal/infrastructure/bindingcipher and
  ToolCall to internal/infrastructure/postgres.
- NewSessionStoreMemory is now a pure in-memory store that passes the shared
  SessionStore conformance suite.
- Tests run against PostgreSQL through internal/infrastructure/postgres/pgstore;
  legacyimport/legacyfixture writes the legacy files the import and the boot
  gate read.
- The cron dialect is owned in internal/domain/scheduling, so
  github.com/pocketbase/pocketbase and pocketbase/dbx leave go.mod.
  modernc.org/sqlite stays for the one-time import (legacyread).
- The UI and Messaging architecture gates ban the Postgres and legacy readers
  in place of internal/store.
2026-09-24 14:43:50 +10:00
sam a3b4ead6a6 refactor(scripts): drop the SQLite task-database backup from the installer and watchdog
The pre-update PostgreSQL snapshot is the only backup; the watchdog restores
binaries and never the database.
2026-09-24 14:43:25 +10:00
sam 879d87831c feat(eda): mappings per event type, match counts, several bindings per source
A mapping belongs to an event type and is checked against its inferred
schema on save. Each event a mapping resolves is counted once in
mapping_matches (migration 0008), read back as match_count and
last_matched_at. Bindings carry an optional CEL filter over the mapping's
parameters, checked on save. The per-source unique indexes are dropped:
every armed binding whose mapping matches the capture's event type and
whose filter admits it dispatches once, claimed in the per-binding ledger
before its task is enqueued. The State Store gains RecordMappingMatch and
the new mapping and binding fields.
2026-09-24 14:42:58 +10:00
sam 8a2b5b95f9 merge: t2db.40 sources and signing into pg/foundation
Resolves against t2db.36: captures carry both event_type and unsigned (proto field 10), the State Store serves sources and event types from the Postgres EDA store, the capture row shows event type and signature together, and the staterpc sentinels map both ways from one table.
2026-09-24 14:38:40 +10:00
sam 57ddd4e002 refactor(gateway): replace the SQLite session store with an in-memory store 2026-09-24 14:26:42 +10:00
sam c485998160 refactor(scheduling): own the cron dialect instead of pocketbase tools/cron 2026-09-24 14:26:41 +10:00
sam 684ec1263a feat(eda): sources own signing, with UUIDv7 paths and approved unsigned option
A source is the capture endpoint: a UUIDv7 path by default or a unique
URL-safe custom path, signed by default with an Archie-generated secret
(encrypted via the bindings cipher), or unsigned after an approval.
Intake verifies X-Hub-Signature-256/X-Signature-256 against the source
secret before redaction; an unauthenticated event on a signed source is
captured but never dispatched, an event on an approved unsigned source
dispatches and is marked unsigned. The binding secret is retired; migration
0006 derives sources from existing binding and capture source strings,
moving each binding's secret onto its source. SourceStore crosses the
State Store gRPC contract. The dashboard gains a sources panel and flags
unsigned in the capture inspector, binding list and task timeline.
2026-09-24 14:26:21 +10:00
sam 75ed54ba7f fix(legacyimport): imported captures take the new event_type column's default 2026-09-24 14:26:18 +10:00
sam 9fac216c21 Merge branch 'worktree-agent-a88b86b51f454341b' into worktree-giggly-marinating-lamport 2026-09-24 14:21:53 +10:00
sam 152cdc6523 feat(webui): inspector shows proposed and unidentified events, names and pastes event types 2026-09-24 14:21:21 +10:00
sam 3878ce0ebb feat(staterpc): event type RPCs and capture event_type on the wire 2026-09-24 14:21:21 +10:00
sam 9f853c839f feat(eventtype): infer event types from captures and gate dispatch on identification 2026-09-24 14:21:20 +10:00
sam 60b88ad6d3 Merge branch 'worktree-agent-a2b33953bdffebf56' into worktree-giggly-marinating-lamport
# Conflicts:
#	docs/data/generated/dev-docs.json
#	internal/app/archied/state_store_pool.go
2026-09-24 10:06:00 +10:00
sam cea88060b1 feat(archied): serve the State Store, event store and Gateway conversations from Postgres
The State Store's task and event-capture stores and the conversation store
read by archied and served by archie-gateway now open on database_url.
Every serving process refuses to start when legacy SQLite data exists and no
import completion is recorded, naming archie-state-store import; a fresh
install records completion itself. The Gateway holds its Postgres serve
claim for its lifetime and releases it before the pool closes.
2026-09-24 10:01:54 +10:00
sam 3fc710fbe7 Merge branch 'worktree-agent-a091696d945ed3377' into worktree-giggly-marinating-lamport
# Conflicts:
#	cmd/archie-state-store/recovery.go
2026-09-24 09:56:10 +10:00
sam d80c01b9fb feat(archied): recovery commands on PostgreSQL; fail-closed pre-update snapshot, no automatic schema rollback
Without -db, backup/restore/validate/rollback resolve database_url from the
config and work on the whole PostgreSQL database. Restore claims every service
role; rollback claims the State Store role; validate refuses a schema newer
than the binary. The installer refuses a daemon update it cannot snapshot
first; the watchdog never restores the database and prints the explicit
offline restore instead.
2026-09-24 09:50:10 +10:00
sam 823a7d0b40 feat(state-store): one-time import of the legacy SQLite stores into Postgres
archie-state-store import copies the State Store task database, the
PocketBase event store and the Gateway conversation store, all named from
db_path, into Postgres in one transaction. The bead asked for an archied
command; offline maintenance of a store belongs to the binary that owns it,
so it sits beside backup/restore/validate/rollback instead.

It refuses rather than guesses: while the state-store or gateway ownership
claim is held, when a domain it imports already holds rows (an allowlist of
that domain's tables, checked after Migrate), on a completed prior import,
on FK orphans, alias case collisions, duplicate bindings, events whose (at,
id) order microsecond storage would change, schema gaps, and any value the
target column cannot hold, naming table, row, value and rule. Historical ids
are kept and every identity sequence is set from AUTOINCREMENT's high-water
mark. After loading it compares key sets, event order, resource blobs and
binding secrets byte for byte and the next id of every sequence, checks the
source files were not written while read, and only then writes an
import_completion row (migration 0005) in the same transaction.

postgres.ImportComplete and legacyimport.HasLegacyData are the two halves
of the boot gate: legacy data without a completed import must not serve.
2026-09-24 09:45:26 +10:00
sam 8c21928140 feat(postgres): whole-database backup and offline restore on pg_dump/pg_restore
Restore claims every service role (state-store and gateway) non-blockingly
and refuses if any is held or the claim cannot be checked; it drops tables a
later migration added before pg_restore --clean so a restored schema matches
the snapshot. Backup validates the archive before replacing the previous one.
Ports the store's write-side clip, park-class and transition-audit coverage
from internal/store.
2026-09-24 09:32:39 +10:00
sam cb518e6f6d feat(webui): serve change statuses and the config schema through /api/task-meta
Closes #865. The dashboard hand-copied the Change* file-status ids and events.ConfigCapturedSchema. /api/task-meta now serves both; the Vue dashboard renders ui/src/lib/task-meta-snapshot.ts until the served catalog lands. The Go handler is pinned to internal/webui/testdata/task_meta.json and the snapshot is pinned to that same fixture, so a one-sided rename fails a test.
2026-09-24 09:31:09 +10:00
sam f9dc76751c feat(legacyread): read-only readers and import verification for the legacy SQLite stores 2026-09-24 03:28:42 +10:00
sam 92c116f606 fix(postgres): lock keys follow D3.7/D4.3; event reader never sees a later id before an earlier one 2026-09-24 03:23:38 +10:00
sam 25e8c8cdd1 feat(postgres): event inserts take a transaction advisory lock so ids commit in order 2026-09-24 03:01:41 +10:00
sam 539c07a5eb feat(postgres): State Store claims serve ownership with a session advisory lock 2026-09-24 02:57:38 +10:00
sam c32db5d935 refactor(postgres): resources and statestore tests use storecontract, not internal/store 2026-09-24 02:47:06 +10:00