A mapping belongs to an event type and is checked against its inferred
schema on save. Each event a mapping resolves is counted once in
mapping_matches (migration 0008), read back as match_count and
last_matched_at. Bindings carry an optional CEL filter over the mapping's
parameters, checked on save. The per-source unique indexes are dropped:
every armed binding whose mapping matches the capture's event type and
whose filter admits it dispatches once, claimed in the per-binding ledger
before its task is enqueued. The State Store gains RecordMappingMatch and
the new mapping and binding fields.
Resolves against t2db.36: captures carry both event_type and unsigned (proto field 10), the State Store serves sources and event types from the Postgres EDA store, the capture row shows event type and signature together, and the staterpc sentinels map both ways from one table.
A source is the capture endpoint: a UUIDv7 path by default or a unique
URL-safe custom path, signed by default with an Archie-generated secret
(encrypted via the bindings cipher), or unsigned after an approval.
Intake verifies X-Hub-Signature-256/X-Signature-256 against the source
secret before redaction; an unauthenticated event on a signed source is
captured but never dispatched, an event on an approved unsigned source
dispatches and is marked unsigned. The binding secret is retired; migration
0006 derives sources from existing binding and capture source strings,
moving each binding's secret onto its source. SourceStore crosses the
State Store gRPC contract. The dashboard gains a sources panel and flags
unsigned in the capture inspector, binding list and task timeline.
The State Store's task and event-capture stores and the conversation store
read by archied and served by archie-gateway now open on database_url.
Every serving process refuses to start when legacy SQLite data exists and no
import completion is recorded, naming archie-state-store import; a fresh
install records completion itself. The Gateway holds its Postgres serve
claim for its lifetime and releases it before the pool closes.
Without -db, backup/restore/validate/rollback resolve database_url from the
config and work on the whole PostgreSQL database. Restore claims every service
role; rollback claims the State Store role; validate refuses a schema newer
than the binary. The installer refuses a daemon update it cannot snapshot
first; the watchdog never restores the database and prints the explicit
offline restore instead.
archie-state-store import copies the State Store task database, the
PocketBase event store and the Gateway conversation store, all named from
db_path, into Postgres in one transaction. The bead asked for an archied
command; offline maintenance of a store belongs to the binary that owns it,
so it sits beside backup/restore/validate/rollback instead.
It refuses rather than guesses: while the state-store or gateway ownership
claim is held, when a domain it imports already holds rows (an allowlist of
that domain's tables, checked after Migrate), on a completed prior import,
on FK orphans, alias case collisions, duplicate bindings, events whose (at,
id) order microsecond storage would change, schema gaps, and any value the
target column cannot hold, naming table, row, value and rule. Historical ids
are kept and every identity sequence is set from AUTOINCREMENT's high-water
mark. After loading it compares key sets, event order, resource blobs and
binding secrets byte for byte and the next id of every sequence, checks the
source files were not written while read, and only then writes an
import_completion row (migration 0005) in the same transaction.
postgres.ImportComplete and legacyimport.HasLegacyData are the two halves
of the boot gate: legacy data without a completed import must not serve.
Restore claims every service role (state-store and gateway) non-blockingly
and refuses if any is held or the claim cannot be checked; it drops tables a
later migration added before pg_restore --clean so a restored schema matches
the snapshot. Backup validates the archive before replacing the previous one.
Ports the store's write-side clip, park-class and transition-audit coverage
from internal/store.
Closes#865. The dashboard hand-copied the Change* file-status ids and events.ConfigCapturedSchema. /api/task-meta now serves both; the Vue dashboard renders ui/src/lib/task-meta-snapshot.ts until the served catalog lands. The Go handler is pinned to internal/webui/testdata/task_meta.json and the snapshot is pinned to that same fixture, so a one-sided rename fails a test.
Wave 1 of the Postgres migration (archie-core-7r0x.9). This is the shared shape
every port lane builds on; nothing here switches a live path to Postgres yet.
- config: add `database_url` beside `db_path` (config.go), mark it bootstrap-only
in denied_keys, expose it in the webui StorageView/config schema, and add it to
config.example.toml plus all deployments/*.toml.
- docker-compose: add a postgres:18 service; `task dev` (dev:postgres) brings it
up and waits for it before the State Store boots.
- archied: `openStateStore` now opens and migrates the process-scoped Postgres
pool via `openStateStorePool`. The State Store fails closed (D1.2) when
database_url is absent or invalid rather than fall back to SQLite; serving
against Postgres later requires an import-completion record (D1.3), which is a
later lane's gate, not this one.
- composition seam: task-store construction (state_store_task.go) and EDA-store
construction (state_store_eda.go) are extracted from openStateStore, with the
pool open in state_store_pool.go. This is a behaviour-preserving refactor only
— no store, interface, error, ordering or logging change — so L1 and L2 build
against separate files instead of sharing state_store.go.
Tests: fail-closed pool behaviour is pinned (absent/invalid URL); the State Store
boot/restart tests now run against a throwaway Postgres 18 container (pgtest
TestMain). Scoped test command and module-wide build pass; see lane report.
CHECKPOINT -- the full State Store port is in progress; this lands the
compile-clean core so the review pass has a diff to work against. Remaining:
advisory-lock ownership, event-insert serialization, maintenance, and the
ported test battery.
Two corrections to behaviour that already shipped, not new features:
- UpdateTask wrote park_reason raw where SQLite applies clip(park_reason,
4000); the PostgreSQL adapter now clips before binding (clip.go).
- tasks.review_cursor existed on the protobuf wire and in workflow.Task but
not in the PostgreSQL schema; 0004 adds it as a NEW migration rather than
amending 0001 because a dev database may already have applied 0001 and
goose tracks version numbers, not checksums.
Also moves the control-plane Resource/ResourceWrite types and their sentinels
into storecontract so the postgres package can stop importing internal/store
(they stay aliased there for daemon callers).
L2 of the PocketBase->Postgres migration (bead archie-core-7r0x.3): a new
Postgres-backed EDA adapter (postgres.EDA) plus its goose migration
(0002_edastore.sql) and sqlc queries (queries/edastore.sql), satisfying the
same storecontract surfaces as internal/infrastructure/edastore.Store so the
cutover wave can swap them behind the contract. The PocketBase opener in
internal/app/archied/state_store.go is untouched: production still opens
SQLite, and the cutover waits for the one-time import.
Three recon-found hazards, each pinned by a test, not a comment:
1. Empty-secret early return. applyBinding deliberately never writes the
secret column when the incoming secret is empty, so an edit form that does
not echo the secret must not blank an armed binding's HMAC key. The
UpdateBinding query keeps the stored secret via
`secret = CASE WHEN $secret = '' THEN secret ELSE $secret END`; pinned by
TestUpdateBindingEmptySecretPreservesStored (raw column unchanged across an
empty-secret edit).
2. Unique-violation detection by SQLSTATE. isUniqueViolation previously grepped
the driver's error text for "unique"/"already exists"; Postgres reports
SQLSTATE 23505 and the text match would not fire, losing ErrAlreadyDispatched.
The adapter detects by pgconn.PgError.Code == "23505"; pinned by
TestIsUniqueViolationDetectsBySQLState (23505 with a message lacking those
words is detected; a non-23505 whose text says "unique" is not).
3. One-binding-per-source / one-armed-binding-per-source were read-then-refuse
with no transaction and a non-unique idx_bindings_source_status. Made them
constraints: UNIQUE(source) plus a partial UNIQUE(source) WHERE
status='armed' (the full unique subsumes the armed rule today, kept explicit
for a future matcher extension). A concurrent insert for one source yields
exactly one success and one ErrBindingOverlap; pinned by
TestConcurrentInsertBindingSameSource.
Dispatch ledgers kept exactly: UNIQUE(binding, capture) and
UNIQUE(playbook_id, playbook_version, event_id, action_id); binding_version is
stored but deliberately not part of the key (pinned by
TestBindingDispatchVersionBumpDoesNotPermitRedispatch). The cipher is
unchanged: AES-256-GCM, key = SHA-256(material), decrypt stays in the store and
the gRPC wire still carries plaintext.
Notify is left as an injected callback (EDA.SetNotify) and is NOT wired to
LISTEN/NOTIFY. It is dead code today: edastore.Config.Notify is omitted at its
only production construction and no consumer maps binding_changed/
mapping_changed, so building a LISTEN/NOTIFY path would be building for a
consumer that does not exist. Wired or not is a separate decision.
Move sessions, messages, turns and full-text search off the per-process
SQLite file onto Postgres behind the existing messaging.SessionStore
contract. New adapter internal/gateway/session_store_postgres.go, sqlc
queries/gateway.sql, migration 0003_gateway.sql, regenerated postgresdb.
FTS5 becomes a STORED generated tsvector column over (sender, text) plus
GIN. plainto_tsquery is the tsvector analog of ftsMatchQuery's
quote-escaping: query text is parsed as plain text, never query syntax, and
the surviving terms are ANDed. MessagePage.Truncated stays always-false on
Postgres too: the result set is counted exactly.
On internal/domain/messaging/session_store.go: this is a comment-only edit
to the MessagePage.Truncated doc comment (the brief's own "change the
comment" instruction). It changes no type, method or value, so L2 and the
import lane still build against an unchanged contract and the wire is
unaffected. No interface widened or altered.
Semantics preserved verbatim:
- The timestamp clamp is SQL-atomic (INSERT ... SELECT MAX(ts)+1). Appends to
one session are serialised by a transaction-scoped advisory lock keyed on
the session so two concurrent appends cannot read the same MAX(ts); a Go
clamp would race (proven by the new cross-handle monotonicity test).
- Session ordering is GREATEST(last_active_at, created_at) with a matching
expression index.
- ClaimTurn is insert-first (ON CONFLICT DO NOTHING), with a race test across
two handles.
State outside the store is NOT made shared by this change, and none of it is
silently improved or worsened:
- sessionTracker.active, Router.titling and per-session persona selection are
in-memory, per-process caches; a second Gateway still diverges from the
first.
- messaging.Turns lanes and the per-process recovery latch stay per-process.
- RecoverTurns still fails every turn owned by another process, so a second
Gateway starting would still fail the first's in-flight turns. This lane
does NOT make that safe and does not try to: the safeguard is D4.1's
single-Gateway advisory serve-ownership lock (not implemented here), which
prevents a second Gateway from serving at all. The turn-recovery semantics
are left exactly as they were.
Production openers still return SQLite; this lane adds the adapter and tests
only. The cutover and the state-sharing questions remain follow-up work.
goose migration 0001 lands the twelve State Store tables on Postgres, and sqlc generates the task queries from it. Two behaviours that genuinely differ from SQLite are pinned by tests against a real PG 18: claiming takes the oldest queued row with FOR UPDATE SKIP LOCKED (without the clause a second claimer blocks, which the test shows by timing out), and every column behind a Go int is bigint -- SQLite's INTEGER is 64-bit, and the chat issue-number allocator seeds at 10^15, which does not fit an int4.
task db:generate writes the committed postgresdb package; task db:check regenerates it in the gate, as proto:check and ui:check do.
The remaining store methods (events, resources, identities, the status tables) still run on SQLite: porting them is the rest of archie-core-7r0x.2.
stateadmin served read-only PocketBase views over the SQLite task file on -admin-addr. PocketBase is SQLite-only and the State Store moves to Postgres (archie-core-7r0x), so the view goes; operators use a Postgres tool of their choice.
internal/infrastructure/postgres opens a pgx pool, checks server_version_num >= 180000, and applies goose migrations with a Postgres session advisory lock so processes starting together apply each migration once. internal/infrastructure/postgres/pgtest starts postgres:18 through testcontainers per test binary and gives each test its own database. Adds jackc/pgx/v5 v5.11.0, pressly/goose/v3 v3.28.0 and testcontainers-go v0.44.0. The database URL config, sqlc, the Compose service and the ownership lock move to archie-core-7r0x.2, where they have consumers.
task test runs -short with the test cache on; task test:full keeps the
uncached run. The 50s keepalive proof skips under -short (the deploy
workflow's full go test still runs it). Sleep-bound archied watch tests
run in parallel, and the read-only gate steps run concurrently after fmt
and proto:check. ui:deps is run: once so the parallel UI steps install once.
Nothing moved a binding out of draft except an unrelated edit, so a newly created binding could not be approved and the dashboard showed no Approve button. Draft is removed; existing draft rows are promoted to pending_approval when the EDA store opens.
Replaces sourcegraph/jsonrpc2 with the go.lsp.dev stack (protocol, jsonrpc2, uri). The exit notification now ends Serve; closing the connection from inside the exit handler deadlocked.
Domain fields ParkClass/RemediationRounds landed in fe2a8e6d; this is
the wire and daemon half. Task gains park_class (32) and
remediation_rounds (33) and maps both directions; the classified park
write crosses the wire as ParkTask on a narrow TaskParker contract
(TaskLifecycle stays under the 8-method cap), with conformance
coverage local + grpc including the stale-guard sentinel and
needs_human normalization. Every daemon park site now states its
class: terminal for a repo that left config and a retired identity,
transient for the environmental parks (worktree, storage, grants,
taskrun), and identityMayAct returns the class beside its reason.
fe2a8e6d added Task.ParkClass and Task.RemediationRounds to the domain
without the wire half; the parity guard went red. Add the two proto
fields (32/33) and map both directions in values.go.
The F2+F3 schema wave from the auto-fix/resume investigation, sequenced
together as one migration. Every failure funneled to one parked status
with free-text ParkReason, and Task.RetryCount doubled as the
operator-retry budget and the review-remediation round budget -- so N
operator retries ate the remediation budget, a repo-level round cap
could re-park a task an operator had just legally retried, and no
surface could answer what a park needed from an operator.
Park class (F2): a producer-recorded vocabulary in taskstate --
needs_human (the default, so an unclassified park always reads as
operator-actionable), transient (environmental; a requeue can succeed),
terminal (retrying is pointless). Recorded at the park site, never
inferred from reason text; NormalizeParkClass makes unknown values
persist as needs_human. Transition-to-parked defaults the class, so
every existing unclassified park site reads correctly with no call-site
churn; the classified ParkTask write follows for parks that know
better. Requeue, RetryTask and BeginRemediation clear the class back to
the default with the reason.
Split budgets (F3): Task.RemediationRounds is the remediation's own
budget; RetryCount stays the operator's. The round-cap stage counts and
caps rounds; operator retry keeps counting RetryCount and can drive a
task to dead without touching rounds. remediation_rounds is a
presence-gated migrator column written through Update.
Known state: proto fields, the ParkTask RPC, the staterpc mapping and
the daemon-side park-site classification follow in the wire half of
this wave (the proto parity test is intentionally red until then).