Restore claims every service role (state-store and gateway) non-blockingly
and refuses if any is held or the claim cannot be checked; it drops tables a
later migration added before pg_restore --clean so a restored schema matches
the snapshot. Backup validates the archive before replacing the previous one.
Ports the store's write-side clip, park-class and transition-audit coverage
from internal/store.
Closes#865. The dashboard hand-copied the Change* file-status ids and events.ConfigCapturedSchema. /api/task-meta now serves both; the Vue dashboard renders ui/src/lib/task-meta-snapshot.ts until the served catalog lands. The Go handler is pinned to internal/webui/testdata/task_meta.json and the snapshot is pinned to that same fixture, so a one-sided rename fails a test.
Wave 1 of the Postgres migration (archie-core-7r0x.9). This is the shared shape
every port lane builds on; nothing here switches a live path to Postgres yet.
- config: add `database_url` beside `db_path` (config.go), mark it bootstrap-only
in denied_keys, expose it in the webui StorageView/config schema, and add it to
config.example.toml plus all deployments/*.toml.
- docker-compose: add a postgres:18 service; `task dev` (dev:postgres) brings it
up and waits for it before the State Store boots.
- archied: `openStateStore` now opens and migrates the process-scoped Postgres
pool via `openStateStorePool`. The State Store fails closed (D1.2) when
database_url is absent or invalid rather than fall back to SQLite; serving
against Postgres later requires an import-completion record (D1.3), which is a
later lane's gate, not this one.
- composition seam: task-store construction (state_store_task.go) and EDA-store
construction (state_store_eda.go) are extracted from openStateStore, with the
pool open in state_store_pool.go. This is a behaviour-preserving refactor only
— no store, interface, error, ordering or logging change — so L1 and L2 build
against separate files instead of sharing state_store.go.
Tests: fail-closed pool behaviour is pinned (absent/invalid URL); the State Store
boot/restart tests now run against a throwaway Postgres 18 container (pgtest
TestMain). Scoped test command and module-wide build pass; see lane report.
CHECKPOINT -- the full State Store port is in progress; this lands the
compile-clean core so the review pass has a diff to work against. Remaining:
advisory-lock ownership, event-insert serialization, maintenance, and the
ported test battery.
Two corrections to behaviour that already shipped, not new features:
- UpdateTask wrote park_reason raw where SQLite applies clip(park_reason,
4000); the PostgreSQL adapter now clips before binding (clip.go).
- tasks.review_cursor existed on the protobuf wire and in workflow.Task but
not in the PostgreSQL schema; 0004 adds it as a NEW migration rather than
amending 0001 because a dev database may already have applied 0001 and
goose tracks version numbers, not checksums.
Also moves the control-plane Resource/ResourceWrite types and their sentinels
into storecontract so the postgres package can stop importing internal/store
(they stay aliased there for daemon callers).
L2 of the PocketBase->Postgres migration (bead archie-core-7r0x.3): a new
Postgres-backed EDA adapter (postgres.EDA) plus its goose migration
(0002_edastore.sql) and sqlc queries (queries/edastore.sql), satisfying the
same storecontract surfaces as internal/infrastructure/edastore.Store so the
cutover wave can swap them behind the contract. The PocketBase opener in
internal/app/archied/state_store.go is untouched: production still opens
SQLite, and the cutover waits for the one-time import.
Three recon-found hazards, each pinned by a test, not a comment:
1. Empty-secret early return. applyBinding deliberately never writes the
secret column when the incoming secret is empty, so an edit form that does
not echo the secret must not blank an armed binding's HMAC key. The
UpdateBinding query keeps the stored secret via
`secret = CASE WHEN $secret = '' THEN secret ELSE $secret END`; pinned by
TestUpdateBindingEmptySecretPreservesStored (raw column unchanged across an
empty-secret edit).
2. Unique-violation detection by SQLSTATE. isUniqueViolation previously grepped
the driver's error text for "unique"/"already exists"; Postgres reports
SQLSTATE 23505 and the text match would not fire, losing ErrAlreadyDispatched.
The adapter detects by pgconn.PgError.Code == "23505"; pinned by
TestIsUniqueViolationDetectsBySQLState (23505 with a message lacking those
words is detected; a non-23505 whose text says "unique" is not).
3. One-binding-per-source / one-armed-binding-per-source were read-then-refuse
with no transaction and a non-unique idx_bindings_source_status. Made them
constraints: UNIQUE(source) plus a partial UNIQUE(source) WHERE
status='armed' (the full unique subsumes the armed rule today, kept explicit
for a future matcher extension). A concurrent insert for one source yields
exactly one success and one ErrBindingOverlap; pinned by
TestConcurrentInsertBindingSameSource.
Dispatch ledgers kept exactly: UNIQUE(binding, capture) and
UNIQUE(playbook_id, playbook_version, event_id, action_id); binding_version is
stored but deliberately not part of the key (pinned by
TestBindingDispatchVersionBumpDoesNotPermitRedispatch). The cipher is
unchanged: AES-256-GCM, key = SHA-256(material), decrypt stays in the store and
the gRPC wire still carries plaintext.
Notify is left as an injected callback (EDA.SetNotify) and is NOT wired to
LISTEN/NOTIFY. It is dead code today: edastore.Config.Notify is omitted at its
only production construction and no consumer maps binding_changed/
mapping_changed, so building a LISTEN/NOTIFY path would be building for a
consumer that does not exist. Wired or not is a separate decision.
Move sessions, messages, turns and full-text search off the per-process
SQLite file onto Postgres behind the existing messaging.SessionStore
contract. New adapter internal/gateway/session_store_postgres.go, sqlc
queries/gateway.sql, migration 0003_gateway.sql, regenerated postgresdb.
FTS5 becomes a STORED generated tsvector column over (sender, text) plus
GIN. plainto_tsquery is the tsvector analog of ftsMatchQuery's
quote-escaping: query text is parsed as plain text, never query syntax, and
the surviving terms are ANDed. MessagePage.Truncated stays always-false on
Postgres too: the result set is counted exactly.
On internal/domain/messaging/session_store.go: this is a comment-only edit
to the MessagePage.Truncated doc comment (the brief's own "change the
comment" instruction). It changes no type, method or value, so L2 and the
import lane still build against an unchanged contract and the wire is
unaffected. No interface widened or altered.
Semantics preserved verbatim:
- The timestamp clamp is SQL-atomic (INSERT ... SELECT MAX(ts)+1). Appends to
one session are serialised by a transaction-scoped advisory lock keyed on
the session so two concurrent appends cannot read the same MAX(ts); a Go
clamp would race (proven by the new cross-handle monotonicity test).
- Session ordering is GREATEST(last_active_at, created_at) with a matching
expression index.
- ClaimTurn is insert-first (ON CONFLICT DO NOTHING), with a race test across
two handles.
State outside the store is NOT made shared by this change, and none of it is
silently improved or worsened:
- sessionTracker.active, Router.titling and per-session persona selection are
in-memory, per-process caches; a second Gateway still diverges from the
first.
- messaging.Turns lanes and the per-process recovery latch stay per-process.
- RecoverTurns still fails every turn owned by another process, so a second
Gateway starting would still fail the first's in-flight turns. This lane
does NOT make that safe and does not try to: the safeguard is D4.1's
single-Gateway advisory serve-ownership lock (not implemented here), which
prevents a second Gateway from serving at all. The turn-recovery semantics
are left exactly as they were.
Production openers still return SQLite; this lane adds the adapter and tests
only. The cutover and the state-sharing questions remain follow-up work.
goose migration 0001 lands the twelve State Store tables on Postgres, and sqlc generates the task queries from it. Two behaviours that genuinely differ from SQLite are pinned by tests against a real PG 18: claiming takes the oldest queued row with FOR UPDATE SKIP LOCKED (without the clause a second claimer blocks, which the test shows by timing out), and every column behind a Go int is bigint -- SQLite's INTEGER is 64-bit, and the chat issue-number allocator seeds at 10^15, which does not fit an int4.
task db:generate writes the committed postgresdb package; task db:check regenerates it in the gate, as proto:check and ui:check do.
The remaining store methods (events, resources, identities, the status tables) still run on SQLite: porting them is the rest of archie-core-7r0x.2.
stateadmin served read-only PocketBase views over the SQLite task file on -admin-addr. PocketBase is SQLite-only and the State Store moves to Postgres (archie-core-7r0x), so the view goes; operators use a Postgres tool of their choice.
internal/infrastructure/postgres opens a pgx pool, checks server_version_num >= 180000, and applies goose migrations with a Postgres session advisory lock so processes starting together apply each migration once. internal/infrastructure/postgres/pgtest starts postgres:18 through testcontainers per test binary and gives each test its own database. Adds jackc/pgx/v5 v5.11.0, pressly/goose/v3 v3.28.0 and testcontainers-go v0.44.0. The database URL config, sqlc, the Compose service and the ownership lock move to archie-core-7r0x.2, where they have consumers.
task test runs -short with the test cache on; task test:full keeps the
uncached run. The 50s keepalive proof skips under -short (the deploy
workflow's full go test still runs it). Sleep-bound archied watch tests
run in parallel, and the read-only gate steps run concurrently after fmt
and proto:check. ui:deps is run: once so the parallel UI steps install once.
Nothing moved a binding out of draft except an unrelated edit, so a newly created binding could not be approved and the dashboard showed no Approve button. Draft is removed; existing draft rows are promoted to pending_approval when the EDA store opens.
Replaces sourcegraph/jsonrpc2 with the go.lsp.dev stack (protocol, jsonrpc2, uri). The exit notification now ends Serve; closing the connection from inside the exit handler deadlocked.
Domain fields ParkClass/RemediationRounds landed in d6f61c2c; this is
the wire and daemon half. Task gains park_class (32) and
remediation_rounds (33) and maps both directions; the classified park
write crosses the wire as ParkTask on a narrow TaskParker contract
(TaskLifecycle stays under the 8-method cap), with conformance
coverage local + grpc including the stale-guard sentinel and
needs_human normalization. Every daemon park site now states its
class: terminal for a repo that left config and a retired identity,
transient for the environmental parks (worktree, storage, grants,
taskrun), and identityMayAct returns the class beside its reason.
d6f61c2c added Task.ParkClass and Task.RemediationRounds to the domain
without the wire half; the parity guard went red. Add the two proto
fields (32/33) and map both directions in values.go.
The F2+F3 schema wave from the auto-fix/resume investigation, sequenced
together as one migration. Every failure funneled to one parked status
with free-text ParkReason, and Task.RetryCount doubled as the
operator-retry budget and the review-remediation round budget -- so N
operator retries ate the remediation budget, a repo-level round cap
could re-park a task an operator had just legally retried, and no
surface could answer what a park needed from an operator.
Park class (F2): a producer-recorded vocabulary in taskstate --
needs_human (the default, so an unclassified park always reads as
operator-actionable), transient (environmental; a requeue can succeed),
terminal (retrying is pointless). Recorded at the park site, never
inferred from reason text; NormalizeParkClass makes unknown values
persist as needs_human. Transition-to-parked defaults the class, so
every existing unclassified park site reads correctly with no call-site
churn; the classified ParkTask write follows for parks that know
better. Requeue, RetryTask and BeginRemediation clear the class back to
the default with the reason.
Split budgets (F3): Task.RemediationRounds is the remediation's own
budget; RetryCount stays the operator's. The round-cap stage counts and
caps rounds; operator retry keeps counting RetryCount and can drive a
task to dead without touching rounds. remediation_rounds is a
presence-gated migrator column written through Update.
Known state: proto fields, the ParkTask RPC, the staterpc mapping and
the daemon-side park-site classification follow in the wire half of
this wave (the proto parity test is intentionally red until then).
Sequencing steps 6 and 7 of pr-review-remediation.md: the consumer
landed in 052e350b, and these are the two sources that feed it.
Webhook (decision 2, low latency): the GitHub receiver decodes
pull_request_review and pull_request_review_comment events behind the
existing HMAC verification and publishes the same typed reaction the
poller produces, keyed source-independently so PublishUnique collapses
the two sources. Only submitted/created carry a verdict to remediate;
edited and dismissed are acknowledged. Reaction deliveries are
rate-limited per source (decision 6's 'add when the receiver gains a
second event family'), and the reaction path carries no dispatch
predicate -- eligibility is the consumer's owned-PR lookup.
Poll (both forges, the no-webhook backstop): one shared scan over all
pr_open tasks, each with the forge client its own identity owns
(reconcilePRs's shape), listing reviews and comments since per-task
persisted high-water marks and skipping the owning identity's bot user
(decision 5's own-comment exclusion, which the poll is positioned to
make identity-correct).
The cursors are two, not one: review IDs and comment IDs are independent
forge sequences, and a combined high-water mark would silently skip
reviews once a larger comment ID landed. watch_comment_id -- dead since
the comment-watch design it was created for never shipped -- carries the
comment cursor; review_cursor is a new presence-gated migrator column,
and SetReviewCursors is its guarded RPC (pr_open only, so cursors never
move under a run that owns the task). forge.ReviewComment gains ReviewID
so a comment is collectable into its review's unit (decision 5) on both
forges, and forge.Review gains Body so poll-delivered reviews carry
their summary.
Note: the store/proto/facade half of this work landed early in a65eae24
under a release session's message -- a shared-checkout collision, kept
rather than rewritten; that commit carries the review_cursor migration
and the SetReviewCursors contract, this one carries the producers.
Beads sync to a Dolt remote, so anything written into one is published.
Two beads filed earlier today named an operator host, its internal
address and its filesystem paths -- the tracker was treated as a private
store, which it is not.
The rule is stated where the tracker is described, next to the existing
one about .beads/ never reaching a commit, because it is the same class:
the tracker is not a private store. Host-specific facts belong in the
operator's memos instance, which a bead may point at.
The remediate workflow was registered but had no trigger: the reaction
stream was provisioned, the envelope, resolver and owned-PR lookup all
landed, yet nothing consumed the stream (pr-review-remediation.md's
sequencing left the consumer for this step). The daemon now drains the
reaction stream each cycle, before the task drain, so a reaction can be
claimed in the same pass that carried it.
Per decision 3, the store's owned-PR lookup is the authorization
boundary: an unmatched, merged or closed PR drops with a counter, and a
public reaction can never start an agent against a foreign repository.
Per decision 5, a review is the unit of remediation -- its comment
events are appended to the unit while it is queued and dropped once the
run is claimed (the round cap bounds the exchange); a standalone comment
is its own unit of one; approved reviews never remediate; and archie's
own comments, matched on the owning identity's bot user, are excluded so
its replies cannot retrigger it.
The guarded BeginRemediation/UpdateReviewPayload store methods are the
dedup: the pr_open -> queued transition carries the review unit in the
same write, so re-delivered reactions fail the status guard instead of
starting a second round. Both are new StateStoreService RPCs following
the Requeue shape, covered by the local/grpc conformance battery.
task check's fmt pass rewrites both, so the committed tree was not
fmt-canonical. Adopted verbatim, per the repository's rule that formatting
changes from the configured formatter are law.
Lane D. The collection existed in the event-capture schema with nothing
writing it; tool calls reached only the on-disk task log, readable as text
and never queryable.
The projection is a decorator on the State Store's task store, wired in
stateStoreDeps. InsertEvent persists the event first and only then, for a
tool_call event, writes one row. A projection failure is counted and logged
but never returned: event persistence is load-bearing, the transcript is
observability, and failing a task write because its shadow failed trades
the fact for the record of it. With no event-capture store the task store
passes through unwrapped.
The event adapts to the existing collection rather than the collection to
the event; args and duration_ms stay reserved. InsertToolCall stays silent
on the notify hook, matching the dispatch tables rather than the
operator-editable ones -- tool calls are high-frequency telemetry, not
dashboard activity.
The tool_calls collection existed in the event-capture schema but nothing
wrote it. Give it its write path:
- edastore gains InsertToolCall and the minimal TaskToolCalls read, writing
only the existing collection. No new index: the collection is an
append-only transcript and the SQLite predecessor had no unique index.
- internal/app/archied composes toolCallProjectingTaskStore, a decorator on
the task store that persists the event first, then -- only for tool_call
events -- writes one row. A projection failure is counted and logged but
never returned from InsertEvent: event persistence is load-bearing,
tool_calls is observability.
- stateStoreDeps wires the decorator around deps.Tasks when both stores are
open, and passes Tasks through untouched otherwise.
The event adapts to the existing schema, not the other way round: the tool
name comes from data, a succeeded call's detail is the row's result, a
failed call's detail its error; args and duration_ms stay reserved for a
producer that reports them.
Implements docs/prds/artifact-sender.md (Approved). archie-core now sends
generated artifacts to the offloaded.dev collaborative editor, whose
POST /api/artifacts route, bearer service token, PocketBase-backed
versions and idempotency semantics are the receiving contract.
- [artifacts] config section: base_url + token SecretRef with a
token_env fallback naming WORKSPACE_INGEST_TOKEN. An empty base_url
disables the sender entirely; a base_url with an unresolvable token
disables it with a warning, because a missing credential disables a
capability rather than stopping the daemon.
- internal/infrastructure/artifactsync: one-method Publish seam in the
Courier posture, full source per publish (versions of record are
snapshots), 503 retried three times with backoff, 401/400 surfaced
immediately with the field reason, deterministic request ids
(artifact:pr-review:<owner>/<repo>#<n>).
- ReviewReport gains a pure markdown renderer in internal/domain/workflow:
summary, one subsection per finding with the failure scenario and
disposition when present, verified properties as a checked-clean list.
- The daemon's PR reviewer publishes after a completed review when the
sender is configured, records the editor deep link on the result, and
degrades a failed publish to a log line -- publishing never fails the
producing work.
The cursor change renamed the field to 'cursor' as well as widening its
type, which buf breaking flags as FIELD_SAME_NAME and FIELD_SAME_JSON_NAME
on top of the type change. The name buys nothing: the field is already
documented as an opaque cursor, and the number and name are the wire
identity. Adopt the gate's verdict and change only the type, keeping the
break confined to the one rule already recorded in proto/buf.yaml.
Lane A. EventsSince no longer relies on events.id being monotonic: it
orders by (at, id), where at is written in a fixed-width layout so
lexicographic order is chronological order, and id is only a
deterministic tie-break. The cursor crosses the wire as an opaque string
and rides the SSE id: field, so EventSource's own Last-Event-ID resume
keeps working untouched. A malformed or legacy cursor degrades to
"from the beginning" rather than failing the feed.
events.id keeps its int64 type and row-identity meaning; no
push/subscribe RPC is added.
EventsSince no longer depends on events.id being monotonic. The feed now
pages a total order over (at, id): at is a fixed-width, lexicographically
sortable UTC timestamp (storecontract.EventCursorLayout) and id is only a
deterministic tie-break for rows sharing one timestamp. The cursor is an
opaque string on the wire (EventsSinceRequest.cursor), carried verbatim as
the SSE id: field and echoed back via Last-Event-ID, so EventSource resume
needs no client change.
A new idempotent migration normalises pre-existing RFC3339Nano at values to
the fixed-width layout before finishTaskMigration, and idx_events_at backs
the new query. Legacy integer and garbage cursors degrade to "from the
beginning", preserving the old failed-ParseInt-yields-0 behaviour.
Amends the "Dashboard live event delivery" decision's cursor-cannot-skip
justification, which previously rested on commit-order id assignment.
Lane C. A successful binding or mapping write on the State Store's
PocketBase tables now emits one activity event on the same task-DB path
capture arrival already uses, so a dashboard edit is visible on the
activity stream without a manual refresh.
The event is an invalidation signal, not a payload: it carries the record
id and the action, and consumers refetch through the authenticated API. A
nil notifier is a silent no-op, so stores composed without one (tests,
tooling) are unchanged.
Each successful write on the State Store's PocketBase bindings/mappings
tables emits one event (binding_changed / mapping_changed) through a
nil-safe Notify func injected at composition, so a dashboard edit is
visible without a manual refresh -- the same task-DB event path capture
arrival uses. Nil notifier keeps every existing constructor silent.
tools/cron ANDs the two day fields where Vixie cron ORs them, so
0 0 1 * 1 means Monday the 1st, not the 1st or any Monday. State the
deviation in the ScheduleCron comment and pin it with a test.