The event-capture tables (captures, mappings, bindings, binding_dispatches,
playbook_dispatches) now live in PocketBase-owned collections and their ids
are opaque record ids rather than autoincrement integers. The task lifecycle
stays on SQLite: tasks/transitions/events carry FIFO claim ordering, id-cursor
pagination and the task-grant map, none of which gain anything from being
hand-editable. binding_dispatches.task_id is the single seam and stays an
integer addressing that store.
BREAKING: CapturedEvent.Id, Mapping.Id, Binding.Id and Binding.MappingId are
strings on the wire, as are the ids in the capture/mapping/binding requests.
RecordDispatchRequest.task_id is deliberately unchanged.
What the collections buy, beyond storage: a payload nobody has modelled yet is
inspectable, a mapping is editable at runtime, and a binding's relation to its
mapping is enforced. The old schema declared no FOREIGN KEY on that link, so a
dangling mapping_id was previously possible and is now refused.
Behaviour preserved deliberately, each recovered from the SQLite version
rather than reimplemented from memory:
- draft -> pending_approval -> armed, with approval only from
pending_approval and an edit always returning a binding for re-approval
- one binding per source, guarded on insert, update and approve
- an empty secret on update keeps the stored one, so an edit form that does
not echo it back cannot silently disarm a binding
- ErrMappingNotFound / ErrBindingNotFound on a missing delete target
- sentinels are re-exported from storecontract, never redefined: staterpc
matches on exact canonical message to rehydrate them, so a second
definition would break errors.Is across the wire invisibly
Two implementation notes worth keeping:
- at-most-once dispatch is now a unique index rather than INSERT OR IGNORE.
Only the unique-violation maps to ErrAlreadyDispatched; any other save
error is returned, because treating them all as "already recorded" would
turn the ledger into a silent at-least-once dispatcher.
- captures store received_at themselves at nanosecond precision in a
fixed-width layout. PocketBase autodates are millisecond and record ids
are random, so same-millisecond captures would otherwise order
arbitrarily; the old store got insertion order free from autoincrement.
- ListUndispatchedCaptures is raw SQL: the anti-join is a subquery and
PocketBase's filter DSL is not SQL. A read fires no hooks either way.
Adds tool_calls, which had no persistence at all: tool calls only ever reached
the on-disk task log, readable as text but never queryable or watchable.
Nothing writes it yet.
The binding secret cipher moves with the bindings it protects.
archie-core-8cda.6.8's carrier. The service now records channel lifecycle and has
nowhere to put it: /api/channels answered with nothing because
webui.Server.Channels had no production source, and reload answered 501 for the
same reason. This adds the surface the register decided on, following the split
archie-core-ymut established for config snapshots -- a pair of administrative RPCs
whose writer is one process and whose reader is the UI -- and wires both ends.
Wire and store: PutChannelStatus / ListChannelStatus, with ChannelStatus carrying
id, name, state, detail, configured, reload_supported and observed_at. The store
holds one row per channel and replaces the reported set, so a channel the reporter
no longer mentions is deleted rather than left standing as a stale "running"; an
empty report clears the table, because hosting no channels is a fact worth
reporting. A row without an id is refused: it could never be replaced or removed
by a later report.
Writer: the service publishes on every lifecycle transition through a coalescing
signal -- a burst of transitions is one write, since a reader only wants the latest
state of every channel -- plus a start-up report and a final report after shutdown,
so a stopped service leaves the store honest. A failed write is logged and retried
on the next signal; the dashboard losing channel state is worth a warning and never
worth failing a channel start over.
Reader: webui.Server.Channels becomes a source rather than a concrete manager, and
webui.RemoteChannelStatus reads the store the way RemoteConfigView reads the config
snapshot. A store it cannot reach reports nothing rather than a stale "running",
and a state the reader does not recognise is carried through rather than dropped.
The contract's own index gains the group in the same change that adds the RPCs --
a surface the grouped table does not list is one nobody finds -- and the count moves
to 51. The register records why the surface is the State Store rather than a
resource kind, and why reload gets its own surface rather than a field here: reading
status must not be able to retrigger an action, a transient request is not
distinguishable from state once anything polls it, and it would invert the
writer/reader relationship the precedent establishes.
Verified: task check exit 0. New coverage: the store's replace-the-set semantics
and its refusal of an id-less row, the reader's mapping including an unknown state,
and the publisher's start-up, transition and final reports.
Add the playbook_dispatches at-most-once ledger (docs/prds/eda-playbook-engine.md
gap 2) and the record-then-invoke coordinator helper, with the cross-process
plumbing the State Store boundary requires.
- proto: RecordPlaybookDispatch and DeletePlaybookDispatches RPCs (string keys)
- storecontract: PlaybookDispatcher interface (no *sql.Tx, reuses ErrAlreadyDispatched)
- store: playbook_dispatches table + INSERT OR IGNORE dedup + delete-by-playbook
- staterpc: client/server methods and Deps field; error sentinel survives the hop
- eda/playbook: InvokeOnce record-before-invoke helper, keyed on
(playbook, version, event, action) with a "1" action-position fallback
The ledger is administrative-only: task-scoped grants never reach it, matching
TaskGrants' Update/Transition/InsertEvent allowlist. Not wired into the daemon
workflow pin; the production call site arrives with the first side-effecting
position.
Adds PutApplyStatus/ListApplyStatus so each process can report which version
of a control-plane resource it is running, and the UI can read it back.
docs/prds/control-plane-apply-status.md is the approved design.
It goes on the State Store contract rather than the control plane, whose
service is for editing a resource: a status report is not an edit. The
config-snapshot pair is the precedent, and the deny-by-default arm of
authorizesTaskScopedCall already refuses both new RPCs to a task grant, which
TestOnlyAdminReportsApplyStatus pins.
The row is keyed on (process, kind) and replaced in place, because a process
re-stamps on an interval and nothing reads a history of re-stamps. An error
leaves applied_version alone, so a rejected edit never erases the fact that an
older version is live. The read is a list and an empty one is not an error,
unlike the config snapshot's found=false singleton.
Contract PRD inventory and RPC count follow in the same series.
Relates to archie-core-pskb.
The dashboard's attempt-log panel answered every read with "task logging is
optional and was not enabled for this run". That message was false: logging is
wired unconditionally in the daemon, and the log files exist. The dashboard
process simply had no reader -- composition left TaskLogs absent because no
contract carried it -- so the handler saw a nil registry and reported the
disabled shape, which the page rendered as a claim about configuration.
internal/logging now answers whether an attempt has a log at all: Tail sets
Result.Found, so "this attempt has no log" is a value rather than an empty
list indistinguishable from a read that never happened. Its registry gains
TaskLog (a decoded page) and TaskLogContent (the file verbatim, streamed,
because a log is unbounded input), and an absent reader reports
ErrTaskLogsUnavailable rather than an empty page -- a process that cannot read
logs must not implicate the attempt.
Those two answers travel the State Store contract as storecontract.TaskLogStore
(ReadTaskLog, StreamTaskLogContent) and are rehydrated on the client by
(Unavailable, canonical message), so archie-ui composes the reader from its own
store client instead of opening another process's files. The daemon's registry
is wired as the server-side reader in the State Store process only.
On top of that read, GET /api/tasks/{id}/logs/download serves the attempt's log
as a Content-Disposition attachment named task-<id>-attempt-<n>.log, and the
panel offers it. The JSON route keeps its shape and gains found, so the page
states the attempt has no log instead of blaming a setting.
Task log files stay owned by their directory's owner: archiving still removes a
task's log directory, and that removal is the daemon's, not the dashboard's.
Completes the objective deletion gate for the UI Service
(archie-core-8cda.5.6): go list -deps ./cmd/archie-ui now links zero
banned packages -- no modernc.org/sqlite, no internal/store, gateway,
channels, memory, secret, agentexec, tools, skill, yaegiutil, curator,
or the workflow engine. The closure shrinks from 68 to 26 archie-core
packages. Each cut is a producer-side move; no consumer-side defensive
shims.
- Store contracts (TaskStore family, value types, sentinels) moved from
internal/store to internal/domain/storecontract -- the approved contract
location per state-store-contract rev. 2d, and NOT internal/contracts:
that tree is buf generate output and tools/proto.sh check deletes
hand-written files there. internal/store keeps aliases; sentinel
message strings are byte-identical and remain the wire contract.
- Chat contracts (ChatContract, sessions, turns, streaming, commands)
moved from internal/gateway to internal/domain/messaging; gateway keeps
aliases. ToolCallEvent rendering helpers become gateway functions
(methods cannot live on alias types).
- Task vocabulary (Task, statuses, mid-run Store, Definition) split into
internal/domain/workflow/task; the pipeline engine stays in package
workflow with aliases, so daemon-side callers are unchanged.
- overlay.DeniedKeys moved to internal/infrastructure/configuration:
webui referenced the constant and that dragged the overlay SQLite
store in.
- webui.Server's concrete runtime handles become webui-owned view
interfaces with daemon-side bootstrap adapters: channel lifecycle
(internal/channels/status), MemoryStatus, CuratorStatus, and a
SkillCatalog seam replacing the direct skill.DefaultRoots/CatalogRoots
calls. Each degrades to an empty page when unwired.
- config.SecretRef moved into internal/config (config vocabulary);
internal/secret keeps an alias and Registry.Resolve absorbs the
zero-value short-circuit.
Docs: state-store-contract rev. 2d relocation record; ui-service-boundary
deletion gate marked satisfied; ARCHITECTURE.md package map rows.
The deletion gate for the UI Service (archie-core-8cda.5.6) is objective:
go list -deps from cmd/archie-ui must show no SQL/store implementation.
internal/store's hand-written contract surfaces (TaskStore and its
component interfaces, CaptureStore, MappingStore, BindingStore,
BindingDispatcher, BindingTaskCreator, ConfigSnapshotStore) move to
internal/contracts/store/v1 (package storev1), alongside the value types
they carry (CapturedEvent, ConfigSnapshot, WorkflowStat, StageStat,
DayTokens) and the sentinel errors -- whose canonical message strings are
the wire contract and are byte-identical after the move.
internal/store keeps type aliases, so daemon-side callers (archied, the
daemon, staterpc's server side, taskactions) are unaffected. The
UI-facing packages switch to storev1 directly: webui, archieui,
captureintake, gatewayrpc, and staterpc's client/values -- which drops
internal/store and modernc.org/sqlite from the archie-ui binary's import
closure.
overlay.DeniedKeys moves to internal/infrastructure/configuration for the
same reason: webui referenced the constant and that dragged the overlay
package's SQLite store in.
The chat-contract and remaining runtime-handle decoupling continues under
the same bead.
archie-core-ymut decided the configuration page reads a projection the daemon
publishes rather than reaching into a shared config.Holder. This is the
carrier: a single-row store surface plus PutConfigSnapshot/GetConfigSnapshot,
taking the contract to 42 RPCs.
The document is opaque here. Its shape belongs to the daemon that builds it
and the page that renders it, so the store neither parses nor validates it,
and schema names the projection so a reader can refuse one it does not
understand. It is secret-free where it is built: the projection carries an API
key's environment variable name and whether it resolved, never a value.
Both RPCs are administrative. authorizesTaskScopedCall is deny-by-default, so
a container's task-scoped grant can neither rewrite what an operator reads nor
read a projection of the deployment it runs inside; a test pins both.
Nothing publishes or reads it yet -- the daemon's publisher and the UI's
read path land with the rest of the cutover.
* feat(state-store): generalize storerpc as the State Store gRPC contract
Implements archie-core-8cda.4.2 against docs/prds/state-store-contract.md
rev. 2c: generate protobuf/gRPC client and server adapters for every
ratified store contract (proto/state/v1/state.proto, 40 RPCs, one
StateStoreService), add internal/infrastructure/staterpc with a bufconn
conformance suite exercising both local and remote adapters, serve the
service in-process (bridge address + mandatory per-task bearer token when
agent containers exist, loopback-only otherwise), and flip archie-agent's
workflow.Store to a long-lived gRPC client when the daemon injects
STATE_STORE_URL/STATE_STORE_TOKEN (falling back to the legacy NATS
storerpc path otherwise -- one agent process is never both).
Also drops the *sql.Tx parameter from store.BindingDispatcher.RecordDispatch
(a required wire-blocker fix: a transaction cannot cross a gRPC boundary,
and production always passed nil).
fix(store): drop RecordDispatch's unusable *sql.Tx parameter
feat(config): add services.state contract-adapter selector
* fix(state-store): apply no-mistakes review decisions
- Drop StreamTokenInterceptor (unused; the State Store service has no
streaming RPCs and nothing calls it).
- Make StateStoreTokens.Validate compare tokens in constant time
(crypto/subtle) per the auto-fix hardening.ing finding.
- Amend config.example.toml / config.Services.State doc text so it frames
[services.state].target as the ratified surface consumed by the upcoming
standalone archie-state-store binary (.4.3), not active today.
* refactor(workflow,store): relocate Task/Status/Source and define workflow.Store
Move store.Task, store.Status*/store.Source*, and the 3-method workflow
store interface out of internal/store into internal/domain/workflow as
workflow.Task/workflow.Status/workflow.Source plus a consumer-owned
workflow.Store, superseding store.WorkflowStore. internal/domain/workflow
now imports zero internal/store symbols; internal/store persists the
domain type instead (infrastructure depends on domain), and its Store
satisfies workflow.Store.
Updates every production and test caller of the relocated symbols across
daemon, webui, gateway, agentworker, agenttransport/nats, storerpc,
taskrun, and worktreerpc, and removes TestRecordDispatchViaExplicitTx per
docs/prds/state-store-contract.md (rev. 2c) sections 4 and 12 step 2.
Two workflow-package integration tests that exercise the real *store.Store
move to an external workflow_test package (store_integration_test.go) to
avoid a workflow(test)->store->workflow(prod) import cycle now that store
depends on workflow.
* no-mistakes(document): docs(state-store): mark §4.8 relocation done in state-store-contract
Implements archie-core-t2db.4: the runtime-editable binding entity that
turns a captured webhook into an archie task, plus the per-source HMAC
verification at handleCapture that makes t2db.5's 'unauthenticated event
cannot trigger a binding' invariant structurally enforceable.
Captures, mappings and bindings are now separate concerns on the same
single *store.Store; the new BindingStore / BindingDispatcher /
BindingTaskCreator split keeps each interface under the interfacebloat
limit (8 methods).
State machine: draft -> pending_approval -> armed; any edit on an armed
binding drops it back to pending_approval, enforced in SQL via
status='pending_approval' on UPDATE so the rule cannot be bypassed by a
direct status field in the request body. Approve is the only sanctioned
transition into armed (WHERE status='pending_approval' guard).
Overlap: two bindings may not share a source in any status; enforced at
InsertBinding and UpdateBinding (transaction-local SELECT). Approve has
a separate TOCTOU guard so a race between Insert and Approve cannot
// bypass the check.
HMAC at handleCapture: reads X-Hub-Signature-256 (then X-Signature-256)
// fallback, verifies via webhookguard.VerifyHMAC against the binding's
// per-source secret, marks CapturedEvent.Authenticated=true on match.
// Captures with no armed binding, missing signature, or wrong signature
all persist as authenticated=false -- the dispatch loop's
binding.Matcher.Matches gate (requires authenticated) is what stops
them from spawning a task. Two armed bindings for one source returns
409 at capture time as a belt-and-braces TOCTOU guard.
Dispatch loop: added Daemon.dispatchBindings, called from both Cycle
and maintainAndDrain between poll and drain. Walks
captured_events where authenticated=1 AND source IN armed bindings
AND id NOT IN binding_dispatches, runs the binding's matcher, resolves
the mapping, enqueues a chat-style task with binding_id /
// binding_version provenance stamped in the same transaction as the
// row insert. The binding_dispatches ledger (composite PK on
(binding_id, capture_id)) is the at-most-once guarantee across cycles
// and daemon restarts.
Tasks schema migration: binding_id INTEGER NOT NULL DEFAULT 0 +
binding_version INTEGER NOT NULL DEFAULT 0 (taskSchemaVersion bumped
1 -> 2). Existing tasks keep 0 / 0.
Out of scope (documented follow-ups):
- secret encryption at rest (plaintext today)
- full dashboard HTML for the binding editor (CRUD API only this bead)
- per-binding rate limits (existing per-remote-address limit applies)
- binding.Owner / binding.Repo fields (dispatch falls back to the
single configured repo; multi-repo deployments log and skip)
Task runs now preserve real model context limits, provider usage, baseline-repair spend, commit identity, durable agent-finish telemetry, and useful timeline details without imposing a token cap. Dashboard forge links are canonical per owning GitHub or Gitea identity. Chat turn claims acquire SQLite writer intent before reading, eliminating the concurrent SQLITE_BUSY snapshot race.
Adds t2db.3: schema-by-example field mapping for the no-code playbook epic.
An operator picks a captured event, clicks through its payload to bind named
fields to JSON paths, and previews the resolution against a real event
before saving -- a missing, null, or type-changed path fails loudly (named
by field and event) rather than handing an agent a blank value.
- internal/domain/mapping: Mapping/Field types, path parsing/walking,
Resolve (per-field failure reporting), Validate (incl. duplicate names)
- internal/store/mappings.go: SQLite persistence, mirrors captures.go
- internal/webui/api_mapping.go: CRUD + preview HTTP API
- ui/src/mappings/: dashboard editor (clickable payload tree, live preview)
Design decisions recorded in docs/prds/payload-field-mapping.md. t2db.4
(binding a mapping + matcher to a workflow) is out of scope here.
Adds POST /webhooks/capture/{source}: an unauthenticated endpoint (mounted
alongside /healthz, bypassing dashboard token auth by design) that persists
any inbound webhook archie has no binding for yet, so it can be inspected
and mapped later (t2db.2/t2db.3).
docs/prds/event-capture-storage.md decides SQLite over NATS JetStream for
storage: NATS is not unconditionally available until 7d5u.2 ships embedded
support, and capture's single-writer/single-reader access pattern doesn't
need JetStream's fan-out/replay machinery anyway.
internal/store/captures.go adds a captured_events table with prune-on-write
(retention age + row-count cap, both configurable). The HTTP handler applies
webhookguard redaction before persisting, and enforces two more independent
disk-bound mechanics before that: a per-request body-size cap and a
per-remote-address rate limit. New [capture] config section with documented
defaults; all fields optional, capture is on by default.
Adversarial review (manual, after the usual fresh-subagent pass hit a
session API limit mid-run) caught and fixed three real defects before this
landed, each with a regression test confirmed to fail against the pre-fix
code:
- the body-size cap silently no-op'd when CaptureMaxBodyBytes was left
unset, the one bound that limits a single payload's size rather than row
count or write rate
- the rate limiter was keyed by the request's own source path segment --
unregistered and attacker-chosen by design for this endpoint, violating
webhookguard.RateLimiter's documented bounded-key-space assumption and
letting a sender bypass the limit by rotating the segment; rekeyed to
remote address
- the retention prune compared time.RFC3339Nano strings, whose trimmed
variable-width fractional seconds break lexicographic ordering within the
same second; switched to fixed-width RFC3339
Closes archie-core-t2db.1.
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.
Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
Add cancel/stop/approve/reject/retry/abandon/archive to the dashboard task
action matrix, each guarded by an optimistic status check against the
store. Retry accounting (RetryCount, max_retries) is now atomic with the
status transition instead of a separate write. Archive removes a terminal
task's local record with a fail-safe audit event first. Add always-labelled
responsive navigation, mobile Jump-to search, and versioned Setup-complete
dismissal.
Fixes#53
- Reword docs and comments to drop vague "legacy" qualifiers where they no longer describe deprecation status
- Remove unused code: RecentEvents, MessageEvent.ToLegacy, index setState, stderr redaction tool, telegramOnlyCommands
- Delete or comment out tests covering removed or superseded behavior
Applies the four-reviewer pass over the forge/dashboard-actions work.
The through-line is that several capabilities were wired but inert, and
one of them could cost real work.
DUPLICATE WORK. "Closes #N" was dropped from PR bodies in favour of
LinkBranch, which is sidebar linkage on Gitea and a no-op stub on
GitHub -- neither closes anything, and nothing else did either. So a
merged task left its issue open, labelled and assigned; the next poll
re-enqueued it, and once the dashboard's Clear removed the task row that
became a second implementation and a second PR for work already merged.
reconcilePRs now closes the issue on merge, and a rejected task closes
its issue too. A PR closed without merging deliberately leaves the issue
open: that usually means "try again".
MAX RETRIES. Configured, defaulted and reported by /api/config, and
enforced nowhere -- markDead was unreachable and EffectiveMaxRetries had
no callers, so the dashboard's retry incremented retry_count forever.
The retry action now retires a task to dead at the cap.
DEAD ON ARRIVAL. Tasks() never selected plan or source, so the
"Decision required" panel could not render and chat tasks -- whose issue
number is synthetic -- linked to forge issues that do not exist.
COSMETIC FAILURE, FATAL RESULT. OpenPR failed the whole stage when
LinkBranch errored, so the pull request was never opened and the task
parked; a retry re-links the same branch, which Gitea answers with a
conflict, so every retry parked again. It is best-effort now.
INVISIBLE ACTIONS. Approve/retry/reject wrote store rows and nothing
else, so the timeline and activity stream never showed that a human had
intervened. They emit events now.
Also: store write failures answer 500 rather than 409; the dashboard
surfaces the server's actual reason instead of "409 Conflict"; action
buttons stop propagation (acting on a task no longer also expands it),
disable in flight, keep per-task errors instead of one shared string,
and confirm before rejecting.
Removed: the maybeRetryParked shim and markDead with their now-vacuous
tests, WaitingTasks (no consumers; its tests were rewritten to assert
the same routing through TaskByID), write-only Daemon.Runtime, and
dir/dirs response bloat from /api/skills. skillSource checked workDir
before sharedDir, mislabelling every shared skill when the two are equal.
DefaultRoots now dedupes roots differing only by a trailing separator.
The air build rule claiming to keep ui/dist current never ran -- it
required one file to satisfy include_dir and include_file together, and
ui/ is excluded from the walk. Deleted with the comment that promised it.
task test:ui joins task check and CI; it ran nowhere before, so the
dashboard suite could rot silently.
Docs corrected where they described behaviour that no longer exists:
parks no longer comment on the issue, state labels no longer drive
retries, and forgerpc's doc claimed four proxied methods while listing
three of five.
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:
- Memory manager (18.7-18.16): background sync pipeline, prefetch with
timeout, memory-write notification, threat scanning, shutdown drain
with abandoned-write tracking, frozen snapshot pattern. Wired
ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
and notification actually apply to writes, not just exist as unused
primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.