feat(webui): deliver live dashboard events by polling the State Store cursor

archie-core-za9f. The UI process owns no event bus, so after the .5.4 cutover
the activity feed would show history at page load and then go silent: live
delivery is a Server.Broadcast call made only from the daemon's bus
subscription.

Decided against the push-subscription RPC the bead assumed, on three findings:

* store.Open sets SetMaxOpenConns(1), so writes serialise and the event id is
  assigned in commit order. The gap-skipping failure that normally rules out
  polling an autoincrement cursor cannot happen here, and SSE catch-up already
  depends on the same property.
* sseStream.drain treats a broadcast as a wakeup, not a payload: it fills the
  persisted gap via catchUp and delivers the event only if catch-up missed it,
  deduplicating on the since watermark. A poller substitutes for the bus with
  no change to sse.go and no new SSE semantics.
* The events table is the only complete fan-out point. ArchiveTask writes an
  audit event inside its transaction and daemon.go inserts binding-dispatch
  failures directly, neither touching the bus. A push hub needs a hook at every
  write site and misses any added later.

So: no proto change, no new auth surface, no backpressure policy. The pump
primes past existing history (a client's own catch-up serves that), then ticks
EventsSince into Broadcast. One pump serves every browser. A failed fetch
leaves the watermark intact, so a State Store restart costs latency, not
events.

Wired into archieui.Run in the background so priming a large events table
cannot delay the listener, with -event-poll-interval (default 1s).

Decision recorded in migration-decisions.md; ui-service-boundary.md's /events
route-owner row moves to RESOLVED, and /api/logs splits out of it as still
unresolved and host-local.
This commit is contained in:
sam committed 2026-09-09 02:53:32 +10:00
1 parent ac7b4492a2
commit 536fa20254
9 files changed
+407 -1

No files matched your search

+1
View File
@@ -39,6 +39,7 @@ func run() int {
flag.StringVar(&options.State.Target, "state-target", "", "archie-state-store gRPC address (defaults to [services.state].target)")
flag.StringVar(&options.State.Token, "state-token", "", "bearer token presented to archie-state-store")
flag.DurationVar(&options.DependencyTimeout, "dependency-timeout", 0, "per-dependency readiness probe timeout (default 5s)")
flag.DurationVar(&options.EventPollInterval, "event-poll-interval", 0, "how often the activity feed polls the State Store for new events (default 1s)")
// BoolFunc rather than BoolVar so an unset flag stays distinguishable
// from an explicit -trust-forwarded-headers=false, which must be able to
// override a configuration file that enables it.
+48
View File
@@ -308,6 +308,54 @@ selection, failure/readiness behavior, and deletion of the shared-holder and
direct-access paths. The extraction cutover removes the in-process UI
listener in the same change; there is no dual-live UI authority.
**Dashboard live event delivery — DECIDED (2026-09-09, `archie-core-za9f`).**
The extracted UI receives live activity by polling the State Store's existing
`EventsSince` cursor, not by a new push-subscription RPC. No proto change, no
contract amendment beyond this record.
The UI process runs one event pump. It primes its watermark by paging
`EventsSince` to the end at startup without delivering, then polls
`EventsSince(watermark, limit)` on an interval and hands each new event to
`webui.Server.Broadcast`, exactly where `internal/app/archied/main.go`'s
`persistAndBroadcastEvents` hands events to it today. One pump serves every
connected browser, so cost is one indexed query per interval regardless of
viewer count, and `Broadcast` is a no-op when nobody is watching.
Three properties make the cursor sufficient, and a push hub unnecessary:
- **The cursor cannot skip.** `internal/store/store.go` opens the database with
`SetMaxOpenConns(1)`, so writes serialise and a row's `id` is assigned in
commit order. There is no window in which a lower id becomes visible after a
higher one, which is the failure mode that normally rules out polling an
autoincrement cursor. SSE catch-up already depends on this same property.
- **A broadcast is a wakeup, not the payload.** `sseStream.drain` treats each
broadcast event as a signal to fill the persisted gap ahead of it via
`catchUp`, then delivers the event only if catch-up did not already cover
it, deduplicating on the `since` watermark. Substituting a poller for the
in-process bus therefore changes no SSE semantics, and needs no change to
`internal/webui/sse.go`. It also means a broadcast dropped to a stalled
client self-heals on the next one.
- **The table is the only complete fan-out point.** Events reach the store
through several paths that never touch the daemon's bus, including the
audit event `store.ArchiveTask` writes inside its transaction
(`internal/store/store.go`) and the binding-dispatch failure
`internal/daemon/daemon.go` inserts directly. A push hub would have to be
hooked at every write site and at post-commit, and would silently miss any
site added later that forgets to notify. Reading the table misses nothing.
The cost is one poll interval of latency on the activity feed, default 1s and
operator-tunable. That is accepted: the feed reports task lifecycle history,
and an operator action already gets its own synchronous HTTP response.
This resolves the `/events` row of the route-owner inventory in
[`ui-service-boundary.md`](../prds/ui-service-boundary.md). `/api/logs` and
`/api/logs/stream` are not resolved by it, they read the daemon's in-process
diagnostic feed and remain host-local (see `archie-core-8cda.5.4`).
Revisit only if sub-second feed latency becomes a requirement, or if a second
consumer needs the same stream, at which point the fan-out belongs in the State
Store process behind a `SubscribeEvents` RPC and this record is superseded.
### 7. Shared mechanics
The following cross-domain mechanics require exact contracts before dependent
+2 -1
View File
@@ -64,7 +64,8 @@ cutover.
|---|---|---|
| `/api/chat/*` | Gateway `ChatContract` and its versioned client | contract seam exists; DTO and stream parity remain |
| `/api/tasks/*`, `/api/workflows`, `/api/setup` | State Store read/action contracts plus an explicit execution/admin action contract | State Store adapter exists; UI-facing narrow surface remains |
| `/api/logs`, `/events` | State/event and log contracts with replay cursor and backpressure rules | local bus/feed only today; contract remains to define |
| `/events` | State Store `EventsSince`, polled by one UI-process pump feeding `Server.Broadcast` | RESOLVED (`archie-core-za9f`): see migration-decisions, "Dashboard live event delivery". No new RPC; replay cursor and drop-recovery are the existing `since` watermark in `sse.go` |
| `/api/logs`, `/api/logs/stream` | daemon diagnostic feed, host-local | unresolved: `logging.Feed` is in-process on the daemon host and has no contract |
| `/api/captures`, `/api/mappings`, `/api/bindings` | State Store contracts, with webhook verification owned by Work Intake/Messaging | local adapter exists; remote UI surface remains to define |
| `/api/config` | daemon/config owner through a versioned admin contract | current app callbacks; shared holder must be removed |
| `/api/channels`, reload, curators, memory, skills, version/update | owning capability contract or an explicitly removed route | owner and failure semantics remain to define |
+3
View File
@@ -133,6 +133,9 @@ func withDefaults(o Options) Options {
if o.DependencyTimeout <= 0 {
o.DependencyTimeout = defaultDependencyTimeout
}
if o.EventPollInterval <= 0 {
o.EventPollInterval = webui.DefaultEventPollInterval
}
if o.ReadHeaderTimeout <= 0 {
o.ReadHeaderTimeout = defaultReadHeaderTimeout
}
+31
View File
@@ -5,6 +5,9 @@ import (
"os"
"path/filepath"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/webui"
)
// uiConfigFile writes a config.toml carrying both the six fields the UI
@@ -190,3 +193,31 @@ target = "127.0.0.1:9090"
t.Errorf("Gateway.Token = %q, want the explicit target_token over env", resolved2.Gateway.Token)
}
}
// The activity feed is a poll in this process, so its interval is a
// process-local setting with a default, like the readiness timeout. Zero
// means "operator did not say", not "never poll".
func TestResolveDefaultsTheEventPollInterval(t *testing.T) {
resolved, err := Resolve(Options{
Gateway: ServiceTarget{Target: "127.0.0.1:8585"},
State: ServiceTarget{Target: "127.0.0.1:9090"},
}, slog.New(slog.DiscardHandler))
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if resolved.EventPollInterval != webui.DefaultEventPollInterval {
t.Fatalf("EventPollInterval = %s, want the default %s", resolved.EventPollInterval, webui.DefaultEventPollInterval)
}
explicit, err := Resolve(Options{
Gateway: ServiceTarget{Target: "127.0.0.1:8585"},
State: ServiceTarget{Target: "127.0.0.1:9090"},
EventPollInterval: 250 * time.Millisecond,
}, slog.New(slog.DiscardHandler))
if err != nil {
t.Fatalf("Resolve with an explicit interval: %v", err)
}
if explicit.EventPollInterval != 250*time.Millisecond {
t.Fatalf("EventPollInterval = %s, want the operator's 250ms", explicit.EventPollInterval)
}
}
+6
View File
@@ -66,6 +66,12 @@ type Options struct {
// DependencyTimeout bounds each readiness probe's call to a dependency.
DependencyTimeout time.Duration
// EventPollInterval bounds how stale the dashboard's activity feed can
// be. This process owns no event bus, so live activity is a poll of the
// State Store's event cursor; see webui.DefaultEventPollInterval and
// docs/architecture/migration-decisions.md, "Dashboard live event
// delivery".
EventPollInterval time.Duration
// ReadHeaderTimeout and ShutdownTimeout bound the HTTP lifecycle.
ReadHeaderTimeout time.Duration
ShutdownTimeout time.Duration
+11
View File
@@ -52,6 +52,17 @@ func Run(ctx context.Context, options Options) error {
Health: newReadinessRegistry(opts, tasks, chat),
})
// Live activity has no in-process bus in this process: the pump reads
// events back out of the State Store over the same cursor SSE catch-up
// uses. Backgrounded so priming past a large events table cannot delay
// the listener (docs/architecture/migration-decisions.md, "Dashboard
// live event delivery").
go func() {
if err := srv.PumpEvents(ctx, opts.EventPollInterval); err != nil {
log.Error("event pump stopped; the activity feed will only show history", "err", err)
}
}()
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", opts.Listen)
if err != nil {
return fmt.Errorf("listen for ui: %w", err)
+141
View File
@@ -0,0 +1,141 @@
package webui
import (
"context"
"errors"
"fmt"
"time"
"github.com/samcharles93/archie-core/internal/events"
)
// DefaultEventPollInterval is how often the UI process asks the State Store
// for events it has not seen. It bounds activity-feed latency; the feed
// reports task lifecycle history, and an operator action already gets its own
// synchronous HTTP response, so a second is not felt.
const DefaultEventPollInterval = time.Second
// Page sizes. Priming walks the whole table once and wants few round trips;
// steady-state ticks expect nothing or a handful and want small messages.
const (
eventPumpPrimePageSize = 5000
eventPumpPageSize = 200
)
// eventPump is the UI process's substitute for the daemon's in-process event
// bus. The daemon hands persisted events to Server.Broadcast from its own bus
// subscription (internal/app/archied/main.go's persistAndBroadcastEvents); a
// process that owns no bus reads the same events back out of the State Store
// instead, over the EventsSince cursor.
//
// Polling is sufficient rather than a compromise, for the reasons recorded in
// docs/architecture/migration-decisions.md ("Dashboard live event delivery"):
// the store serialises writes on a single connection, so the id cursor is
// assigned in commit order and cannot skip; sseStream.drain treats a broadcast
// as a wakeup and re-reads the gap from the store, so nothing here changes SSE
// semantics; and reading the table catches writers that never touch a bus,
// including the audit event ArchiveTask writes inside its transaction.
type eventPump struct {
store eventReader
log func(string, ...any)
broadcast func(events.Event)
// watermark is the highest event id already handed to broadcast. It only
// ever moves forward, and only past an event that was delivered.
watermark int64
}
// eventReader is the one method the pump needs, so a test can drive it
// without a whole TaskStore.
type eventReader interface {
EventsSince(ctx context.Context, sinceID int64, limit int) ([]events.Event, error)
}
func (s *Server) newEventPump() *eventPump {
return &eventPump{store: s.Store, log: s.logf, broadcast: s.Broadcast}
}
// PumpEvents delivers events persisted after it starts to every connected SSE
// client, until ctx ends. It is the UI process's only live-delivery path;
// history is served by each client's own catch-up, not by this.
func (s *Server) PumpEvents(ctx context.Context, interval time.Duration) error {
if s.Store == nil {
return errors.New("event pump needs a store to read events from")
}
if interval <= 0 {
interval = DefaultEventPollInterval
}
pump := s.newEventPump()
if err := pump.prime(ctx); err != nil {
return fmt.Errorf("prime event pump: %w", err)
}
pump.run(ctx, interval)
return nil
}
// prime advances the watermark past everything already persisted, without
// delivering any of it. Without this the pump would replay the entire events
// table into every browser the first time it ticks; catch-up already serves a
// client the history it asked for.
func (p *eventPump) prime(ctx context.Context) error {
for {
batch, err := p.store.EventsSince(ctx, p.watermark, eventPumpPrimePageSize)
if err != nil {
return err
}
before := p.watermark
for _, e := range batch {
if e.ID > p.watermark {
p.watermark = e.ID
}
}
// A short page is the end of the table. The watermark check is the
// guard against a full page that advanced nothing, which would
// otherwise spin forever.
if len(batch) < eventPumpPrimePageSize || p.watermark == before {
return nil
}
}
}
// deliver broadcasts every event after the watermark and reports how many.
// A fetch that fails leaves the watermark where it was, so the next tick
// retries the same range: a State Store restart costs latency, not events.
func (p *eventPump) deliver(ctx context.Context) (int, error) {
delivered := 0
for {
batch, err := p.store.EventsSince(ctx, p.watermark, eventPumpPageSize)
if err != nil {
return delivered, err
}
before := p.watermark
for _, e := range batch {
if e.ID <= p.watermark {
continue
}
p.broadcast(e)
p.watermark = e.ID
delivered++
}
if len(batch) < eventPumpPageSize || p.watermark == before {
return delivered, nil
}
}
}
// run ticks until ctx ends. A failed tick is logged and retried on the next
// one rather than ending the pump: the feed is a view, and losing it for a
// poll interval is better than losing it for the life of the process.
func (p *eventPump) run(ctx context.Context, interval time.Duration) {
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
if _, err := p.deliver(ctx); err != nil && ctx.Err() == nil {
p.log("event pump fetch failed", "since", p.watermark, "err", err)
}
}
}
}
+164
View File
@@ -0,0 +1,164 @@
package webui
import (
"bufio"
"context"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/samcharles93/archie-core/internal/events"
)
// newTestPump builds a pump over the remote State Store contract, which is
// the composition the UI process runs: the pump's only view of an event is
// the one that came back over the wire.
func newTestPump(t *testing.T, srv *Server) *eventPump {
t.Helper()
return srv.newEventPump()
}
// Priming is what keeps the pump from replaying the whole events table into
// every connected browser on startup. History is a client's own catch-up
// (sseStream.catchUp), not the pump's job.
func TestEventPumpPrimesPastHistoryThenDeliversNewEvents(t *testing.T) {
srv := newRemoteTestServer(t)
for _, detail := range []string{"first", "second"} {
if _, err := srv.Store.InsertEvent(t.Context(), events.Event{Kind: "task_created", Detail: detail}); err != nil {
t.Fatalf("seed history: %v", err)
}
}
pump := newTestPump(t, srv)
if err := pump.prime(t.Context()); err != nil {
t.Fatalf("prime: %v", err)
}
delivered, err := pump.deliver(t.Context())
if err != nil {
t.Fatalf("deliver after prime: %v", err)
}
if delivered != 0 {
t.Fatalf("pump delivered %d pre-existing events; priming must advance past history", delivered)
}
id, err := srv.Store.InsertEvent(t.Context(), events.Event{Kind: "task_started", Detail: "after the pump"})
if err != nil {
t.Fatalf("insert live event: %v", err)
}
delivered, err = pump.deliver(t.Context())
if err != nil {
t.Fatalf("deliver live event: %v", err)
}
if delivered != 1 {
t.Fatalf("pump delivered %d events, want the 1 inserted after priming", delivered)
}
if pump.watermark != id {
t.Fatalf("watermark = %d, want %d: the pump must not refetch a delivered event", pump.watermark, id)
}
}
// A store failure must not kill the pump: the next tick retries from the
// same watermark, so a State Store restart costs latency, not the feed.
func TestEventPumpKeepsItsWatermarkAcrossAStoreFailure(t *testing.T) {
srv := newRemoteTestServer(t)
pump := newTestPump(t, srv)
if err := pump.prime(t.Context()); err != nil {
t.Fatalf("prime: %v", err)
}
before := pump.watermark
cancelled, cancel := context.WithCancel(t.Context())
cancel()
if _, err := pump.deliver(cancelled); err == nil {
t.Fatal("deliver against a cancelled context returned no error")
}
if pump.watermark != before {
t.Fatalf("watermark moved to %d on a failed fetch, want %d", pump.watermark, before)
}
}
// The bead's headline claim: an event written to the State Store by another
// process reaches a browser hanging on /events, with no in-process bus.
func TestEventPumpReachesALiveSSEClient(t *testing.T) {
srv := newRemoteTestServer(t)
pump := newTestPump(t, srv)
if err := pump.prime(t.Context()); err != nil {
t.Fatalf("prime: %v", err)
}
// handleSSE writes no response headers until its first send, so the
// client blocks on Do() until something is on the wire. Seed one event
// for catch-up to flush, then prove the pump delivers what comes after.
if _, err := srv.Store.InsertEvent(t.Context(), events.Event{Kind: "task_created", Detail: "flushes the headers"}); err != nil {
t.Fatalf("seed catch-up event: %v", err)
}
ts := newSSETestServer(t, srv)
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
lines := openSSEStream(t, ctx, ts)
go pump.run(ctx, 5*time.Millisecond)
if _, err := srv.Store.InsertEvent(t.Context(), events.Event{Kind: "task_started", Detail: "over the wire"}); err != nil {
t.Fatalf("insert live event: %v", err)
}
deadline := time.After(5 * time.Second)
for {
select {
case line := <-lines:
if strings.Contains(line, "over the wire") {
return
}
case <-deadline:
t.Fatal("live event never reached the SSE client; the pump is the only delivery path in the UI process")
}
}
}
// newSSETestServer serves the dashboard's real route set, so /events goes
// through requireToken and the mux exactly as a browser reaches it.
func newSSETestServer(t *testing.T, srv *Server) *httptest.Server {
t.Helper()
ts := httptest.NewServer(srv.Handler())
t.Cleanup(ts.Close)
return ts
}
// openSSEStream connects to /events and relays each response line on a
// channel. The request context ends the handler, which is what lets the
// test server close.
func openSSEStream(t *testing.T, ctx context.Context, ts *httptest.Server) <-chan string {
t.Helper()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, ts.URL+"/events", nil)
if err != nil {
t.Fatalf("sse request: %v", err)
}
resp, err := ts.Client().Do(req)
if err != nil {
t.Fatalf("GET /events: %v", err)
}
// Closed at cleanup rather than by the reader goroutine: the body is the
// live stream and must outlive this function, but it still has exactly
// one owner and one close.
t.Cleanup(func() { _ = resp.Body.Close() })
if resp.StatusCode != http.StatusOK {
t.Fatalf("GET /events = %d, want 200", resp.StatusCode)
}
lines := make(chan string, 64)
go func() {
defer close(lines)
scanner := bufio.NewScanner(resp.Body)
for scanner.Scan() {
select {
case lines <- scanner.Text():
case <-ctx.Done():
return
}
}
}()
return lines
}