refactor(state-store): migrate capture/mapping/binding consumers to State Store (archie-core-8cda.4.4) (#772)

* feat(state-store): add daemon composition path for remote store adapter ($services.state)

Add the additive composition path the daemon's own capture consumers use
(archie-core-8cda.4.4, docs/prds/state-store-contract.md §10/§12 step 6).
bootstrap.go opens production store then resolves b.stateStore: the local
*store.Store by default, or a remote *staterpc.Client dialed to the
standalone archie-state-store gRPC service when [services.state].target is
set. Wire the dashboard CaptureStore surface from b.stateStore.

The in-process StateStore gRPC server keeps serving the daemon's SAME local
store (stateStoreDeps from b.st), not the consumer-facing b.d.* adapters:
task lifecycle (b.st) stays in-process until .4.5/.4.6 and there is no
dual-store ownership. Fail-closed on a non-loopback target without a bearer
token mirrors the standalone server's confinement (§9).

* feat(state-store): route daemon mapping consumers through the state store adapter

Migrate the MappingStore consumer to the State Store contract adapter
(archie-core-8cda.4.4 step 6, Mapping swap): the dashboard's mapping editor
(web.Mappings) and the daemon's dispatch loop (d.Mappings) resolve from
b.stateStore instead of reaching into b.st directly. Local by default,
remote *staterpc.Client when [services.state].target is set.

* feat(state-store): route daemon binding consumers through the state store adapter

Migrate the BindingStore/BindingDispatcher/BindingTaskCreator consumers to
the State Store contract adapter (archie-core-8cda.4.4 step 6, Binding
swap): the dashboard's binding editor (web.Bindings / web.BindingDispatcher)
and the daemon's dispatch loop (d.Bindings / d.BindingDispatcher /
d.BindingTaskCreator) resolve from b.stateStore instead of b.st. Local by
default, remote *staterpc.Client when [services.state].target is set.

* refactor(state-store): resolve daemon state store adapter only in daemon composition

openStores is shared with the standalone archie-gateway (RunGateway), which
does not consume the capture/mapping/binding surfaces. Resolve b.stateStore
from the daemon's own Run() composition instead, so a gateway config that
happens to carry [services.state].target can never dial (or fail closed
against) a state store it does not use. openStateStoreAdapter takes no ctx:
grpc.NewClient connects lazily, and the adapter resolution is synchronous.
This commit is contained in:
sam authored and GitHub committed 2026-09-07 04:04:34 +10:00
1 parent a59505816a
commit 5fe5ff6626
5 files changed
+239 -33

No files matched your search

+8 -4
View File
@@ -272,10 +272,14 @@ gate_max_failures = 5
# # archie-state-store server validates this; a loopback
# # bind (127.0.0.1/::1/localhost) needs no token. Non-loopback
# # without a token fails closed (docs/prds/state-store-contract.md §9).
# # This surface is consumed by the standalone
# # archie-state-store binary (.4.3); the
# # daemon does not read it yet -- it still serves the
# # State Store gRPC service in-process today.
# # The standalone archie-state-store binary (.4.3) consumes
# # this surface; the daemon reads [services.state].target to
# # route its own capture/mapping/binding consumers to the
# # remote store (docs/prds/state-store-contract.md §10/§12 step 6).
# # Task lifecycle stays in-process and the daemon still serves
# # the State Store gRPC service in-process until that path is
# # deleted (.4.6); flipping this during .4.4 routes only the
# # capture/mapping/binding contracts remotely.
# NATS is the sole autonomous workflow handoff. mode changes only where the
# broker runs; every workflow still runs in a managed task container:
+77 -29
View File
@@ -18,6 +18,7 @@ import (
"path/filepath"
"slices"
"sort"
"strings"
"sync/atomic"
"syscall"
"time"
@@ -100,7 +101,17 @@ type boot struct {
playbooks *playbook.Store
st store.TaskStore
st store.TaskStore
// stateStore is the State Store contract adapter the daemon's own
// capture/mapping/binding consumers use. It is the in-process
// *store.Store (the same one b.st holds) unless [services.state].target
// is set, in which case it is a remote *staterpc.Client dialed to the
// standalone archie-state-store gRPC service. Task lifecycle (b.st)
// stays in-process until .4.5/.4.6; only capture/mapping/binding
// resolve through this field, so a consumer mixes in-process and remote
// contracts per contract-code while the daemon's own store stays local
// (docs/prds/state-store-contract.md §12 step 6).
stateStore store.TaskStore
chatSessionStore gateway.SessionStore
catalog modelcatalog.Snapshot
@@ -287,6 +298,33 @@ func (b *boot) openStores(ctx context.Context) error {
return b.openChatSessions(ctx)
}
// openStateStoreAdapter selects the State Store contract adapter the daemon's
// own capture/mapping/binding consumers will use. The base path is the local
// *store.Store opened by openStores; when [services.state].target is set the
// daemon instead dials the standalone archie-state-store gRPC service and
// uses *staterpc.Client. Empty target keeps the in-process adapter (the
// default -- the State Store service is not yet extracted into its own
// process), so this is additive and behaviour-neutral until an operator flips
// the seam (docs/prds/state-store-contract.md §10). Task lifecycle (b.st) is
// untouched here -- only the capture/mapping/binding contracts route through
// the adapter during .4.4. Called only from the daemon's composition (the
// gateway does not consume these surfaces), so it runs after openStores has
// opened b.st and resolved b.secrets.
func (b *boot) openStateStoreAdapter() error {
if strings.TrimSpace(b.cfg.Services.State.Target) == "" {
b.stateStore = b.st
return nil
}
client, cleanup, err := composeStateStoreClient(b.cfg.Services.State, b.secrets)
if err != nil {
b.log.Error("state store adapter", "err", err)
return err
}
b.stateStore = client
b.addCleanup(cleanup)
return nil
}
func (b *boot) openChatSessions(_ context.Context) error { //nolint:unparam // context keeps the composition phase contract aligned with other openers
chatSessionStore, err := makeTelegramSessionStore(b.cfg) //nolint:contextcheck // SQLite schema initialization is synchronous and has no context-aware API
if err != nil {
@@ -378,39 +416,46 @@ func (b *boot) setupObservability() {
}
// wireWebStoreSurfaces attaches the dashboard's optional storage surfaces.
// openProductionTaskStore's declared return type is the narrow
// store.TaskStore, so each wider surface (all implemented by the same
// *store.Store) needs its own assertion here rather than a direct field
// reuse. A store that does not implement one degrades that dashboard feature
// with a warning instead of aborting bootstrap. The BindingStore/
// BindingDispatcher/BindingTaskCreator split keeps each interface under the
// interfacebloat limit. See docs/prds/event-capture-storage.md,
// docs/prds/payload-field-mapping.md and docs/prds/webhook-intake-security.md.
// b.stateStore is declared as the narrow store.TaskStore, so each wider
// capture/mapping/binding surface needs its own assertion here rather than a
// direct field reuse. b.stateStore is the local *store.Store by default and a
// remote *staterpc.Client when [services.state].target is set, so the same
// assertion resolves either adapter. A store that does not implement one
// degrades that dashboard feature with a warning instead of aborting
// bootstrap. The BindingStore/BindingDispatcher/BindingTaskCreator split
// keeps each interface under the interfacebloat limit. See
// docs/prds/event-capture-storage.md,
// docs/prds/payload-field-mapping.md and docs/prds/webhook-intake-security.md, and
// docs/prds/state-store-contract.md §10 for the adapter selection.
func (b *boot) wireWebStoreSurfaces() {
cfg, log := b.cfg, b.log
if cs, ok := b.st.(store.CaptureStore); ok {
// Capture/mapping/binding surfaces resolve from b.stateStore (the State
// Store contract adapter): local by default, remote *staterpc.Client when
// [services.state].target is set. See wireWebStoreSurfaces' doc and
// docs/prds/state-store-contract.md §10.
if cs, ok := b.stateStore.(store.CaptureStore); ok {
b.web.Captures = cs
} else {
log.Warn("capture storage unavailable: task store does not implement CaptureStore")
log.Warn("capture storage unavailable: state store does not implement CaptureStore")
}
b.web.CaptureRetention = cfg.Capture.Retention.Std()
b.web.CaptureMaxEvents = cfg.Capture.MaxEvents
b.web.CaptureMaxBodyBytes = int64(cfg.Capture.MaxBodyBytes)
b.web.CaptureLimiter = webhookguard.NewRateLimiter(cfg.Capture.RatePerSecond, cfg.Capture.RateBurst, time.Now)
if ms, ok := b.st.(store.MappingStore); ok {
if ms, ok := b.stateStore.(store.MappingStore); ok {
b.web.Mappings = ms
} else {
log.Warn("mapping storage unavailable: task store does not implement MappingStore")
log.Warn("mapping storage unavailable: state store does not implement MappingStore")
}
if bs, ok := b.st.(store.BindingStore); ok {
if bs, ok := b.stateStore.(store.BindingStore); ok {
b.web.Bindings = bs
} else {
log.Warn("binding storage unavailable: task store does not implement BindingStore")
log.Warn("binding storage unavailable: state store does not implement BindingStore")
}
if bd, ok := b.st.(store.BindingDispatcher); ok {
if bd, ok := b.stateStore.(store.BindingDispatcher); ok {
b.web.BindingDispatcher = bd
} else {
log.Warn("binding dispatcher unavailable: task store does not implement BindingDispatcher")
log.Warn("binding dispatcher unavailable: state store does not implement BindingDispatcher")
}
}
@@ -1256,16 +1301,19 @@ func (b *boot) buildDaemon() {
// this point /api/config and the daemon can never disagree about the
// published config.
b.web.Cfg = b.d.Cfg
if ms, ok := b.st.(store.MappingStore); ok {
// Consumer mapping/binding surfaces resolve from b.stateStore (the State
// Store contract adapter): local by default, remote *staterpc.Client when
// [services.state].target is set. See docs/prds/state-store-contract.md §10.
if ms, ok := b.stateStore.(store.MappingStore); ok {
b.d.Mappings = ms
}
if bs, ok := b.st.(store.BindingStore); ok {
if bs, ok := b.stateStore.(store.BindingStore); ok {
b.d.Bindings = bs
}
if bd, ok := b.st.(store.BindingDispatcher); ok {
if bd, ok := b.stateStore.(store.BindingDispatcher); ok {
b.d.BindingDispatcher = bd
}
if btc, ok := b.st.(store.BindingTaskCreator); ok {
if btc, ok := b.stateStore.(store.BindingTaskCreator); ok {
b.d.BindingTaskCreator = btc
}
b.setupForgeWebhook()
@@ -1283,14 +1331,14 @@ func (b *boot) buildDaemon() {
// .4.2, not yet load-bearing.
func (b *boot) startStateStoreServer(ctx context.Context) {
log := b.log
deps := staterpc.Deps{
Tasks: b.st, Mappings: b.d.Mappings, Bindings: b.d.Bindings,
BindingDispatcher: b.d.BindingDispatcher, BindingTaskCreator: b.d.BindingTaskCreator,
Log: log,
}
if cs, ok := b.st.(store.CaptureStore); ok {
deps.Captures = cs
}
// The in-process server fronts the daemon's LOCAL store surfaces, not the
// consumer-facing b.d.* adapters: during .4.4 the daemon may route its own
// capture/mapping/binding consumers to a remote *staterpc.Client via
// [services.state].target, but the daemon still OWNS archie.db in-process
// and the agent's gRPC target must keep answering from that same local
// file (no dual-store ownership, docs/prds/state-store-contract.md §9/§12
// step 6). Deletion of the in-process serving path is .4.6.
deps := b.stateStoreDeps()
host := "127.0.0.1"
var opts []grpc.ServerOption
+8
View File
@@ -378,6 +378,14 @@ func Run() int { //nolint:cyclop // the composition root's setup sequence is del
if err := b.openStores(ctx); err != nil {
return 1
}
// Resolve the State Store contract adapter the daemon's own
// capture/mapping/binding consumers use. This is daemon-only (the gateway
// does not consume those surfaces), so it runs after openStores has opened
// b.st and resolved b.secrets and before setupObservability wires the
// dashboard's storage surfaces (docs/prds/state-store-contract.md §10).
if err := b.openStateStoreAdapter(); err != nil {
return 1
}
if exit, err := b.handleRequeue(ctx, args.requeue, args.once); err != nil {
return 1
} else if exit {
@@ -0,0 +1,53 @@
package archied
import (
"fmt"
"strings"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/infrastructure/staterpc"
"github.com/samcharles93/archie-core/internal/secret"
"github.com/samcharles93/archie-core/internal/store"
)
// composeStateStoreClient dials the State Store gRPC service the daemon's own
// capture/mapping/binding consumers will call when [services.state].target is
// set. It mirrors the gateway's composeChatContract (internal/app/archied/
// chat_service.go), but the fail-closed rule for non-loopback targets (§9) is
// the State Store's own and requires a bearer token. The token comes from
// [services.state].target_token, falling back to the STATE_STORE_TOKEN
// secret/env. The returned cleanup closes the connection; Close() on the
// adapter itself is a no-op, because the store service owns its own DB
// lifecycle (§11).
func composeStateStoreClient(settings config.ServiceConnection, secrets *secret.Registry) (store.TaskStore, func(), error) {
target := strings.TrimSpace(settings.Target)
if target == "" {
return nil, nil, fmt.Errorf("services.state.target is required")
}
token := settings.TargetToken
if token == "" {
token = secrets.Getenv("STATE_STORE_TOKEN")
}
loopback, err := stateStoreListenIsLoopback(target)
if err != nil {
return nil, nil, fmt.Errorf("services.state.target must be host:port: %w", err)
}
if !loopback && token == "" {
return nil, nil, fmt.Errorf(
"services.state.target %q is non-loopback; non-loopback exposure requires a bearer token ([services.state].target_token / STATE_STORE_TOKEN) or TLS, mirroring the gateway's --listen confinement (docs/prds/state-store-contract.md §9)",
target,
)
}
opts := []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}
if token != "" {
opts = append(opts, grpc.WithUnaryInterceptor(staterpc.UnaryClientTokenInterceptor(token)))
}
conn, err := grpc.NewClient(target, opts...)
if err != nil {
return nil, nil, fmt.Errorf("create state store client: %w", err)
}
return staterpc.NewClient(conn), func() { _ = conn.Close() }, nil
}
@@ -0,0 +1,93 @@
package archied
import (
"path/filepath"
"strings"
"testing"
"github.com/samcharles93/archie-core/internal/config"
"github.com/samcharles93/archie-core/internal/secret"
"github.com/samcharles93/archie-core/internal/store"
)
func TestComposeStateStoreClientRequiresTarget(t *testing.T) {
_, _, err := composeStateStoreClient(config.ServiceConnection{}, &secret.Registry{})
if err == nil {
t.Fatal("empty target should error")
}
if !strings.Contains(err.Error(), "services.state.target is required") {
t.Fatalf("unexpected error: %v", err)
}
}
func TestComposeStateStoreClientNonLoopbackFailsClosed(t *testing.T) {
for _, tt := range []struct {
name string
target string
}{
{name: "wildcard without token", target: "0.0.0.0:9090"},
{name: "hostname without token", target: "store.example.com:9090"},
} {
t.Run(tt.name, func(t *testing.T) {
_, _, err := composeStateStoreClient(config.ServiceConnection{Target: tt.target}, &secret.Registry{})
if err == nil {
t.Fatalf("non-loopback %q without a token should fail closed (docs/prds/state-store-contract.md §9)", tt.target)
}
})
}
}
func TestComposeStateStoreClientLoopbackSucceeds(t *testing.T) {
st, cleanup, err := composeStateStoreClient(config.ServiceConnection{Target: "127.0.0.1:9090"}, &secret.Registry{})
if err != nil {
t.Fatalf("loopback without token should succeed: %v", err)
}
if st == nil {
t.Fatal("expected a non-nil store adapter")
}
if cleanup == nil {
t.Fatal("expected a cleanup func")
}
cleanup() // must not panic or fail; Close() on the adapter is a no-op (§11)
}
func TestComposeStateStoreClientTokenFromTargetToken(t *testing.T) {
// Non-loopback with a configured token must not fail closed; the interceptor
// is what carries it, so the client is dialable.
st, _, err := composeStateStoreClient(config.ServiceConnection{Target: "0.0.0.0:9090", TargetToken: "secret"}, &secret.Registry{})
if err != nil {
t.Fatalf("non-loopback with a token should succeed: %v", err)
}
if st == nil {
t.Fatal("expected a non-nil store adapter")
}
}
func TestOpenStateStoreAdapterLocalUsesOpenedStore(t *testing.T) {
dir := t.TempDir()
st, err := store.Open(t.Context(), filepath.Join(dir, "tasks.sqlite"))
if err != nil {
t.Fatalf("open temp store: %v", err)
}
defer st.Close()
b := newBootstrap()
b.st = st
b.secrets = &secret.Registry{}
// Empty [services.state].target → the base path: the local *store.Store.
if err := b.openStateStoreAdapter(); err != nil {
t.Fatalf("openStateStoreAdapter(local): %v", err)
}
if b.stateStore != st {
t.Fatal("local adapter should be the opened *store.Store")
}
}
func TestOpenStateStoreAdapterRemoteFailsClosed(t *testing.T) {
b := newBootstrap()
b.secrets = &secret.Registry{}
b.cfg.Services.State = config.ServiceConnection{Target: "0.0.0.0:9090"}
if err := b.openStateStoreAdapter(); err == nil {
t.Fatal("non-loopback without a token should fail closed")
}
}