Reaction delivery via embedded NATS, no external server required #95

Closed
opened 2026-10-27 13:34:55 +00:00 by sam · 0 comments
Owner

Reactions need at-least-once delivery distinct from the lossy events.Bus (see epic description). The delivery mechanism is NATS -- that part is settled, not a design question. What's open is deployment shape.

VERIFIED 2026-08-22: config.NATSConfig.URL empty currently means 'no NATS, use SQLite ClaimNext' (internal/config/config.go:573-582). That fallback was designed for task distribution working without a broker at all. It was never meant to generalise to 'archie has no NATS' as a permanent architectural stance -- the intended second deployment shape alongside an external NATS server was always NATS EMBEDDED (in-process), not a different eventbus.

github.com/nats-io/nats-server/v2 is already a direct dependency and is proven to work in this codebase: internal/infrastructure/agenttransport/nats/transport_test.go and others already start a real embedded server via nats-server/v2/test for tests. It has just never been wired into production startup -- cmd/archied only knows 'external URL configured' vs 'fall back to SQLite'. No new library, no new bus, no architectural gap to design around: this is 'wire the dependency we already have and already test against into the daemon's own startup path.'

Design questions that remain, now correctly scoped: does an embedded NATS instance replace the SQLite fallback outright for single-process deployments, or run alongside it; what config controls embedded vs external vs disabled; does the daemon own the embedded server's lifecycle the same way it owns everything else (start/stop paired with daemon start/stop, per the shutdown-ordering rules in ARCHITECTURE.md).

Acceptance criteria

A single-process deployment with no external NATS URL configured still gets reaction delivery via an embedded NATS instance; the daemon starts and stops the embedded server as part of its own lifecycle; no second eventbus type is introduced; internal/infrastructure/eventbus/nats is reused as the transport rather than duplicated.

Notes

2026-08-22, from docs/prds/event-sources-and-reactions.md: do not reuse the existing nats.Client's stream creation as-is. internal/infrastructure/eventbus/nats/client.go:65 hardcodes Retention: jetstream.WorkQueuePolicy -- correct for ARCHIE_TASKS (task distribution, one consumer claims each message) and wrong for reactions, which need fan-out (every registered reaction sees every matching event). Two reactions on overlapping subjects under WorkQueuePolicy hit the exact trap CLAUDE.md documents: the second consumer silently gets nothing. Config.StreamName/Config.Subjects are already composition-supplied (config.go:30-38); Retention is not parameterized and needs to become a Config field, defaulting to today's WorkQueuePolicy so ARCHIE_TASKS is unaffected, with reactions getting a second stream under a fan-out-capable policy and their own subject namespace.

Close reason

Embedded NATS wired into production startup: mode=embedded/external/off resolves from url, daemon owns the in-process server lifecycle, Retention parameterized for a future fan-out reaction stream. task check + lint clean; adversarial review zero surviving findings. Committed 2c7db34.


Imported from beads archie-core-7d5u.2 · GitHub #545 · type feature · priority P1 · created 2026-08-21 · closed 2026-08-21

Reactions need at-least-once delivery distinct from the lossy events.Bus (see epic description). The delivery mechanism is NATS -- that part is settled, not a design question. What's open is deployment shape. VERIFIED 2026-08-22: config.NATSConfig.URL empty currently means 'no NATS, use SQLite ClaimNext' (internal/config/config.go:573-582). That fallback was designed for task distribution working without a broker at all. It was never meant to generalise to 'archie has no NATS' as a permanent architectural stance -- the intended second deployment shape alongside an external NATS server was always NATS EMBEDDED (in-process), not a different eventbus. github.com/nats-io/nats-server/v2 is already a direct dependency and is proven to work in this codebase: internal/infrastructure/agenttransport/nats/transport_test.go and others already start a real embedded server via nats-server/v2/test for tests. It has just never been wired into production startup -- cmd/archied only knows 'external URL configured' vs 'fall back to SQLite'. No new library, no new bus, no architectural gap to design around: this is 'wire the dependency we already have and already test against into the daemon's own startup path.' Design questions that remain, now correctly scoped: does an embedded NATS instance replace the SQLite fallback outright for single-process deployments, or run alongside it; what config controls embedded vs external vs disabled; does the daemon own the embedded server's lifecycle the same way it owns everything else (start/stop paired with daemon start/stop, per the shutdown-ordering rules in ARCHITECTURE.md). ### Acceptance criteria A single-process deployment with no external NATS URL configured still gets reaction delivery via an embedded NATS instance; the daemon starts and stops the embedded server as part of its own lifecycle; no second eventbus type is introduced; internal/infrastructure/eventbus/nats is reused as the transport rather than duplicated. ### Notes 2026-08-22, from docs/prds/event-sources-and-reactions.md: do not reuse the existing nats.Client's stream creation as-is. internal/infrastructure/eventbus/nats/client.go:65 hardcodes Retention: jetstream.WorkQueuePolicy -- correct for ARCHIE_TASKS (task distribution, one consumer claims each message) and wrong for reactions, which need fan-out (every registered reaction sees every matching event). Two reactions on overlapping subjects under WorkQueuePolicy hit the exact trap CLAUDE.md documents: the second consumer silently gets nothing. Config.StreamName/Config.Subjects are already composition-supplied (config.go:30-38); Retention is not parameterized and needs to become a Config field, defaulting to today's WorkQueuePolicy so ARCHIE_TASKS is unaffected, with reactions getting a second stream under a fan-out-capable policy and their own subject namespace. ### Close reason Embedded NATS wired into production startup: mode=embedded/external/off resolves from url, daemon owns the in-process server lifecycle, Retention parameterized for a future fan-out reaction stream. task check + lint clean; adversarial review zero surviving findings. Committed 2c7db34. --- _Imported from beads `archie-core-7d5u.2` · GitHub #545 · type feature · priority P1 · created 2026-08-21 · closed 2026-08-21_
sam added the highfeature labels 2026-10-27 13:34:55 +00:00
sam self-assigned this 2026-10-27 13:34:55 +00:00
sam closed this issue 2026-10-27 13:34:55 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: sam/archie#95