a7d6f71d5a2c9da170a46bfc4471f9979189de87
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
800352a692 |
feat(gateway): give /status a real daemon-health surface (#892)
* feat(gateway): give /status a real daemon-health surface Phase 2 of archie-core-wp9s. Phase 1 split the commands apart (health vs work) but left /status with one aggregate queue line and a static runtime block; it could not answer the questions an operator actually asks when something feels wrong. /status now reports, between the queue line and the runtime block: - Broker: the live state of this process's own NATS connection (eventbus/nats.Client.Connected), not a fresh dial -- a probe would report the probe's reachability, not the daemon's. - Containers: the pool's own active counter against the configured cap (Pool.Active/Cap), not a Docker listing, which would include containers this pool does not own. A zero cap means unlimited, so it is not shown as a ratio. - Channels: the channel manager's real lifecycle state, including a failed adapter's reason (the documented decision: read the existing manager rather than re-deriving health per adapter). - Chat model: last-known outcome of every chat-model call this process made, recorded once in sendChatTurn (the single choke point) and read back by /status. The second documented decision: never a live probe, so /status stays instant and reports what happened -- or says plainly that nothing has been attempted yet. - Last poll: when the most recent poll pass began, stamped on both poll paths (Run's single-identity loop and runIdentities' per-identity goroutines). A pass that hangs leaves the stamp stale, which is the signal; stamping completion would leave a wedged poller looking healthy. Truthful or absent: a section this process has no source for is omitted rather than rendered as a zero. The standalone Gateway owns no poll loop, pool or channel manager, so its /status reports broker and chat-model health only; aggregating across processes would need a new daemon RPC and is deliberately not built. Version stays out (/version owns it). /status and /tasks descriptions now describe what each command actually does on both published surfaces (Telegram menu/help and the shared specs), pinned by a test that also fails if the two surfaces drift. Sources are read through functions, not captured values: occupancy and channel state move, and the daemon is built after the gateways, so a source captured at construction would freeze /status on boot state. Left in place deliberately: the Runtime (provider/model) block, which Phase 1 flagged and which duplicates /model and /whoami. Removing it is a product call on a command Sam uses daily, not a mechanical follow-up. * docs(prds): record the two open /status health decisions as settled Both were flagged in PR #892 as needing the maintainer's call; both are now answered, and the PRD is the place a follow-up would look: - The Runtime (provider/model) block stays. It is the one line in /status the ticket's own rule says should not be there, but /status is read daily to see the active model at a glance, so the duplication is deliberate. Recorded as a settled product decision so it is not removed later as an oversight. - Health stays per-process. Pooling the daemon's and the Gateway's chat-model outcomes would need a new daemon<->gateway health RPC for one status line, and a per-process line is true where an aggregate would be a claim about a process the answering one cannot see. /agents' published description is still left to archie-core-mxls, unchanged. |
||
|
|
7b7401d98b
|
fix(nats): bound the ARCHIE_REACTIONS stream by age
Review fix for PR #875. ARCHIE_REACTIONS is provisioned under LimitsPolicy, which retains every message until a limit is reached. With no limit configured, acknowledged reactions accumulated without bound and continuous traffic could grow the stream until it exhausted disk. nats.Config gains MaxAge *time.Duration, using the same pointer form as Retention so "unset" (nil) is distinguishable from an explicit zero. It is applied to the stream config only when set, so every existing caller keeps its current behaviour. ARCHIE_REACTIONS sets MaxAge = 24h. A reaction is a producer-only wake event whose authoritative state is re-read at pass time, so a dropped reaction delays work until the next poll (default 60s) rather than losing it; 24h tolerates a consumer down for a full maintenance window and matches the package's existing DefaultInactiveTTL. ARCHIE_TASKS is deliberately left unbounded. It uses WorkQueuePolicy, which discards a message on acknowledgement, so its size is bounded by outstanding work; a limit there could discard a task that has not been delivered yet, which is a worse failure than the disk risk. The table asserts that non-change. |
||
|
|
9e8427bc8e
|
feat(nats): provision the ARCHIE_REACTIONS fan-out stream
archie.reaction.* had no matching stream. bootstrap.go:501-502 bound only
ARCHIE_TASKS to archie.task.> under WorkQueuePolicy, so any publish to a
reaction subject made js.PublishMsg fail and the publisher saw a publish error
-- the reason a previously landed reaction producer would have answered HTTP 500
on every webhook delivery.
ARCHIE_TASKS must stay WorkQueuePolicy: under it a second consumer on an
overlapping filter subject silently receives nothing, which is the trap recorded
in CLAUDE.md. Reactions need the opposite property, so they get a second stream:
- workintake.SubjectReactionWildcard ("archie.reaction.>")
- nats.DefaultReactionStreamName ("ARCHIE_REACTIONS")
- nats.FanOutRetention(), a pointer helper so composition can select
LimitsPolicy without importing the NATS SDK, since nil Retention already means
WorkQueuePolicy and the zero value (LimitsPolicy) is a valid explicit choice
- a second nats.Connect in connectNATS carrying that stream, closed on failure
and registered for shutdown cleanup
Red-then-green: TestConnectNATSCreatesReactionStream failed with "publish to
archie.reaction.* = nats: no response from stream, want it to land in
ARCHIE_REACTIONS" before the stream existed.
Step 1 of docs/prds/pr-review-remediation.md (bead archie-core-8li9.1).
|
||
|
|
3a0dfa924d |
refactor(state-store): cut daemon/Gateway over to the remote State Store (archie-core-8cda.4.6) (#774)
* refactor(state-store): delete NATS storerpc; agent store is gRPC-only (archie-core-8cda.4.6) The agent is fully on the gRPC State Store contract, so the legacy NATS storerpc path is dead (docs/prds/state-store-contract.md §12 step 4): - delete internal/storerpc (Server, Client, subjects) and its tests - drop the storerpc registration from registerTaskRPCServers and the now-unused store param - make the agent transport's State Store URL required and Store() return only the gRPC staterpc.Client (no NATS fallback) - rework agentworker/agent transport tests to back the store with a real gRPC State Store server instead of the deleted NATS storerpc server The agent binary now fails fast when STATE_STORE_URL is absent. Agent store calls (workflow.Store) go over gRPC to the archie-state-store service. * refactor(state-store): delete daemon/gateway in-process store ownership (archie-core-8cda.4.6) The daemon and gateway no longer open archie.db in-process: the single SQLite file is owned by the standalone archie-state-store process, and both consumers dial its gRPC State StateStore contract (docs/prds/state-store-contract.md §12 step 7, no dual-store ownership). - openStores no longer opens the production task store or seeds b.st; it only wires secrets/forge and the gateway conversation store - openStateStoreAdapter now requires [services.state].target (the old local default is deleted) and resolves b.stateStore as the remote *staterpc.Client - delete the in-process startStateStoreServer (the multiplexed gRPC server and its per-task bearer-token registry) from the daemon composition - RunGateway now resolves the same remote State Store adapter; its session SQLite stays Gateway-owned and untouched - daemon.ConnectedStateStore carries the configured target plus the resolved bearer token, which containerEnv injects as STATE_STORE_URL / STATE_STORE_TOKEN so archie-agent authenticates to the same remote service - readiness store probe reads b.stateStore PRD-consistent choice (noted): with the in-process server gone, per-task token generation is removed; the daemon/gateway/agent all present the single [services.state].target_token (or STATE_STORE_TOKEN secret) that the standalone store validates. * docs(webui): require [services.state].target in config and deployment templates (archie-core-8cda.4.6) After the in-process store path is deleted, archied and archie-gateway dial the standalone archie-state-store gRPC service via [services.state].target and never open archie.db themselves (docs/prds/state-store-contract.md §12 step 7). - config.example.toml documents [services.state].target as required, with the loopback vs Docker-bridge gateway + target_token rule - deployment templates set [services.state] per topology: bridge gateway + token for container-mode agents, loopback for host-only - deployments/README.md and systemd-user-service.md describe start commands for both topologies and drop the stale "not yet consumed by the daemon" note * style: gofumpt alignment for State Store daemon literal; appease noctx/staticcheck (archie-core-8cda.4.6) * docs(state-store): refresh stale in-process references post-cutover (archie-core-8cda.4.6) |
||
|
|
1f05dbb33c |
feat(deployment): default to managed NATS workers
Autonomous workflows now always hand a complete task to a scoped archie-agent container. Embedded NATS is the default and external NATS changes only broker placement; legacy agent and containers.enabled inputs decode for migration but cannot select an execution path. The daemon flushes task-event subscriptions before a worker can publish, and runtime updates snapshot and restore the configured worker image. |
||
|
|
d3da153768 |
refactor(agentexec): remove legacy execution paths
Autonomous workflows now have exactly one production boundary: a task-scoped archie-agent receives a full-task core-NATS request and runs the worker-local stage loop. The legacy [agent] section remains decode-only for migration; it no longer selects runtime behavior. |
||
|
|
9d73bc7157 | feat(nats): expose embedded broker to managed workers | ||
|
|
8a799cd8cd |
feat(forge): observable webhook receiver + dedup regression proof; decide adversarial review design
archie-core-7d5u.5 (idempotency and poll/webhook parity): the dedup mechanism (TaskEnvelope.IdempotencyKey + PublishUnique + JetStream Nats-Msg-Id dedup) and the shared dispatch predicate were already fully wired by 7d5u.4. What was missing: proof the dedup actually holds, and observability for a wedged receiver. - internal/infrastructure/eventbus/nats/dedup_test.go: regression test proving PublishUnique suppresses a duplicate publish on the same WorkQueuePolicy stream ARCHIE_TASKS uses in production. - internal/forge/webhook/receiver.go: Receiver.Status() tracks authenticated deliveries separately from successful publishes; a GET to the receiver's listen address (GitHub only ever POSTs) now returns it as JSON, so a wedged/non-delivering receiver is observable instead of indistinguishable from an idle repo. Poll/webhook parity (disabling webhook still polls) needed no code change: Cycle()/Run() never reference cfg.Forge.Intake anywhere in internal/daemon, only setupForgeWebhook gates the webhook HTTP server. archie-core-h019.1 (design: adversarial self-review stage): one-page decision at docs/prds/adversarial-self-review.md. Reviewer runs in a second workspace pointed at a .git-stripped snapshot of the reviewed commit, never the implementer's worktree, so isolation is a property of what's mounted rather than a prompt instruction. Findings contract is workflow.ReviewFinding, owned by internal/domain/workflow. A surviving error-level finding sets StatusParked, not WaitingHuman -- the epic's informal wording named two different states in the actual machine; this resolves which. Both changes reviewed by a fresh subagent with no memory of the implementation, per CLAUDE.md's adversarial-verification requirement; zero blocking findings, two minor coverage gaps closed. |
||
|
|
51bb80f6a1 |
feat(nats): run an embedded server when no external NATS is configured
Wire github.com/nats-io/nats-server/v2 into production startup so a single-process deployment gets reaction delivery and task distribution over an in-process server rather than a separate one (docs/prds/embedded-nats.md). Meaning change: [nats].url empty used to mean "no NATS, use the SQLite ClaimNext flow"; it now means "start an embedded NATS server". The old behaviour is the explicit opt-out [nats].mode = "off". mode resolves from url when unset (url set -> external, url empty -> embedded), and agent.mode="nats" / [containers] now require mode="external" (an embedded loopback server cannot be reached by agent workers or containers). The daemon owns the embedded server's lifecycle: StartEmbedded before dialing the client, Shutdown after the client closes (LIFO cleanups). The embedded server binds loopback on a random port, enables JetStream in a store dir under the data directory, and leaves its logger nil so a startup failure surfaces as a Connect error rather than an os.Exit. Retention becomes a nats.Config field (default WorkQueuePolicy) so a future reaction stream can fan out under LimitsPolicy instead of the hardcoded competing-consumer policy that starves a second consumer on overlapping subjects. |
||
|
|
9d5040fd65 |
refactor: add internal/eventbus as the broker-neutral messaging contract
dependencies-and-contracts.md rule 1 says domains must not import
internal/infrastructure. internal/daemon and internal/agentexec both did,
and for exactly the thing the doc carves out: "internal/eventbus is an
approved shared application contract ... it MUST remain broker-neutral."
The Publisher/Consumer/Requester interfaces existed but lived inside the
NATS package, so a caller could depend on the behaviour yet still had to
import the implementation to name it. They now live in internal/eventbus,
and nats.Client asserts against eventbus.Bus.
The larger violation was ownership of meaning. The doc says "message schemas
stay with their owning domains; the event bus transports messages, it does
not own their meaning." The bus exported TaskEnvelope and a PublishTask
method, so the transport knew what a unit of work was, and it exported the
archie.agent.* subject helpers, so it also owned the agent protocol.
- TaskEnvelope, its routing Kind, the label vocabulary and the task
subjects move to internal/domain/workintake, which encodes and decodes
its own bytes. This starts internal/domain/ per organisation.md.
- The agent request/response/system subjects move to internal/agentexec,
which defines what those messages mean.
- internal/workflow keeps only the kind-to-registry mapping; the label
table stays single and lives with the domain that reads forge issues.
- The bus now carries opaque payloads on plain-string subjects.
Consumers declare the smallest interface they need rather than taking the
whole Bus: daemon.TaskBus is publish-unique/fetch/request, agentexec.
RequestBus is publish-request/reply-inbox, and archie-agent's stageBus is
fetch/respond. None can reach what it does not use.
Two defects this surfaced:
- archie-agent declared its own JetStream stream with a second copy of the
name, subjects, retention, dedup window, MaxDeliver and TTLs that
nats.Connect already sets -- two definitions of one stream, free to
diverge. Config now takes Subjects and FilterSubject from composition,
which is what let the agent use the shared client and delete its
hand-rolled setup.
- Replying to a request needs core NATS, not the JetStream publish path:
a reply inbox is ephemeral and belongs to no stream, so a durable
publish to it fails. The old code hid this by using the raw connection.
The contract now distinguishes Respond from Publish. The agentexec
round-trip test caught it after I had made the same mistake in
archie-agent.
Also translated the SDK's ErrNoResponders into eventbus.ErrNoResponders, so
the daemon's taskrun retry no longer imports the NATS SDK to name a
transient condition. The SDK is now confined to the nats package, the *rpc
packages (themselves NATS infrastructure), and cmd/*.
The agentexec test fake previously hand-rolled a JetStream consumer; it now
drains the client's own consumer, which is both closer to production and
avoids conflicting consumers on a work-queue stream.
Verified: go build, go vet, gofmt clean; full test suite passes.
|
||
|
|
a8efbf5046 |
refactor: route tasks on a typed kind, not forge labels
The event bus was inspecting forge issue labels ("bug", "feature",
"bootstrap") to pick a subject. That put a forge vocabulary inside
infrastructure and duplicated the label table already owned by
workflow.Route -- adding a label meant remembering both places, and
nothing would have caught it if you forgot.
The label table now lives once, in internal/workflow, which already owns
label semantics:
- workflow.Kind names the routing category, with labelKinds as the
single label vocabulary and kindWorkflows mapping kind to registry
entry. Route consumes the ordered kind list so a "bug,feature" task
still falls through to feasibility when no tdd workflow is
registered, exactly as the switch did.
- nats.TaskKind is a closed, validated set of subject tokens.
PublishTask rejects an unknown kind rather than silently delivering
it to the default queue; the zero kind means default, so messages
queued before this field decode unchanged.
- The daemon classifies at the publish site and sets TaskEnvelope.Kind.
Also fixes two defects this exposed:
- drainNATS never reached its SQLite fallback. Replacing the old
(nil, nil) no-message signal with ErrNoMessage in
|
||
|
|
492ca209f9 |
refactor(nats): rebuild as infrastructure/eventbus/nats with real boundaries
Relocating internal/nats as-is would have cemented its problems into the approved layout, so this is a rewrite rather than a move. What was wrong: - Conn() *nats.Conn, PublishMsg(*nats.Msg) and Fetch() jetstream.Msg pushed SDK types into all four consumers, so the broker was not replaceable -- the opposite of what dependencies-and-contracts.md requires. - PublishTask took seven positional parameters, six of them strings, where a transposed argument was a silent bug. - Labels travelled as a comma-separated string and were re-split on every publish. - Fetch returned (nil, nil) for 'no message', which every caller had to remember to special-case. - The logger field was stored and never used. - Stream name, consumer name and every timeout were hardcoded constants. - Errors were inconsistently wrapped; several paths returned bare errors. - Connect repeated nc.Close() on each failure branch. What replaces it: - One concern per file: config, errors, subjects, message, client, publisher, consumer. - Publisher/Consumer/Requester interfaces with compile-time assertions, so callers depend on behaviour and can be faked without a broker. - A Message wrapper keeps jetstream types behind the boundary; consumers no longer import the NATS SDK at all. - TaskEnvelope with []string labels and JSON methods preserving the comma-separated wire form for already-queued messages. - Sentinel errors (ErrNoMessage, ErrNotConnected, ErrNoReplyAddress, ErrInvalidConfig) and %w wrapping with operation context throughout. - Config with validation and defaults; a single deferred cleanup in Connect. - CoreConn() as the one documented escape hatch, for the *rpc packages that are themselves NATS infrastructure. Consumers migrated: cmd/archied, cmd/archie-agent, internal/daemon, internal/agentexec. Behaviour, subjects and wire format unchanged. |