Commit Graph
4 Commits
Author SHA1 Message Date
sam 924b02a1b0 feat(controlplane): runtime administration over the State Store's gRPC server
The Web UI and messaging channels now read and write Archie's runtime
settings through one versioned, audited control-plane API hosted on the
State Store's existing gRPC server: catalog, query, command, watch. Each
feature owns its resource, validation and commands; storage only writes
versions and audit records.

Workflows move with it. Definitions become portable YAML compiled through
a closed, typed step registry and stored as a control-plane resource, so
the yaegi interpretation paths (internal/gate, wfeval, wfextract,
gateeval, gateextract, skillbuild) are deleted rather than carried.

The UI process reads definitions as data: the JSON projection lives in
internal/domain/workflow/task so a dashboard decodes the stored
collection without linking the workflow engine, which cmd/archie-ui's
deletion gate forbids. Shipped definitions ride the catalog descriptor
rather than a bespoke REST route, keeping the API to its four operations.

archie-messaging gains -state-store-target/-state-store-token and reads
channel settings from the database, failing closed when the State Store
is unreachable: the database is authoritative for those settings.
2026-09-21 08:17:42 +10:00
sam f6397af7f4 fix: address the Greptile findings on the merged PRs (#902)
* fix: address the Greptile findings on the merged PRs

Four valid review findings from PRs #892, #893, #898 and #899, which were
merged without the comments being read.

- Config.Clone left the new diff-cap pointers aliased (#899, mine). Clone
  deep-copies every reference field, including the *bool beside it, so a
  clone could write the cap through to the published snapshot. Both the
  top-level field and the identity overrides are copied, with the clone test
  extended to cover them.
- providerOutcomeRecorder.record could report an older call as the latest
  (#892). The completion time is taken before the lock, so two concurrent
  calls can reach the lock in the opposite order to their completion and the
  unconditional store overwrote the newer outcome. It now keeps the newer,
  via a recordAt seam that makes the reordering testable.
- Curator model calls bypassed the recorder (#892). curatorLLMRunner calls
  the runtime directly, so a daemon whose only recent model traffic was
  curator work reported "no calls attempted yet" while the provider was
  demonstrably reachable. It records at its own adapter, and sendChatTurn's
  "single choke point" comment is corrected to say there are two.
- A provider error reached /status raw (#892), unlike channel details, so a
  multiline or oversized error broke the one-line-per-fact layout. Both now
  share one bounded formatter.

Found while fixing the curator path: agentexec.NewRuntime returns nil when no
provider is configured, and a nil *runtime.Runtime panics on its first method
call, so a curator asking for a completion took the daemon down. It now
returns an error, and records nothing -- a local misconfiguration is not a
provider call.

Not fixed, filed instead: the shared lastPollAt masking a stuck identity, the
container max-uptime callback resurrecting teardown bookkeeping, the
dashboard dropping a served-only status filter on cold load, and the route
registry test's line-by-line parser.

* fix(build): drop goimports from the gate's formatter set

`task check` spent 136 of its 167 seconds in `task fmt`, and 134 of those
136 in one formatter: goimports alone over the tree.

golangci-lint's goimports formatter calls imports.Process once per file,
which builds a fresh x/tools ProcessEnv each time, so
internal/imports.getFixesWithSource re-reads modindex.Read($GOMODCACHE) per
file. On this machine that index is 62.7 MiB / 2,081,070 lines (an 11 GiB
module cache), and modindex's in-process memo does not take effect: 3 files
read 187.9 MiB back, 15 files open the payload 15 times. That is ~53 GiB of
I/O and CSV parsing per run, and it is linear at ~155 ms per file (122 files
-> 18.8s, 864 -> 133.8s). Pointing the index cache elsewhere drops the same
command to 3.9s.

The same analyzer runs under `golangci-lint run`, so a cold cache pays it
again: 169s cold with goimports, 31s without -- which is what CI and archied's
own PR gate see, since only `fmt` never caches locally.

Removing it costs no enforcement. gofumpt and gci remain, `task lint` still
reports formatter drift as an issue, and the whole-tree fmt is now 2.2s with
the tree byte-identical. What is given up is goimports' convenience of
adding and removing import lines, which compile errors already cover.

The examples/secret-engines exclusion stays: gofumpt and gci leave those
Yaegi programs alone today, but the exclusion is the stated invariant that a
formatter must not rewrite the imports of files it cannot type-check, and
TestFormattersExcludeInterpretedSecretEngines holds it. Its comment no longer
pins the reason to the formatter that was dropped.

* fix(gateway): stop /status promising and mis-reporting health it cannot see

Two findings from the cloud review of #892.

Title generation no longer records chat-model health. It runs detached after
a first turn under its own 30s bound and its error is swallowed by the
caller, but it shared the process-wide outcome recorder, so a title that
merely timed out overwrote the health line with "failed" -- reporting a
broken provider on the strength of a call no user was waiting for.

The /status command description no longer claims "workers" and "channels".
Neither can render: newStatusHealth reads the container pool and the poll
loop from boot, but the only production router.Health assignment is in
setupGatewayChat, which runs solely in the archie-gateway process, where
neither exists; channels is nil since the Messaging Service took the
front-ends. Only the broker and chat-model lines actually appear, so that is
what the help text now says.

The underlying split -- daemon owns the facts, Gateway owns the Router -- is
filed rather than patched here: it needs either a cross-process health
contract or a ratified two-section /status, not a quick wire-up.
2026-09-20 02:42:36 +10:00
sam 800352a692 feat(gateway): give /status a real daemon-health surface (#892)
* feat(gateway): give /status a real daemon-health surface

Phase 2 of archie-core-wp9s. Phase 1 split the commands apart (health vs
work) but left /status with one aggregate queue line and a static runtime
block; it could not answer the questions an operator actually asks when
something feels wrong. /status now reports, between the queue line and the
runtime block:

- Broker: the live state of this process's own NATS connection
  (eventbus/nats.Client.Connected), not a fresh dial -- a probe would
  report the probe's reachability, not the daemon's.
- Containers: the pool's own active counter against the configured cap
  (Pool.Active/Cap), not a Docker listing, which would include containers
  this pool does not own. A zero cap means unlimited, so it is not shown
  as a ratio.
- Channels: the channel manager's real lifecycle state, including a
  failed adapter's reason (the documented decision: read the existing
  manager rather than re-deriving health per adapter).
- Chat model: last-known outcome of every chat-model call this process
  made, recorded once in sendChatTurn (the single choke point) and read
  back by /status. The second documented decision: never a live probe,
  so /status stays instant and reports what happened -- or says plainly
  that nothing has been attempted yet.
- Last poll: when the most recent poll pass began, stamped on both poll
  paths (Run's single-identity loop and runIdentities' per-identity
  goroutines). A pass that hangs leaves the stamp stale, which is the
  signal; stamping completion would leave a wedged poller looking
  healthy.

Truthful or absent: a section this process has no source for is omitted
rather than rendered as a zero. The standalone Gateway owns no poll loop,
pool or channel manager, so its /status reports broker and chat-model
health only; aggregating across processes would need a new daemon RPC and
is deliberately not built. Version stays out (/version owns it).

/status and /tasks descriptions now describe what each command actually
does on both published surfaces (Telegram menu/help and the shared
specs), pinned by a test that also fails if the two surfaces drift.

Sources are read through functions, not captured values: occupancy and
channel state move, and the daemon is built after the gateways, so a
source captured at construction would freeze /status on boot state.

Left in place deliberately: the Runtime (provider/model) block, which
Phase 1 flagged and which duplicates /model and /whoami. Removing it is a
product call on a command Sam uses daily, not a mechanical follow-up.

* docs(prds): record the two open /status health decisions as settled

Both were flagged in PR #892 as needing the maintainer's call; both are now
answered, and the PRD is the place a follow-up would look:

- The Runtime (provider/model) block stays. It is the one line in /status the
  ticket's own rule says should not be there, but /status is read daily to see
  the active model at a glance, so the duplication is deliberate. Recorded as a
  settled product decision so it is not removed later as an oversight.
- Health stays per-process. Pooling the daemon's and the Gateway's chat-model
  outcomes would need a new daemon<->gateway health RPC for one status line, and
  a per-process line is true where an aggregate would be a claim about a process
  the answering one cannot see.

/agents' published description is still left to archie-core-mxls, unchanged.
2026-09-20 00:30:23 +10:00
sam da28ddb5e6 refactor(ui): sever the archie-ui binary from every daemon runtime package
Completes the objective deletion gate for the UI Service
(archie-core-8cda.5.6): go list -deps ./cmd/archie-ui now links zero
banned packages -- no modernc.org/sqlite, no internal/store, gateway,
channels, memory, secret, agentexec, tools, skill, yaegiutil, curator,
or the workflow engine. The closure shrinks from 68 to 26 archie-core
packages. Each cut is a producer-side move; no consumer-side defensive
shims.

- Store contracts (TaskStore family, value types, sentinels) moved from
  internal/store to internal/domain/storecontract -- the approved contract
  location per state-store-contract rev. 2d, and NOT internal/contracts:
  that tree is buf generate output and tools/proto.sh check deletes
  hand-written files there. internal/store keeps aliases; sentinel
  message strings are byte-identical and remain the wire contract.
- Chat contracts (ChatContract, sessions, turns, streaming, commands)
  moved from internal/gateway to internal/domain/messaging; gateway keeps
  aliases. ToolCallEvent rendering helpers become gateway functions
  (methods cannot live on alias types).
- Task vocabulary (Task, statuses, mid-run Store, Definition) split into
  internal/domain/workflow/task; the pipeline engine stays in package
  workflow with aliases, so daemon-side callers are unchanged.
- overlay.DeniedKeys moved to internal/infrastructure/configuration:
  webui referenced the constant and that dragged the overlay SQLite
  store in.
- webui.Server's concrete runtime handles become webui-owned view
  interfaces with daemon-side bootstrap adapters: channel lifecycle
  (internal/channels/status), MemoryStatus, CuratorStatus, and a
  SkillCatalog seam replacing the direct skill.DefaultRoots/CatalogRoots
  calls. Each degrades to an empty page when unwired.
- config.SecretRef moved into internal/config (config vocabulary);
  internal/secret keeps an alias and Registry.Resolve absorbs the
  zero-value short-circuit.

Docs: state-store-contract rev. 2d relocation record; ui-service-boundary
deletion gate marked satisfied; ARCHITECTURE.md package map rows.
2026-09-10 03:03:33 +10:00