Commit Graph
7 Commits
Author SHA1 Message Date
sam a19cd24b83 feat(controlplane): derive resource schemas from the documents they describe
Twelve definitions each advertised a bare {"type":"object"} or {"type":"array"},
so no resource descriptor carried a single field: the Web UI had nothing to
label, describe or format a field with, and the one component that parsed a
schema was dead code. The constants could not be kept honest either -- a schema
written beside a struct drifts the moment a field is added, and nothing compares
the two.

Definition now names its document type and Descriptor derives the schema from it.
json tags give the keys, falling back to the Go field name because that is what
encoding/json stores; the Go type gives type and format; and a field's own
title, format and doc tags override. A duration is a string with format
"duration", which is how every duration field in the catalogue -- poll_interval,
max_uptime, rate_limit.window -- gets a client-side affordance from one rule
instead of twelve hand-written ones.

Deliberately structural: it emits no prose of its own. Field descriptions already
have one home (internal/webui/config_schema.go, keyed by dotted config path), and
a second would be two places to keep the same fact true. It also never emits
"required", because an empty string is legitimate for nearly every field here --
a blank workspace, a disabled rate limit -- so requiring most of them would
invite a client to enforce something the document does not mean.

Guarded by TestEveryResourceDescriptorDerivesProperties, which walks the catalogue
the production constructor builds and fails if any kind describes no fields --
the assertion that would have caught all twelve. It accepts a keyed collection's
additionalProperties as described, because a map document has no fixed keys.

Verified: controlplane, workflow, webui, archieui and staterpc suites, go vet,
go build ./..., golangci-lint run ./... (0 issues), docs:check.
2026-09-22 03:50:53 +10:00
sam 7f7eb573a1 fix(controlplane): a kind with no stored value leaves the file's value in force
The fix-closure review found the previous pass moved the failure instead of
ending it. A seed the resource validator refuses leaves the kind ABSENT, and
RuntimeConfig treated an absent kind as fatal, so archied and the Gateway
refused to start with "control-plane resource not found" -- a database-named
error that editing config.toml cannot clear. Reproduced before the fix: every
kind in the new table failed that way.

Two changes, one on each side of the seam:

- Client.RuntimeConfig (and RuntimeChatConfig) now treat a kind with no stored
  value as "the file document's value stays in effect", which is what the
  skip-and-report comment already claimed. The version map records no version
  for such a kind, so nothing is reported as applied. A value the file layer
  also refuses still fails closed, at configuration.Validate, with a message
  naming the file's own field -- and the file is the fix.

- Which repository lists are valid can no longer differ between the layers.
  configuration.ValidateRepositories is now the single rule set (owner and name
  present, no duplicates, compilable test glob) and the repository-policies
  resource validator delegates to it. The file layer had no duplicate rule
  while the resource layer did, so a duplicate [[repos]] entry passed the loader
  and then vanished into a skipped seed.

Also, from the same review: the scheduling validator's "0s" carve-out is gone,
so a zero poll interval is refused at write time as configuration.Validate
refuses it (there is no defaulting behind a stored value); applyOverlayFile's
doc now says which overlay forms fold (a conf.d extra does not, and nothing
reads cfg.Extra); and plugin-settings' no-op validator is a named function with
the reason it accepts everything.
2026-09-21 22:12:32 +10:00
sam 9a45a35c27 fix(config): judge a database-owned setting at the layer that owns it
Review of the two previous commits found the split was incomplete in both
directions: the checks moved off the file load path were still enforced
anywhere the value could enter the system, and the fold reached only one of the
three overlay paths.

A stale database-owned TOML value could still stop archie-state-store. The
process seeds every control-plane resource from the same file config on a fresh
database, and the seed runs the resource's own validator; three of the five
values the loader now accepts aborted the boot there (verified by booting
RunStateStore before the fix). A seed that does not validate is now SKIPPED and
reported instead of fatal: the resource stays absent, so the file's value is the
one in effect and editing config.toml is still the fix -- the daemon and the
Gateway are where it fails closed, at boot.runtimeConfig.

The resource validators were weaker than configuration.Validate for exactly the
fields the split reclassified, so a value the file path used to refuse at load
became storable in the database, and then un-fixable by editing config.toml.
The trigger enum and the test-glob rule now come from the configuration package
rather than being restated, the container image is required there as it is in the
file, and a table test ties the two layers together value by value.

The directory form of -config-overlay kept the archie-core-e2e2 failure: it
decoded each overlay file straight into the populated config, so a map-valued
entry it only partly addressed still cleared the fields it did not name. Both
the overlay layer's main file and its feature files now fold, which is what
Loader.Dir's documented parity with Loader.Overlay has always claimed.

The overlay file's key report is derived from the apply's own view of the file as
well as the file decode's, because TOML matches keys case-insensitively and the
fold's yaml decode does not: a key only the file decode matched was consumed by
the report and dropped by the apply, a silent no-op that worked in the base file.

Tests: the state-store boot is asserted against the real process, the directory
overlay and the case-mismatched key each have one, and ImportConfig pins that a
refused seed is not stored.
2026-09-21 22:12:32 +10:00
sam 08df95684c fix(statestore): read a kind the store does not hold as its seed, and stop validate touching the daemon's log
Four defects from the re-run contract review, all in the recovery path.

A store missing a control-plane kind was refused while a store with no
resources table at all was accepted -- two stances inside validateStore for
states the daemon cannot tell apart. The State Store seeds every kind from
its own config before it serves (RunStateStore -> ImportConfig), so a kind
the file does not hold yet is one archied reads the seed for: an upgrade that
adds a kind, and a validate before the State Store has started again, exited
1 with "read provider-settings: resource not found" for a store the daemon
boots on fine. StoredRuntimeConfig now reads a missing kind as the seed the
State Store would write for it (Definition.seededValue, shared with
ImportConfig so the two cannot drift), and the missing-table branch is the
same stance taken to its extreme.

defaultConfigPath() derived the daemon's config with os.UserConfigDir, which
disagrees with archied's configHome on darwin and rejects a relative
XDG_CONFIG_HOME outright -- leaving validate with an empty path and no
verdict, on a host where archied boots from relative-dir/archie/config.toml.
It is now the daemon's own rule, exported as archied.DefaultConfigPath and
used by the daemon's own -config default too, with a test pinning the
equality.

-h exited 2, the code documented as "the command line was wrong", while the
serve path in the same binary exits 0 and main.go advertises -h for these
subcommands. ErrHelp now returns 0.

validate resolved the boot config through loadConfig -> setupLogging, which
created cfg.Log.File's directory, opened the log, and wrote a line stamped
component="daemon" -- a diagnostic editing the deployment it inspects, and
rotating a log the daemon is writing. boot.stderrLog keeps setupLogging on
the stderr logger for the recovery path: no feed, no task-log registry, no
durable copy.

Tests: TestRecoveryValidateRunsTheGateBootRuns now asserts the unseeded and
the missing-kind stores are accepted (both exited 1 before), while the
bogus-policy case still refuses; TestRecoveryHelpExitsZero,
TestRecoveryValidateLeavesTheDaemonLogAlone and
TestRecoveryDefaultConfigFollowsTheDaemonsConfigHome are new. Each was run
red first, and the seed fallback and stderrLog were mutation-checked.
2026-09-21 22:04:42 +10:00
sam e7e5ddfba5 fix(statestore): make validate run boot's gate, and stop backup replacing the database
Two P1 findings from review, both about a claim outrunning the code.

validate answered a different question from boot's. Boot refuses on
configuration.Validate over the file config with every stored resource
layered onto it (internal/app/archied/control_plane.go:27), which is a
stricter check than the write path's per-resource Decode: validateScheduling
never inspects dispatch.trigger and explicitly permits "0s", while boot
rejects an unknown trigger and a non-positive poll interval. A store holding
such a value printed "is a valid store ... N stored resources validate" and
exited 0 while archied exited 1 on it -- a false green pushing an operator
toward the blunt restore instead of the surgical rollback.

validate now runs both layers and reports both, in boot's order:
store.ValidateFile, then the write path's ValidateStored, then the config
gate. The layering itself is now one implementation: controlplane's
runtimeConfigFrom takes a resourceReader, which the gRPC client and a new
Server.StoredRuntimeConfig (reading its own store) both provide, so the
offline check cannot drift from the daemon's. validate takes -config /
-config-overlay (default: the path the serve subcommand uses), because the
question is about the stored values layered onto that document.

backup had no guard against -out naming the database. It renames its
snapshot over the target, so `backup -db F -out F` replaced a database a
serving State Store may still have open and orphaned the WAL of the file it
just unlinked -- the state Restore refuses up front and maintenance.go's own
comment calls unrecoverable. backup now refuses with the same message
Restore uses, which is what makes its exemption from the ownership lock
sound.

Claims corrected where they ran ahead of the code: ValidateStored now says it
is the write path's check and not boot's, the Restore comment no longer
promises "never a mixture" for the sidecars it removes first, the usage text
advertises validate's -config, and the architecture table drops "must be
stopped" for validate (nothing enforces it; read-only opens skip the lock by
design) and states which two layers validate reports.

Tests: TestRecoveryValidateRunsTheGateBootRuns fails on the pre-fix code
("validate accepted a stored policy the daemon refuses to boot with") and
covers a store the State Store never seeded; TestRecoveryBackupRefuses-
TheDatabaseAsItsOwnSnapshot fails on the pre-fix code ("backup over the
database itself succeeded"). The validate-accept test now seeds the store
through ImportConfig and asserts the reported count is non-zero, and the
stored-resource case uses a value both gates refuse instead of one only the
write path does.
2026-09-21 22:04:42 +10:00
sam b8b477fb32 feat(statestore): offline backup, restore, validate and rollback
archie-state-store gains the offline recovery commands
docs/prds/runtime-control-plane.md ("Bootstrap, migration, and recovery")
asks for. They work on the task database file directly -- <db_path>-tasks.sqlite,
not db_path -- and need neither archied nor the Web UI, which matters because
the failure they exist for is the one that stops archied starting:

  backup   -db F -out F     consistent snapshot (VACUUM INTO), verified as a
                            usable store before it is published; safe against
                            a serving store
  restore  -db F -from F    replace the database with a snapshot, dropping the
                            replaced file's -wal/-shm
  validate -db F            readable, not corrupt, schema no newer than this
                            binary, every stored resource decoding under the
                            definition that owns it
  rollback -db F -kind K    replay an earlier revision of a stored resource
           [-revision N]    through the ordinary replace, recorded as a new
                            revision

No rollback RPC is added and none is needed: a revision carries its own value,
so restoring one is the same replace the dashboard and the channels already
use, by hand, with the store stopped.

"The State Store is stopped" is checked rather than assumed. The serving
process now takes an exclusive flock on <db>.lock for its whole life
(store.AcquireOwnership), and restore/rollback refuse with
`task database is owned by another process` while it holds it. backup is the
deliberate exception: the update installer runs inside the process it is
replacing and cannot stop the State Store.

scripts/archie-update-install takes its pre-flight snapshot through
`go run ./cmd/archie-state-store backup` from the release being installed
instead of its own sqlite3 shell, so the installer and an operator run one
implementation. The watchdog's refusal semantics are unchanged: an update that
cannot back up still must not start.

Docs: safe-change-and-recovery.md's "Recovery: not yet built" now describes
what exists.
2026-09-21 22:04:41 +10:00
sam 924b02a1b0 feat(controlplane): runtime administration over the State Store's gRPC server
The Web UI and messaging channels now read and write Archie's runtime
settings through one versioned, audited control-plane API hosted on the
State Store's existing gRPC server: catalog, query, command, watch. Each
feature owns its resource, validation and commands; storage only writes
versions and audit records.

Workflows move with it. Definitions become portable YAML compiled through
a closed, typed step registry and stored as a control-plane resource, so
the yaegi interpretation paths (internal/gate, wfeval, wfextract,
gateeval, gateextract, skillbuild) are deleted rather than carried.

The UI process reads definitions as data: the JSON projection lives in
internal/domain/workflow/task so a dashboard decodes the stored
collection without linking the workflow engine, which cmd/archie-ui's
deletion gate forbids. Shipped definitions ride the catalog descriptor
rather than a bespoke REST route, keeping the API to its four operations.

archie-messaging gains -state-store-target/-state-store-token and reads
channel settings from the database, failing closed when the State Store
is unreachable: the database is authoritative for those settings.
2026-09-21 08:17:42 +10:00