Commit Graph
20 Commits
Author SHA1 Message Date
sam ffdb1ce583 feat(configuration): name the instance with ARCHIE_INSTANCE and scope default paths by it 2026-10-19 10:12:48 +11:00
sam 0cc8e5e676 refactor: delete wrappers superseded by live entry points
Remove controlplanerpc.NewClient (superseded by NewRPCClient), Loader.Overlay
(-config-overlay reaches Resolve directly), and setup.Run (RunParams is
the live entry point). Move ResolverFunc into a test file, since only
tests adapt functions to the egress Resolver.
2026-10-14 23:10:35 +11:00
sam 8b1fafd1d6 fix(configuration): reject unknown conf.d files 2026-10-09 09:57:56 +11:00
sam 0ec8e6f272 refactor(configuration): delete the dead runtime config overlay
The file overlay path has folded partial map-valued entries since
b1d28164, answering xrux's open question: a decode that zeroes omitted
fields is the old behaviour, kept only by the now callerless runtime
overlay. Loader.ApplyOverlay, ApplyOverlayValues and
applySchedulingOverlay had no callers anywhere in the tree, so the fold
machinery stays and the dead runtime path goes.
2026-10-05 19:21:05 +11:00
sam 0a140a9c87 fix(configuration): an overlay stands in for a missing config file; dev overlay is complete 2026-10-04 03:47:17 +11:00
sam 02537366eb refactor(configuration,builtin): cut comments down to what the code does 2026-10-03 21:58:23 +10:00
sam 96e5d7b84b refactor: strip doc and issue narration from code comments
Remove PRD/doc paths, bead IDs, phase notes and deleted-test references
from comments; drop test-only hooks left by the test deletion. Includes
in-progress archied bootstrap, gateway and query changes.
2026-10-03 21:47:46 +10:00
sam 0263bf4e0d fix(configuration): resolve the default config path in one place
Four binaries derived the default with os.UserConfigDir while the daemon
resolved it from its own config home. The two agree on Linux with an
absolute XDG_CONFIG_HOME and diverge everywhere else, so one deployment
booted from a different config file depending on which binary started.

Move the rule into the package that owns configuration resolution
(internal/infrastructure/configuration, which already owns xdgDataHome and
DefaultWorkDir) as DefaultConfigDir/DefaultConfigPath, and have every
-config default take it from there: archie-gateway, archie-messaging,
archie-state-store (serve and recovery) and archie-ui, plus the daemon's
own parseArgs, setup generator and drain-marker path.

Fence the claim at the source, since the resolved value is only observable
by starting a process: no source under cmd/ or internal/app/archied may
call os.UserConfigDir, and every -config registration must default to the
helper.
2026-09-22 01:57:49 +10:00
sam 9a45a35c27 fix(config): judge a database-owned setting at the layer that owns it
Review of the two previous commits found the split was incomplete in both
directions: the checks moved off the file load path were still enforced
anywhere the value could enter the system, and the fold reached only one of the
three overlay paths.

A stale database-owned TOML value could still stop archie-state-store. The
process seeds every control-plane resource from the same file config on a fresh
database, and the seed runs the resource's own validator; three of the five
values the loader now accepts aborted the boot there (verified by booting
RunStateStore before the fix). A seed that does not validate is now SKIPPED and
reported instead of fatal: the resource stays absent, so the file's value is the
one in effect and editing config.toml is still the fix -- the daemon and the
Gateway are where it fails closed, at boot.runtimeConfig.

The resource validators were weaker than configuration.Validate for exactly the
fields the split reclassified, so a value the file path used to refuse at load
became storable in the database, and then un-fixable by editing config.toml.
The trigger enum and the test-glob rule now come from the configuration package
rather than being restated, the container image is required there as it is in the
file, and a table test ties the two layers together value by value.

The directory form of -config-overlay kept the archie-core-e2e2 failure: it
decoded each overlay file straight into the populated config, so a map-valued
entry it only partly addressed still cleared the fields it did not name. Both
the overlay layer's main file and its feature files now fold, which is what
Loader.Dir's documented parity with Loader.Overlay has always claimed.

The overlay file's key report is derived from the apply's own view of the file as
well as the file decode's, because TOML matches keys case-insensitively and the
fold's yaml decode does not: a key only the file decode matched was consumed by
the report and dropped by the apply, a silent no-op that worked in the base file.

Tests: the state-store boot is asserted against the real process, the directory
overlay and the case-mismatched key each have one, and ImportConfig pins that a
refused seed is not stored.
2026-09-21 22:12:32 +10:00
sam 60292e3e88 fix(config): stop judging database-owned settings on the file load path
Loader.finalize ran the whole validation over the file document, so a stale
TOML value in a setting the control plane owns failed startup -- including
archie-state-store, which resolves its file config and opens the store without
ever layering a database resource over it. docs/prds/runtime-control-plane.md,
"Bootstrap, migration, and recovery": after migration, settings in TOML are
ignored and cannot block State Store startup.

The checks over the settings a control plane resource is seeded from and
replaces wholesale now run only on the effective document: providers
(provider-settings), poll_interval and dispatch (scheduling-policy), containers
(container-runtime-policies) and the repository lists (repository-policies).
They moved from validate to validateDatabaseOwned; validateBootstrap keeps the
checks over settings no resource carries (forge intake, identities, nats,
memory, image, capture). File sources pass validateBootstrap and
Loader.ApplyOverlay, which layers overrides onto an effective document, passes
Validate.

Nothing stops being judged: the daemon and the Gateway both run the full
Validate in boot.runtimeConfig, the one place database resources are layered
in, before any capability starts -- so a stored value that will not validate
still stops the process rather than being published. Cases that asserted the
load path rejects a database-owned value now assert both layers, and
TestRuntimeConfigFailsClosedOnAnInvalidDatabaseValue pins the second.
2026-09-21 22:09:06 +10:00
sam b1d28164cc fix(config): keep the fields a file overlay omits inside a map entry
Loader.overlayFile decoded the overlay file straight into the config the
base file produced, and a decode replaces a map-valued entry wholesale:
[services.state] target = ... in an overlay cleared the target_token the
base file set. That is archie-core-e2e2 on the user-visible -config-overlay
path -- deployments/dev.toml is the documented argument and sets
[services.state] target/listen, while target_token lives in the installed
config.toml. An enabled image.hosted entry lost its class and api_key_env
the same way, silently turning the provider off.

apply.go's foldOverrides is the fix that path was missing, so the file
overlay now goes through the same fold as the map overlay: the overlay file
parses to its raw mapping (decodeFileMapping), the fields of each partially
addressed map entry are folded forward out of the base config, and the
completed mapping is decoded over it. Loader.ApplyOverlay, ApplyOverlayValues
and apply.go stay: the hazard they were written for is this one.

An overlay file still reports its own unknown keys, from a separate decode
into a scratch value.
2026-09-21 22:09:06 +10:00
sam 6bdbd5e31e feat(config): key services by name through a registry
Services was a struct with one field per service, so each service's name was
stated three times: the field, the loader's defaulting branch, and every
consumer's field access. Adding one meant editing all three, and nothing
stopped them disagreeing. The drift was already there: stateStoreResolvedToken
and gatewayResolvedToken were identical apart from the env var each hard-coded.

Services becomes map[string]ServiceConnection, and RegisterService declares a
service once with the five fields it has. Everything that differs between
services is now an argument rather than a branch elsewhere: an empty registered
target means the operator must supply one, which is how services.state.target
stays required while the gateway's is defaulted, and the registered token env
var replaces the two copied resolvers with Services.ResolvedToken.

Defaulting iterates the registry, so a new service needs no loader edit.
Consumers read Services.Get(name).

Keeping a typo'd section reported took explicit work. [services.gatway] used to
land in toml.MetaData.Undecoded() because the struct had no such field; a map
decodes any section, so it would have loaded silently and left the daemon
dialing the default. unregisteredServiceKeys asks the registry which names are
real, and the test for it fails with an empty UnknownKeys when that check is
removed.

The TOML shape is unchanged, so no deployment profile, config.example.toml or
install.sh edit is needed.

Refs: archie-core-u23g
2026-09-18 23:32:36 +10:00
sam 8139c0a3bf feat(configuration): report config keys no decode target consumed
decodeTOML discarded toml.DecodeFile's metadata, so a misspelled or
renamed key parsed, validated and silently did nothing. Document.UnknownKeys
now carries the dotted key paths a loaded file's decode targets left
undecoded.

One file feeds two decode targets (&doc.Config and &doc.Scheduling), so a
key legitimately owned by one shows up as undecoded in the other -- a key
is only reported when both leave it undecoded (unknownKeys intersects the
two sets), which is what keeps a valid [scheduling] block from being
misreported as unknown on every config that uses it.

YAML config sources are unchanged for now (no equivalent metadata hook in
the yaml decoder); this covers the TOML path, which is what
config.example.toml, every shipped deployments/*.toml profile, and
install.sh's generated config.toml all use.
2026-09-15 05:07:54 +10:00
sam dd3ae02472 fix(scheduling): respect configuration ownership 2026-09-14 23:25:09 +10:00
sam 63cff4fe53 refactor: flatten nested config and tool fallback logic 2026-08-16 01:20:48 +10:00
sam 479ed95a91 fix(config): close review blocker on the PATCH path; deep-copy snapshots; atomics for shared state
Second adversarial pass findings, fixed:

BLOCKER -- PATCH silently dropped every dotted key. The UI sends
dotted keys ('budgets.max_steps') but ApplyOverlayValues fed them
straight to yaml.Unmarshal, which cannot decode a dotted key as a
struct field and silently ignored it: 200 OK, zero effect, and the row
still persisted (poisoning later reloads with an unvalidated value).
The reload path worked only because overlay.Snapshot nests first. Fix:
applyDottedOverlay nests dotted keys via overlay.Nest before decode --
the same nesting Snapshot performs -- and is pinned by
TestApplyDottedOverlayNestsBeforeDecode, which fails against the old
behaviour.

MAJOR -- shallow copies let overlay decode mutate the published
snapshot. Holder.Get and *doc are shallow; yaml decodes maps in place,
so a crafted nested update (or a failed boot overlay apply) mutated
shared maps of the published config before validation. Fix: config.Clone
deep-copies every reference field (Models, Providers, Repos, Identities,
Dispatch.Labels, Agent.Env, Chat/Telegram slices, Tools.MCPServers,
Memory.ProviderConfig, WebFetch.Enabled pointer, Extra), used by the
PATCH path (next := d.Cfg.Get().Clone()) and by Loader.ApplyOverlay,
which now never mutates its input (clones Config and the provenance
chain internally), so boot and reload pass the base document directly
and a failed apply leaves it exactly as it was. Pinned by
TestConfigCloneDeepCopiesReferenceFields.

MAJOR -- data races on captured main.go variables. bootOverlayErr and
currentProvenance were plain values written by the SIGHUP goroutine and
the PATCH/Reset handlers and read by every GET /api/config; -race passed
only because no test exercised the composition root concurrently. Both
are now atomic.Pointer; the LastReload closure and the provenance
dedup read them atomically.

MINOR -- MaxConcurrency removed from the reloadable allowlist. The
dispatchers re-read it but the startup-built container pool captures it
(container/pool.go:94,163), so 'every consumer re-reads it' is false and
a change only partially applies. It now warns requires-restart; the
criterion comment on reloadableFields documents the pool capture.

MINOR -- ResetConfig now resolves and validates the target state BEFORE
deleting the row, so a broken file cannot leave the store and the
running config disagreeing; the provenance chain no longer grows
unboundedly (the runtime-overlay origin replaces, not appends).

Also: holder.go no longer references a file watcher that does not exist.

Verified: task check clean; -race clean on cmd/archied, webui, daemon,
config; new findings from the second adversarial pass are closed.
2026-08-10 01:23:34 +10:00
sam da2d93a830 feat(config): runtime config overlay store and PATCH /api/config
The second half of #348's storage layer: dashboard-edited overrides
live in their own SQLite file, separate from the task and conversation
stores so each file owns its user_version and migrator with no
contention (configDBPath -> cfg.DBPath + "-config.sqlite", matching
the taskDBPath / conversationDBPath convention).

Store (internal/infrastructure/configuration/overlay):
- config_overlay(key TEXT PRIMARY KEY, value TEXT NOT NULL,
  updated_at, updated_by), own migration, WAL.
- Set refuses DeniedKeys (db_path, work_dir) at write time as well as
  at the API layer; the map is exported so the dashboard can show the
  reason instead of silently omitting the edit affordance.
- Snapshot JSON-decodes each value and nests dotted keys
  ("budgets.max_steps" -> map[b][m]) ready to decode into Config.

Loader (ApplyOverlay / ApplyOverlayValues):
- Decodes a nested overlay map with the same field-level precedence as
  a file overlay, records the runtime-overlay origin, then applies
  defaults and validates. Used at boot, on SIGHUP reload (ReloadController
  gained an optional WithOverlay source) and by the PATCH path.

Boot and reload:
- A broken overlay (unopenable store, unreadable snapshot, values that
  fail validation) degrades to file config alone -- it must never brick
  the daemon. --no-config-overlay / ARCHIE_SKIP_CONFIG_OVERLAY=1 skips
  the store entirely; the flag is now a real gate, not a log-only stub.

PATCH /api/config (webui):
- handleConfigUpdate decodes {"updates": {dotted.key: value}}, calls
  the composition-wired UpdateConfig seam, and classifies errors via
  webui sentinels: ErrConfigUpdateInvalid -> 400 (denylisted key,
  failed validation, unparseable value), ErrConfigUpdateUnavailable ->
  503 (overlay skipped by recovery flag), anything else -> 500.
- The seam (main.go) applies updates to a COPY of the published
  config, validates the materialised result, persists to the overlay,
  then publishes through the same path as reload -- it never calls
  Set directly from the handler, since web.Cfg is the daemon's own
  Holder.
- ConfigView gains Locked (dotted key -> reason) so the UI disables
  denylisted rows explicitly.

Verified: task check clean; new tests cover the store round-trip,
denied keys, loader overlay layering/validation/provenance, reload
with and without overlay, and the PATCH handler's 200/400/503 paths.
2026-08-10 00:54:23 +10:00
sam 472d8d717c feat(config,channels): format-neutral config resolution and truthful channel state
Loader.Resolve() selects TOML/YAML files or a config directory as the
production entry point, replacing the TOML-only Overlay() as the default
path. Add internal/channels.Manager, reporting each adapter's actual
runtime lifecycle (configured/starting/running/degraded/failed) instead of
inferring health from configuration presence alone. Wire channel reload
through a per-adapter seam.

Fixes #55
2026-08-09 01:25:58 +10:00
sam 1579b027b8 feat(configuration): export Validate for archied setup to call
Loader.finalize's decode-then-validate sequence called an unexported
validate(cfg *config.Config), so nothing outside the package -- e.g.
archied setup, which builds a config.Config in memory before writing
it to disk -- could run the same checks. Validate wraps it; Loader now
calls the exported form internally (single call site, no behavior
change).

Documented the sharp edge this alone doesn't solve: Validate doesn't
apply defaults (agent.mode, dispatch.trigger, forge.type all only pass
once defaulted), and defaulting is still an unexported method on
*Loader. A cfg setup builds by hand needs those fields filled in
itself, or -- safer, since it can't drift from the real load path --
setup should render to TOML and load it back through a Loader rather
than call Validate on the in-memory value directly.

Refs archie-core-ewk.1
2026-08-05 12:44:42 +10:00
sam c66c0a2a38 refactor(config): move loading into infrastructure/configuration
internal/config both defined the types every package imports and performed
the file I/O to produce them. Importing a settings struct therefore pulled
in a TOML decoder and a YAML decoder, and the shared model owned discovery,
decoding, overlays, defaulting and validation all at once. See
docs/architecture/configuration.md, which requires loading to move out and
internal/config to be dissolved.

internal/config is now types only -- no os, no filepath, no toml. The new
package owns loading, one concern per file:

  features.go    the closed set of recognised feature files
  discovery.go   directory scan and classification
  decode.go      TOML/YAML decoding, format dispatch
  defaults.go    filling absent values, no errors
  validate.go    judging supplied values, no mutation
  provenance.go  which file contributed what
  loader.go      Loader wiring the above; File/Overlay/Dir

Beyond relocating, this fixes what the old code conflated:

  - Defaulting and validation were interleaved. applyAgentDefaults and
    applyDispatchDefaults returned validation errors; validateContainers and
    validateSingleIdentity quietly mutated the config. There was no point at
    which you could see what the operator actually supplied. They are now
    separate passes: applyDefaults never fails, validate never writes.
  - Errors were fmt.Errorf strings prefixed "config: ", so a caller could not
    distinguish operator-fixable input from an unreadable file without
    matching message text. Five sentinels now carry that distinction.
  - dataHome() read XDG_DATA_HOME directly and discarded the error from
    os.UserHomeDir, so defaulting could not be tested without the
    environment. It is injectable on Loader and no longer swallows the error.
  - Nothing recorded where a value came from. With overlays, feature files
    and conf.d/ all able to set the same field, "which file set this?" was
    answerable only by re-deriving precedence by hand. Provenance records it.
  - Feature files were applied in map iteration order, making load order and
    any resulting error arbitrary. Discovery now returns them sorted.
  - loadMainConfig reported "no config.yaml or config.toml found in %s" with
    an empty string: the directory was lost when it was extracted from
    loadDirInto, so the error named no directory.
  - Unreadable or malformed conf.d/ extras were skipped silently, which
    presents as a setting the operator wrote simply not taking effect.
  - Decode errors from the main config were returned bare, without naming
    the file that failed to parse.

The 40 loader tests moved with the code and their assertions are unchanged,
so the defaulting and validation rewrite is behaviour-preserving.

Document.Config is the temporary seam: still internal/config.Config, named
as a field so the compromise is visible at each use site, with its deletion
condition recorded in doc.go. Building typed per-domain sections now, while
every consumer still reads config.Config, would mean two shapes kept in sync
by hand with no consumer for the new one; that belongs with the dissolution
work, not ahead of it.

No consumer outside cmd/archied changed.

Verified: go build, go vet, gofmt clean; go test ./internal/config/... and
./internal/infrastructure/configuration/... pass. Pre-existing gateway and
telegram failures are unchanged.
2026-07-29 11:07:41 +10:00