Commit Graph
15 Commits
Author SHA1 Message Date
sam 042a86ce4a fix(config): judge a database-owned setting at the layer that owns it
Review of the two previous commits found the split was incomplete in both
directions: the checks moved off the file load path were still enforced
anywhere the value could enter the system, and the fold reached only one of the
three overlay paths.

A stale database-owned TOML value could still stop archie-state-store. The
process seeds every control-plane resource from the same file config on a fresh
database, and the seed runs the resource's own validator; three of the five
values the loader now accepts aborted the boot there (verified by booting
RunStateStore before the fix). A seed that does not validate is now SKIPPED and
reported instead of fatal: the resource stays absent, so the file's value is the
one in effect and editing config.toml is still the fix -- the daemon and the
Gateway are where it fails closed, at boot.runtimeConfig.

The resource validators were weaker than configuration.Validate for exactly the
fields the split reclassified, so a value the file path used to refuse at load
became storable in the database, and then un-fixable by editing config.toml.
The trigger enum and the test-glob rule now come from the configuration package
rather than being restated, the container image is required there as it is in the
file, and a table test ties the two layers together value by value.

The directory form of -config-overlay kept the archie-core-e2e2 failure: it
decoded each overlay file straight into the populated config, so a map-valued
entry it only partly addressed still cleared the fields it did not name. Both
the overlay layer's main file and its feature files now fold, which is what
Loader.Dir's documented parity with Loader.Overlay has always claimed.

The overlay file's key report is derived from the apply's own view of the file as
well as the file decode's, because TOML matches keys case-insensitively and the
fold's yaml decode does not: a key only the file decode matched was consumed by
the report and dropped by the apply, a silent no-op that worked in the base file.

Tests: the state-store boot is asserted against the real process, the directory
overlay and the case-mismatched key each have one, and ImportConfig pins that a
refused seed is not stored.
2026-09-21 22:12:32 +10:00
sam 13efe4fd1a fix(config): stop judging database-owned settings on the file load path
Loader.finalize ran the whole validation over the file document, so a stale
TOML value in a setting the control plane owns failed startup -- including
archie-state-store, which resolves its file config and opens the store without
ever layering a database resource over it. docs/prds/runtime-control-plane.md,
"Bootstrap, migration, and recovery": after migration, settings in TOML are
ignored and cannot block State Store startup.

The checks over the settings a control plane resource is seeded from and
replaces wholesale now run only on the effective document: providers
(provider-settings), poll_interval and dispatch (scheduling-policy), containers
(container-runtime-policies) and the repository lists (repository-policies).
They moved from validate to validateDatabaseOwned; validateBootstrap keeps the
checks over settings no resource carries (forge intake, identities, nats,
memory, image, capture). File sources pass validateBootstrap and
Loader.ApplyOverlay, which layers overrides onto an effective document, passes
Validate.

Nothing stops being judged: the daemon and the Gateway both run the full
Validate in boot.runtimeConfig, the one place database resources are layered
in, before any capability starts -- so a stored value that will not validate
still stops the process rather than being published. Cases that asserted the
load path rejects a database-owned value now assert both layers, and
TestRuntimeConfigFailsClosedOnAnInvalidDatabaseValue pins the second.
2026-09-21 22:09:06 +10:00
sam fad7cdc92e fix(config): keep the fields a file overlay omits inside a map entry
Loader.overlayFile decoded the overlay file straight into the config the
base file produced, and a decode replaces a map-valued entry wholesale:
[services.state] target = ... in an overlay cleared the target_token the
base file set. That is archie-core-e2e2 on the user-visible -config-overlay
path -- deployments/dev.toml is the documented argument and sets
[services.state] target/listen, while target_token lives in the installed
config.toml. An enabled image.hosted entry lost its class and api_key_env
the same way, silently turning the provider off.

apply.go's foldOverrides is the fix that path was missing, so the file
overlay now goes through the same fold as the map overlay: the overlay file
parses to its raw mapping (decodeFileMapping), the fields of each partially
addressed map entry are folded forward out of the base config, and the
completed mapping is decoded over it. Loader.ApplyOverlay, ApplyOverlayValues
and apply.go stay: the hazard they were written for is this one.

An overlay file still reports its own unknown keys, from a separate decode
into a scratch value.
2026-09-21 22:09:06 +10:00
sam c75ed44721 feat(telegram): support secret refs for bot tokens (#705) 2026-09-03 14:52:31 +10:00
sam 972fb5b8e1 feat(deployment): default to managed NATS workers
Autonomous workflows now always hand a complete task to a scoped archie-agent container. Embedded NATS is the default and external NATS changes only broker placement; legacy agent and containers.enabled inputs decode for migration but cannot select an execution path. The daemon flushes task-event subscriptions before a worker can publish, and runtime updates snapshot and restore the configured worker image.
2026-08-24 14:56:57 +10:00
sam 5d00ec7b0c refactor(agentexec): remove legacy execution paths
Autonomous workflows now have exactly one production boundary: a task-scoped archie-agent receives a full-task core-NATS request and runs the worker-local stage loop. The legacy [agent] section remains decode-only for migration; it no longer selects runtime behavior.
2026-08-24 14:56:57 +10:00
sam 691e10c251 fix(telegram): compact tool output and drop turn caps
Render completed ToolCallEvents as Telegram-safe streaming blocks
with compact name/status and a short cleaned preview. Summarize
file contents and listings instead of dumping source or quoted
JSON. Collapse repeated legacy turn-budget failures.

Remove max_tokens turn gating and the 200000 turn_budget_chars
default so absent or commented config cannot restore an aggregate
cap. Keep per-result max_result_chars and spill.

Confine find/grep discovery to the active workspace/worktree.

Refs #604
2026-08-23 16:40:16 +00:00
sam 61b4b7b0cf refactor(store): replace legacy persistence with SQLite
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.

Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
2026-08-09 22:27:25 +10:00
sam 472d8d717c feat(config,channels): format-neutral config resolution and truthful channel state
Loader.Resolve() selects TOML/YAML files or a config directory as the
production entry point, replacing the TOML-only Overlay() as the default
path. Add internal/channels.Manager, reporting each adapter's actual
runtime lifecycle (configured/starting/running/degraded/failed) instead of
inferring health from configuration presence alone. Wire channel reload
through a per-adapter seam.

Fixes #55
2026-08-09 01:25:58 +10:00
sam e10cbbc92c refactor: pre-cleanup. 2026-08-06 01:40:15 +10:00
sam 0ddc43081e feat(setup): implement the interactive setup steps (archie-core-cbk)
internal/infrastructure/configuration/setup.Run drives operator
identity, LLM provider, models, forge, and Telegram chat channel
prompts and returns TOML edits -- not a config.Config. Building a
parallel in-memory representation to validate against was rejected
during design review: it's a second source of truth about the exact
thing (text vs. struct disagreeing) this whole feature exists to stop.
Tests instead render the edits against the real embedded template and
load the result through a real configuration.Loader, proving the
actual defaults/decode/validate path accepts it.

Two boot-time landmines in config.example.toml surfaced by that
approach, both fixed here:

- [providers.openai] ships active with api_key pointed at the bws
  engine. cmd/archied/provider_secrets.go resolves every entry in
  cfg.Providers at startup, not just the ones [models] references, and
  refuses to start the daemon if any fails -- so picking any other
  provider (or picking openai but skipping the key) left a config that
  wouldn't boot on a machine without the bws CLI installed (the common
  case). stepProvider now neutralises providers.openai.api_key to the
  zero-value secret ref whenever openai wasn't the actual choice.
- [[repos]] shipped active with a placeholder owner/name that
  validates cleanly (owner/name are only checked non-empty) and would
  have archied silently polling a repo that doesn't exist. Commented
  out in the template with an explanation; setup does not write this
  section since tomlwrite cannot target array-of-tables.
  TestExampleConfigLoads updated to expect zero repos.

Also: forge and provider secret env-var names are each defined in one
table (forgeToken, cloudProviders) read by both the config edit and
the secret sink, so the two can't name a secret differently -- the
exact bug class this feature exists to close for the installer, tested
directly (TestRun_ForgeTokenKeyMatchesStoredSecretKey compares the
decoded key against the key the secret was actually stored under, not
a hardcoded literal on both sides). Telegram writes chat.telegram.
token_env, not a secret.SecretRef -- TelegramConfig has no such field.
Telegram also requires at least one allowed_user_id: AllowedUserIDs is
deny-by-default, so a bot token with none configured answers nobody,
which reads as broken rather than as the safe default it is.

SecretSink is a caller-supplied sink (Put buffers, Commit persists),
not a return value: Run never hands back plaintext secrets, and the
caller is expected to call Commit only after proving the generated
config loads -- so a validation failure never leaves a secret written
without the config that references it.

Refs archie-core-cbk
2026-08-05 12:58:53 +10:00
sam 6c32550345 chore: replace personal owner names in tests and config with generic values
Test fixtures used 'sam', 'samcharles93', and 'sam/<repo>' as owner
names. These are personal and machine-specific — the repo should use
generic placeholders like 'acme' (matching the existing convention in
worktree_test.go).

config.example.toml: samcharles93 -> your-github-username
Test owner: sam -> acme
Repo refs: sam/archie-core -> acme/archie-core, etc.
Example skills: sam -> acme
2026-07-31 02:15:41 +10:00
sam bc5d8dcdc9 test: drop tests that grep source text instead of behaviour
These asserted on file contents rather than what the code does:
Dockerfile strings, deploy.yml YAML, CHANGELOG headings, and greps of
main.go for identifiers. Two also referenced config.docker.toml and
docker-compose.yml, which were never committed, so they could not pass
at all.

The worst compared strings.Index byte offsets in main.go to assert
startup ordering -- it would pass on code that never runs and fail on
a correct refactor that moved a line.
2026-07-31 01:13:16 +10:00
sam 6108fc5548 fix: use errors.Is/As for context errors and cleanup 2026-07-30 00:06:10 +10:00
sam c66c0a2a38 refactor(config): move loading into infrastructure/configuration
internal/config both defined the types every package imports and performed
the file I/O to produce them. Importing a settings struct therefore pulled
in a TOML decoder and a YAML decoder, and the shared model owned discovery,
decoding, overlays, defaulting and validation all at once. See
docs/architecture/configuration.md, which requires loading to move out and
internal/config to be dissolved.

internal/config is now types only -- no os, no filepath, no toml. The new
package owns loading, one concern per file:

  features.go    the closed set of recognised feature files
  discovery.go   directory scan and classification
  decode.go      TOML/YAML decoding, format dispatch
  defaults.go    filling absent values, no errors
  validate.go    judging supplied values, no mutation
  provenance.go  which file contributed what
  loader.go      Loader wiring the above; File/Overlay/Dir

Beyond relocating, this fixes what the old code conflated:

  - Defaulting and validation were interleaved. applyAgentDefaults and
    applyDispatchDefaults returned validation errors; validateContainers and
    validateSingleIdentity quietly mutated the config. There was no point at
    which you could see what the operator actually supplied. They are now
    separate passes: applyDefaults never fails, validate never writes.
  - Errors were fmt.Errorf strings prefixed "config: ", so a caller could not
    distinguish operator-fixable input from an unreadable file without
    matching message text. Five sentinels now carry that distinction.
  - dataHome() read XDG_DATA_HOME directly and discarded the error from
    os.UserHomeDir, so defaulting could not be tested without the
    environment. It is injectable on Loader and no longer swallows the error.
  - Nothing recorded where a value came from. With overlays, feature files
    and conf.d/ all able to set the same field, "which file set this?" was
    answerable only by re-deriving precedence by hand. Provenance records it.
  - Feature files were applied in map iteration order, making load order and
    any resulting error arbitrary. Discovery now returns them sorted.
  - loadMainConfig reported "no config.yaml or config.toml found in %s" with
    an empty string: the directory was lost when it was extracted from
    loadDirInto, so the error named no directory.
  - Unreadable or malformed conf.d/ extras were skipped silently, which
    presents as a setting the operator wrote simply not taking effect.
  - Decode errors from the main config were returned bare, without naming
    the file that failed to parse.

The 40 loader tests moved with the code and their assertions are unchanged,
so the defaulting and validation rewrite is behaviour-preserving.

Document.Config is the temporary seam: still internal/config.Config, named
as a field so the compromise is visible at each use site, with its deletion
condition recorded in doc.go. Building typed per-domain sections now, while
every consumer still reads config.Config, would mean two shapes kept in sync
by hand with no consumer for the new one; that belongs with the dissolution
work, not ahead of it.

No consumer outside cmd/archied changed.

Verified: go build, go vet, gofmt clean; go test ./internal/config/... and
./internal/infrastructure/configuration/... pass. Pre-existing gateway and
telegram failures are unchanged.
2026-07-29 11:07:41 +10:00