diff --git a/docs/architecture/configuration.md b/docs/architecture/configuration.md index 4d8d8077f..fe7f27980 100644 --- a/docs/architecture/configuration.md +++ b/docs/architecture/configuration.md @@ -244,7 +244,7 @@ The application currently changes behavioural values through unrelated paths: | Active provider selection | `chatModelManager.SetActiveProvider` changes provider selection. | A conversation/session command changing runtime state, with the same distinction from provider-instance configuration. | | Persona selection | `PersonaRegistry.SetActive` mutates an in-memory per-session choice. | Agent-system session state. Persistence and audit requirements are owned by that domain, not the configuration loader. | | File and environment overlays | Applied while loading `internal/config`; defaults are filled by mutation in `finalize`. | Configuration-infrastructure input processing that produces separately typed candidates. Defaults and derivations remain visible in provenance. | -| SIGHUP config reload (2026-08) | Re-resolves the file config, validates, then atomically republishes through `config.Holder`. A failed reload keeps the running config and records `last_error`/`last_error_at` in `/api/config`. It does not re-read the control plane, so database-owned settings revert to their file values until restart (`archie-core-ju85`). | Candidate promotion with last-known-good retention: the running snapshot is untouched on failure. Reload must re-apply the database layer. A bounded observation period and actor audit are not yet implemented. | +| SIGHUP config reload (2026-08) | Re-resolves the file config, re-applies the database layer over it, validates, then atomically republishes through `config.Holder`. A failed reload, whether the file or the control plane caused it, keeps the running config and records `last_error`/`last_error_at` in `/api/config`. | Candidate promotion with last-known-good retention: the running snapshot is untouched on failure. A bounded observation period and actor audit are not yet implemented. | | Control-plane resource command (2026-09) | `ControlPlaneService.Command` validates a whole resource document against its owning feature, then writes it with an audit record (actor, source, request ID, expected version) under optimistic concurrency. Hosted on the State Store's gRPC server; the Web UI and messaging channels are its only clients. See `docs/prds/runtime-control-plane.md`. | Met for validate-stage-promote, actor audit and versioned history. Per-field health observation and validate-before-switch on live apply are not yet implemented. | Any further mutation seam discovered during migration MUST be added here before @@ -307,11 +307,13 @@ deliberately. A reload that changes any non-allowlisted field logs ### SIGHUP reload -SIGHUP re-resolves the file config, validates, and atomically republishes -through the shared Holder. A failed reload keeps the running config and records -`last_error`/`last_error_at` in `/api/config` so the stale state is visible -where the operator is looking. The three poll tickers re-read `PollInterval` per iteration and reset on change, -so a reloaded interval genuinely takes effect. +SIGHUP re-resolves the file config, re-applies the database layer over it +(`boot.runtimeConfig`, the same call boot makes), validates, and atomically +republishes through the shared Holder. A failed reload keeps the running config +and records `last_error`/`last_error_at` in `/api/config` so the stale state is +visible where the operator is looking. The three poll tickers re-read +`PollInterval` per iteration and reset on change, so a reloaded interval +genuinely takes effect. ### Settings writes go through the control plane @@ -327,9 +329,13 @@ Two properties matter to anyone changing this area: layers database resources over it at boot (`boot.loadRuntimeConfig`). Values the daemon needs before it can reach the State Store — the database path, listen addresses, credentials — stay file-only by design. -- **Reload does not re-read them.** SIGHUP republishes the file document alone, - which reverts database-owned settings until restart. Tracked as - `archie-core-ju85`; see the ledger above. +- **Reload re-reads them.** SIGHUP resolves the file document and then runs the + same `boot.runtimeConfig` layering boot does, so a reload cannot revert a + database-owned setting to its file value. A control plane that cannot answer + fails the reload rather than publishing the file's values over the running + ones. Workflow execution budgets arrive on a watch rather than in a resource + query, so boot records the last published settings and the reload re-applies + them from there. ## Completion criteria diff --git a/docs/architecture/safe-change-and-recovery.md b/docs/architecture/safe-change-and-recovery.md index d3f3bae6a..564e27e00 100644 --- a/docs/architecture/safe-change-and-recovery.md +++ b/docs/architecture/safe-change-and-recovery.md @@ -113,7 +113,7 @@ posture is the opposite: **the daemon fails closed.** | --- | --- | --- | | State Store unreachable | `boot.loadRuntimeConfig` returns the dial error and `archied` exits non-zero | stderr: `runtime settings unavailable` | | A stored resource fails validation | Same: `validate database settings` wraps the error and the daemon exits | Same | -| SIGHUP reload with a bad file | Running config is kept; the reload records `last_error` / `last_error_at` | `/api/config` → `reload.last_error` → dashboard banner | +| SIGHUP reload with a bad file, or a control plane that cannot be read | Running config is kept; the reload records `last_error` / `last_error_at`. The reload re-applies the database layer, so it never publishes file values over the running database-owned ones | `/api/config` → `reload.last_error` → dashboard banner | Failing closed is deliberate for settings the database owns: falling back to the file values would restore the second source of truth the control plane exists to diff --git a/internal/app/archied/bootstrap.go b/internal/app/archied/bootstrap.go index f908046f3..2a18593bc 100644 --- a/internal/app/archied/bootstrap.go +++ b/internal/app/archied/bootstrap.go @@ -115,7 +115,11 @@ type boot struct { stateStoreGrants *staterpc.GrantIssuer stateStoreToken string controlPlane *controlplane.Client - chatSessionStore gateway.SessionStore + // executionSettings is the last workflow execution settings the control + // plane published. They arrive on a watch rather than in the file + // document, so reloadConfig re-applies them from here. + executionSettings atomic.Pointer[workflow.ExecutionSettings] + chatSessionStore gateway.SessionStore catalog modelcatalog.Snapshot catalogModels []string @@ -1391,23 +1395,7 @@ func (b *boot) publishConfigSnapshot(ctx context.Context) { // /api/config. func (b *boot) wireConfigPublishing(ctx context.Context, cfgPath, overlayPath string) { log := b.log - reloadController := newReloadController(b.loader, cfgPath, overlayPath, func(doc *configuration.Document) { - // Boot merges catalog-discovered providers into cfg before the - // Holders are seeded; publishing the raw reloaded document would - // drop them from the running config even though the file is - // unchanged. Re-apply the same merge (idempotent: a catalog that - // failed to load merges to identity). - applyModelCatalog(&doc.Config, b.catalog) - old := b.d.Cfg.Get() - b.currentProvenance.Store(&doc.Provenance) - b.publishConfig(ctx, doc.Config) - if fields := changedNonReloadableFields(old, doc.Config); len(fields) > 0 { - log.Warn("config reloaded; some changes require a restart", - "fields", fields, "paths", doc.Provenance.Paths()) - } else { - log.Info("config reloaded", "paths", doc.Provenance.Paths()) - } - }) + reloadController := newReloadController(b.loader, cfgPath, overlayPath, b.reloadConfig) b.lastReload = func() config.ReloadStatus { st := reloadController.Status() return st diff --git a/internal/app/archied/control_plane.go b/internal/app/archied/control_plane.go index 142ac9ab9..297cf8879 100644 --- a/internal/app/archied/control_plane.go +++ b/internal/app/archied/control_plane.go @@ -3,25 +3,75 @@ package archied import ( "context" "fmt" + "time" "github.com/samcharles93/archie-core/internal/config" "github.com/samcharles93/archie-core/internal/domain/workflow" "github.com/samcharles93/archie-core/internal/infrastructure/configuration" ) +// runtimeConfig layers every database-owned setting over a file config and +// validates the result. Boot and the SIGHUP reload both go through it: the +// reload re-resolves the file document alone, so republishing that document +// as it stands reverts each of these layers to its file value until the +// process restarts (archie-core-ju85). +func (b *boot) runtimeConfig(ctx context.Context, base config.Config) (config.Config, error) { + cfg, err := b.controlPlane.RuntimeConfig(ctx, base) + if err != nil { + return config.Config{}, err + } + if settings := b.executionSettings.Load(); settings != nil { + applyExecutionBudgets(&cfg, *settings) + } + if err := configuration.Validate(&cfg); err != nil { + return config.Config{}, fmt.Errorf("validate database settings: %w", err) + } + return cfg, nil +} + func (b *boot) loadRuntimeConfig(ctx context.Context) error { - cfg, err := b.controlPlane.RuntimeConfig(ctx, b.cfg) + cfg, err := b.runtimeConfig(ctx, b.cfg) if err != nil { return err } - if err := configuration.Validate(&cfg); err != nil { - return fmt.Errorf("validate database settings: %w", err) - } b.cfg = cfg b.cfgHolder.Set(cfg) return nil } +// reloadConfig publishes a freshly resolved file document as the running +// config. The database layer is re-applied first, and a control plane that +// cannot answer aborts the reload: the running config already holds those +// values, so publishing the file's over them is the worse outcome. +func (b *boot) reloadConfig(ctx context.Context, doc *configuration.Document) error { + // Boot merges catalog-discovered providers into cfg before the + // Holders are seeded; publishing the raw reloaded document would + // drop them from the running config even though the file is + // unchanged. Re-apply the same merge (idempotent: a catalog that + // failed to load merges to identity). + applyModelCatalog(&doc.Config, b.catalog) + // Bounded: this runs on the signal loop, which handles nothing else + // while it waits, and a State Store that has stopped answering must + // surface as a failed reload rather than a SIGHUP that never returns. + queryCtx, cancel := context.WithTimeout(ctx, 10*time.Second) + defer cancel() + cfg, err := b.runtimeConfig(queryCtx, doc.Config) + if err != nil { + return fmt.Errorf("runtime settings unavailable: %w", err) + } + doc.Config = cfg + old := b.cfgHolder.Get() + b.currentProvenance.Store(&doc.Provenance) + b.publishConfig(ctx, cfg) + if fields := changedNonReloadableFields(old, cfg); len(fields) > 0 { + b.log.Warn("config reloaded; some changes require a restart", + "fields", fields, "paths", doc.Provenance.Paths()) + } else { + b.log.Info("config reloaded", "paths", doc.Provenance.Paths()) + } + return nil +} + func (b *boot) startWorkflowExecutionSettings(ctx context.Context) error { settings, version, err := b.controlPlane.WorkflowExecutionSettings(ctx) if err != nil { @@ -44,18 +94,22 @@ func (b *boot) startWorkflowExecutionSettings(ctx context.Context) error { return nil } +// applyWorkflowExecutionSettings records the settings as well as applying +// them: they arrive on a watch rather than in the file document, so a reload +// has nowhere else to read them back from. func (b *boot) applyWorkflowExecutionSettings(settings workflow.ExecutionSettings, version int64) { + b.executionSettings.Store(&settings) cfg := b.cfgHolder.Get().Clone() - cfg.Budgets = executionBudgets(settings) - for i := range cfg.Identities { - cfg.Identities[i].Budgets = cfg.Budgets - } + applyExecutionBudgets(&cfg, settings) b.cfgHolder.Set(cfg) b.log.Info("workflow execution settings applied", "version", version) } -func executionBudgets(settings workflow.ExecutionSettings) config.Budgets { - return config.Budgets{ +func applyExecutionBudgets(cfg *config.Config, settings workflow.ExecutionSettings) { + cfg.Budgets = config.Budgets{ MaxSteps: settings.MaxModelToolSteps, WallClock: config.Duration(settings.MaxRuntime), GateMaxFailures: settings.MaxConsecutiveGateFailures, } + for i := range cfg.Identities { + cfg.Identities[i].Budgets = cfg.Budgets + } } diff --git a/internal/app/archied/control_plane_reload_test.go b/internal/app/archied/control_plane_reload_test.go new file mode 100644 index 000000000..75f3267a6 --- /dev/null +++ b/internal/app/archied/control_plane_reload_test.go @@ -0,0 +1,157 @@ +package archied + +import ( + "context" + "encoding/json" + "log/slog" + "testing" + "time" + + "google.golang.org/grpc" + + "github.com/samcharles93/archie-core/internal/app/controlplane" + "github.com/samcharles93/archie-core/internal/config" + pb "github.com/samcharles93/archie-core/internal/contracts/controlplane/v1" + "github.com/samcharles93/archie-core/internal/domain/workflow" + "github.com/samcharles93/archie-core/internal/infrastructure/configuration" +) + +// controlPlaneStub answers Query from a fixed resource map. queryErr, when +// set, fails every Query so a test can drive the unreachable-State-Store +// path. +type controlPlaneStub struct { + values map[string]any + queryErr error +} + +func (*controlPlaneStub) Catalog(context.Context, *pb.CatalogRequest, ...grpc.CallOption) (*pb.CatalogResponse, error) { + return &pb.CatalogResponse{}, nil +} + +func (c *controlPlaneStub) Query(_ context.Context, request *pb.QueryRequest, _ ...grpc.CallOption) (*pb.QueryResponse, error) { + if c.queryErr != nil { + return nil, c.queryErr + } + value, err := json.Marshal(c.values[request.Kind]) + return &pb.QueryResponse{Resource: &pb.Resource{Kind: request.Kind, Version: 3, ValueJson: value}}, err +} + +func (*controlPlaneStub) History(context.Context, *pb.HistoryRequest, ...grpc.CallOption) (*pb.HistoryResponse, error) { + panic("unexpected History") +} + +func (*controlPlaneStub) Command(context.Context, *pb.CommandRequest, ...grpc.CallOption) (*pb.CommandResponse, error) { + panic("unexpected Command") +} + +func (*controlPlaneStub) Watch(context.Context, *pb.WatchRequest, ...grpc.CallOption) (grpc.ServerStreamingClient[pb.WatchResponse], error) { + panic("unexpected Watch") +} + +// databaseOwnedResources is a control-plane projection whose every value +// differs from fileConfig below, so any field that comes back with the file +// value after a reload is a field the reload reverted. +func databaseOwnedResources() map[string]any { + return map[string]any{ + controlplane.ProviderSettingsKind: map[string]any{"main": map[string]any{"class": "openai", "base_url": "https://db.example.com"}}, + controlplane.ModelRoleAssignmentsKind: map[string]string{"builder": "main/db-model"}, + controlplane.RepositoryPoliciesKind: []config.Repo{{Owner: "acme", Name: "from-database"}}, + controlplane.ChannelSettingsKind: map[string]any{"operator": "database", "max_steps": 20, "models": []string{"main/db-model"}, "email": map[string]any{}, "webhook": map[string]any{}, "telegram": map[string]any{"allowed_user_ids": []int64{42}}, "rate_limit": map[string]any{}, "workspace": "/workspace"}, + controlplane.SchedulingPolicyKind: map[string]any{"poll_interval": "2m", "max_retries": 7, "dispatch": map[string]any{"trigger": "assignee"}}, + controlplane.ToolSettingsKind: map[string]any{"mcp_servers": []map[string]any{}, "policy": map[string]any{}, "web_fetch": map[string]any{}, "minimax": map[string]any{}}, + controlplane.PluginSettingsKind: map[string]any{"plugin_dir": "/db/plugins", "module_dir": "/db/modules", "secret_engine_dir": "/db/secrets", "skills_dir": "/db/skills"}, + controlplane.ContainerRuntimePoliciesKind: config.ContainerConfig{Image: "archie:from-database", PullPolicy: "missing"}, + } +} + +// fileConfig is what the loader produces from config.toml alone: the layer a +// SIGHUP reload re-resolves, holding none of the database's values. +func fileConfig() config.Config { + return config.Config{ + BotUser: "widget", + Forge: config.Forge{Type: "github"}, + Providers: map[string]config.Provider{"file": {Class: "openai"}}, + Models: map[string]string{"builder": "file/model"}, + Repos: []config.Repo{{Owner: "acme", Name: "from-file"}}, + Chat: config.ChatConfig{Operator: "file"}, + PollInterval: config.Duration(time.Minute), + MaxRetries: 1, + PluginDir: "/file/plugins", + Containers: config.ContainerConfig{Image: "archie:from-file", PullPolicy: "missing"}, + Budgets: config.Budgets{MaxSteps: 1, WallClock: config.Duration(time.Minute)}, + } +} + +func newReloadBoot(t *testing.T, stub *controlPlaneStub) *boot { + t.Helper() + b := &boot{ + cfg: fileConfig(), + log: slog.New(slog.DiscardHandler), + controlPlane: controlplane.NewRPCClient(stub), + } + b.cfgHolder = config.NewHolder(b.cfg) + return b +} + +// TestReloadConfigKeepsDatabaseOwnedSettings is archie-core-ju85. The SIGHUP +// callback re-resolves the file document alone; without re-applying the +// control plane over it, every database-owned setting reverts to its file +// value until the process restarts, and the reload still logs success. +func TestReloadConfigKeepsDatabaseOwnedSettings(t *testing.T) { + b := newReloadBoot(t, &controlPlaneStub{values: databaseOwnedResources()}) + if err := b.loadRuntimeConfig(t.Context()); err != nil { + t.Fatalf("loadRuntimeConfig: %v", err) + } + b.applyWorkflowExecutionSettings(workflow.ExecutionSettings{ + MaxModelToolSteps: 99, MaxRuntime: time.Hour, MaxConsecutiveGateFailures: 5, + }, 4) + + if err := b.reloadConfig(t.Context(), &configuration.Document{Config: fileConfig()}); err != nil { + t.Fatalf("reloadConfig: %v", err) + } + + got := b.cfgHolder.Get() + for _, tt := range []struct { + field string + got any + want any + }{ + {"Providers", got.Providers["main"].BaseURL, "https://db.example.com"}, + {"Models", got.Models["builder"], "main/db-model"}, + {"Repos", got.Repos[0].Name, "from-database"}, + {"Chat", got.Chat.Operator, "database"}, + {"PollInterval", time.Duration(got.PollInterval), 2 * time.Minute}, + {"MaxRetries", got.MaxRetries, 7}, + {"PluginDir", got.PluginDir, "/db/plugins"}, + {"Containers", got.Containers.Image, "archie:from-database"}, + {"Budgets", got.Budgets.MaxSteps, 99}, + } { + if tt.got != tt.want { + t.Errorf("%s reverted to its file value after reload: got %v, want %v", tt.field, tt.got, tt.want) + } + } + if got.BotUser != "widget" { + t.Errorf("file-owned BotUser = %q, want the reloaded file value", got.BotUser) + } +} + +// TestReloadConfigFailsWhenControlPlaneUnreachable pins the direction of the +// failure: the running config already holds the database's values, so a +// control plane that cannot answer must abort the reload rather than publish +// the file's values over them. +func TestReloadConfigFailsWhenControlPlaneUnreachable(t *testing.T) { + stub := &controlPlaneStub{values: databaseOwnedResources()} + b := newReloadBoot(t, stub) + if err := b.loadRuntimeConfig(t.Context()); err != nil { + t.Fatalf("loadRuntimeConfig: %v", err) + } + + stub.queryErr = context.DeadlineExceeded + err := b.reloadConfig(t.Context(), &configuration.Document{Config: fileConfig()}) + if err == nil { + t.Fatal("reloadConfig with an unreachable control plane: expected an error") + } + if got := b.cfgHolder.Get(); got.Models["builder"] != "main/db-model" { + t.Fatalf("running config was replaced on the error path: Models = %v", got.Models) + } +} diff --git a/internal/app/archied/reload.go b/internal/app/archied/reload.go index c5d2fff48..747ed1c55 100644 --- a/internal/app/archied/reload.go +++ b/internal/app/archied/reload.go @@ -14,10 +14,11 @@ import ( // ReloadController re-runs the configuration load on demand and // publishes the result through apply. A failed reload never publishes: -// apply is called only with a Document that passed validation, and the -// running config is left untouched on error. That "Set is not called on -// the error path" property is the safety core of the reload feature -- -// a bad file must leave the daemon running on the previous config. +// apply is called only with a Document that passed validation, and apply +// itself reports failure rather than publishing a half-built config. That +// "Set is not called on the error path" property is the safety core of the +// reload feature -- a bad file, or a database layer that cannot be read, +// must leave the daemon running on the previous config. type ReloadController struct { loader *configuration.Loader base string @@ -26,18 +27,14 @@ type ReloadController struct { // apply receives a freshly loaded, validated Document. It is the // composition root's job to wire this to whatever owns the running // config (the daemon's Holder, the dashboard's provenance). Must be - // non-nil; newReloadController requires it. - apply func(*configuration.Document) - - // overlayValues, when non-nil, returns the runtime config overlay - // (dotted keys -> typed values) to layer over the file config before - // publishing. A failed overlay read or validation aborts the reload - // exactly like a bad file. + // non-nil; newReloadController requires it. An error from apply fails + // the reload and leaves the running config alone. + apply func(context.Context, *configuration.Document) error status atomic.Pointer[config.ReloadStatus] } -func newReloadController(loader *configuration.Loader, base, overlay string, apply func(*configuration.Document)) *ReloadController { +func newReloadController(loader *configuration.Loader, base, overlay string, apply func(context.Context, *configuration.Document) error) *ReloadController { c := &ReloadController{loader: loader, base: base, overlay: overlay, apply: apply} c.status.Store(&config.ReloadStatus{}) return c @@ -45,10 +42,13 @@ func newReloadController(loader *configuration.Loader, base, overlay string, app // Reload re-runs the configuration load. On validation failure it // records the error in Status and returns it; apply is not called and -// the running configuration is unchanged. On success it calls apply -// with the fresh Document and records the reload time. -func (c *ReloadController) Reload() error { +// the running configuration is unchanged. An error from apply is +// recorded the same way. On success it records the reload time. +func (c *ReloadController) Reload(ctx context.Context) error { doc, err := c.loader.Resolve(c.base, c.overlay) + if err == nil { + err = c.apply(ctx, doc) + } if err != nil { c.status.Store(&config.ReloadStatus{ LastError: err.Error(), @@ -56,7 +56,6 @@ func (c *ReloadController) Reload() error { }) return err } - c.apply(doc) c.status.Store(&config.ReloadStatus{ LastReloadAt: time.Now().UTC().Format(time.RFC3339), }) @@ -80,7 +79,7 @@ func reloadLoop(ctx context.Context, ch <-chan os.Signal, c *ReloadController, l case <-ctx.Done(): return case <-ch: - if err := c.Reload(); err != nil { + if err := c.Reload(ctx); err != nil { log.Error("config reload failed; keeping running config", "err", err) } } diff --git a/internal/app/archied/reload_test.go b/internal/app/archied/reload_test.go index ec558742b..cd4072483 100644 --- a/internal/app/archied/reload_test.go +++ b/internal/app/archied/reload_test.go @@ -2,6 +2,7 @@ package archied import ( "context" + "errors" "log/slog" "os" "path/filepath" @@ -40,11 +41,12 @@ func TestReloadAppliesAndPublishes(t *testing.T) { loader := configuration.New(slog.New(slog.DiscardHandler)) var published config.Config - controller := newReloadController(loader, path, "", func(doc *configuration.Document) { + controller := newReloadController(loader, path, "", func(_ context.Context, doc *configuration.Document) error { published = doc.Config + return nil }) - if err := controller.Reload(); err != nil { + if err := controller.Reload(t.Context()); err != nil { t.Fatalf("first reload: %v", err) } if published.BotUser != "first" { @@ -52,7 +54,7 @@ func TestReloadAppliesAndPublishes(t *testing.T) { } writeConfig(t, path, minimalConfigTOML("second")) - if err := controller.Reload(); err != nil { + if err := controller.Reload(t.Context()); err != nil { t.Fatalf("second reload: %v", err) } if published.BotUser != "second" { @@ -74,9 +76,9 @@ func TestReloadValidationFailureDoesNotApply(t *testing.T) { loader := configuration.New(slog.New(slog.DiscardHandler)) applied := 0 - controller := newReloadController(loader, path, "", func(*configuration.Document) { applied++ }) + controller := newReloadController(loader, path, "", func(context.Context, *configuration.Document) error { applied++; return nil }) - if err := controller.Reload(); err != nil { + if err := controller.Reload(t.Context()); err != nil { t.Fatalf("first reload: %v", err) } if applied != 1 { @@ -84,7 +86,7 @@ func TestReloadValidationFailureDoesNotApply(t *testing.T) { } writeConfig(t, path, invalidConfigTOML()) - err := controller.Reload() + err := controller.Reload(t.Context()) if err == nil { t.Fatal("reload with invalid config: expected an error") } @@ -97,7 +99,7 @@ func TestReloadValidationFailureDoesNotApply(t *testing.T) { // A subsequent valid reload clears the error. writeConfig(t, path, minimalConfigTOML("widget")) - if err := controller.Reload(); err != nil { + if err := controller.Reload(t.Context()); err != nil { t.Fatalf("recovery reload: %v", err) } if st := controller.Status(); st.LastError != "" || st.LastReloadAt == "" { @@ -105,6 +107,33 @@ func TestReloadValidationFailureDoesNotApply(t *testing.T) { } } +// TestReloadApplyFailureIsRecorded pins the other half of the safety +// property: a file that loads cleanly can still fail to apply, because the +// database layer reloadConfig re-applies comes from the control plane. That +// failure must be recorded and returned, not reported as a successful +// reload. +func TestReloadApplyFailureIsRecorded(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "config.toml") + writeConfig(t, path, minimalConfigTOML("widget")) + + loader := configuration.New(slog.New(slog.DiscardHandler)) + controller := newReloadController(loader, path, "", func(context.Context, *configuration.Document) error { + return errors.New("runtime settings unavailable") + }) + + if err := controller.Reload(t.Context()); err == nil { + t.Fatal("reload with a failing apply: expected an error") + } + st := controller.Status() + if st.LastError == "" || st.LastErrorAt == "" { + t.Fatalf("Status after a failed apply = %+v, want LastError and LastErrorAt set", st) + } + if st.LastReloadAt != "" { + t.Fatalf("Status after a failed apply recorded a reload time: %+v", st) + } +} + func TestReloadLoopAppliesOnSignalAndStopsOnCancel(t *testing.T) { dir := t.TempDir() path := filepath.Join(dir, "config.toml") @@ -112,8 +141,9 @@ func TestReloadLoopAppliesOnSignalAndStopsOnCancel(t *testing.T) { loader := configuration.New(slog.New(slog.DiscardHandler)) applied := make(chan struct{}, 1) - controller := newReloadController(loader, path, "", func(*configuration.Document) { + controller := newReloadController(loader, path, "", func(context.Context, *configuration.Document) error { applied <- struct{}{} + return nil }) ctx, cancel := context.WithCancel(context.Background())